Abstract: Embodiments described herein provides a training mechanism that transfers the knowledge from a trained BERT model into a much smaller model to approximate the behavior of BERT. Specifically, the BERT model may be treated as a teacher model, and a much smaller student model may be trained using the same inputs to the teacher model and the output from the teacher model. In this way, the student model can be trained within a much shorter time than the BERT teacher model, but with comparable performance with BERT.

Abstract: A hydrocarbon resin polymer is provided. The hydrocarbon resin polymer includes: 0.15-15 mol % of repeating units (A), which are derived from bridged ring monomer compounds; 15-90 mol % of repeating units (B), which are derived from monovinyl aromatic compounds; and 8-80 mol % of repeating units (C), which are derived from divinyl aromatic compounds.

Abstract: Embodiments provide a software framework for evaluating and troubleshooting real-world task-oriented bot systems. Specifically, the evaluation framework includes a generator that infers dialog acts and entities from bot definitions and generates test cases for the system via model-based paraphrasing. The framework may also include a simulator for task-oriented dialog user simulation that supports both regression testing and end-to-end evaluation. The framework may also include a remediator to analyze and visualize the simulation results, remedy some of the identified issues, and provide actionable suggestions for improving the task-oriented dialog system.

Abstract: An embodiment proposed herein uses sparsification techniques to train the neural network with a high feature dimension that may yield desirable in-domain detection accuracy but may prune away dimensions in the output that are less important. Specifically, a sparsification vector is generated based on Gaussian distribution (or other probabilistic distribution) and is used to multiply with the higher dimension output to reduce the number of feature dimensions. The pruned output may be then used for the neural network to learn the sparsification vector. In this way, out-of-distribution detection accuracy can be improved.

Abstract: An embodiment provided herein preprocesses the input samples to the classification neural network, e.g., by adding Gaussian noise to word/sentence representations to make the function of the neural network satisfy Lipschitz property such that a small change in the input does not cause much change to the output if the input sample is in-distribution. Method to induce properties in the feature representation of neural network such that for out-of-distribution examples the feature representation magnitude is either close to zero or the feature representation is orthogonal to all class representations. Method to generate examples that are structurally similar to in-domain and semantically out-of domain for use in out-of-domain classification training. Method to prune feature representation dimension to mitigate long tail error of unused dimension in out-of-domain classification. Using these techniques, the accuracy of both in-domain and out-of-distribution identification can be improved.

Abstract: Methods, systems, and devices for data retention handling are described. In some data storage systems, data objects are stored in a non-relational database schema. The system may support configurable data retention policies for different tenants, users, or applications. For example, a data store may receive retention requests, where the retention requests may specify deletion or exportation actions to perform on records contained within data objects. The data store may determine retention rules based on these retention requests, and may periodically or aperiodically evaluate the rules to determine active actions to perform. To improve the efficiency of the system, the data store may aggregate the active actions (e.g., according to the dataset to perform the actions on), and may generate work items corresponding to the aggregate actions. A work processor may retrieve these work items and may efficiently perform the data retention actions on datasets stored in the data object store.

Abstract: An embodiment proposed herein uses sparsification techniques to train the neural network with a high feature dimension that may yield desirable in-domain detection accuracy but may prune away dimensions in the output that are less important. Specifically, a sparsification vector is generated based on Gaussian distribution (or other probabilistic distribution) and is used to multiply with the higher dimension output to reduce the number of feature dimensions. The pruned output may be then used for the neural network to learn the sparsification vector. In this way, out-of-distribution detection accuracy can be improved.

Abstract: Embodiments described herein provides a training mechanism that transfers the knowledge from a trained BERT model into a much smaller model to approximate the behavior of BERT. Specifically, the BERT model may be treated as a teacher model, and a much smaller student model may be trained using the same inputs to the teacher model and the output from the teacher model. In this way, the student model can be trained within a much shorter time than the BERT teacher model, but with comparable performance with BERT.

Abstract: An embodiment provided herein preprocesses the input samples to the classification neural network, e.g., by adding Gaussian noise to word/sentence representations to make the function of the neural network satisfy Lipschitz property such that a small change in the input does not cause much change to the output if the input sample is in-distribution. Method to induce properties in the feature representation of neural network such that for out-of-distribution examples the feature representation magnitude is either close to zero or the feature representation is orthogonal to all class representations. Method to generate examples that are structurally similar to in-domain and semantically out-of domain for use in out-of-domain classification training. Method to prune feature representation dimension to mitigate long tail error of unused dimension in out-of-domain classification. Using these techniques, the accuracy of both in-domain and out-of-distribution identification can be improved.

Abstract: Methods, systems, and devices for data access and processing are described. To set up secure environments for data processing (e.g., including machine learning), an access control system may first receive approval from an authorized user (e.g., an approver) granting access to data objects in a multi-tenant data store. The system may determine tenant-specific paths for retrieving the data objects from the data store, and may initialize a number of virtual computing engines for accessing the data. Each computing engine may be tenant-specific based on the path(s) used by that computing engine, and each may include an access role defining the data objects or data object types accessible by that computing engine. By accessing the requested data objects according to the tenant-specific path prefixes and access roles, the virtual computing engines may securely maintain separate environments for different tenants and may only allow user access to approved tenant data.

Abstract: Methods, systems, and devices for data access and processing are described. To set up secure environments for data processing (e.g., including machine learning), an access control system may first receive approval from an authorized user (e.g., an approver) granting access to data objects in a multi-tenant data store. The system may determine tenant-specific paths for retrieving the data objects from the data store, and may initialize a number of virtual computing engines for accessing the data. Each computing engine may be tenant-specific based on the path(s) used by that computing engine, and each may include an access role defining the data objects or data object types accessible by that computing engine. By accessing the requested data objects according to the tenant-specific path prefixes and access roles, the virtual computing engines may securely maintain separate environments for different tenants and may only allow user access to approved tenant data.

Abstract: Methods, systems, and devices for data retention handling are described. In some data storage systems, data objects are stored in a non-relational database schema. The system may support configurable data retention policies for different tenants, users, or applications. For example, a data store may receive retention requests, where the retention requests may specify deletion or exportation actions to perform on records contained within data objects. The data store may determine retention rules based on these retention requests, and may periodically or aperiodically evaluate the rules to determine active actions to perform. To improve the efficiency of the system, the data store may aggregate the active actions (e.g., according to the dataset to perform the actions on), and may generate work items corresponding to the aggregate actions. A work processor may retrieve these work items and may efficiently perform the data retention actions on datasets stored in the data object store.