Abstract: Embodiments described herein provide regression testing using artificial intelligence. A regression testing network model for a first plurality of organizations using a common codebase is provided. The regression testing network model provides an organization finite state machine (FSM) model for each organization. A first dataset including samples of the organization FSM models based on regression testing for one or more previous releases of the common codebase prior to a first release of the common codebase is received. A training dataset is generated based on the first dataset. The regression testing network model using the training dataset. A second plurality of organizations for regression testing for the first release is determined, from the first plurality of organizations, using the trained regression testing network model.

Abstract: Embodiments described herein provides a training mechanism that transfers the knowledge from a trained BERT model into a much smaller model to approximate the behavior of BERT. Specifically, the BERT model may be treated as a teacher model, and a much smaller student model may be trained using the same inputs to the teacher model and the output from the teacher model. In this way, the student model can be trained within a much shorter time than the BERT teacher model, but with comparable performance with BERT.

Abstract: Embodiments described herein provide safe policy improvement (SPI) in a batch reinforcement learning framework for a task-oriented dialogue. Specifically, a batch reinforcement learning framework for dialogue policy learning is provided, which improves the performance of the dialogue and learns to shape a reward that reasons the invention behind human response rather than just imitating the human demonstration.

Abstract: Methods, apparatuses, and computer-program products are disclosed. The method may include inputting one or more subcomponent training datasets into the plurality of subcomponent models of the machine learning model, the machine learning model may be configured to perform a final task, and the plurality of subcomponent models may be configured to perform sequential subtasks that result in the final task. The method may include computing one or more weights for data points of the one or more subcomponent training datasets and the one or more weights may be based on a contribution of the data points to an end-to-end error loss measurement associated with performing the final task of the machine learning model. The method may include training the plurality of subcomponent models based on the one or more weights for the data points of the one or more subcomponent training datasets.

Abstract: A system performs group testing on a population of items. The group testing identifies items satisfying particular criteria from a population of items, for example, defective items from the population. The group testing may be performed for software or hardware testing, for testing a human population, for training of deep learning applications, and so on. The system trains a machine learning based model, for example, a reinforcement learning based model to evaluate groups. The model may further determine system dynamics that may represent priors of items. An agent treats the population and groups of items being tested as the environment and performs actions, for example, adjusting the groups. The system also performs a non-adaptive strategy based on monte carlo simulation of tests based on a simulation results.

Abstract: An embodiment proposed herein uses sparsification techniques to train the neural network with a high feature dimension that may yield desirable in-domain detection accuracy but may prune away dimensions in the output that are less important. Specifically, a sparsification vector is generated based on Gaussian distribution (or other probabilistic distribution) and is used to multiply with the higher dimension output to reduce the number of feature dimensions. The pruned output may be then used for the neural network to learn the sparsification vector. In this way, out-of-distribution detection accuracy can be improved.

Abstract: An embodiment provided herein preprocesses the input samples to the classification neural network, e.g., by adding Gaussian noise to word/sentence representations to make the function of the neural network satisfy Lipschitz property such that a small change in the input does not cause much change to the output if the input sample is in-distribution. Method to induce properties in the feature representation of neural network such that for out-of-distribution examples the feature representation magnitude is either close to zero or the feature representation is orthogonal to all class representations. Method to generate examples that are structurally similar to in-domain and semantically out-of domain for use in out-of-domain classification training. Method to prune feature representation dimension to mitigate long tail error of unused dimension in out-of-domain classification. Using these techniques, the accuracy of both in-domain and out-of-distribution identification can be improved.

Abstract: A method for natural language processing includes receiving, by one or more processors, an unstructured text input. An entity classifier is used to identify entities in the unstructured text input. The identifying the entities includes generating, using a plurality of sub-classifiers of a hierarchical neural network classifier of the entity classifier, a plurality of lower-level entity identifications associated with the unstructured text input. The identifying the entities further includes generating, using a combiner of the hierarchical neural network classifier, a plurality of higher-level entity identifications associated with the unstructured text input based on the plurality of lower-level entity identifications. Identified entities are provided based on the plurality of higher-level entity identifications.

Abstract: Embodiments described herein provide safe policy improvement (SPI) in a batch reinforcement learning framework for a task-oriented dialogue. Specifically, a batch reinforcement learning framework for dialogue policy learning is provided, which improves the performance of the dialogue and learns to shape a reward that reasons the invention behind human response rather than just imitating the human demonstration.

Abstract: Embodiments described herein provide safe policy improvement (SPI) in a batch reinforcement learning framework for a task-oriented dialogue. Specifically, a batch reinforcement learning framework for dialogue policy learning is provided, which improves the performance of the dialogue and learns to shape a reward that reasons the invention behind human response rather than just imitating the human demonstration.

Abstract: A method for using a neural network to generate an improved graph model includes receiving, by the neural network, a graph model. The graph model is based on data relating to an environment for allocating resources to a first group and a second group. The method further includes receiving, by the neural network, a budget for editing the graph model based on a cost of corresponding modification to the environment, and determining, by the neural network, a fairness representation based on a fairness requirement between the first and second groups. It is determined by the neural network, a utility function for the graph model based on first and second group utilities representing resource allocation to the first and second groups respectively. Reinforcement learning is performed on the neural network to generate the improved graph model using the utility function and the fairness representation.

Abstract: An embodiment proposed herein uses sparsification techniques to train the neural network with a high feature dimension that may yield desirable in-domain detection accuracy but may prune away dimensions in the output that are less important. Specifically, a sparsification vector is generated based on Gaussian distribution (or other probabilistic distribution) and is used to multiply with the higher dimension output to reduce the number of feature dimensions. The pruned output may be then used for the neural network to learn the sparsification vector. In this way, out-of-distribution detection accuracy can be improved.

Abstract: Embodiments described herein provides a training mechanism that transfers the knowledge from a trained BERT model into a much smaller model to approximate the behavior of BERT. Specifically, the BERT model may be treated as a teacher model, and a much smaller student model may be trained using the same inputs to the teacher model and the output from the teacher model. In this way, the student model can be trained within a much shorter time than the BERT teacher model, but with comparable performance with BERT.

Abstract: An embodiment provided herein preprocesses the input samples to the classification neural network, e.g., by adding Gaussian noise to word/sentence representations to make the function of the neural network satisfy Lipschitz property such that a small change in the input does not cause much change to the output if the input sample is in-distribution. Method to induce properties in the feature representation of neural network such that for out-of-distribution examples the feature representation magnitude is either close to zero or the feature representation is orthogonal to all class representations. Method to generate examples that are structurally similar to in-domain and semantically out-of domain for use in out-of-domain classification training. Method to prune feature representation dimension to mitigate long tail error of unused dimension in out-of-domain classification. Using these techniques, the accuracy of both in-domain and out-of-distribution identification can be improved.

Abstract: Associating job responsibilities with job titles is described. A database system identifies a job level and a job department associated with a job title stored by an object. The database system identifies level-based job responsibilities associated with the job level. The database system identifies department-based job responsibilities associated with the job department. The database system identifies at least one job responsibility associated with the job title based on the level-based job responsibilities and the department-based job responsibilities. The database system stores each identified job responsibility in metadata and/or a field associated with the object. The database system outputs a message based on the object to a user device, in response to a search that specifies any identified job responsibility.

Abstract: For a database system accessible by one or more users, a neural network model and related method are provided that allow a user of the database system to provide unstructured input in the form of a verbal or textual narrative or utterance that expresses the information in a language and manner that is more comfortable for the user. A portion of the narrative or utterance may relate to one or action items that the user intends to be taken with respect to the database system, such as creating, updating, modifying, or deleting a database item (e.g., contact, calendar item, deal, etc.). The neural model processes the unstructured input (narrative or utterance) and determines or classifies the intent with respect to the action item for the database.

Abstract: A method for natural language processing includes receiving, by one or more processors, an unstructured text input. An entity classifier is used to identify entities in the unstructured text input. The identifying the entities includes generating, using a plurality of sub-classifiers of a hierarchical neural network classifier of the entity classifier, a plurality of lower-level entity identifications associated with the unstructured text input. The identifying the entities further includes generating, using a combiner of the hierarchical neural network classifier, a plurality of higher-level entity identifications associated with the unstructured text input based on the plurality of lower-level entity identifications. Identified entities are provided based on the plurality of higher-level entity identifications.

Abstract: Contact recommendations based on purchase history are described. A system creates a directed graph of nodes in which at least some of the nodes are connected by directed arcs, wherein a directed arc from a first node to a second node represents a conditional probability that previous users who purchased a first contact also purchased a second contact. The system identifies a set of contacts purchased by a current user. The system estimates a prospective purchase probability based on a historical probability that previous users purchased a specific contact and a related probability that previous users who purchased the specific contact also purchased a contact in the set of contacts, for each candidate contact. The system outputs a recommendation for the current user to purchase a recommended candidate contact based on a corresponding prospective purchase probability.

Abstract: Identifying entities in semi-structured content is described. A system assigns a corresponding entity type based on a corresponding entity type score for each token in a sequence of tokens in semi-structured content, based on multiple entity types, wherein each token is a corresponding character set. The system assigns a corresponding boundary type based on a corresponding boundary type score for each token in the sequence of tokens, based on a begin boundary type or a continue boundary type. The system identifies an entity based on a corresponding entity type score and a corresponding boundary type for each token in the sequence of tokens. The system outputs the sequence of tokens as an identified set of entities based on the identified entity.

Abstract: Associating job responsibilities with job titles is described. A database system identifies a job level and a job department associated with a job title stored by an object. The database system identifies level-based job responsibilities associated with the job level. The database system identifies department-based job responsibilities associated with the job department. The database system identifies at least one job responsibility associated with the job title based on the level-based job responsibilities and the department-based job responsibilities. The database system stores each identified job responsibility in metadata and/or a field associated with the object. The database system outputs a message based on the object to a user device, in response to a search that specifies any identified job responsibility.