Abstract: Embodiments described herein provides a training mechanism that transfers the knowledge from a trained BERT model into a much smaller model to approximate the behavior of BERT. Specifically, the BERT model may be treated as a teacher model, and a much smaller student model may be trained using the same inputs to the teacher model and the output from the teacher model. In this way, the student model can be trained within a much shorter time than the BERT teacher model, but with comparable performance with BERT.

Abstract: Methods, systems, and devices for analyzing communication messages (e.g., emails or activities) to determine relationship strength using a distributed graph are described. In some systems, a user may be associated with a specific tenant. A database server of the system may receive communication messages associated with the user and a target user. The server may perform a natural language processing (NLP) analysis on the communication messages to extract metadata, and may generate or update a distributed graph indicating connections between users based on the extracted metadata. Using the connections of the graph, the server may calculate a closeness score between the user and the target user. Additionally, the server may calculate closeness scores between the target and other users associated with the tenant, and may determine the users with the greatest closeness scores. The server may send a suggestion for the determined users to initiate communication with the target.

Abstract: An embodiment proposed herein uses sparsification techniques to train the neural network with a high feature dimension that may yield desirable in-domain detection accuracy but may prune away dimensions in the output that are less important. Specifically, a sparsification vector is generated based on Gaussian distribution (or other probabilistic distribution) and is used to multiply with the higher dimension output to reduce the number of feature dimensions. The pruned output may be then used for the neural network to learn the sparsification vector. In this way, out-of-distribution detection accuracy can be improved.

Abstract: An embodiment provided herein preprocesses the input samples to the classification neural network, e.g., by adding Gaussian noise to word/sentence representations to make the function of the neural network satisfy Lipschitz property such that a small change in the input does not cause much change to the output if the input sample is in-distribution. Method to induce properties in the feature representation of neural network such that for out-of-distribution examples the feature representation magnitude is either close to zero or the feature representation is orthogonal to all class representations. Method to generate examples that are structurally similar to in-domain and semantically out-of domain for use in out-of-domain classification training. Method to prune feature representation dimension to mitigate long tail error of unused dimension in out-of-domain classification. Using these techniques, the accuracy of both in-domain and out-of-distribution identification can be improved.

Abstract: Methods, systems, and devices for analyzing communication messages (e.g., emails) and selecting corresponding actions are described. In some database systems, a user may receive multiple messages at a user device. To efficiently determine responses to these messages, the user device may send the messages to a backend server for analysis. The server may perform natural language processing (NLP) to classify the message with one or more binary classifications and may extract metadata from each message. Based on the classifications and the metadata, the server may determine one or more actions the user device may perform to respond to each message. The server may send instructions to the user device indicating the suggested actions, and the user device may display these actions as options to a user. Additionally, the user device may use the classifications and metadata to automatically generate one or more communication templates in response to the message.

Abstract: An embodiment proposed herein uses sparsification techniques to train the neural network with a high feature dimension that may yield desirable in-domain detection accuracy but may prune away dimensions in the output that are less important. Specifically, a sparsification vector is generated based on Gaussian distribution (or other probabilistic distribution) and is used to multiply with the higher dimension output to reduce the number of feature dimensions. The pruned output may be then used for the neural network to learn the sparsification vector. In this way, out-of-distribution detection accuracy can be improved.

Abstract: Embodiments described herein provides a training mechanism that transfers the knowledge from a trained BERT model into a much smaller model to approximate the behavior of BERT. Specifically, the BERT model may be treated as a teacher model, and a much smaller student model may be trained using the same inputs to the teacher model and the output from the teacher model. In this way, the student model can be trained within a much shorter time than the BERT teacher model, but with comparable performance with BERT.

Abstract: An embodiment provided herein preprocesses the input samples to the classification neural network, e.g., by adding Gaussian noise to word/sentence representations to make the function of the neural network satisfy Lipschitz property such that a small change in the input does not cause much change to the output if the input sample is in-distribution. Method to induce properties in the feature representation of neural network such that for out-of-distribution examples the feature representation magnitude is either close to zero or the feature representation is orthogonal to all class representations. Method to generate examples that are structurally similar to in-domain and semantically out-of domain for use in out-of-domain classification training. Method to prune feature representation dimension to mitigate long tail error of unused dimension in out-of-domain classification. Using these techniques, the accuracy of both in-domain and out-of-distribution identification can be improved.

Abstract: A database server may identify mentioned names in a body of a message and extract the names using name identification heuristics and algorithms. The service retrieves or utilizes a distributed connection graph to identify contacts associated with the parties to the conversation that may match or be similar to the mentioned name. Contacts may be scored based on similarities between the extracted name the names associated with nodes of the graph, as well as other factors. The highest scoring contact may be surfaced or displayed to one or more of the parties to the communication message.

Abstract: A database server may analyze interaction data including communication to generate a graph representation of various users and connections between the users. The database server may utilize the graph representation of connections to identify sufficiently connected target user identifiers in one or more external organizations. A connection metric may be assigned to each user identifier of one or more groups of user identifiers generated using the graph representation, and the target user identifiers may be identified based on the connection metrics.

Abstract: A database server may analyze interaction data including communication to generate a graph representation of various users and connections between the users. The database server may utilize the graph representation of connections to identify sufficiently connected target user identifiers in one or more external organizations. A connection metric may be assigned to each user identifier of one or more groups of user identifiers generated using the graph representation, and the target user identifiers may be identified based on the connection metrics.

Abstract: A database server may identify mentioned names in a body of a message and extract the names using name identification heuristics and algorithms. The service retrieves or utilizes a distributed connection graph to identify contacts associated with the parties to the conversation that may match or be similar to the mentioned name. Contacts may be scored based on similarities between the extracted name the names associated with nodes of the graph, as well as other factors. The highest scoring contact may be surfaced or displayed to one or more of the parties to the communication message.

Abstract: Methods, systems, and devices for analyzing communication messages (e.g., emails) and selecting corresponding actions are described. In some database systems, a user may receive multiple messages at a user device. To efficiently determine responses to these messages, the user device may send the messages to a backend server for analysis. The server may perform natural language processing (NLP) to classify the message with one or more binary classifications and may extract metadata from each message. Based on the classifications and the metadata, the server may determine one or more actions the user device may perform to respond to each message. The server may send instructions to the user device indicating the suggested actions, and the user device may display these actions as options to a user. Additionally, the user device may use the classifications and metadata to automatically generate one or more communication templates in response to the message.

Abstract: Methods, systems, and devices for analyzing communication messages (e.g., emails or activities) to determine relationship strength using a distributed graph are described. In some systems, a user may be associated with a specific tenant. A database server of the system may receive communication messages associated with the user and a target user. The server may perform a natural language processing (NLP) analysis on the communication messages to extract metadata, and may generate or update a distributed graph indicating connections between users based on the extracted metadata. Using the connections of the graph, the server may calculate a closeness score between the user and the target user. Additionally, the server may calculate closeness scores between the target and other users associated with the tenant, and may determine the users with the greatest closeness scores. The server may send a suggestion for the determined users to initiate communication with the target.