Abstract: Techniques are disclosed for providing a scalable multi-tenant serve pool for chatbot systems. A query serving system (QSS) receives a request to serve a query for a skillbot. The QSS includes: (i) a plurality of deployments in a serving pool, and (ii) a plurality of deployments in a free pool. The QSS determines whether a first deployment from the plurality of deployments in the serving pool can serve the query based on an identifier of the skillbot. In response to determining that the first deployment cannot serve the query, the QSS selects a second deployment from the plurality of deployments in the free pool to be assigned to the skillbot, and loads a machine-learning model associated with the skillbot into the second deployment, wherein the machine-learning model is trained to serve the query for the skillbot. The query is served using the machine-learning model loaded into the second deployment.

Abstract: Techniques are disclosed herein for transforming natural language conversations into a visual output. In one aspect, a computer-implement method includes generating an input string by concatenating a natural language utterance with a schema representation comprising a set of entities for visualization actions, generating, by a first encoder of a machine learning model, one or more embeddings of the input string, encoding, by a second encoder of the machine learning model, relations between elements in the schema representation and words in the natural language utterance based on the one or more embeddings, generating, by a grammar-based decoder of the machine learning model and based on the encoded relations and the one or more embeddings, an intermediate logical form that represents at least the query, the one or more visualization actions, or the combination thereof, and generating, based on the intermediate logical form, a command for a computing system.

Abstract: The present disclosure relates to manufacturing training data by leveraging an automated pipeline that manufactures visualization training datasets to train a machine learning model to convert a natural language utterance into meaning representation language logical form that includes one or more visualization actions. Aspects are directed towards accessing an original training dataset, a visualization query dataset, an incremental visualization dataset, a manipulation visualization dataset, or any combination thereof. One or more visualization training datasets are generated by: (i) modifying examples in the original training dataset, the visualization query dataset, or both to include visualization actions, (ii) generating examples, using the incremental visualization dataset, the manipulation visualization dataset, or both, that include visualization actions, or (iii) both (i) and (ii).

Abstract: The present disclosure relates to chatbot systems, and more particularly, to batching techniques for handling unbalanced training data when training a model such that bias is removed from the trained machine learning model when performing inference. In an embodiment, a plurality of raw utterances is obtained. A bias eliminating distribution is determined and a subset of the plurality of raw utterances is batched according to the bias-reducing distribution. The resulting unbiased training data may be input into a prediction model for training the prediction model. The trained prediction model may be obtained and utilized to predict unbiased results from new inputs received by the trained prediction model.

Abstract: Techniques for automatically switching between chatbot skills in the same domain. In one particular aspect, a method is provided that includes receiving an utterance from a user within a chatbot session, where a current skill context is a first skill and a current group context is a first group, inputting the utterance into a candidate skills model for the first group, obtaining, using the candidate skills model, a ranking of skills within the first group, determining, based on the ranking of skills, a second skill is a highest ranked skill, changing the current skill context of the chatbot session to the second skill, inputting the utterance into a candidate flows model for the second skill, obtaining, using the candidate flows model, a ranking of intents within the second skill that match the utterance, and determining, based on the ranking of intents, an intent that is a highest ranked intent.

Abstract: Techniques for using logit values for classifying utterances and messages input to chatbot systems in natural language processing. A method can include a chatbot system receiving an utterance generated by a user interacting with the chatbot system. The chatbot system can input the utterance into a machine-learning model including a set of binary classifiers. Each binary classifier of the set of binary classifiers can be associated with a modified logit function. The method can also include the machine-learning model using the modified logit function to generate a set of distance-based logit values for the utterance. The method can also include the machine-learning model applying an enhanced activation function to the set of distance-based logit values to generate a predicted output. The method can also include the chatbot system classifying, based on the predicted output, the utterance as being associated with the particular class.

Abstract: A training request including an identifier that is indicative of a type of a machine learning (ML) model that is to be trained is received. A plurality of workers are maintained in a training pool, and a plurality of jobs are maintained in a queue of training jobs. Each worker is configured to train a particular type of ML model. Upon the training request being validated, a training job is created for the request and submitted to the queue of training jobs. For each type of ML model, a first metric and a second metric is obtained. A target metric is computed based on the first and the second metrics. The number of workers included in the training pool is modified based on the target metric.

Abstract: A method includes receiving an indication of a first coverage value corresponding to a desired overlap between a dataset of natural language phrases and a training dataset for training a machine learning model; determining a second coverage value corresponding to a measured overlap between the dataset of natural language phrases and the training dataset; determining a coverage delta value based on a comparison between the first coverage value and the second coverage value; modifying, based on the coverage delta value, the dataset of natural language phrases; and processing, utilizing a machine learning model including the modified dataset of natural language phrases, an input dataset including a set of input features. The machine learning model processes the input dataset based at least in part on the dataset of natural language phrases to generate an output dataset.

Abstract: A data corpus is partitioned into text strings for header classification. A group characteristic is computed for a text string, and whether the group characteristic satisfies a group characteristic criterion is determined. The text string may be disqualified from header classification if the group characteristic criterion is not satisfied, or one or more font characteristics may be determined for the text string if the group characteristic criterion is satisfied. A font characteristic that meets one or more prevalence criteria may be identified and evaluated to determine whether the font characteristic meets at least one font characteristic criterion. The text string may be disqualified from header classification if the font characteristic criterion is not satisfied, or if the font characteristic meets the font characteristic criterion, the text string is classified as a header, and tagged content is generated by applying a header tag to the text string.

Abstract: Techniques are disclosed for providing a scalable multi-tenant serve pool for chatbot systems. A query serving system (QSS) receives a request to serve a query for a skillbot. The QSS includes: (i) a plurality of deployments in a serving pool, and (ii) a plurality of deployments in a free pool. The QSS determines whether a first deployment from the plurality of deployments in the serving pool can serve the query based on an identifier of the skillbot. In response to determining that the first deployment cannot serve the query, the QSS selects a second deployment from the plurality of deployments in the free pool to be assigned to the skillbot, and loads a machine-learning model associated with the skillbot into the second deployment, wherein the machine-learning model is trained to serve the query for the skillbot. The query is served using the machine-learning model loaded into the second deployment.

Abstract: Techniques are disclosed for systems including techniques for multi-feature balancing for natural langue processors. In an embodiment, a method includes receiving a natural language query to be processed by a machine learning model, the machine learning model utilizing a dataset of natural language phrases for processing natural language queries, determining, based on the machine learning model and the natural language query, a feature dropout value, generating, and based on the natural language query, one or more contextual features and one or more expressional features that may be input to the machine learning model, modifying at least one or the one or more contextual features and the one or more expressional features based on the feature dropout value to generate a set of input features for the machine learning model, and processing the set of input features to cause generating an output dataset for corresponding to the natural language query.

Abstract: Techniques for keyword data augmentation for training chatbot systems in natural language processing. In one particular aspect, a method is provided that includes receiving a training set of utterances for training a machine-learning model to identify one or more intents for one or more utterances, augmenting the training set of utterances with out-of-domain (OOD) examples. The augmenting includes: identifying keywords within utterances of the training set of utterances, generating a set of OOD examples with the identified keywords, filtering out OOD examples from the set of OOD examples that have a context substantially similar to context of the utterances of the training set of utterances, and incorporating the set of OOD examples without the filtered OOD examples into the training set of utterances to generate an augmented training set of utterances. Thereafter, the machine-learning model is trained using the augmented training set of utterances.

Abstract: A chatbot system is configured to execute code to perform determining, by the chatbot system, a classification result for an utterance and one or more anchors each anchor of the one or more anchors corresponding to one or more anchor words of the utterance. For each anchor of the one or more anchors, one or more synthetic utterances are generated, and one or more classification results for the one or more synthetic utterances are determined. A report is generated by the chatbot system including a representation of a particular anchor of the one or more anchors, the particular anchor corresponding to a highest confidence value among the one or more anchors. The one or more synthetic utterances may be used to generate a new training dataset for training a machine-learning model. The training dataset may be refined according to a threshold confidence values to filter out datasets for training.

Abstract: Techniques for multi-layer training of a machine learning model are disclosed. A system pre-trains a machine learning model on training data obtained from unlabeled document graph data by executing unsupervised pre-training tasks on the unlabeled document graph data to generate a labeled pre-training data set. The system modifies document graphs to change attributes of nodes in the document graphs. The system pre-trains the machine learning model with a data set including the modified document graphs and un-modified document graphs to generate prediction associated with the modifications to the document graphs. Subsequent to pre-training, the system fine-tunes the machine learning model with a set of labeled training data to generate predictions associated with a specific attribute of a document graph.

Abstract: A training request including an identifier that is indicative of a type of a machine learning (ML) model that is to be trained is received. A plurality of workers are maintained in a training pool, and a plurality of jobs are maintained in a queue of training jobs. Each worker is configured to train a particular type of ML model. Upon the training request being validated, a training job is created for the request and submitted to the queue of training jobs. For each type of ML model, a first metric and a second metric is obtained. A target metric is computed based on the first and the second metrics. The number of workers included in the training pool is modified based on the target metric.

Abstract: A chatbot system is configured to execute code to perform determining, by the chatbot system, a classification result for an utterance and one or more anchors each anchor of the one or more anchors corresponding to one or more anchor words of the utterance. For each anchor of the one or more anchors, one or more synthetic utterances are generated, and one or more classification results for the one or more synthetic utterances are determined. A report is generated by the chatbot system comprising a representation of a particular anchor of the one or more anchors, the particular anchor corresponding to a highest confidence value among the one or more anchors. The one or more synthetic utterances may be used to generate a new training dataset for training a machine-learning model. The training dataset may be refined according to a threshold confidence values to filter out datasets for training.

Abstract: Techniques are disclosed for systems including techniques for multi-factor modelling for training and utilizing chatbot systems for natural language processing. In an embodiment, a method includes receiving a set of utterance data corresponding to a natural language-based query, determining one or more intents for the chatbot corresponds to a possible context for the natural language-based query and associated with a skill for the chatbot, generating one or more intent classification datasets, each intent classification dataset associated with a probability that the natural language query corresponds to an intent of the one or more intents, generating one or more transformed datasets each corresponding to a skill of one or more skills, determining a first skill of the one or more skills based on the one or more transformed datasets and processing, based on the determined first skill, the set of utterance data to resolve the natural language-based query.

Abstract: The present disclosure relates to techniques for identifying out-of-domain utterances.

Abstract: Techniques for generating formatting tags for textual content obtained from a source electronic document are disclosed. A system parses a digital file to obtain information about characters in an electronic document. The system applies tags to text generated based on the textual content of the electronic document by creating segments of textually-consecutive characters and applying corresponding text formatting style tags to the segments. The system further identifies segments of text overlapping bounding boxes in the electronic document. The system generates textual content including a segment of text and a corresponding hyperlink associated with the segment of text. The system further generates textual content by selectively applying line breaks from the source electronic document in the textual content.

Abstract: A method includes receiving a training set of utterances for training a machine-learning model to identify one or more intents for one or more utterances, and augmenting the training set of utterances with out-of-domain (OOD) examples. The augmenting includes: generating a data set of OOD examples, filtering out OOD examples from the data set of OOD examples, determining a difficulty value for each OOD example remaining within the filtered data set of the OOD examples, and generating augmented batches of utterances including utterances from the training set of utterances and utterances from the filtered data set of the OOD based on the difficulty value for each OOD. Thereafter, the machine-learning model is trained using the augmented batches of utterances in accordance with a curriculum training protocol.