Abstract: Techniques are provided for using context tags in named-entity recognition (NER) models. In one particular aspect, a method is provided that includes receiving an utterance, generating embeddings for words of the utterance, generating a regular expression and gazetteer feature vector for the utterance, generating a context tag distribution feature vector for the utterance, concatenating or interpolating the embeddings with the regular expression and gazetteer feature vector and the context tag distribution feature vector to generate a set of feature vectors, generating an encoded form of the utterance based on the set of feature vectors, generating log-probabilities based on the encoded form of the utterance, and identifying one or more constraints for the utterance.

Abstract: Techniques for natural language processing include accessing an input string comprising a natural language utterance and a database schema representation for a database; providing the natural language utterance to a first encoder to generate one or more embeddings of the natural language utterance; providing the database schema representation to the first encoder to generate one or more embeddings of the database schema representation; encoding, by a second encoder, relations between elements in the database schema representation and words in the natural language utterance based on the one or more embeddings of the natural language utterance and the one or more embeddings of the database schema representation; and generating a logical form for the natural language utterance based on the encoded relations, the one or more embeddings of the natural language utterance, and the one or more embeddings of the database schema representation.

Abstract: Techniques are disclosed herein for resolving date/time expressions while transforming natural language to a logical form such as a meaning representation language. A class label for a token in a natural language utterance and a meaning representation for the natural language utterance can be predicted. The class label can be associated with a date/time expression. The meaning representation can include an operator and a value. When the value associated with the class label matches a predetermined value type or the operator matches a predetermined operator, the value and/or the operator can be modified, and an executable statement can be generated for the meaning representation. A query on a computing system can be executed using the executable statement.

Abstract: Techniques are disclosed herein for focused training of language models and end-to-end hypertuning of the framework. In one aspect, a method is provided that includes obtaining a machine learning model pre-trained for language modeling, and post-training the machine learning model for various tasks to generate a focused machine learning model. The post-training includes: (i) training the machine learning model on an unlabeled set of training data pertaining to a task that the machine learning model was pre-trained for as part of the language modeling, and the unlabeled set of training data is obtained with respect to a target domain, a target task, or a target language, and (ii) training the machine learning model on a labeled set of training data that pertains to another task that is an auxiliary task related to a downstream task to be performed using the machine learning model or output from the machine learning model.

Abstract: Techniques are described for invoking and switching between chatbots of a chatbot system. In some embodiments, the chatbot system is capable of routing an utterance received while a user is already interacting with a first chatbot in the chatbot system. For instance, the chatbot system may identify a second chatbot based on determining that (i) such an utterance is an invalid input to the first chatbot or (ii) that the first chatbot is attempting to route the utterance to a destination associated with the first chatbot. Identifying the second chatbot can involve computing, using a predictive model, separate confidence scores for the first chatbot and the second chatbot, and then determining that a confidence score for the second chatbot satisfies one or more confidence score thresholds. The utterance is then routed to the second chatbot based on the identifying of the second chatbot.

Abstract: Techniques are disclosed herein for improving model robustness on operators and triggering keywords in natural language to a meaning representation language system. The techniques include augmenting an original set of training data for a target robustness bucket by leveraging a combination of two training data generation techniques: (1) modification of existing training examples and (2) synthetic template-based example generation. The resulting set of augmented data examples from the two training data generation techniques are appended to the original set of training data to generate an augmented training data set and the augmented training data set is used to train a machine learning model to generate logical forms for utterances.

Abstract: The present disclosure relates to techniques for identifying out-of-domain utterances.

Abstract: A method includes receiving a training set of utterances for training a machine-learning model to identify one or more intents for one or more utterances, and augmenting the training set of utterances with out-of-domain (OOD) examples. The augmenting includes: generating a data set of OOD examples, filtering out OOD examples from the data set of OOD examples, determining a difficulty value for each OOD example remaining within the filtered data set of the OOD examples, and generating augmented batches of utterances including utterances from the training set of utterances and utterances from the filtered data set of the OOD based on the difficulty value for each OOD. Thereafter, the machine-learning model is trained using the augmented batches of utterances in accordance with a curriculum training protocol.

Abstract: Techniques are disclosed herein for focused training of language models and end-to-end hypertuning of the framework. In one aspect, a method is provided that includes obtaining a machine learning model pre-trained for language modeling, and post-training the machine learning model for various tasks to generate a focused machine learning model. The post-training includes: (i) training the machine learning model on an unlabeled set of training data pertaining to a task that the machine learning model was pre-trained for as part of the language modeling, and the unlabeled set of training data is obtained with respect to a target domain, a target task, or a target language, and (ii) training the machine learning model on a labeled set of training data that pertains to another task that is an auxiliary task related to a downstream task to be performed using the machine learning model or output from the machine learning model.

Abstract: Techniques for maintaining list-type text formatting when converting content from a source content format to a destination content format are disclosed. A system generates text content by applying text formatting tags to segments of characters obtained from a source electronic document. The system parses a static-display type source electronic document to obtain character data of the characters in the source document. The system analyzes the parsed data to identify text arranged in a list-type text format in the source document. The system generates text content in a destination content format different from the source format by applying tags to segments of the text content designating the segments items in a list.

Abstract: Techniques for using logit values for classifying utterances and messages input to chatbot systems in natural language processing. A method can include a chatbot system receiving an utterance generated by a user interacting with the chatbot system. The chatbot system can input the utterance into a machine-learning model including a set of binary classifiers. Each binary classifier of the set of binary classifiers can be associated with a modified logit function. The method can also include the machine-learning model using the modified logit function to generate a set of distance-based logit values for the utterance. The method can also include the machine-learning model applying an enhanced activation function to the set of distance-based logit values to generate a predicted output. The method can also include the chatbot system classifying, based on the predicted output, the utterance as being associated with the particular class.

Abstract: Techniques are disclosed herein for executing an execution plan for a digital assistant with generative artificial intelligence (genAI). A first genAI model can generate a list of executable actions based on an utterance provided by a user. An execution plan can be generated to include the executable actions. The execution plan can be executed by performing an iterative process for each of the executable actions. The iterative process can include identifying an action type, invoking one or more states, and executing, by the one or more states, the executable action using an asset to obtain an output. A second prompt can be generated based on the output obtained from executing each of the executable actions. A second genAI model can generate a response to the utterance based on the second prompt.

Abstract: Techniques are disclosed for fine-tuning a pre-trained machine learning model to be used by a digital assistant for supporting a user's interactions. In one aspect, a method includes accessing a set of training examples, generating a set of synthesized training examples using an iterative process including accessing a dialog script and corresponding prompt template and response template for a predefined scenario, generating one or more prompts based on the dialog script and corresponding prompt template, generating one or more responses associated with each of the one or more prompts based on the dialog script and the response template, and linking each of the responses with the associated prompts to generate one or more synthesized training examples in the set of synthesized training examples. The pre-trained machine learning model is then fine-tuned using the set of training examples and the set of synthesized training examples.

Abstract: Techniques are disclosed herein for routing an utterance to action for a digital assistant with generative artificial intelligence. An input query comprising particular data can be received from a user. An action and a set of input argument slots within a schema associated with the action can be identified based on the input query. The input argument slots can be filled by determining whether one or more parameters are derivable from the particular data and filling the input argument slot with a version of the parameters that conforms to the schema. An execution plan that comprises the action that includes the set of filled input argument sots can be sent to an execution engine configured to execute the action for generating a response to the input query.

Abstract: Techniques are disclosed herein for managing ambiguous date mentions in natural language utterances in transforming natural language utterances to logical forms by encoding the uncertainties of the ambiguous date mentions and including the encoded uncertainties in the logical forms. In a training phase, training examples including natural language utterances, logical forms, and database schema information are automatically augmented and used to train a machine learning model to convert natural language utterances to logical form. In an inference phase, input database schema information is augmented and used by the trained machine learning model to convert an input natural language utterance to logical form.

Abstract: Techniques are disclosed herein for configuring agents for use by digital assistants that use generative artificial intelligence. An agent may be in the form of a container that is configured to have one or more actions that can be executed by a digital assistant. The agent may be configured by initially defining specification parameters for the agent based on natural language input from a user. Configuration information for the one or more assets can be imported into the agent. One or more actions may then be defined for the agent based on importing of the configuration information, the natural language input from the user, or both. A specification document can be generated for the agent and can comprise various description metadata, such as agent, asset, or action metadata, or combinations thereof. The specification document may be stored in a data store that is communicatively coupled to the digital assistant.

Abstract: Techniques are disclosed herein for managing date-time intervals in transforming natural language utterances to logical forms by providing an enhanced grammar, a natural language utterance comprising a date-time interval, and database schema information to a machine learning model that has been trained to convert natural language utterances to logical forms; and using the machine learning model to convert the natural language utterance to an output logical form, wherein the output logical form comprises at least one of the date-time interval and an extraction function for extracting date-time information corresponding to the date-time interval from at least one date-time attribute of the database schema information.

Abstract: Techniques are disclosed herein for implementing digital assistants using generative artificial intelligence. An input prompt comprising a natural language utterance and candidate agents and associated actions can be constructed. An execution plan can be generated using a first generative artificial model based on the input prompt. The execution plan can be executed to perform actions included in the execution plan using agents indicated by the execution plan. A response to the natural language utterance can be generated by a second generative artificial intelligence model using one or more outputs from executing the execution plan.

Abstract: Techniques are disclosed herein for contextual query rewriting. The techniques include inputting a first user utterance and a conversation history to a first language model. The first language model identifies an ambiguity in the first user utterance and one or more terms in the conversation history to resolve the ambiguity, modifies the first user utterance to include the one or more terms identified to resolve the ambiguity to generate a modified utterance, and outputs the modified utterance. The computing system provides the modified utterance as input to a second language model. The second language model performs a natural language processing task based on the input modified utterance and outputs a result. The computing system outputs a response to the first user utterance based on the result.

Abstract: Techniques are described for invoking and switching between chatbots of a chatbot system. In some embodiments, the chatbot system is capable of routing an utterance received while a user is already interacting with a first chatbot in the chatbot system. For instance, the chatbot system may identify a second chatbot based on determining that (i) such an utterance is an invalid input to the first chatbot or (ii) that the first chatbot is attempting to route the utterance to a destination associated with the first chatbot. Identifying the second chatbot can involve computing, using a predictive model, separate confidence scores for the first chatbot and the second chatbot, and then determining that a confidence score for the second chatbot satisfies one or more confidence score thresholds. The utterance is then routed to the second chatbot based on the identifying of the second chatbot.