Abstract: A method includes receiving a training set of utterances for training a machine-learning model to identify one or more intents for one or more utterances, and augmenting the training set of utterances with out-of-domain (OOD) examples. The augmenting includes: generating a data set of OOD examples, filtering out OOD examples from the data set of OOD examples, determining a difficulty value for each OOD example remaining within the filtered data set of the OOD examples, and generating augmented batches of utterances including utterances from the training set of utterances and utterances from the filtered data set of the OOD based on the difficulty value for each OOD. Thereafter, the machine-learning model is trained using the augmented batches of utterances in accordance with a curriculum training protocol.

Abstract: Techniques are disclosed herein for focused training of language models and end-to-end hypertuning of the framework. In one aspect, a method is provided that includes obtaining a machine learning model pre-trained for language modeling, and post-training the machine learning model for various tasks to generate a focused machine learning model. The post-training includes: (i) training the machine learning model on an unlabeled set of training data pertaining to a task that the machine learning model was pre-trained for as part of the language modeling, and the unlabeled set of training data is obtained with respect to a target domain, a target task, or a target language, and (ii) training the machine learning model on a labeled set of training data that pertains to another task that is an auxiliary task related to a downstream task to be performed using the machine learning model or output from the machine learning model.

Abstract: Techniques for maintaining list-type text formatting when converting content from a source content format to a destination content format are disclosed. A system generates text content by applying text formatting tags to segments of characters obtained from a source electronic document. The system parses a static-display type source electronic document to obtain character data of the characters in the source document. The system analyzes the parsed data to identify text arranged in a list-type text format in the source document. The system generates text content in a destination content format different from the source format by applying tags to segments of the text content designating the segments items in a list.

Abstract: Techniques for using logit values for classifying utterances and messages input to chatbot systems in natural language processing. A method can include a chatbot system receiving an utterance generated by a user interacting with the chatbot system. The chatbot system can input the utterance into a machine-learning model including a set of binary classifiers. Each binary classifier of the set of binary classifiers can be associated with a modified logit function. The method can also include the machine-learning model using the modified logit function to generate a set of distance-based logit values for the utterance. The method can also include the machine-learning model applying an enhanced activation function to the set of distance-based logit values to generate a predicted output. The method can also include the chatbot system classifying, based on the predicted output, the utterance as being associated with the particular class.

Abstract: Techniques are disclosed herein for routing an utterance to action for a digital assistant with generative artificial intelligence. An input query comprising particular data can be received from a user. An action and a set of input argument slots within a schema associated with the action can be identified based on the input query. The input argument slots can be filled by determining whether one or more parameters are derivable from the particular data and filling the input argument slot with a version of the parameters that conforms to the schema. An execution plan that comprises the action that includes the set of filled input argument sots can be sent to an execution engine configured to execute the action for generating a response to the input query.

Abstract: Techniques are disclosed herein for executing an execution plan for a digital assistant with generative artificial intelligence (genAI). A first genAI model can generate a list of executable actions based on an utterance provided by a user. An execution plan can be generated to include the executable actions. The execution plan can be executed by performing an iterative process for each of the executable actions. The iterative process can include identifying an action type, invoking one or more states, and executing, by the one or more states, the executable action using an asset to obtain an output. A second prompt can be generated based on the output obtained from executing each of the executable actions. A second genAI model can generate a response to the utterance based on the second prompt.

Abstract: Techniques are disclosed herein for configuring agents for use by digital assistants that use generative artificial intelligence. An agent may be in the form of a container that is configured to have one or more actions that can be executed by a digital assistant. The agent may be configured by initially defining specification parameters for the agent based on natural language input from a user. Configuration information for the one or more assets can be imported into the agent. One or more actions may then be defined for the agent based on importing of the configuration information, the natural language input from the user, or both. A specification document can be generated for the agent and can comprise various description metadata, such as agent, asset, or action metadata, or combinations thereof. The specification document may be stored in a data store that is communicatively coupled to the digital assistant.

Abstract: Techniques are disclosed herein for managing ambiguous date mentions in natural language utterances in transforming natural language utterances to logical forms by encoding the uncertainties of the ambiguous date mentions and including the encoded uncertainties in the logical forms. In a training phase, training examples including natural language utterances, logical forms, and database schema information are automatically augmented and used to train a machine learning model to convert natural language utterances to logical form. In an inference phase, input database schema information is augmented and used by the trained machine learning model to convert an input natural language utterance to logical form.

Abstract: Techniques are disclosed for fine-tuning a pre-trained machine learning model to be used by a digital assistant for supporting a user's interactions. In one aspect, a method includes accessing a set of training examples, generating a set of synthesized training examples using an iterative process including accessing a dialog script and corresponding prompt template and response template for a predefined scenario, generating one or more prompts based on the dialog script and corresponding prompt template, generating one or more responses associated with each of the one or more prompts based on the dialog script and the response template, and linking each of the responses with the associated prompts to generate one or more synthesized training examples in the set of synthesized training examples. The pre-trained machine learning model is then fine-tuned using the set of training examples and the set of synthesized training examples.

Abstract: Techniques are disclosed herein for managing date-time intervals in transforming natural language utterances to logical forms by providing an enhanced grammar, a natural language utterance comprising a date-time interval, and database schema information to a machine learning model that has been trained to convert natural language utterances to logical forms; and using the machine learning model to convert the natural language utterance to an output logical form, wherein the output logical form comprises at least one of the date-time interval and an extraction function for extracting date-time information corresponding to the date-time interval from at least one date-time attribute of the database schema information.

Abstract: Techniques are disclosed herein for contextual query rewriting. The techniques include inputting a first user utterance and a conversation history to a first language model. The first language model identifies an ambiguity in the first user utterance and one or more terms in the conversation history to resolve the ambiguity, modifies the first user utterance to include the one or more terms identified to resolve the ambiguity to generate a modified utterance, and outputs the modified utterance. The computing system provides the modified utterance as input to a second language model. The second language model performs a natural language processing task based on the input modified utterance and outputs a result. The computing system outputs a response to the first user utterance based on the result.

Abstract: Techniques are disclosed herein for implementing digital assistants using generative artificial intelligence. An input prompt comprising a natural language utterance and candidate agents and associated actions can be constructed. An execution plan can be generated using a first generative artificial model based on the input prompt. The execution plan can be executed to perform actions included in the execution plan using agents indicated by the execution plan. A response to the natural language utterance can be generated by a second generative artificial intelligence model using one or more outputs from executing the execution plan.

Abstract: Techniques are described for invoking and switching between chatbots of a chatbot system. In some embodiments, the chatbot system is capable of routing an utterance received while a user is already interacting with a first chatbot in the chatbot system. For instance, the chatbot system may identify a second chatbot based on determining that (i) such an utterance is an invalid input to the first chatbot or (ii) that the first chatbot is attempting to route the utterance to a destination associated with the first chatbot. Identifying the second chatbot can involve computing, using a predictive model, separate confidence scores for the first chatbot and the second chatbot, and then determining that a confidence score for the second chatbot satisfies one or more confidence score thresholds. The utterance is then routed to the second chatbot based on the identifying of the second chatbot.

Abstract: The present disclosure relates to manufacturing training data by leveraging an automated pipeline that manufactures visualization training datasets to train a machine learning model to convert a natural language utterance into meaning representation language logical form that includes one or more visualization actions. Aspects are directed towards accessing an original training dataset, a visualization query dataset, an incremental visualization dataset, a manipulation visualization dataset, or any combination thereof. One or more visualization training datasets are generated by: (i) modifying examples in the original training dataset, the visualization query dataset, or both to include visualization actions, (ii) generating examples, using the incremental visualization dataset, the manipulation visualization dataset, or both, that include visualization actions, or (iii) both (i) and (ii).

Abstract: Techniques are disclosed herein for transforming natural language conversations into a visual output. In one aspect, a computer-implement method includes generating an input string by concatenating a natural language utterance with a schema representation comprising a set of entities for visualization actions, generating, by a first encoder of a machine learning model, one or more embeddings of the input string, encoding, by a second encoder of the machine learning model, relations between elements in the schema representation and words in the natural language utterance based on the one or more embeddings, generating, by a grammar-based decoder of the machine learning model and based on the encoded relations and the one or more embeddings, an intermediate logical form that represents at least the query, the one or more visualization actions, or the combination thereof, and generating, based on the intermediate logical form, a command for a computing system.

Abstract: Techniques are disclosed for providing a scalable multi-tenant serve pool for chatbot systems. A query serving system (QSS) receives a request to serve a query for a skillbot. The QSS includes: (i) a plurality of deployments in a serving pool, and (ii) a plurality of deployments in a free pool. The QSS determines whether a first deployment from the plurality of deployments in the serving pool can serve the query based on an identifier of the skillbot. In response to determining that the first deployment cannot serve the query, the QSS selects a second deployment from the plurality of deployments in the free pool to be assigned to the skillbot, and loads a machine-learning model associated with the skillbot into the second deployment, wherein the machine-learning model is trained to serve the query for the skillbot. The query is served using the machine-learning model loaded into the second deployment.

Abstract: The present disclosure relates to chatbot systems, and more particularly, to batching techniques for handling unbalanced training data when training a model such that bias is removed from the trained machine learning model when performing inference. In an embodiment, a plurality of raw utterances is obtained. A bias eliminating distribution is determined and a subset of the plurality of raw utterances is batched according to the bias-reducing distribution. The resulting unbiased training data may be input into a prediction model for training the prediction model. The trained prediction model may be obtained and utilized to predict unbiased results from new inputs received by the trained prediction model.

Abstract: Techniques for automatically switching between chatbot skills in the same domain. In one particular aspect, a method is provided that includes receiving an utterance from a user within a chatbot session, where a current skill context is a first skill and a current group context is a first group, inputting the utterance into a candidate skills model for the first group, obtaining, using the candidate skills model, a ranking of skills within the first group, determining, based on the ranking of skills, a second skill is a highest ranked skill, changing the current skill context of the chatbot session to the second skill, inputting the utterance into a candidate flows model for the second skill, obtaining, using the candidate flows model, a ranking of intents within the second skill that match the utterance, and determining, based on the ranking of intents, an intent that is a highest ranked intent.

Abstract: Techniques for using logit values for classifying utterances and messages input to chatbot systems in natural language processing. A method can include a chatbot system receiving an utterance generated by a user interacting with the chatbot system. The chatbot system can input the utterance into a machine-learning model including a set of binary classifiers. Each binary classifier of the set of binary classifiers can be associated with a modified logit function. The method can also include the machine-learning model using the modified logit function to generate a set of distance-based logit values for the utterance. The method can also include the machine-learning model applying an enhanced activation function to the set of distance-based logit values to generate a predicted output. The method can also include the chatbot system classifying, based on the predicted output, the utterance as being associated with the particular class.

Abstract: A training request including an identifier that is indicative of a type of a machine learning (ML) model that is to be trained is received. A plurality of workers are maintained in a training pool, and a plurality of jobs are maintained in a queue of training jobs. Each worker is configured to train a particular type of ML model. Upon the training request being validated, a training job is created for the request and submitted to the queue of training jobs. For each type of ML model, a first metric and a second metric is obtained. A target metric is computed based on the first and the second metrics. The number of workers included in the training pool is modified based on the target metric.