Abstract: Techniques are described for a contextual natural language understanding (cNLU) framework that is able to incorporate contextual signals of variable history length to perform joint intent classification (IC) and slot labeling (SL) tasks. A user utterance provided by a user within a multi-turn chat dialog between the user and a conversational agent is received. The user utterance and contextual information associated with one or more previous turns of the multi-turn chat dialog is provided to a machine learning (ML) model. An intent classification and one or more slot labels for the user utterance are then obtained from the ML model. The cNLU framework described herein thus uses, in addition to a current utterance itself, various contextual signals as input to a model to generate IC and SL predictions for each utterance of a multi-turn chat dialog.

Abstract: Techniques for hosting adding and warming a host are described. In some instances, a method of determining that at least one group of hosts is to be increased by adding an additional host to the group of hosts; sending a request to the group of hosts for a list of machine learning models loaded per host of the group of hosts; receiving, from each host, the list of loaded machine learning models; loading at least a proper subset of list of loaded machine learning models into random access memory of the at least one group; receiving a request to perform an inference; routing the request to the additional host of the group of hosts; performing an inference using the additional host of the group of hosts; and providing a result of the inference to an external entity is described.

Abstract: Techniques for hosting machine learning models are described. In some instances, a method of receiving a request to perform an inference using a particular machine learning model; determining a group of hosts to route the request to, the group of hosts to host a plurality of machine learning models including the particular machine learning model; determining a path to the determined group of hosts; determining a particular host of the group of hosts to perform an analysis of the request based on the determined path, the particular host having the particular machine learning model in memory; routing the request to the particular host of the group of hosts; performing inference on the request using the particular host; and providing a result of the inference to a requester is performed.

Abstract: Techniques for resolving a label are described. For example, a method of generating one or more labels corresponding to a substring of a tokenized utterance, each of the labels including one or more slot types, resolving the one or more generated labels into machine-readable values by: generating a tree of the generated one or more labels, the graph including a plurality of leaf nodes, wherein each rule of associated with a leaf node references a resolver, and resolving the generated tree from the leaf nodes to a root of the generated tree, by at each node, calling the referenced resolver and passing a result from each resolver to a subsequent resolver of the tree; and outputting a result of the resolving to an entity as dictated by the request is described.

Abstract: Techniques for building slot types and slot type artifacts are described. For example, in some implementations, a natural language understanding (NLU) service is implemented by instructions that upon execution are to cause: receiving a request to build a slot type, the request including an indication of one or more locations storing information need to build a slot type, a slot type being a list of one or more values used to train a machine learning model to recognize a value for a slot; retrieving a slot type definition bundle associated with the request, the slot type definition bundle at least including a slot type definition defining resolution activities for the slot type; building the slot type using the retrieved slot type definition bundle; and storing into the storage slot type artifacts produced from building the slot type to be used during an analysis of an utterance.

Abstract: Techniques for hosting machine learning models are described. In some instances, a method of receiving a request to perform an inference using a particular machine learning model; determining a group of hosts to route the request to, the group of hosts to host a plurality of machine learning models including the particular machine learning model; determining a path to the determined group of hosts; determining a particular host of the group of hosts to perform an analysis of the request based on the determined path, the particular host having the particular machine learning model in memory; routing the request to the particular host of the group of hosts; performing inference on the request using the particular host; and providing a result of the inference to a requester is performed.

Abstract: Techniques for hosting adding and warming a host are described. In some instances, a method of determining that at least one group of hosts is to be increased by adding an additional host to the group of hosts; sending a request to the group of hosts for a list of machine learning models loaded per host of the group of hosts; receiving, from each host, the list of loaded machine learning models; loading at least a proper subset of list of loaded machine learning models into random access memory of the at least one group; receiving a request to perform an inference; routing the request to the additional host of the group of hosts; performing an inference using the additional host of the group of hosts; and providing a result of the inference to an external entity is described.