Abstract: Examples of the present disclosure describe systems and methods of personalizing natural language systems. In aspects, personal data may be uploaded to a personalization server. Upon receiving a data request, a server device may query the personalization server using a user's login information. The login data and the associated personal data may be paired and provided to the personalization server. The paired data may then be provided to a language understanding model to generate a response to the data request. The data in the response may be used to train the language understanding model.

Abstract: Aspects herein provide third party application authors with a user interface authoring platform that automates and simplifies a task definition process while also providing the ability to leverage pre-existing language understanding models and canonicalization and resolution modules that are provided by the operating system on which the CU system resides or as provided by other third parties. In particular, the present disclosure provides a method and system for authoring a task using a user interface authoring platform.

Abstract: Examples of the present disclosure describe systems and methods of transfer learning techniques for disparate label sets. In aspects, a data set may be accessed on a server device. The data set may comprise labels and word sets associated with the labels. The server device may induce label embedding within the data set. The embedded labels may be represented by multi-dimensional vectors that correspond to particular labels. The vectors may be used to construct label mappings for the data set. The label mappings may be used to train a model to perform domain adaptation or transfer learning techniques. The model may be used to provide results to a statement/query or to train a different model.

Abstract: Techniques for optimizing a system to improve an overall user satisfaction in a speech controlled system are described. A user speaks an utterance and the system compares an expected sum of user satisfaction values for each action to make a decision as to how best to process the utterance. As a result, the system may make a decision that decreases user satisfaction in the short term but increases user satisfaction in the long term. The system may estimate a user satisfaction value and associate the estimated user satisfaction value with a current dialog state. By tracking user satisfaction values over time, the system may train machine learning models to optimize the expected sum of user satisfaction values. This improves how the system selects an action or application to which to dispatch the dialog state and how a specific application selects an action or intent corresponding to the command.

Abstract: A system for processing user utterances and/or text based queries that tracks entities and other context data of a current dialog between the system and the user and can fill slots for new intents of the dialog by performing statistical processing on previously mentioned entities with respect to current slots to be filled. The system may compare a previously mentioned entity to a current slot to be filled using vector representations, such as word embeddings, of the current utterance, dialog history, current intent, name of an entity under consideration, category of the current slot to be filled, distance between the current dialog turn and the dialog turn that mentioned the entity, and other considerations. The individual vectors may be weighted according to an attention operation and processed by a trained decoder to output a score indicating whether the entity in consideration is relevant to the particular slot.

Abstract: Techniques for performing command processing are described. A system receives, from a device, input data corresponding to a command. The input data may originate as audio data, as text data, or as other data. The system determines NLU processing results corresponding to the input data. The NLU processing results may be associated with multiple speechlets. The system also determines NLU confidences for the NLU processing results for each speechlet. The system sends NLU processing results and an indication to provide potential results to a portion of the multiple speechlets, and receives potential results from the portion of the speechlets. The system also receives indications whether the speechlets need to be re-called if the speechlets are selected to execute with respect to the command. The system ranks the portion of the speechlets based at least in part on the NLU processing results as well as the potential results provided by the portion of the speechlets.

Abstract: Multi-turn cross-domain natural language understanding (NLU) systems and platforms for building the multi-turn cross-domain NLU system are provided. Further, methods for using and building the multi-turn cross-domain NLU system are provided. More specifically, the multi-turn cross-domain NLU system supports multi-turn bot/agent/application scenarios for new domains without having to select a task definition and/or define a new schema during the building of the NLU system. Accordingly, the platform for building the multi-turn cross-domain NLU system that does not require the builder to select a task and/or build a schema for a selected task provides an easy to use, cost effective, and efficient service for building a NLU system. Further, the multi-turn cross-domain NLU system provides a more versatile NLU system than previously utilized NLU systems that were trained for and limited to a selected task and/or domain.

Abstract: In non-limiting examples of the present disclosure, systems, methods and devices for providing personalized experiences to a computing device based on user input such as voice, text and gesture input are provided. Acoustic patterns associated with voice input, speech patterns, language patterns and natural language processing may be used to identify a specific user providing input from a plurality of users, identify user background characteristics and traits for the specific user, and topically categorize user input in a tiered hierarchical index. Topically categorized user input may be supplemented with user data and world knowledge and personalized responses and feedback for an identified specific user may be provided reactively and proactively.

Abstract: A system for processing user utterances and/or text based queries that tracks entities and other context data of a current dialog between the system and the user and can fill slots for new intents of the dialog by performing statistical processing on previously mentioned entities with respect to current slots to be filled. The system may compare a previously mentioned entity to a current slot to be filled using vector representations, such as word embeddings, of the current utterance, dialog history, current intent, name of an entity under consideration, category of the current slot to be filled, distance between the current dialog turn and the dialog turn that mentioned the entity, and other considerations. The individual vectors may be weighted according to an attention operation and processed by a trained decoder to output a score indicating whether the entity in consideration is relevant to the particular slot.

Abstract: Technologies pertaining to slot filling are described herein. A deep neural network, a recurrent neural network, and/or a spatio-temporally deep neural network are configured to assign labels to words in a word sequence set forth in natural language. At least one label is a semantic label that is assigned to at least one word in the word sequence.

Abstract: Techniques for optimizing a system to improve an overall user satisfaction in a speech controlled system are described. A user speaks an utterance and the system compares an expected sum of user satisfaction values for each action to make a decision as to how best to process the utterance. As a result, the system may make a decision that decreases user satisfaction in the short term but increases user satisfaction in the long term. The system may estimate a user satisfaction value and associate the estimated user satisfaction value with a current dialog state. By tracking user satisfaction values over time, the system may train machine learning models to optimize the expected sum of user satisfaction values. This improves how the system selects an action or application to which to dispatch the dialog state and how a specific application selects an action or intent corresponding to the command.

Abstract: In non-limiting examples of the present disclosure, systems, methods and devices for providing personalized experiences to a computing device based on user input such as voice, text and gesture input are provided. Acoustic patterns associated with voice input, speech patterns, language patterns and natural language processing may be used to identify a specific user providing input from a plurality of users, identify user background characteristics and traits for the specific user, and topically categorize user input in a tiered hierarchical index. Topically categorized user input may be supplemented with user data and world knowledge and personalized responses and feedback for an identified specific user may be provided reactively and proactively.

Abstract: Technology is provided for improving digital assistant performance by generating and presenting suggestions to users for completing a task or a session. To generate the suggestions, a machine learned language prediction model is trained with features extracted from multiple sources, such as log data and session context. When input is received from a user, the trained machine learned language prediction model is used to determine the most likely suggestion to present to the user to lead to successful task completion. In generating the suggestion, intermediate suggestion data, such as a domain, intent, and/or slot, is generated for the suggestion. From the generated intermediate suggestion data for the suggestion, a surface form of the suggestion is generated that can be presented to the user. The resulting suggestion and related context may further be used to continue training the machine learned language prediction model.

Abstract: Aspects herein provide third party application authors with a user interface authoring platform that automates and simplifies a task definition process while also providing the ability to leverage pre-existing language understanding models and canonicalization and resolution modules that are provided by the operating system on which the CU system resides or as provided by other third parties. In particular, the present disclosure provides a method and system for authoring a task using a user interface authoring platform.

Abstract: Technology is provided for improving digital assistant performance by generating and presenting suggestions to users for completing a task or a session. To generate the suggestions, a machine learned language prediction model is trained with features extracted from multiple sources, such as log data and session context. When input is received from a user, the trained machine learned language prediction model is used to determine the most likely suggestion to present to the user to lead to successful task completion. In generating the suggestion, intermediate suggestion data, such as a domain, intent, and/or slot, is generated for the suggestion. From the generated intermediate suggestion data for the suggestion, a surface form of the suggestion is generated that can be presented to the user. The resulting suggestion and related context may further be used to continue training the machine learned language prediction model.

Abstract: Techniques for limiting natural language processing performed on input data are described. A system receives input data from a device. The input data corresponds to a command to be executed by the system. The system determines applications likely configured to execute the command. The system performs named entity recognition and intent classification with respect to only the applications likely configured to execute the command.

Abstract: A computer-implemented technique is described for facilitating the creation of a language understanding (LU) component for use with an application. The technique allows a developer to select a subset of parameters from a larger set of parameters. The subset of parameters pertains to a LU scenario to be handled by the application. The larger set of parameters pertains to a plurality of LU scenarios handled by an already-existing generic LU model. The technique creates a constrained LU component that is based on the subset of parameters in conjunction with the generic LU model. At runtime, the constrained LU component interprets input language items using the generic LU model in a manner that is constrained by the subset of parameters that have been selected, to provide an output result. The technique also allows the developer to create new rules and/or supplemental models.

Abstract: Conversational understanding systems allow users to conversationally interface with a computing device. In examples, a query may be received that includes a request for execution of a task. A data exchange task definition may be accessed. The data exchange task definition assists a conversational understanding system in managing task state tracking for information needed for task execution. Using the data exchange task definition, a per-turn policy for interacting with the user computing device is generated based on the state of a dialogue with a computing device and an evaluation of a process flow chart provided by a task owner resource. The task owner resource may be independent from the conversational understanding system. A response to the query may be generated and output based on the per-turn policy. In examples, the per-turn policy is used to generate one or more responses during a dialogue with a user via a computing device.

Abstract: Aspects herein provide third party application authors with a user interface authoring platform that automates and simplifies a task definition process while also providing the ability to leverage pre-existing language understanding models and canonicalization and resolution modules that are provided by the operating system on which the CU system resides or as provided by other third parties. In particular, the present disclosure provides a method and system for authoring a task using a user interface authoring platform.

Abstract: Techniques for performing command processing are described. A system receives, from a device, input data corresponding to a command. The system determines NLU processing results associated with multiple applications. The system also determines NLU confidences for the NLU processing results for each application. The system sends NLU processing results to a portion of the multiple applications, and receives output data or instructions from the portion of the applications. The system ranks the portion of the applications based at least in part on the NLU processing results associated with the portion of the applications as well as the output data or instructions provided by the portion of the applications. The system may also rank the portion of the applications using other data. The system causes content corresponding to output data or instructions provided by the highest ranked application to be output to a user.