Abstract: Techniques for optimizing a system to improve an overall user satisfaction in a speech controlled system are described. A user speaks an utterance and the system compares an expected sum of user satisfaction values for each action to make a decision as to how best to process the utterance. As a result, the system may make a decision that decreases user satisfaction in the short term but increases user satisfaction in the long term. The system may estimate a user satisfaction value and associate the estimated user satisfaction value with a current dialog state. By tracking user satisfaction values over time, the system may train machine learning models to optimize the expected sum of user satisfaction values. This improves how the system selects an action or application to which to dispatch the dialog state and how a specific application selects an action or intent corresponding to the command.

Abstract: A speech-processing system may provide access to one or more skills via spoken commands and/or responses in the form of synthesized speech. The system may be capable of keeping one or more skills active in the background while a user interacts (e.g., provides inputs to and/or receives outputs from) with a skill running in the foreground. A background skill may receive some trigger data, and determine to request the system to return the background skill to the foreground to, for example, request a user input regarding an action previously requested by the user. In some cases, the user may invoke a background skill to continue a previous interaction. The system may return the background skill to the foreground. The resumed skill may continue a previous interaction to, for example, to query the user for instructions, provide an update or alert, or continue a previous output.

Abstract: Techniques for generating a personalization value that measures how tailored certain system interactions are for a user are described. A dialog exchange between a user and a skill may be determined, with the dialog exchange including user input data and system output data. It may be determined that the system output data was generated without respect to at least one previous user input or system output of the dialog exchanges. Based on this, a personalization value may be generated and sent to the skill.

Abstract: Described herein is a system for enabling a user to multitask by allowing a user to pause or interrupt an on-going interaction with a skill. The system monitors a state of a skill session, and updates the state to allow the user or system to suspend the session. The user may provide an instruction to pause an active session, causing the system to place the session in a suspended state. The user may then provide an instruction to resume the suspended session, causing the system to place the session in an active state. In other cases, the user input may be a request during an active session that requires invoking another skill. The system may place the current session in a suspended state, and invoke a second skill session to monitor the interaction with a second skill. When the interaction with the second skill is completed, the system may resume the previous session by placing it in an active state.

Abstract: The present disclosure provides various metrics that may be used to generate a score representing a conversational quality of a user/computing system dialog. User input data, corresponding to a dialog with a natural language processing system, may be determined and associated with a dialog identifier. System output data, corresponding to the dialog, may also be determined and associated with the dialog identifier. After the system output data is output to a user, the user input data and system output data may be processed to determine a score corresponding to the dialog exchange.

Abstract: Techniques for generating a conformance score for a system/user dialog are described. A conformance score may represent a degree to which output data, provided by a skill, conforms to various policies (e.g., the data includes content appropriate for the age of the user, the data does not include profanity, etc.). User input data and system output data, corresponding to a dialog exchange between a user and a skill, ma be determined. A user type associated with the dialog exchange may also be determined. Based on the user type and the system output data, it may be determined that one or more filtering resources are to be assigned to process future data, received from the skill, prior to the future data being presented to a user.

Abstract: Techniques for generating a prompt coverage score, which measures an extent to which data output to a user during a dialog is repetitive and monotonous, are described. User input data and system output, corresponding to a dialog exchange between a user and a skill, may be determined. A portion of the system output data, corresponding to a system prompt representing default output data, may be determined. A first number, representing possible variants of the prompt, may be determined along with a second number, representing variants of the prompt output during the dialog exchange. A prompt coverage score may be determined based on the first and second numbers.

Abstract: Techniques for optimizing a system to improve an overall user satisfaction in a speech controlled system are described. A user speaks an utterance and the system compares an expected sum of user satisfaction values for each action to make a decision as to how best to process the utterance. As a result, the system may make a decision that decreases user satisfaction in the short term but increases user satisfaction in the long term. The system may estimate a user satisfaction value and associate the estimated user satisfaction value with a current dialog state. By tracking user satisfaction values over time, the system may train machine learning models to optimize the expected sum of user satisfaction values. This improves how the system selects an action or application to which to dispatch the dialog state and how a specific application selects an action or intent corresponding to the command.

Abstract: Described herein is a system for allocating resources among multiple skills to enable multitasking. The system tracks use of resources using skill sessions. In one case, the system suspends a skill session to release a resource for allocation to another resource. In another case, the system determines if multiple skill sessions can remain active and use resources to provide output to the user at the same time.

Abstract: Techniques for optimizing a system to improve an overall user satisfaction in a speech controlled system are described. A user speaks an utterance and the system compares an expected sum of user satisfaction values for each action to make a decision as to how best to process the utterance. As a result, the system may make a decision that decreases user satisfaction in the short term but increases user satisfaction in the long term. The system may estimate a user satisfaction value and associate the estimated user satisfaction value with a current dialog state. By tracking user satisfaction values over time, the system may train machine learning models to optimize the expected sum of user satisfaction values. This improves how the system selects an action or application to which to dispatch the dialog state and how a specific application selects an action or intent corresponding to the command.