Abstract: A system and method are disclosed for device-agnostic synchronizing of audio and physical actions at a client device. Different client devices may synthesize audio from text at different rates, and different client devices may perform physical actions such as gestures and other physical movements at different rates. The system of the present technology enables synchronization of audio with physical actions at different client devices, where audio and/or physical actions may be synthesized at different rates.

Abstract: Systems perform methods of interpreting spoken utterances from a user and responding to the utterances by providing requested information or performing a requested action. The utterances are interpreted in the context of multiple domains. Each interpretation is assigned a relevancy score based on how well the interpretation represents what the speaker intended. Interpretations having a relevancy score below a threshold for its associated domain are discarded. A remaining interpretation is chosen based on choosing the most relevant domain for the utterance. The user may be prompted to provide disambiguation information that can be used to choose the best domain. Storing past associations of utterance representation and domain choice allows for measuring the strength of correlation between uttered words and phrases with relevant domains. This correlation strength information may allow the system to automatically disambiguate alternate interpretations without requiring user input.

Abstract: Systems perform methods of interpreting spoken utterances from a user and responding to the utterances by providing requested information or performing a requested action. The utterances are interpreted in the context of multiple domains. Each interpretation is assigned a relevancy score based on how well the interpretation represents what the speaker intended. Interpretations having a relevancy score below a threshold for its associated domain are discarded. A remaining interpretation is chosen based on choosing the most relevant domain for the utterance. The user may be prompted to provide disambiguation information that can be used to choose the best domain. Storing past associations of utterance representation and domain choice allows for measuring the strength of correlation between uttered words and phrases with relevant domains. This correlation strength information may allow the system to automatically disambiguate alternate interpretations without requiring user input.

Abstract: A speech and natural language-based virtual assistant parses user utterances and analyzes them in the context of recent prior actions to detect sentiment and indicators of satisfaction or dissatisfaction. Indicators are stored in a database in association with the prior command and resulting action. Databases can include timestamps, clarifications made by users, and a knowledge graph of facts. Machine learning, applied to the database, train models to deliver improved results in future user engagements.

Abstract: A system, method, and non-transitory computer readable medium provide for a visual display of a user interface for a voice-based virtual assistant system. After displaying a transcription of user speech and performing requested actions, the system allows the user to provide, by speech or manual input, an indication of satisfaction or dissatisfaction. For transcription errors, the user is presented an opportunity to correct the transcription text. The system can present several transcription hypotheses to the user, and allow the user to choose among them, or to edit one of them, as the intended transcription. A back-end server system uses the corrected transcription to train a machine learning model to perform more accurate speech recognition or provide more useful actions for future users. A system can save one or more speech recognition transcription hypotheses and check corrected results against the other transcriptions to further improve models.

Abstract: Virtual assistants provide results in response to user commands and analyze user utterances in response to the result. The analysis can interpret words, recognized from the utterance, as being negative indicators that imply user dissatisfaction. Virtual assistants request follow-up information from users. Analysis also interprets words as indicators of clarification and collect information to add to a knowledgebase. Machine learning algorithms use recognized words to train a behavioral model to improve results. Virtual assistants also infer, from replacement of words in successive commands, that earlier commands had word recognition errors and infer, from addition of words, that earlier commands had interpretation errors. Virtual assistants act locally or as devices in communication with servers.

Abstract: Systems perform methods of interpreting spoken utterances from a user and responding to the utterances by providing requested information or performing a requested action. The utterances are interpreted in the context of multiple domains. Each interpretation is assigned a relevancy score based on how well the interpretation represents what the speaker intended. Interpretations having a relevancy score below a threshold for its associated domain are discarded. A remaining interpretation is chosen based on choosing the most relevant domain for the utterance. The user may be prompted to provide disambiguation information that can be used to choose the best domain. Storing past associations of utterance representation and domain choice allows for measuring the strength of correlation between uttered words and phrases with relevant domains. This correlation strength information may allow the system to automatically disambiguate alternate interpretations without requiring user input.