Abstract: Features are disclosed for automatically identifying a speaker. Artifacts of automatic speech recognition (“ASR”) and/or other automatically determined information may be processed against individual user profiles or models. Scores may be determined reflecting the likelihood that individual users made an utterance. The scores can be based on, e.g., individual components of Gaussian mixture models (“GMMs”) that score best for frames of audio data of an utterance. A user associated with the highest likelihood score for a particular utterance can be identified as the speaker of the utterance. Information regarding the identified user can be provided to components of a spoken language processing system, separate applications, etc.

Abstract: In one embodiment, a method includes receiving a user request from a first user at a client system, wherein the user request is associated with a semantic-intent, identifying dialog-intents associated with the user request by the client system based on the semantic-intent and context information associated with the user request, wherein each dialog-intent is a sub-intent of the semantic-intent; determining agents for executing tasks associated with the dialog-intents by the client system, and presenting information returned from the agents responsive to executing the tasks at the client system.

Abstract: In one embodiment, a method includes, by a client system, receiving, by an assistant xbot of the client system, a request from a first user for a summary of user content from a first content source, retrieving, from the first content source, a plurality of content items corresponding to the request, generating a personalized summary of the retrieved content items, wherein the personalization of the summary is based on a user profile of the first user, and presenting, by the assistant xbot, the personalized summary responsive to the request within a separate communication interface between the assistant xbot and the first user, wherein the personalized summary is interactable by the first user to react to one or more of the plurality of content items.

Abstract: In one embodiment, a system includes an automatic speech recognition (ASR) module, a natural-language understanding (NLU) module, a dialog manager, one or more agents, an arbitrator, a delivery system, one or more processors, and a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to receive a user input, process the user input using the ASR module, the NLU module, the dialog manager, one or more of the agents, the arbitrator, and the delivery system, and provide a response to the user input.

Abstract: In one embodiment, a method includes receiving a user input at a client system, wherein the user input is associated with one or more intents and one or more slots, generating one or more first dialog acts based on the user input, calculating a task-confidence score based on one or more intent-confidence scores associated with the one or more intents, respectively, and one or more slot-confidence scores associated with the one or more slots, respectively, generating one or more second dialog acts modifying the one or more first dialog acts responsive to the task-confidence score being less than a threshold score, and presenting a response to the user input at the client system, wherein the response is based on one or more of the first dialog acts or the second dialog acts.

Abstract: In one embodiment, a method includes receiving a user request from a first user at a client system, wherein the user request is associated with a semantic-intent, identifying dialog-intents associated with the user request by the client system based on the semantic-intent and context information associated with the user request, wherein each dialog-intent is a sub-intent of the semantic-intent; determining agents for executing tasks associated with the dialog-intents by the client system, and presenting information returned from the agents responsive to executing the tasks at the client system.

Abstract: In one embodiment, a method includes receiving a user request from a client system associated with a first user, wherein the user request is associated with a semantic-intent, identifying one or more dialog-intents associated with the user request based on the semantic-intent and context information associated with the user request, wherein each dialog-intent is a sub-intent of the semantic-intent, determining one or more agents for executing one or more tasks associated with the one or more dialog-intents, and sending instructions for presenting information returned from the one or more agents responsive to executing the one or more tasks to the client system.

Abstract: In one embodiment, a method includes, by one or more computing systems, receiving, from a client system, an input in a multi-turn message thread, parsing the input to identify a plurality of hypothesis dialog states associated with the input, generating a plurality of functions corresponding to the plurality of hypothesis dialog states, calculating a plurality of probability scores for the plurality of functions, respectively, based on a prior dialog state associated with the message thread and a current user context, wherein each probability score represents a likelihood that the respective function is associated with an intended dialog state, and storing the plurality of functions if each of the probability scores is below a threshold probability score, wherein the plurality of functions is stored as a hierarchical set of functions that is used as the prior dialog state in response to a next input in the message thread.

Abstract: In one embodiment, a method includes receiving a user input from a client system associated with a user, determining a task based on the user input and a confidence score associated with the task, generating one or more first dialog acts based on a task policy which specifies dialog acts associated with the task, generating one or more second dialog acts based on an override policy responsive to the confidence score being less than a threshold score, wherein the override policy specifies dialog acts that modify dialog acts specified by the task policy; and sending instructions for presenting a response to the user input to the client system, wherein the response is based on one or more of the first dialog acts or the second dialog acts.

Abstract: In one embodiment, a method includes receiving a user input at a client system, wherein the user input is associated with one or more intents and one or more slots, generating one or more first dialog acts based on the user input, calculating a task-confidence score based on one or more intent-confidence scores associated with the one or more intents, respectively, and one or more slot-confidence scores associated with the one or more slots, respectively, generating one or more second dialog acts modifying the one or more first dialog acts responsive to the task-confidence score being less than a threshold score, and presenting a response to the user input at the client system, wherein the response is based on one or more of the first dialog acts or the second dialog acts.

Abstract: In one embodiment, a method includes, by one or more computing systems, receiving, from a client system, an input in a multi-turn message thread, parsing the input to identify a plurality of hypothesis dialog states associated with the input, generating a plurality of functions corresponding to the plurality of hypothesis dialog states, calculating a plurality of probability scores for the plurality of functions, respectively, based on a prior dialog state associated with the message thread and a current user context, wherein each probability score represents a likelihood that the respective function is associated with an intended dialog state, and storing the plurality of functions if each of the probability scores is below a threshold probability score, wherein the plurality of functions is stored as a hierarchical set of functions that is used as the prior dialog state in response to a next input in the message thread.

Abstract: In one embodiment, a method includes establishing a video call between a plurality of client systems, wherein access to an assistant system is persistently maintained during the video call, receiving, from a first client system of the plurality of client systems, a request by a first user to be performed by the assistant system during the video call, wherein the request references one or more activities associated with one or more users associated with the plurality of client systems, analyzing, by a context engine of the assistant system, images of a scene of the video call to identify the one or more activities within the scene, instructing the assistant system to execute the request based on the identified one or more activities, and sending, to one or more of the plurality of client systems, a response to the request while maintaining the video call between the plurality of client systems.

Abstract: In one embodiment, a method includes establishing a video call between multiple client systems while persistently maintaining access to an assistant system during the video call. A request to be performed by the assistant system during the video call may then be received from a first client system; this request may reference one or more second users in the video call. An intent of the request and one or more user identifiers of these one or more second users referenced by the request may be determined, and the assistant system may be instructed to execute the request based on the determined intent and user identifiers. Finally, a response to the request may be sent to one more of the multiple client systems while maintaining the video call between these client systems.

Abstract: Features are disclosed for automatically identifying a speaker. Artifacts of automatic speech recognition (“ASR”) and/or other automatically determined information may be processed against individual user profiles or models. Scores may be determined reflecting the likelihood that individual users made an utterance. The scores can be based on, e.g., individual components of Gaussian mixture models (“GMMs”) that score best for frames of audio data of an utterance. A user associated with the highest likelihood score for a particular utterance can be identified as the speaker of the utterance. Information regarding the identified user can be provided to components of a spoken language processing system, separate applications, etc.

Abstract: In one embodiment, a method includes receiving, from a client system of a user, a user input comprising a plurality of n-grams, parsing the user input to identify one or more overall intents, hidden intents, and slots associated with the one or more n-grams, wherein at least one of the hidden intents is non-resolvable for being associated with partial slot information corresponding to an n-gram that has not been resolved to a particular entity identifier, wherein the partial slot information is associated with two more entity identifiers of two or more entities, respectively, sending, to the client system, instructions for prompting the user to select one of the entities to be associated with the non-resolvable hidden intent, resolving the non-resolvable hidden intent based on the entity identifier of the entity selected by the user, and generating a response to the user input based on the resolved hidden intent.

Abstract: Exemplary embodiments relate to improvements in spoken language understanding (SLU) systems. Conventionally, SLU systems include an automatic speech recognition (ASR) component configured to receive an input of audio data and to generate a textual representation of the audio data. Conventional SLU systems also include a natural language understanding (NLU) component configured to receive a text-based transcript and perform language-based tasks such as domain classification, intent determination, and slot-filling. However, these two components are typically trained separately based on different metrics. In real-world situations, errors in the ASR component propagate to the NLU component, which degrades the performance of the overall system. Exemplary embodiments described herein perform SLU in an end-to-end manner that infers semantic meaning directly from audio features without an intermediate text representation.

Abstract: In one embodiment, a method includes, by one or more computing systems, receiving a user input comprising a plurality of n-grams from a user of a client system, generating a tree-structured representation for the user input based on a parsing by a compositional model, resolving the tree-structured representation by applying a depth-first search algorithm, wherein the tree-structured representation comprises one or more non-resolvable non-terminal nodes associated with one or more slots, and wherein each non-terminal parent node of a non-resolvable non-terminal node is partially resolved based on partial slot information associated with the non-resolvable non-terminal node, and wherein each non-resolvable non-terminal node is resolved based on the respective partially resolved non-terminal parent node of the non-resolvable non-terminal node, generating a response to the user input based on the resolved tree-structured representation, sending instructions for presenting the response to the client system of the

Abstract: In one embodiment, a method includes, by a client system, receiving, by an assistant xbot of the client system, a request from a first user for a summary of user content from a first content source, retrieving, from the first content source, a plurality of content items corresponding to the request, generating a personalized summary of the retrieved content items, wherein the personalization of the summary is based on a user profile of the first user, and presenting, by the assistant xbot, the personalized summary responsive to the request within a separate communication interface between the assistant xbot and the first user, wherein the personalized summary is interactable by the first user to react to one or more of the plurality of content items.

Abstract: In one embodiment, a method includes by a client system associated with a user, receiving, at the client system, a user input from the user, parsing, by the client system, the first user input to identify a request to execute a function to be performed by an assistant system of several assistant systems associated with the client system, determining whether the user is authorized to access the assistant system by comparing a voiceprint of the user to several voiceprints stored on the client system, sending, from the client system to the assistant system in response to determining the user is authorized to access the assistant system, a request to set an assistant xbot of the assistant system into a listening mode, and receiving, at the client system from the assistant system, an indication that the assistant xbot is in listening mode.

Abstract: In one embodiment, a method includes receiving a user input by the first user from a client system associated with a first user, parsing the user input to identify one or more n-grams associated with the user input, accessing a user profile associated with the first user, wherein the user profile is stored in a first data store, accessing ontology data based on the one or more identified n-grams from one or more information graphs, wherein the one or more information graphs are stored in one or more second data stores, respectively, determining contextual information associated with the user input, generating semantic information by aggregating the user profile, ontology data, and contextual information, generating a feature representation for the identified one or more n-grams based on the semantic information, and resolving one or more entities associated with the one or more n-grams based on the feature representation.