Abstract: In one embodiment, a method includes establishing a video call between multiple client systems while persistently maintaining access to an assistant system during the video call. A request to be performed by the assistant system during the video call may then be received from a first client system; this request may reference one or more second users in the video call. An intent of the request and one or more user identifiers of these one or more second users referenced by the request may be determined, and the assistant system may be instructed to execute the request based on the determined intent and user identifiers. Finally, a response to the request may be sent to one more of the multiple client systems while maintaining the video call between these client systems.

Abstract: In one embodiment, a method includes, by one or more computing systems, receiving, from a client system associated with a user, a request for a summary of user communications from a content source, accessing a plurality of user communications from the content source, identifying a plurality of segments associated with the plurality of user communications, wherein the plurality of segments is associated with a plurality of topics, respectively, calculating, for each segment of the plurality of segments, a user interest score for the segment, selecting one or more of the segments for summarization based on their user interest scores, generating one or more personalized summaries of the one or more selected segments, wherein the personalization of the summary is based on the user profile of the user and sending, to the client system, instructions to present the personalized summaries to the user responsive to the request.

Abstract: In one embodiment, a method includes receiving an instruction to add an assistant xbot as a participant in a conversation thread from a first user of a plurality of users participating in the conversation thread, monitoring the conversation thread including user inputs by one or more users of the plurality of users via the assistant xbot, analyzing the user inputs to identify intents based on a natural-language understanding module, sending instructions for prompting one or more users of the plurality of users to provide information for completing tasks associated with the intents via the assistant xbot within the conversation thread, executing the tasks based on the information provided by one or more agents, and sending instructions for presenting information associated with one or more of the executed tasks via the assistant xbot within the conversation thread.

Abstract: In one embodiment, a method includes by a client system associated with a user, receiving, at the client system associated with the user, a user input, parsing the user input to identify an n-gram associated with a wake word from a plurality of wake words corresponding to a plurality of assistant systems associated with the client system, wherein each assistant system provides a particular set of functions, determining that the wake word corresponds to a first assistant system of the plurality of assistant systems, wherein the first assistant system provides a first set of functions, sending, to the first assistant system, a request to set an assistant xbot of the first assistant system into a listening mode, and receiving, from the first assistant system, an indication that the assistant xbot is in listening mode responsive to a determination that the user has permission to access the first assistant system.

Abstract: In one embodiment, a method includes receiving from a client system a user request from a first user, determining a necessity for resolving the first user to a known entity to execute one or more tasks associated with the user request based on privacy restrictions associated with the user request, determining a set of candidate entities for the first user based on one or more machine-learning models, each candidate entity being associated with a respective confidence score greater than a threshold score, sending instructions for prompting the first user to select a candidate entity from the set of candidate entities, resolving the first user to a selected candidate entity responsive to receiving a selection from the first user, and executing the one or more tasks associated with the user request based on a user profile associated with the selected candidate entity.

Abstract: In one embodiment, a method includes receiving a user request from a client system associated with a first user, wherein the user request is associated with a semantic-intent, identifying one or more dialog-intents associated with the user request based on the semantic-intent and context information associated with the user request, wherein each dialog-intent is a sub-intent of the semantic-intent, determining one or more agents for executing one or more tasks associated with the one or more dialog-intents, and sending instructions for presenting information returned from the one or more agents responsive to executing the one or more tasks to the client system.

Abstract: Features are disclosed for automatically identifying a speaker. Artifacts of automatic speech recognition (“ASR”) and/or other automatically determined information may be processed against individual user profiles or models. Scores may be determined reflecting the likelihood that individual users made an utterance. The scores can be based on, e.g., individual components of Gaussian mixture models (“GMMs”) that score best for frames of audio data of an utterance. A user associated with the highest likelihood score for a particular utterance can be identified as the speaker of the utterance. Information regarding the identified user can be provided to components of a spoken language processing system, separate applications, etc.

Abstract: In one embodiment, a method includes receiving a user request associated with one or more domains from a client system associated with a first user, parsing the user request to identify one or more semantic-intents are associated with the one or more domains and one or more slots, identifying, based on a ranker model, one or more dialog-intents associated with the user request based on the one or more semantic-intents and slots and context information associated with the user request, wherein each dialog-intent is a sub-intent of one or more of the semantic-intents, determining one or more agents for executing one or more tasks associated with the one or more dialog-intents respectively, and sending instructions for presenting a communication content information returned from the one or more agents responsive to executing the one or more tasks responsive to the user input to the client system.

Abstract: Exemplary embodiments relate to improvements in digital assistants incorporating personalization based on social network data. Various aspects of the agent, such as the agent's voice, language style, and avatar may be personalized. Personalization may be applied to components of an agent's architecture (e.g., the virtual agent's language model, natural language generator, voice generation component, etc.). Moreover, by interfacing with the social network's social graph, the agent may be provided with information useful to performing certain tasks (e.g., a calendar for scheduling, food preferences for ordering tasks, etc.). An agent may be provided (and personalized) for a single user, or a group of users (e.g., a family). The agent can be personalized to anyone, which may allow (e.g.) for the agent to represent a celebrity or a person who is not currently available in interactions with others. Different agents can talk to each other, e.g. for purposes of scheduling meetings.

Abstract: Features are disclosed for automatically identifying a speaker. Artifacts of automatic speech recognition (“ASR”) and/or other automatically determined information may be processed against individual user profiles or models. Scores may be determined reflecting the likelihood that individual users made an utterance. The scores can be based on, e.g., individual components of Gaussian mixture models (“GMMs”) that score best for frames of audio data of an utterance. A user associated with the highest likelihood score for a particular utterance can be identified as the speaker of the utterance. Information regarding the identified user can be provided to components of a spoken language processing system, separate applications, etc.

Abstract: Features are disclosed for automatically identifying a speaker. Artifacts of automatic speech recognition (“ASR”) and/or other automatically determined information may be processed against individual user profiles or models. Scores may be determined reflecting the likelihood that individual users made an utterance. The scores can be based on, e.g., individual components of Gaussian mixture models (“GMMs”) that score best for frames of audio data of an utterance. A user associated with the highest likelihood score for a particular utterance can be identified as the speaker of the utterance. Information regarding the identified user can be provided to components of a spoken language processing system, separate applications, etc.

Abstract: In one embodiment, a method includes receiving a user request associated with one or more domains from a client system associated with a first user, parsing the user request to identify one or more semantic-intents are associated with the one or more domains and one or more slots, identifying, based on a ranker model, one or more dialog-intents associated with the user request based on the one or more semantic-intents and slots and context information associated with the user request, wherein each dialog-intent is a sub-intent of one or more of the semantic-intents, determining one or more agents for executing one or more tasks associated with the one or more dialog-intents respectively, and sending instructions for presenting a communication content information returned from the one or more agents responsive to executing the one or more tasks responsive to the user input to the client system.

Abstract: In one embodiment, a method includes receiving a user input by the first user from a client system associated with a first user, parsing the user input to identify one or more n-grams associated with the user input, accessing a user profile associated with the first user, wherein the user profile is stored in a first data store, accessing ontology data based on the one or more identified n-grams from one or more information graphs, wherein the one or more information graphs are stored in one or more second data stores, respectively, determining contextual information associated with the user input, generating semantic information by aggregating the user profile, ontology data, and contextual information, generating a feature representation for the identified one or more n-grams based on the semantic information, and resolving one or more entities associated with the one or more n-grams based on the feature representation.

Abstract: Features are disclosed for automatically identifying a speaker. Artifacts of automatic speech recognition (“ASR”) and/or other automatically determined information may be processed against individual user profiles or models. Scores may be determined reflecting the likelihood that individual users made an utterance. The scores can be based on, e.g., individual components of Gaussian mixture models (“GMMs”) that score best for frames of audio data of an utterance. A user associated with the highest likelihood score for a particular utterance can be identified as the speaker of the utterance. Information regarding the identified user can be provided to components of a spoken language processing system, separate applications, etc.

Abstract: Exemplary embodiments relate to improvements in digital assistants incorporating personalization based on social network data. Various aspects of the agent, such as the agent's voice, language style, and avatar may be personalized. Personalization may be applied to components of an agent's architecture (e.g., the virtual agent's language model, natural language generator, voice generation component, etc.). Moreover, by interfacing with the social network's social graph, the agent may be provided with information useful to performing certain tasks (e.g., a calendar for scheduling, food preferences for ordering tasks, etc.). An agent may be provided (and personalized) for a single user, or a group of users (e.g., a family). The agent can be personalized to anyone, which may allow (e.g.) for the agent to represent a celebrity or a person who is not currently available in interactions with others. Different agents can talk to each other, e.g. for purposes of scheduling meetings.

Abstract: Features are disclosed for error tolerant model compression. Such features could be used to reduce the size of a deep neural network model including several hidden node layers. The size reduction in an error tolerant fashion ensures predictive applications relying on the model do not experience performance degradation due to model compression. Such predictive applications include automatic recognition of speech, image recognition, and recommendation engines. Partially quantized models are re-trained such that any degradation of accuracy is “trained out” of the model providing improved error tolerance with compression.

Abstract: An automatic speech recognition (ASR) system detects an endpoint of an utterance using the active hypotheses under consideration by a decoder. The ASR system calculates the amount of non-speech detected by a plurality of hypotheses and weights the non-speech duration by the probability of each hypotheses. When the aggregate weighted non-speech exceeds a threshold, an endpoint may be declared.

Abstract: A voice controlled assistant having a housing to hold one or more microphones, one or more speakers, and various computing components. The voice controlled assistant facilitates transactions and other functions primarily through verbal interactions with a user. In some situations, a transaction may require entry of a code, which the user may wish to enter in a non-verbal way. The voice controlled assistant is configured to analyze an audio signal to detect user interactions with the surface of the voice controlled assistant and to interpret the detected interactions as entry of the code.

Abstract: Features are disclosed for automatically identifying a speaker. Artifacts of automatic speech recognition (“ASR”) and/or other automatically determined information may be processed against individual user profiles or models. Scores may be determined reflecting the likelihood that individual users made an utterance. The scores can be based on, e.g., individual components of Gaussian mixture models (“GMMs”) that score best for frames of audio data of an utterance. A user associated with the highest likelihood score for a particular utterance can be identified as the speaker of the utterance. Information regarding the identified user can be provided to components of a spoken language processing system, separate applications, etc.

Abstract: Features are disclosed for automatically identifying a speaker. Artifacts of automatic speech recognition (“ASR”) and/or other automatically determined information may be processed against individual user profiles or models. Scores may be determined reflecting the likelihood that individual users made an utterance. The scores can be based on, e.g., individual components of Gaussian mixture models (“GMMs”) that score best for frames of audio data of an utterance. A user associated with the highest likelihood score for a particular utterance can be identified as the speaker of the utterance. Information regarding the identified user can be provided to components of a spoken language processing system, separate applications, etc.