Abstract: In one embodiment, a method includes receiving a voice input having acoustic features from a first client system associated with a first user, determining emotions associated with the voice input based on one or more of the acoustic features by machine-learning models, determining facial features for a first extended-reality (XR) avatar representing the first user based on the emotions, and sending instructions for rendering the first XR avatar representing the first user to a second client system associated with a second user, wherein the first XR avatar is rendered with the determined facial features.

Abstract: In one embodiment, a method includes receiving a first input by a user from a client system associated with the user, wherein the first input is in a voice modality, analyzing the first input to generate one or more candidate hypotheses, determining one or more modalities for presenting output generated by the one or more computing systems to the user at the client system, and sending instructions to the client system for presenting one or more suggested auto-completions corresponding to one or more of the candidate hypotheses, respectively, wherein each suggested auto-completion comprises the corresponding candidate hypothesis, and wherein the one or more suggested auto-completions are presented in the one or more determined modalities.

Abstract: In one embodiment, a method includes accessing visual signals comprising images portraying textual content in a real-world environment associated with a first user from a client system associated with the first user, recognizing the textual content based on machine-learning models and the visual signals, determining a context associated with the first user with respect to the real-world environment based on the visual signals, executing tasks determined based on the textual content and the determined context for the first user, and sending instructions for presenting execution results of the tasks to the first user to the client system.

Abstract: In one embodiment, a method includes receiving a voice input having acoustic features from a first client system associated with a first user, determining emotions associated with the voice input based on one or more of the acoustic features by machine-learning models, determining facial features for a first extended-reality (XR) avatar representing the first user based on the emotions, and sending instructions for rendering the first XR avatar representing the first user to a second client system associated with a second user, wherein the first XR avatar is rendered with the determined facial features.

Abstract: In one embodiment, a method includes accessing visual signals comprising images portraying textual content in a real-world environment associated with a first user from a client system associated with the first user, recognizing the textual content based on machine-learning models and the visual signals, determining a context associated with the first user with respect to the real-world environment based on the visual signals, executing tasks determined based on the textual content and the determined context for the first user, and sending instructions for presenting execution results of the tasks to the first user to the client system.

Abstract: In one embodiment, a method includes receiving, by a first client system, from one or more remote servers, a current version of a neural network model including multiple model parameters, training the neural network model on multiple examples retrieved from a local data store to generate multiple updated model parameters, wherein each of the examples includes one or more features and one or more labels, calculating a user valuation associated with the first client system, wherein the user valuation represents a measure of utility of training the neural network model on the multiple examples, and sending, to one or more of the remote servers, the trained neural network model and the user valuation, wherein the user valuation is associated with a likelihood of the first client system being selected for a subsequent training of the neural network model.

Abstract: In one embodiment, a system includes an automatic speech recognition (ASR) module, a natural-language understanding (NLU) module, a dialog manager, one or more agents, an arbitrator, a delivery system, one or more processors, and a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to receive a user input, process the user input using the ASR module, the NLU module, the dialog manager, one or more of the agents, the arbitrator, and the delivery system, and provide a response to the user input.

Abstract: In one embodiment, a method includes accessing visual signals comprising images portraying textual content in a real-world environment associated with a first user from a client system associated with the first user, recognizing the textual content based on machine-learning models and the visual signals, determining a context associated with the first user with respect to the real-world environment based on the visual signals, executing tasks determined based on the textual content and the determined context for the first user, and sending instructions for presenting execution results of the tasks to the first user to the client system.

Abstract: Exemplary embodiments relate to methods, mediums, and systems for moving language models from a server to the client device. Such embodiments may be deployed in an environment where the server is not able to provide modeling services to the clients, such as an end-to-end encrypted (E2EE) environment. Several different techniques are described to address issues of size and complexity reduction, model architecture optimization, model training, battery power reduction, and latency reduction.

Abstract: Exemplary embodiments relate to methods, mediums, and systems for moving language models from a server to the client device. Such embodiments may be deployed in an environment where the server is not able to provide modeling services to the clients, such as an end-to-end encrypted (E2EE) environment. Several different techniques are described to address issues of size and complexity reduction, model architecture optimization, model training, battery power reduction, and latency reduction.

Abstract: In one embodiment, a method includes receiving a first input by a user from a client system associated with the user, wherein the first input is in a voice modality, analyzing the first input to generate one or more candidate hypotheses, determining one or more modalities for presenting output generated by the one or more computing systems to the user at the client system, and sending instructions to the client system for presenting one or more suggested auto-completions corresponding to one or more of the candidate hypotheses, respectively, wherein each suggested auto-completion comprises the corresponding candidate hypothesis, and wherein the one or more suggested auto-completions are presented in the one or more determined modalities.

Abstract: Exemplary embodiments relate to methods, mediums, and systems for moving language models from a server to the client device. Such embodiments may be deployed in an environment where the server is not able to provide modeling services to the clients, such as an end-to-end encrypted (E2EE) environment. Several different techniques are described to address issues of size and complexity reduction, model architecture optimization, model training, battery power reduction, and latency reduction.

Abstract: In one embodiment, a method includes receiving, by a first client system, from one or more remote servers, a current version of a global neural network model including multiple federated model parameters, accessing, from a local data store, multiple examples and a local personalization model including multiple local model parameters, wherein each of the examples includes one or more features and one or more labels, training the global neural network model and the local personalization model together on the examples to generate multiple updated federated model parameters and multiple updated local model parameters, storing, in the local data store, the trained local personalization model including the updated local model parameters, and sending, to one or more of the remote servers, the trained global neural network model including the updated federated model parameters.