Abstract: The present disclosure is generally directed to a data processing system for customizing content in a voice activated computer network environment. With user consent, the data processing system can improve the efficiency and effectiveness of auditory data packet transmission over one or more computer networks by, for example, increasing the accuracy of the voice identification process used in the generation of customized content. The present solution can make accurate identifications while generating fewer audio identification models, which are computationally intensive to generate.

Abstract: The present disclosure provides an on-device machine learning platform that enables sharing of machine-learned models between applications on a computing device. For example, a first application which has a machine-learned model for a specific task can expose the model to other applications through a system level application programming interface (API) for the other applications to use. Communications using the API can be handled by the on-device machine learning platform. In some implementations, some exchange of resources (e.g., computing resources) can be provided so that the first application is compensated for sharing the machine-learned model (e.g., on a per model invocation basis).

Abstract: Methods, apparatus, and computer readable media related to receiving textual input of a user during a dialog between the user and an automated assistant (and optionally one or more additional users), and generating responsive reply content based on the textual input and based on user state information. The reply content is provided for inclusion in the dialog. In some implementations, the reply content is provided as a reply, by the automated assistant, to the user's textual input and may optionally be automatically incorporated in the dialog between the user and the automated assistant. In some implementations, the reply content is suggested by the automated assistant for inclusion in the dialog and is only included in the dialog in response to further user interface input.

Abstract: The present disclosure provides systems and methods for on-device machine learning. In particular, the present disclosure is directed to an on-device machine learning platform and associated techniques that enable on-device prediction, training, example collection, and/or other machine learning tasks or functionality. The on-device machine learning platform can include a mode controller that allows a user to provide data input indicating whether to operate one or more applications on the device in a first collection mode (e.g., permission mode) for storing training examples or a second collection mode for (e.g., incognito mode) for not storing training examples. The training examples can be generated based on user interaction with the one or more applications and used to personalize one or more machine-learned models used by the application(s) by retraining the models using the user-specific training examples.

Abstract: The present disclosure provides systems and methods that leverage machine learning to predict multiple touch interpretations. In particular, the systems and methods of the present disclosure can include and use a machine-learned touch interpretation prediction model that has been trained to receive touch sensor data indicative of one or more locations of one or more user input objects relative to a touch sensor at one or more times and, in response to receipt of the touch sensor data, provide one or more predicted touch interpretation outputs. Each predicted touch interpretation output corresponds to a different type of predicted touch interpretation based at least in part on the touch sensor data. Predicted touch interpretations can include a set of touch point interpretations, a gesture interpretation, and/or a touch prediction vector for one or more future times.

Abstract: Methods, systems, and apparatuses for implementing advanced content retrieval are described. Machine learning methods may be implemented so that a system may predict when a user device may experience network disconnections. The system may also predict the type of content one or more applications on the user device may seek to download during the network disconnection period. Neural networks may be trained based on user activity log data and may implement machine-learning techniques to determine user preferences and settings for advanced content retrieval. The system may predict when a user may want to download content in advance, the type of content the user may be interested in, anticipated network connectivity, and anticipated battery consumption. The system may then generate recommendations for the user device based on the predictions. If a user agrees with the recommendations, the system may obtain and cache the content.

Abstract: Methods, systems, and apparatuses, including computer programs encoded on a computer-readable storage medium for implementing advanced information retrieval are described. A user may provide fetching parameter values to acquire content. The system may determine whether or not the user-provided fetching parameter values can be satisfied. If the fetching parameters can be satisfied, the system obtains and caches the information.

Abstract: Methods, apparatus, and computer readable media related to receiving textual input of a user during a dialog between the user and an automated assistant (and optionally one or more additional users), and generating responsive reply content based on the textual input and based on user state information. The reply content is provided for inclusion in the dialog. In some implementations, the reply content is provided as a reply, by the automated assistant, to the user's textual input and may optionally be automatically incorporated in the dialog between the user and the automated assistant. In some implementations, the reply content is suggested by the automated assistant for inclusion in the dialog and is only included in the dialog in response to further user interface input.

Abstract: The present disclosure provides systems and methods that leverage machine learning to refine and/or predict sensor outputs for multiple sensors. In particular, systems and methods of the present disclosure can include and use a machine-learned virtual sensor model that has been trained to receive sensor data from multiple sensors that is indicative of one or more measured parameters in each sensor's physical environment, recognize correlations among sensor outputs of the multiple sensors, and in response to receipt of the sensor data from multiple sensors, output one or more virtual sensor output values. The one or more virtual sensor output values can include one or more of refined sensor output values and one or more predicted future sensor output value.

Abstract: The present disclosure provides systems and methods that leverage machine learning to predict multiple touch interpretations. In particular, the systems and methods of the present disclosure can include and use a machine-learned touch interpretation prediction model that has been trained to receive touch sensor data indicative of one or more locations of one or more user input objects relative to a touch sensor at one or more times and, in response to receipt of the touch sensor data, provide one or more predicted touch interpretation outputs. Each predicted touch interpretation output corresponds to a different type of predicted touch interpretation based at least in part on the touch sensor data. Predicted touch interpretations can include a set of touch point interpretations, a gesture interpretation, and/or a touch prediction vector for one or more future times.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for collaboration between multiple voice controlled devices are disclosed. In one aspect, a method includes the actions of identifying, by a first computing device, a second computing device that is configured to respond to a particular, predefined hotword; receiving audio data that corresponds to an utterance; receiving a transcription of additional audio data outputted by the second computing device in response to the utterance; based on the transcription of the additional audio data and based on the utterance, generating a transcription that corresponds to a response to the additional audio data; and providing, for output, the transcription that corresponds to the response.

Abstract: A social network server system may receive a social media message that is to be posted at the social network server system, the social media message being authored by a user of the social network server system. Prior to posting the social media message at the social network server system, the social network server system may determine, based at least in part on applying one or more rules to content of the social media message, a likelihood that the user would modify the content of the social media message after it is posted at the social network server system, wherein the one or more rules are generated based at least in part on previous actions taken by the user on previous social media messages authored by the user and posted at the social network server system and may, responsive to determining that the likelihood exceeds a threshold, generate an alert message.

Abstract: Systems and methods enable a computing system to recognize a sequence of repeated actions and offer to automatically repeat any such recognized actions. An example method includes determining a current sequence of user actions is similar to a previous sequence of user actions, determining whether the previous sequence is reproducible and, when reproducible, initiating display of a prompt that requests approval for completing the current sequence based on the previous sequence and, responsive to receiving an indication of approval, completing the previous sequence. Another example method includes determining that a first current sequence of user interactions is complete and is not similar to any saved sequence of user interactions, saving the first current sequence as a previous sequence, identifying a second current sequence as satisfying a similarity threshold with the previous sequence, and initiating display of a prompt that requests approval for saving the previous sequence as a shortcut.

Abstract: The present disclosure provides systems and methods that leverage machine-learned models (e.g., neural networks) to provide enhanced communication assistance. In particular, the systems and methods of the present disclosure can include or otherwise leverage a machine-learned communication assistance model to detect problematic statements included in a communication and/or provide suggested replacement statements to respectively replace the problematic statements. In one particular example, the communication assistance model can include a long short-term memory recurrent neural network that detects an inappropriate tone or unintended meaning within a user-composed communication and provides one or more suggested replacement statements to replace the problematic statements.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer-readable storage medium for implementing one or more application programming interfaces (APIs) that configure applications stored in an electronic device are described. An application may be configured to receive event information from various sources based on user preferences and application permissions. In response to receiving the event information, the app may determine whether a notification should be issued to a user. This determination may be made based on various factors such as the type of event, user history, contextual data, ranking data, and application permissions. The notifications may include one or more of messages to the user and recommended actions for consideration by the user. The actions may include sharing data with other users who share a presence or interest in an event with the user.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing dynamic, stroke-based alignment of touch displays. In one aspect, a method include providing, for output by a first mobile computing device that (i) has a first proximity sensitive display and (ii) has been designated a primary display device, a primary alignment user interface. The methods also includes transmitting, by the first mobile computing device to a second mobile computing device that (i) has a second proximity sensitive display and (ii) has been designated a secondary display device, an instruction to output a secondary alignment user interface.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for providing actionable suggestions are disclosed. In one aspect, a method includes receiving (i) an indication that an event detection module has determined that a shared event of a particular type is presently occurring or has occurred, and (ii) data referencing an attribute associated with the shared event. The method includes selecting, from among multiple output templates that are each associated with a different type of shared event, a particular output template associated with the particular type of shared event detected by the module. The method generates a notification for output using at least (i) the selected particular output template, and (ii) the data referencing the attribute associated with the shared event. The method then provides, for output to a user device, the notification that is generated.

Abstract: A first handwriting input is received comprising strokes corresponding to a set of first characters comprising one or more first characters forming a first language model unit. A set of candidate first characters and a set of candidate first language model units with corresponding probability scores are determined based on an analysis of the one or more sets of candidate first characters using the first language model and a corresponding first character recognition model. When no first probability score satisfies a threshold, one or more sets of candidate second characters and a set of candidate second language model units are determined based on an analysis of the first handwriting input using a second language model and a corresponding second character recognition model. A first candidate list is then output comprising at least one of the set of candidate second language model units.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing event detection are disclosed. In one aspect, a method a computing system that receives data from a first computing device associated with a first user that indicates a current context of the first user. The method includes identifying a subset of users associated with the first user based on the current context of the first user, and receiving data indicating a current context of the at least one other user. The method compares the current context of the first user with the current context of the at least one other user and determines that a shared event is presently occurring or has occurred. The shared event can be an event associated with the first user and the at least one other user of the subset of users. The method then indicates, at least to the first user, that the shared event is presently occurring or has occurred.

Abstract: Methods, apparatus, and computer readable media related to receiving textual input of a user during a dialog between the user and an automated assistant (and optionally one or more additional users), and generating responsive reply content based on the textual input and based on user state information. The reply content is provided for inclusion in the dialog. In some implementations, the reply content is provided as a reply, by the automated assistant, to the user's textual input and may optionally be automatically incorporated in the dialog between the user and the automated assistant. In some implementations, the reply content is suggested by the automated assistant for inclusion in the dialog and is only included in the dialog in response to further user interface input.