Abstract: Systems and methods for learning differentially private machine-learned models are provided. A computing system can include one or more server computing devices comprising one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors cause the one or more server computing devices to perform operations. The operations can include selecting a subset of client computing devices from a pool of available client computing devices; providing a machine-learned model to the selected client computing devices; receiving, from each selected client computing device, a local update for the machine-learned model; determining a differentially private aggregate of the local updates; and determining an updated machine-learned model based at least in part on the data-weighted average of the local updates.

Abstract: The present disclosure provides systems and methods for on-device machine learning. In particular, the present disclosure is directed to an on-device machine learning platform and associated techniques that enable on-device prediction, training, example collection, and/or other machine learning tasks or functionality. The on-device machine learning platform can include a context provider that securely injects context features into collected training examples and/or client-provided input data used to generate predictions/inferences. Thus, the on-device machine learning platform can enable centralized training example collection, model training, and usage of machine-learned models as a service to applications or other clients.

Abstract: The present disclosure provides systems and methods for on-device machine learning. In particular, the present disclosure is directed to an on-device machine learning platform and associated techniques that enable on-device prediction, training, example collection, and/or other machine learning tasks or functionality. The on-device machine learning platform can include a context provider that securely injects context features into collected training examples and/or client-provided input data used to generate predictions/inferences. Thus, the on-device machine learning platform can enable centralized training example collection, model training, and usage of machine-learned models as a service to applications or other clients.

Abstract: The present disclosure provides an image capture, curation, and editing system that includes a resource-efficient mobile image capture device that continuously captures images. In particular, the present disclosure provides low power frameworks for controlling image sensor mode in a mobile image capture device. On example low power frame work includes a scene analyzer that analyzes a scene depicted by a first image and, based at least in part on such analysis, causes an image sensor control signal to be provided to an image sensor to adjust at least one of the frame rate and the resolution of the image sensor.

Abstract: The present disclosure relates to a method for controlling a digital photography system. The method includes obtaining, by a device, image data and audio data. The method also includes identifying one or more objects in the image data and obtaining a transcription of the audio data. The method also includes controlling a future operation of the device based at least on the one or more objects identified in the image data, and the transcription of the audio data.

Abstract: The present disclosure provides an image capture, curation, and editing system that includes a resource-efficient mobile image capture device that continuously captures images. The mobile image capture device is operable to input an image into at least one neural network and to receive at least one descriptor of the desirability of a scene depicted by the image as an output of the at least one neural network. The mobile image capture device is operable to determine, based at least in part on the at least one descriptor of the desirability of the scene of the image, whether to store a second copy of such image and/or one or more contemporaneously captured images in a non-volatile memory of the mobile image capture device or to discard a first copy of such image from a temporary image buffer without storing the second copy of such image in the non-volatile memory.

Abstract: The present disclosure provides systems and methods for the management and/or evaluation of machine-learned models based on locally logged data. In one example, a user computing device can obtain a machine-learned model (e.g., from a server computing device) and can evaluate at least one performance metric for the machine-learned model. In particular, the at least one performance metric for the machine-learned model can be evaluated relative to data that is stored locally at the user computing device. The user computing device and/or the server computing device can determine whether to activate the machine-learned model on the user computing device based at least in part on the at least one performance metric. In another example, the user computing device can evaluate a plurality of machine-learned models against locally stored data. At least one of the models can be selected based on the evaluated performance metrics.

Abstract: The present disclosure relates to a method for controlling a digital photography system. The method includes obtaining, by a device, image data and audio data. The method also includes identifying one or more objects in the image data and obtaining a transcription of the audio data. The method also includes controlling a future operation of the device based at least on the one or more objects identified in the image data, and the transcription of the audio data.

Abstract: The present disclosure provides an image capture, curation, and editing system that includes a resource-efficient mobile image capture device that continuously captures images. The mobile image capture device is operable to input an image into at least one neural network and to receive at least one descriptor of the desirability of a scene depicted by the image as an output of the at least one neural network. The mobile image capture device is operable to determine, based at least in part on the at least one descriptor of the desirability of the scene of the image, whether to store a second copy of such image and/or one or more contemporaneously captured images in a non-volatile memory of the mobile image capture device or to discard a first copy of such image from a temporary image buffer without storing the second copy of such image in the non-volatile memory.

Abstract: The present disclosure provides an image capture, curation, and editing system that includes a resource-efficient mobile image capture device that continuously captures images. The mobile image capture device is operable to input an image into at least one neural network and to receive at least one descriptor of the desirability of a scene depicted by the image as an output of the at least one neural network. The mobile image capture device is operable to determine, based at least in part on the at least one descriptor of the desirability of the scene of the image, whether to store a second copy of such image in a non-volatile memory of the mobile image capture device or to discard a first copy of such image from a temporary image buffer without storing the second copy of such image in the non-volatile memory.

Abstract: The present disclosure relates to a method for controlling a digital photography system. The method includes obtaining, by a device, image data and audio data. The method also includes identifying one or more objects in the image data and obtaining a transcription of the audio data. The method also includes controlling a future operation of the device based at least on the one or more objects identified in the image data, and the transcription of the audio data.

Abstract: A messaging application may automatically analyze content of one or more messages and/or user information to automatically provide suggestions to a user within a messaging application. The suggestions may automatically incorporate particular non-messaging functionality into the messaging application. The automatic suggestions may suggest one or more appropriate responses to be selected by a user to respond in the messaging application, and/or may automatically send one or more appropriate responses on behalf of a user.

Abstract: Systems and methods of determining a global model are provided. In particular, one or more local updates can be received from a plurality of user devices. Each local update can be determined by the respective user device based at least in part on one or more data examples stored on the user device. The one or more data examples stored on the plurality of user devices are distributed on an uneven basis, such that no user device includes a representative sample of the overall distribution of data examples. The local updates can then be aggregated to determine a global model.

Abstract: Systems and methods are provided for suggesting actions for selected text based on content displayed on a mobile device. An example method can include converting a selection made via a display device into a query, providing the query to an action suggestion model that is trained to predict an action given a query, each action being associated with a mobile application, receiving one or more predicted actions, and initiating display of the one or more predicted actions on the display device. Another example method can include identifying, from search records, queries where a website is highly ranked, the website being one of a plurality of websites in a mapping of websites to mobile applications. The method can also include generating positive training examples for an action suggestion model from the identified queries, and training the action suggestion model using the positive training examples.

Abstract: The present disclosure relates to a method for controlling a digital photography system. The method includes obtaining, by a device, image data and audio data. The method also includes identifying one or more objects in the image data and obtaining a transcription of the audio data. The method also includes controlling a future operation of the device based at least on the one or more objects identified in the image data, and the transcription of the audio data.

Abstract: A system stores a table mapping users to attributes, and stores a second table mapping the users to products associated with a source domain. The system determines a set of top scoring products for each of the attributes, and creates, using the top scoring products, a model that is predictive of an activity in a target domain, the target domain being separate from the source domain. The system detects a behavior from a particular user accessing the target domain, and generates a personalized prediction for the particular user based on the model, in response to the detecting the behavior.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for obtaining a global model for a particular activity, the global model derived based on input data representing multiple observations associated with the particular activity performed by a collection of users; determining, using the global model, expected data representing an expected observation associated with the particular activity performed by a particular user; receiving, by a computing device operated by the particular user, particular data representing an actual observation associated with the particular activity performed by the particular user; determining, by the computing device and using (i) the expected data and (ii) the particular data, residual data of the particular user; and deriving a local model of the particular user based on the residual data.

Abstract: A system stores a table mapping users to attributes, and stores a second table mapping the users to products associated with a source domain. The system determines a set of top scoring products for each of the attributes, and creates, using the top scoring products, a model that is predictive of an activity in a target domain, the target domain being separate from the source domain. The system detects a behavior from a particular user accessing the target domain, and generates a personalized prediction for the particular user based on the model, in response to the detecting the behavior.

Abstract: A latent topic labels text mining system and method to mine and analyze the content of textual data. Embodiments of the system and method are particularly well suited for use on microblog data to help people identify posts they want to read and to find people that they want to follow. Embodiments of the system and method use a modified Labeled LDA technique (called an L+LDA technique) that analyzes content using a combination of labeled and latent topics. The resultant data is assigned labels one of four labels to generate a lower-dimensional representation of the data that the individual words in a microblog post. This learned topic representation is used to characterize, summarize, filter, find, suggest, and compare the content of microblog posts. Embodiments of the system and method also include visualization techniques such as a tag cloud visualization that is used to visualize microblogging data.