Abstract: User interaction may be supported with an audio presentation by an automated assistant, and in particular with the spoken content of such an audio presentation that is presented at particular points within the audio presentation. Analysis of an audio presentation may be performed to identify one or more entities addressed by, mentioned by, or otherwise associated with the audio presentation, and utterance classification may be performed to determine whether an utterance received during playback of the audio presentation is directed to the audio presentation, and in some instances, to a particular entity and/or point of playback in the audio presentation, thereby enabling a suitable response to be generated to the utterance.

Abstract: Implementations set forth herein relate to an automated assistant that can control graphical user interface (GUI) elements via voice input using natural language understanding of GUI content in order to resolve ambiguity and allow for condensed GUI voice input requests. When a user is accessing an application that is rendering various GUI elements at a display interface, the automated assistant can operate to process actionable data corresponding to the GUI elements. The actionable data can be processed in order to determine a correspondence between GUI voice input requests to the automated assistant and at least one of the GUI elements rendered at the display interface. When a particular spoken utterance from the user is determined to correspond to multiple GUI elements, an indication of ambiguity can be rendered at the display interface in order to encourage the user to provide a more specific spoken utterance.

Abstract: The present disclosure provides systems and methods that perform structure-based access control. In particular, rather than relying upon a user-specific credential scheme, which can require manual sharing of user-specific credentials and/or switching between the multiple accounts to access the particular devices, applications, or services associated with such accounts, the systems and methods of the present disclosure facilitate user credentials to be inherited by or otherwise assigned to a structure identifier associated with a structure (e.g., a home in which the user resides), thereby generating a set of structure credentials. This enables other users in the structure, who may be part of a collaborative user group, to access devices, applications, and/or services using the structure credentials.

Abstract: Implementations relate to receiving natural language input that requests an automated assistant to provide information and processing the natural language input to identify the requested information and to identify one or more predicted actions. Those implementations further cause a computing device, at which the natural language input is received, to render the requested information and the one or more predicted actions in response to the natural language input. Yet further, those implementations, in response to the user confirming a rendered predicted action, cause the automated assistant to initialize the predicted action.

Abstract: Selection and serving of content items may include receiving data indicative of a status of an energy source of a device with a request for a content item. A first received content item may be associated with a first energy consumption level and a second received content item may be associated with a second energy consumption level. The accessed content items are responsive to the request for a content item. The first energy consumption level may be higher than the second energy consumption level. The first content item or the second content item may be selected based, at least in part, on the received data indicative of the status of the energy source of the device, and data to display the selected content item may be provided to the device.

Abstract: This document describes techniques and systems that enable privacy controls for sharing embeddings for searching and indexing media content. A set of images of a user's face are obtained and a machine-learned model is applied to the set of images to generate a user-specific dataset of face embeddings for the user. Media content stored in a media storage is indexed by applying the machine-learned model to the media content to provide indexed media information identifying one or more faces shown in the media content. Access to the indexed media information by another user querying the media content for images or videos depicting the user is controlled based on a digital key shared by the user with the other user, where the digital key is associated with the user-specific dataset and the user-specific dataset is usable to identify the images or videos depicting the user.

Abstract: Techniques of generating a wavefront modulating element (WME) for imaging an object over a large image field include (i) designing a WME by breaking a large image into smaller sub-images and then applying an inverse imaging operation to find a segment of a plurality of segments (324-1 to-334-n) of a WME (in the from of DOE) producing a sub-image, and (ii) specifying an optical system to illuminate the WME resulting from each of the plurality of segments in such a way that the large image is reproduced as closely as possible. Along these lines, given a large target image in the far-field, a WME generation system decomposes the target image into sub-images. From this decomposition, the WME generation system then produces WME segments corresponding to the sub-images.

Abstract: Training and/or utilizing a hierarchical reinforcement learning (HRL) model for robotic control. The HRL model can include at least a higher-level policy model and a lower-level policy model. Some implementations relate to technique(s) that enable more efficient off-policy training to be utilized in training of the higher-level policy model and/or the lower-level policy model. Some of those implementations utilize off-policy correction, which re-labels higher-level actions of experience data, generated in the past utilizing a previously trained version of the HRL model, with modified higher-level actions. The modified higher-level actions are then utilized to off-policy train the higher-level policy model. This can enable effective off-policy training despite the lower-level policy model being a different version at training time (relative to the version when the experience data was collected).

Abstract: Coordinating processing of audio queries is provided. A system receives a query. The system provides the query to a first digital assistant component and a second digital assistant component for processing. The system receives a first response to the query from the first digital assistant component, and a second response to the query from the second digital assistant component. The first digital assistant component can be authorized to access a database the second digital assistant component is prohibited from accessing. The system determines, based on a ranking decision function, to select the second response to the query from the second digital assistant component. The system provides, responsive to the selection, the second response from the second digital assistant to a computing device.

Abstract: Methods, systems, and apparatus for determining that a native application limits access to the native application using account credential requirements, the native application generating an application environment for display on a user device within the native application and operating independent of a browser application that can operate on the user device; obtaining a set of account credentials for indexing environment instances of the native application; instantiating the native application with the set of account credentials; and accessing environment instances of the native application, and for each of the environment instances: generating environment instance data describing content of the environment instance, the content described by the environment instance data including text that a user device displays on the environment instance when the user device displays the environment instance; and indexing the environment instance data for the native application in an index that is searchable by a search en

Abstract: A transform type is obtained for decoding the transform block of transform coefficients. A template for entropy-decoding values related to the transform coefficients is selected based on the transform type. The template indicates, for a to-be-coded value, positions of already coded values. A context for selecting a probability distribution for entropy decoding a current value of the values is determined using the template. The current value is entropy decoded from a compressed bitstream using the probability distribution.

Abstract: A method in a user device that supports an air interface for connecting to core networks (CNs) of different types includes receiving system information from a first base station or a second base station (1102), determining, based on the received system information, at least two CN types of respective CNs to which the first base station is connected (1104), choosing a CN type from among the at least two CN types (1106), and performing a mobility management procedure with a CN of the chosen CN type via the first base station (1108).

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for an automated calling system are disclosed. In one aspect, a method includes the actions of receiving audio data of an utterance spoken by a user who is having a telephone conversation with a bot. The actions further include determining a context of the telephone conversation. The actions further include determining a user intent of a first previous portion of the telephone conversation spoken by the user and a bot intent of a second previous portion of the telephone conversation outputted by a speech synthesizer of the bot. The actions further include, based on the audio data of the utterance, the context of the telephone conversation, the user intent, and the bot intent, generating synthesized speech of a reply by the bot to the utterance. The actions further include, providing, for output, the synthesized speech.

Abstract: Implementations relate to automatic generation of speaker features for each of one or more particular text-dependent speaker verifications (TD-SVs) for a user. Implementations can generate speaker features for a particular TD-SV using instances of audio data that each capture a corresponding spoken utterance of the user during normal non-enrollment interactions with an automated assistant via one or more respective assistant devices. For example, a portion of an instance of audio data can be used in response to: (a) determining that recognized term(s) for the spoken utterance captured by that the portion correspond to the particular TD-SV; and (b) determining that an authentication measure, for the user and for the spoken utterance, satisfies a threshold. Implementations additionally or alternatively relate to utilization of speaker features, for each of one or more particular TD-SVs for a user, in determining whether to authenticate a spoken utterance for the user.

Abstract: The present disclosure provides systems and methods for on-device machine learning. In particular, the present disclosure is directed to an on-device machine learning platform and associated techniques that enable on-device prediction, training, example collection, and/or other machine learning tasks or functionality. The on-device machine learning platform can include a mode controller that allows a user to provide data input indicating whether to operate one or more applications on the device in a first collection mode (e.g., permission mode) for storing training examples or a second collection mode for (e.g., incognito mode) for not storing training examples. The training examples can be generated based on user interaction with the one or more applications and used to personalize one or more machine-learned models used by the application(s) by retraining the models using the user-specific training examples.

Abstract: The present disclosure provides systems and methods that generate query templates that are expressed in a generic schema-agnostic language. The query templates can be generated “from scratch” or can be automatically generated from existing queries, a process which may be referred to as “templatizing” the existing queries. As one example, generation of query templates can be performed through an iterative process that iteratively generates candidate templates over time to optimize a coverage over a set of existing queries. After generation of the schema-agnostic query templates, the systems and methods described herein can automatically translate/map the templatized queries into “concrete,” schema-specific queries that can be evaluated over specific customer schemas/datasets. In this manner, a query template for a given semantic query (e.g., “return the names of all employees”), is required to be written only once.

Abstract: A method for determining playback behavior of a media item including a video component and an audio component includes determining a current event that causes a playing of the video component of the media item on a screen of a user device to be suspended. The method further includes predicting user playback preferences for the media item using a behavioral model trained on previous events corresponding to the current event, and determining, based on the predicted user playback preferences, whether to continue playing the audio component of the media item.