Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a prediction of an audio signal. One of the methods includes receiving a request to generate an audio signal conditioned on an input; processing the input using an embedding neural network to map the input to one or more embedding tokens; generating a semantic representation of the audio signal; generating, using one or more generative neural networks and conditioned on at least the semantic representation and the embedding tokens, an acoustic representation of the audio signal; and processing at least the acoustic representation using a decoder neural network to generate the prediction of the audio signal.

Abstract: Methods, systems, devices, and tangible non-transitory computer readable media for navigation are provided. The disclosed technology can include accessing navigation data that includes information associated with a navigation request from a user. Based on the navigation data, a determination of whether the navigation request indicates a specific location or a deferred travel time can be made. Based on the navigation request, one or more locations and one or more travel times associated with fulfilling the navigation request can be determined. The one or more locations can be based on whether the navigation request indicates a specific location. The one or more travel times can be based on whether the navigation request indicates a deferred travel time. Furthermore, output including a time window of the one or more travel times for the user to travel to at least one of the one or more locations can be generated.

Abstract: Systems and methods for media aggregation are disclosed herein. The system includes a media system that can transform media items into one aggregated media item. A synchronization component synchronizes media items with respect to time. The synchronized media items can be analyzed and transformed into an aggregated media item for storage and/or display. In one implementation, the aggregated media item is capable of being displayed in multiple ways to create an enhanced and customizable viewing and/or listening experience.

Abstract: Implementations relate to generating a proficiency measure, and utilizing the proficiency measure to adapt one or more automated assistant functionalities. The generated proficiency measure is for a particular class of automated assistant actions, and is specific to an assistant device and/or is specific to a particular user. A generated proficiency measure for a class can reflect a degree of proficiency, of a user and/or of an assistant device, for that class. Various automated assistant functionalities can be adapted, for a particular class, responsive to determining the proficiency measure satisfies a threshold, or fails to satisfy the threshold (or an alternate threshold). The adaptation(s) can make automated assistant processing more efficient and/or improve (e.g., shorten the duration of) user-assistant interaction(s).

Abstract: Systems and methods are provided for suggesting actions for selected text based on content displayed on a mobile device. An example method can include converting a selection made via a display device into a query, providing the query to an action suggestion model that is trained to predict an action given a query, each action being associated with a mobile application, receiving one or more predicted actions, and initiating display of the one or more predicted actions on the display device. Another example method can include identifying, from search records, queries where a website is highly ranked, the website being one of a plurality of websites in a mapping of websites to mobile applications. The method can also include generating positive training examples for an action suggestion model from the identified queries, and training the action suggestion model using the positive training examples.

Abstract: A computing device may implement a method for creating and providing reminders during navigation sessions. The method may include initiating a navigation session for providing a set of navigation instructions to a user from a starting location to a destination location along a route, and receiving, from a user during the navigation session, an input including a request to create a reminder. The method may further include analyzing the input to determine: (i) a content of the reminder, and (ii) a trigger condition for providing the reminder, the trigger condition including a trigger location on the route, and storing the reminder, including the content and the trigger condition. The method may also include, detecting, during a subsequent navigation session, the trigger condition, and, in response to detecting the trigger condition, providing the reminder.

Abstract: Implementations set forth herein relate to an automated assistant that can perform operations to revert various applications to prior states that the applications may have arrived at via certain user inputs. The user can provide a spoken utterance such as, “undo,” in order to cause the automated assistant to identify a particular application that the user may want to affect with the “undo” command. When the particular application is identified, the automated assistant can identify one or more operations recently performed using the particular application. In some implementations, the automated assistant can provide the user with a variety of undo options in response to an “undo” command. For instance, the automated assistant can prompt the user to select one of a first cluster of operations and/or a second cluster of operations to be undone, and each cluster can refer to different operations.

Abstract: A computing device may implement a method for determining places and routes through natural conversation. The method may include receiving, from a user, a speech input including a search query to initiate a navigation session; and generating a set of navigation search results responsive to the search query. The set of navigation search results include a plurality of destinations or a plurality of routes corresponding to one or more destinations. The method further includes providing an audio request to the user for refining the set of navigation search results, and in response to the audio request, receiving, from the user, a subsequent speech input including a refined search query. The method further includes providing one or more refined navigation search results responsive to the refined search query including a subset of the plurality of destinations or the plurality of routes.

Abstract: Implementations set forth herein relate to an automated assistant that facilitates the creation of complex messages from user input(s) to the automated assistant. Each message can be created according to a respective template that is selected based on user input that directs the automated assistant to communicate a message to a recipient. Furthermore, sections of a template can be designated for certain content based on prior messages communicated by one or more users to one or more recipients. In this way, in response to a user requesting that the automated assistant send a message, the automated assistant can select a related template and fill out the template accordingly. In some instances, content that is assigned to certain sections of the selected template can come from a variety of different sources and/or may not be explicitly specified in the request from the user to the automated assistant.

Abstract: Implementations set forth herein relate to an automated assistant application that can be accessible via a virtual environment and can streamline certain assistant interactions by rendering certain virtual features within the virtual environment. When the automated assistant determines that a user is accessing a portion of a virtual environment that includes a particular virtual object, the automated assistant can identify application operations that may be associated with the particular virtual object. Based on these identified operations, the automated assistant can cause rendering of certain virtual features in the virtual environment for initializing and/or otherwise controlling the operations. In some instances, these operations can affect the virtual environment and/or devices in a physical environment, as well as any other users that may be accessing the virtual environment.

Abstract: Techniques are described herein for collaborative search sessions through an automated assistant. A method includes: receiving, from a first user of a first client device, a first query in a query session; providing, to the first user, a first set of search results; determining, based on at least one term in the first query, that the first query is relevant to a second user of the first client device; providing, to the second user, a selectable option to join the query session; in response to receiving, from the second user, an indication of acceptance of the selectable option, adding the second user to the query session; receiving, from the second user, additional input; generating, based on the additional input received from the second user, a modified set of search results; and providing, to the first user and the second user, the modified set of search results.

Abstract: Implementations related to selecting a primary automated assistant for a given automated assistant device that is connected to one or more other automated assistant devices in an ecosystem of connected devices. An affinity score is generated for each of a plurality of candidate automated assistants that are capable of executing on the automated assistant device. The affinity score of a given automated assistant for an automated assistant device is indicative of suitability of the automated assistant for the automated assistant device. One of the candidate automated assistants is selected as the primary automated assistant for the device. The primary automated assistant is prioritized when resources of the automated assistant device are allocated to automated assistants executing on the automated assistant device.

Abstract: Implementations set forth herein relate to an automated assistant that can be customized by a user to provide custom assistant responses to certain assistant queries, which may originate from other users. The user can establish certain custom assistant responses by providing an assistant response request to the automated assistant and/or responding to a request from the automated assistant to establish a particular custom assistant response. In some instances, a user can elect to establish a custom assistant response when the user determines or acknowledges that certain common queries are being submitted to the automated assistant—but the automated assistant is unable to resolve the common query. Establishing such custom assistant responses can therefore condense interactions between other users and the automated assistant.

Abstract: Implementations relate to automatically translating a customized automated assistant from a first language to a new language, so that the automated assistant can interpret spoken utterances in the new language and respond to such spoken utterances in the new language. For example, a customized automated assistant can be configured for use in a first language through the developer(s) providing input(s) that are in the first language, and thereafter automatically translated to a distinct second language for which no developer input is provided. The deployment of the customized automated assistant for utilization with the second language can be selective. For example, it can be selective in that it is only automatically deployed and/or is only suggested for deployment in response to determining that one or more objective criteria, that indicate accuracy and/or robustness of the second language translation of the customized automated assistant, are satisfied.

Abstract: Spatial audio may be generated by a speaker array that is switched according to rows and/or columns to reduce its cost and complexity. The speaker array may include a row of speakers that are each coupled to a different column channel. The rows of speakers can receive portions of the spatial audio on a row-by-row basis as each row is activated to couple the speakers in a row to their respective column. This switched approach reduces a number of required audio sources. The audio sources may generate PWM signals for each column using an approach that is similar to that found in Class-D amplification or sigma-delta Modulation. Analog signals may be recovered from the PWM signals using a low-pass filter positioned before each speaker in the array.

Abstract: A method includes, while a user is wearing stereo headphones in an environment, obtaining, from a target digital assistant, a response to a query issued by the user, and obtaining spatial audio preferences of the user. Based on the spatial audio preferences of the user, the method also includes determining a spatially disposed location within a playback sound-field for the user to perceive as a sound-source of the response to the query. The method further includes rendering output audio signals characterizing the response to the query through the stereo headphones to produce the playback sound-field. Here, the user perceives the response to the query as emanating from the sound-source at the spatially disposed location within the playback sound-field.

Abstract: Implementations described herein relate to pairing a location-based automated assistant with a user device. The user device can include, for example, a headphones apparatus and/or a device that is paired with the headphones apparatus. The user device provides an indication that it is present at a location that is associated with a location-based automated assistant. A trust measure is determined that is indicative of trust between the user device and the location-based automated assistant. User information is provided by the user device to the location-based automated assistant. The location-based automated assistant determines response data to provide, via one or more speakers associated with the user device, that is specific to the location and further based on the user information.

Abstract: A method for combining hotwords in a single utterance receives, at a first assistant-enabled device (AED), audio data corresponding to an utterance directed toward the first AED and a second AED among two or more AEDs where the audio data includes a query specifying an operation to perform. The method also detects, using a hotword detector, a first hotword assigned to the first AED that is different than a second hotword assigned to the second AED. In response to detecting the first hotword, the method initiates processing on the audio data to determine that the audio data includes a term preceding the query that at least partially matches the second hotword assigned. Based on the at least partial match, the method executes a collaboration routine to cause the first AED and the second AED to collaborate with one another to fulfill the query.

Abstract: Implementations are directed to enabling a representative associated with an entity to quickly and efficiently modify a voice bot associated with the entity. The voice bot can be previously trained to communicate with user(s) on behalf of the entity through various communication channels (e.g., a telephone communication channel, a software application communication channel, a messaging communication channel, etc.). Processor(s) of a computing device can receive, from the representative, representative input to modify behavior(s) and/or parameter(s) that the voice bot utilizes in communicating with the plurality of users via the communication channels, determine whether the representative is authorized to cause the behavior(s) and/or parameter(s) to be modified, and cause the behavior(s) and/or parameter(s) to be modified in response to determining that the representative is authorized.

Abstract: A method using media arbitration includes, while a first assistant-enabled device is performing a first long-standing operation, determining the first assistant-enabled device satisfies a co-presence condition with a second assistant-enabled device, and determining that the second assistant-enabled device is performing a second long-standing operation that conflicts with the first long-standing operation performed by the first assistant-enabled device. Based on determining that the first long-standing operation and the second long-standing operation conflict, the method also includes executing an operation arbitration routine to identify one or more compromise operations for at least one of the first assistant-enabled device or the second assistant-enabled device to perform, and instructing the first assistant-enabled device or the second assistant-enabled device to perform a selected compromise operation among the identified compromise operations.