Abstract: A method includes instructing an always-on first processor to operate in a follow-on query detection mode, and while the always-on first processor operates in the follow-on query detection mode: receiving follow-on audio data captured by the assistant-enabled device; determining, using a voice activity detection (VAD) model executing on the always-on first processor, whether or not the VAD model detects voice activity in the follow-on audio data; performing, using a speaker identification (SID) model executing on the always-on first processor, speaker verification on the follow-on audio data to determine whether the follow-on audio data includes an utterance spoken by the same user. The method also includes initiating a wake-up process on a second processor to determine whether the utterance includes a follow-on query.

Abstract: Methods, systems, and apparatus for receiving a request that includes a user identifier of a user that submitted a search query and an entity identifier of an entity that is referenced by the search query, identifying a plurality of knowledge elements that are related to the entity, identifying, in a consumption database, one or more items that have been indicated as consumed by the user and that are associated with the entity that is referenced by the search query, assigning rank scores to the plurality of knowledge elements, based at least on identifying the one or more items, selecting one or more of the knowledge elements from among the knowledge elements based at least on the rank scores assigned to the knowledge elements, and providing, in response to the request, information associated with the entity and the one or more selected knowledge elements.

Abstract: A computing system and method that can be used for a mapping system that can recommend paths for navigational routing to a primary user. In particular, a primary user may be interested in navigational routes that secondary users, who may be the same or different from the primary user, have taken in the past. Even more particularly, the mapping systems described herein can provide improved user navigational services by leveraging the insight that users who have previously been in a similar situation (e.g., on vacation, in icy conditions, etc.) may wish to take the same route as they did the last time. Alternatively, users may desire to take routes based on routes which were taken by other users who are connected (e.g., via social media, address books, etc.).

Abstract: Implementations described herein relate to pairing a location-based automated assistant with a user device. The user device can include, for example, a headphones apparatus and/or a device that is paired with the headphones apparatus. The user device provides an indication that it is present at a location that is associated with a location-based automated assistant. A trust measure is determined that is indicative of trust between the user device and the location-based automated assistant. User information is provided by the user device to the location-based automated assistant. The location-based automated assistant determines response data to provide, via one or more speakers associated with the user device, that is specific to the location and further based on the user information.

Abstract: Systems and methods for generating a domain-specific conversational automated assistant. In some examples, a conversational language model is used to generate a target answer and a target action recommendation in response to each of a set of in-domain training questions. In some examples, the conversational language model is further used to generate follow-up questions to one or more of its generated target answers, and to then generate a target answer and target action recommendation to each generated follow-up question. In some examples, the processing system also generates a set of out-of-domain training examples including an out-of-domain question, a predetermined target answer, and a predetermined target action recommendation. The automated assistant may then be trained to predict the generated target answers and target action recommendations based on the associated training question or generated follow-up question, as well as any prior questions and answers in the conversation.

Abstract: Implementations relate to an automated assistant that can respond to communications received via a third party application and/or other third party communication modality. The automated assistant can determine that the user is participating in multiple different conversations via multiple different third party communication services. In some implementations, conversations can be processed to identify particular features of the conversations. When the automated assistant is invoked to provide input to a conversation, the automated assistant can compare the input to the identified conversation features in order to select the particular conversation that is most relevant to the input. In this way, the automated assistant can assist with any of multiple disparate conversations that are each occurring via a different third party application.

Abstract: Implementations set forth herein relate to a robotic computing device that can perform certain operations, such as communicating between users in a common space, according to certain preferences of the users. When interacting with a particular user, the robotic computing device can perform an operation at a preferred location relative to the particular user based on an express or implied preference of that particular user. For instance, certain types of operations can be performed at a first location within a room, and other types of operations can be performed at a second location within the room. When an operation involves following or guiding a user, parameters for driving the robotic computing device can be selected based on preferences of the user and/or a context in which the robotic computing device is interacting with the user (e.g., whether or not the context indicates some amount of urgency).

Abstract: A method includes detecting multiple users, receiving a first query issued by a first user, the first query including a command for a digital assistant to perform a first action, and enabling a round robin mode to control performance of actions commanded by queries. The method also includes, while performing the first action, receiving audio data corresponding to a second query including a command to perform a second action, performing speaker identification on the audio data, determining that the second query was spoken by the first user, preventing performing the second action, and prompting at least another user to issue a query. The method further includes receiving a third query issued by a second user, the third query including a command for the digital assistant to perform a third action, and when the digital assistant completes performing the first action, executing performance of the third action.

Abstract: A computing system receives requests from client devices to process voice queries that have been detected in local environments of the client devices. The system identifies that a value that is based on a number of requests to process voice queries received by the system during a specified time interval satisfies one or more criteria. In response, the system triggers analysis of at least some of the requests received during the specified time interval to trigger analysis of at least some received requests to determine a set of requests that each identify a common voice query. The system can generate an electronic fingerprint that indicates a distinctive model of the common voice query. The fingerprint can then be used to detect an illegitimate voice query identified in a request from a client device at a later time.

Abstract: Systems and methods are provided for a personalized entity repository. For example, a computing device comprises a personalized entity repository having fixed sets of entities from an entity repository stored at a server, a processor, and memory storing instructions that cause the computing device to identify fixed sets of entities that are relevant to a user based on context associated with the computing device, rank the fixed sets by relevancy, and update the personalized entity repository using selected sets determined based on the rank and on set usage parameters applicable to the user. In another example, a method includes generating fixed sets of entities from an entity repository, including location-based sets and topic-based sets, and providing a subset of the fixed sets to a client, the client requesting the subset based on the client's location and on items identified in content generated for display on the client.

Abstract: Techniques are described herein for warm word arbitration between automated assistant devices. A method includes: determining that warm word arbitration is to be initiated between a first assistant device and one or more additional assistant devices, including a second assistant device; broadcasting, by the first assistant device, to the one or more additional assistant devices, an active set of warm words for the first assistant device; for each of the one or more additional assistant devices, receiving, from the additional assistant device, an active set of warm words for the additional assistant device; identifying a matching warm word included in the active set of warm words for the first assistant device and included in the active set of warm words for the second assistant device; and enabling or disabling detection of the matching warm word by the first assistant device, in response to identifying the matching warm word.

Abstract: Implementations relate to an automated assistant that can automate repeatedly performed procedures. The automation can involve communicating with different users, organizations, and/or other automated assistants. The automated assistant, with prior permission from respective user(s), can detect repeated performance of a particular series of manually initiated computational actions. Based on this determination, the automated assistant can determine automated assistant computational action(s) that can be performed by the automated assistant in order to reduce latency in performing a procedure, reduce quantity and/or size of transmissions in performing the procedure, and/or reduce an amount of client device resources required for performing the procedure. Such actions can include communicating with an additional automated assistant that may be associated with another user and/or organization.

Abstract: In some implementations, a language proficiency of a user of a client device is determined by one or more computers. The one or more computers then determines a text segment for output by a text-to-speech module based on the determined language proficiency of the user. After determining the text segment for output, the one or more computers generates audio data including a synthesized utterance of the text segment. The audio data including the synthesized utterance of the text segment is then provided to the client device for output.

Abstract: A computing device may implement a method for providing route information regarding a completed or ongoing trip by a user without the user having previously initiated a navigation session. The method may include receiving a query regarding a previous or ongoing trip by a user prior to the user initiating a navigation session; determining an origin for the previous or ongoing trip; obtaining route information for the previous or ongoing trip; generating one or more route attributes associated with the query based at least on the origin for the previous or ongoing trip and the route information for the previous or ongoing trip; and providing a response to the query based at least on the one or more route attributes.

Abstract: Implementations set forth herein relate to an automated assistant that can jointly process spoken utterances that can include at least one corrective utterance and one utterance that may have been misinterpreted by the automated assistant. In this way, the automated assistant does not need to exclusively rely on the corrective utterance to determine a request of a user but, rather, can utilize data capturing any prior spoken utterance to more accurately determine the request. The automated assistant can process utterances jointly after determining that they are acoustically similar and/or received within a threshold duration of time. The joint processing can involve generating a single input, or multiple inputs, to characterize the series of utterances and processing the single input, or multiple inputs, with one or more neural network models and/or machine learning models. The automated assistant can then proactively correct any previous responses to misinterpreted utterances.

Abstract: To present a navigation directions preview, a server device receives a request for navigation directions from a starting location to a destination location and generates a set of navigation directions in response to the request. The set of navigation directions includes a set of route segments for traversing from the starting location to the destination location. The server device selects a subset of the route segments based on characteristics of each route segment in the set of route segments. For each selected route segment, the server device provides a preview of the route segment to be displayed on a client device. The preview of the route segment includes panoramic street level imagery depicting the route segment.

Abstract: Some implementations relate to performing speech biasing, NLU biasing, and/or other biasing based on historical assistant interaction(s). It can be determined, for one or more given historical interactions of a given user, whether to affect future biasing for (1) the given user account, (2) additional user account(s), and/or (3) the shared assistant device as a whole. Some implementations disclosed herein additionally and/or alternatively relate to: determining, based on utterance(s) of a given user to a shared assistant device, an association of first data and second data; storing the association as accessible to a given user account of the given user; and determining whether to store the association as also accessible by additional user account(s) and/or the shared assistant device.

Abstract: Systems and methods for steganographic embedding of metadata in media, and improved generation of synthetic media files. In some examples, a steganography encoder may be trained to encode a media file with data such that it will be more likely to be accurately decoded, and/or less likely to be perceptible to a user or other applications. In some examples, the media file may be a synthetically generated media file, and the data may be some or all of the data used to generate the synthetically generated media file. In some examples, a generative model may be trained to create synthetically generated media files that are more likely to be accurately interpreted by an interpretive model. In some examples, data encoded into a synthetically generated media file may be used to output an indication that the file was synthetically generated.

Abstract: Implementations described herein include detecting a stream of audio data that captures a spoken utterance of the user and that captures ambient noise occurring within a threshold time period of the spoken utterance being spoken by the user. Implementations further include processing a portion of the audio data that includes the ambient noise to determine ambient noise classification(s), processing a portion of the audio data that includes the spoken utterance to generate a transcription, processing both the transcription and the ambient noise classification(s) with a machine learning model to generate a user intent and parameter(s) for the user intent, and performing one or more automated assistant actions based on the user intent and using the parameter(s).

Abstract: Implementations are directed to biasing speaker authentication on a per-user basis and on a device-by-device basis and/or contextual feature(s) basis. In some of those implementations, in performing speaker authentication based on a spoken utterance, different biasing parameters are determined for each of multiple different registered users of an assistant device at which the spoken utterance was detected. In those implementations, each of the biasing parameters can be used to make it more likely or less likely (in dependence of the biasing parameter) that a corresponding registered user will be verified using the speaker authentication. Through utilization of biasing parameter(s) in performing speaker authentication, accuracy and/or robustness of speaker authentication can be increased.