Abstract: Implementations relate to encrypting audio data utilizing utterance features generated from the audio data. Some of those implementations include generating utterance features from a portion of the audio data, encrypting at least part of the audio data using the utterance features, and providing the encrypted audio data to one or more applications for decryption. Speaker features previously generated from utterances are utilized to decrypt the audio data for further processing. Other implementations relate to receiving speaker features and comparing the speaker features to utterance features generated from audio data. The audio data is provided to target applications that provide speaker features that match the utterance features generated from the audio data.

Abstract: Methods, systems, and computer readable media related to generating a combined search query based on search parameters of a current search query of a user and search parameters of one or more previously submitted search quer(ies) of the user that are determined to be of the same line of inquiry as the current search query. Two or more search queries may be determined to share a line of inquiry when it is determined that they are within a threshold level of semantic similarity to one another. Once a shared line of inquiry has been identified and a combined search query generated, users may interact with the search parameters and/or the search results to update the search parameters of the combined search query.

Abstract: Implementations are directed to receiving a search query from a user, obtaining environmental signal(s) associated with an environment in which the user is located when the search query is received, processing the environmental signal(s) to generate a privacy measure associated with submission of the search query, obtaining additional environmental signal(s) associated with the environment in which the user is located when user input directed to a search interface is received, processing the additional environmental signal(s) to generate an additional privacy measure associated with the user input, selecting, from a superset of historical search queries of the user, a subset of the historical search queries based on at least the privacy measure and the additional privacy measure, and causing the subset of the historical search queries to be presented to the user in response to receiving the user input directed to the search interface.

Abstract: Computer-implemented techniques can include obtaining, by a client computing device, a digital media item and a request for a processing task on the digital item and determining a set of operating parameters based on (i) available computing resources at the client computing device and (ii) a condition of a network. Based on the set of operating parameters, the client computing device or a server computing device can select one of a plurality of artificial neural networks (ANNs), each ANN defining which portions of the processing task are to be performed by the client and server computing devices. The client and server computing devices can coordinate processing of the processing task according to the selected ANN. The client computing device can also obtain final processing results corresponding to a final evaluation of the processing task and generate an output based on the final processing results.

Abstract: Implementations set forth herein relate to an automated assistant that can bias speech processing towards certain requests according to whether those requests are relevant to content that is being rendered, or is expected to be rendered, at a computing device. In this way, speech processing can be dynamically biased according to features of content that may be rendered by a particular application and/or a particular device. Biasing can be performed during rendering of a portion of content determined to be relevant to a particular request by adjusting a score threshold that is used for determining whether a particular request was received. When the portion of content is no longer being rendered, the threshold can return to a particular value, or be adjusted again according to a subsequent portion of the content.

Abstract: A method using media arbitration includes, while a first assistant-enabled device is performing a first long-standing operation, determining the first assistant-enabled device satisfies a co-presence condition with a second assistant-enabled device, and determining that the second assistant-enabled device is performing a second long-standing operation that conflicts with the first long-standing operation performed by the first assistant-enabled device. Based on determining that the first long-standing operation and the second long-standing operation conflict, the method also includes executing an operation arbitration routine to identify one or more compromise operations for at least one of the first assistant-enabled device or the second assistant-enabled device to perform, and instructing the first assistant-enabled device or the second assistant-enabled device to perform a selected compromise operation among the identified compromise operations.

Abstract: A method for adapting hotword recognition includes receiving audio data characterizing a hotword event detected by a first stage hotword detector in streaming audio captured by a user device. The method also includes processing, using a second stage hotword detector, the audio data to determine whether a hotword is detected by the second stage hotword detector in a first segment of the audio data. When the hotword is not detected by the second stage hotword detector, the method includes, classifying the first segment of the audio data as containing a negative hotword that caused a false detection of the hotword event in the streaming audio by the first stage hotword detector. Based on the first segment of the audio data classified as containing the negative hotword, the method includes updating the first stage hotword detector to prevent triggering the hotword event in subsequent audio data that contains the negative hotword.

Abstract: Techniques are described herein for identifying a failed hotword attempt. A method includes: receiving first audio data; processing the first audio data to generate a first predicted output; determining that the first predicted output satisfies a secondary threshold but does not satisfy a primary threshold; receiving second audio data; processing the second audio data to generate a second predicted output; determining that the second predicted output satisfies the secondary threshold but does not satisfy the primary threshold; in response to the first predicted output and the second predicted output satisfying the secondary threshold but not satisfying the primary threshold, and in response to the first spoken utterance and the second spoken utterance satisfying one or more temporal criteria relative to one another, identifying a failed hotword attempt; and in response to identifying the failed hotword attempt, providing a hint that is responsive to the failed hotword attempt.

Abstract: Implementations are directed to dynamically adapting which assistant on-device model(s) are locally stored at assistant devices of an assistant device group and/or dynamically adapting the assistant processing role(s) of the assistant device(s) of the assistant device group. In some of those implementations, the corresponding on-device model(s) and/or corresponding processing role(s), for each of the assistant devices of the group, is determined based on collectively considering individual processing capabilities of the assistant devices of the group. Implementations are additionally or alternatively directed to cooperatively utilizing assistant devices of a group, and their associated post-adaptation on-device model(s) and/or post-adaptation processing role(s), in cooperatively processing assistant requests that are directed to any one of the assistant devices of the group.

Abstract: Systems and methods are provided for a personalized entity repository. For example, a computing device comprises a personalized entity repository having fixed sets of entities from an entity repository stored at a server, a processor, and memory storing instructions that cause the computing device to identify fixed sets of entities that are relevant to a user based on context associated with the computing device, rank the fixed sets by relevancy, and update the personalized entity repository using selected sets determined based on the rank and on set usage parameters applicable to the user. In another example, a method includes generating fixed sets of entities from an entity repository, including location-based sets and topic-based sets, and providing a subset of the fixed sets to a client, the client requesting the subset based on the client's location and on items identified in content generated for display on the client.

Abstract: Implementations can receive audio data corresponding to a spoken utterance of a user, process the audio data to generate a plurality of speech hypotheses, determine an action to be performed by an automated assistant based on the speech hypotheses, and cause the computing device to render an indication of the action. In response to the computing device rendering the indication, implementations can receive additional audio data corresponding to an additional spoken utterance of the user, process the additional audio data to determine that a portion of the spoken utterance is similar to an additional portion of the additional spoken utterance, supplant the action with an alternate action, and cause the automated assistant to initiate performance of the alternate action. Some implementations can determine whether to render the indication of the action based on a confidence level associated with the action.

Abstract: Implementations are directed to biasing speaker authentication on a per-user basis and on a device-by-device basis and/or contextual feature(s) basis. In some of those implementations, in performing speaker authentication based on a spoken utterance, different biasing parameters are determined for each of multiple different registered users of an assistant device at which the spoken utterance was detected. In those implementations, each of the biasing parameters can be used to make it more likely or less likely (in dependence of the biasing parameter) that a corresponding registered user will be verified using the speaker authentication. Through utilization of biasing parameter(s) in performing speaker authentication, accuracy and/or robustness of speaker authentication can be increased.

Abstract: Implementations relate to an automated assistant that can respond to communications received via a third party application and/or other third party communication modality. The automated assistant can determine that the user is participating in multiple different conversations via multiple different third party communication services. In some implementations, conversations can be processed to identify particular features of the conversations. When the automated assistant is invoked to provide input to a conversation, the automated assistant can compare the input to the identified conversation features in order to select the particular conversation that is most relevant to the input. In this way, the automated assistant can assist with any of multiple disparate conversations that are each occurring via a different third party application.

Abstract: Methods, systems, and apparatus for receiving a request that includes a user identifier of a user that submitted a search query and an entity identifier of an entity that is referenced by the search query, identifying a plurality of knowledge elements that are related to the entity, identifying, in a consumption database, one or more items that have been indicated as consumed by the user and that are associated with the entity that is referenced by the search query, assigning rank scores to the plurality of knowledge elements, based at least on identifying the one or more items, selecting one or more of the knowledge elements from among the knowledge elements based at least on the rank scores assigned to the knowledge elements, and providing, in response to the request, information associated with the entity and the one or more selected knowledge elements.

Abstract: Some implementations relate to performing speech biasing, NLU biasing, and/or other biasing based on historical assistant interaction(s). It can be determined, for one or more given historical interactions of a given user, whether to affect future biasing for (1) the given user account, (2) additional user account(s), and/or (3) the shared assistant device as a whole. Some implementations disclosed herein additionally and/or alternatively relate to: determining, based on utterance(s) of a given user to a shared assistant device, an association of first data and second data; storing the association as accessible to a given user account of the given user; and determining whether to store the association as also accessible by additional user account(s) and/or the shared assistant device.

Abstract: To identify driving event sounds during navigation, a client device in a vehicle provides a set of navigation directions for traversing from a starting location to a destination location along a route. During navigation to the destination location, the client device identifies audio that includes a driving event sound from within the vehicle or an area surrounding the vehicle. In response to determining that the audio includes the driving event sound, the client device determines whether the driving event sound is artificial. In response to determining that the driving event sound is artificial, the client device presents a notification to the driver indicating that the driving event sound is artificial or masks the driving event sound to prevent the driver from hearing the driving event sound.

Abstract: A computing device may receive inputted text and perform, using one or more neural networks, on-device grammar checking of a sequence of words in the inputted text, including determining, using the one or more neural networks, a grammatically correct version of the sequence of words and determining that the sequence of words does not match the grammatically correct version of the sequence of words. The computing device may, in response to determining that the sequence of words does not match the grammatically correct version of the sequence of words, output, for display at a display device, at least a portion of the grammatically correct version of the sequence of words as a suggested replacement for at least a sequence of the sequence of words in the inputted text.

Abstract: A method of training a speech model includes receiving, at a voice-enabled device, a fixed set of training utterances where each training utterance in the fixed set of training utterances includes a transcription paired with a speech representation of the corresponding training utterance. The method also includes sampling noisy audio data from an environment of the voice-enabled device. For each training utterance in the fixed set of training utterances, the method further includes augmenting, using the noisy audio data sampled from the environment of the voice-enabled device, the speech representation of the corresponding training utterance to generate noisy audio samples and pairing each of the noisy audio samples with the corresponding transcription of the corresponding training utterance. The method additionally includes training a speech model on the noisy audio samples generated for each speech representation in the fixed set of training utterances.

Abstract: Methods, systems, and apparatus for receiving a query image, receiving one or more entities that are associated with the query image, identifying, for one or more of the entities, one or more candidate search queries that are pre-associated with the one or more entities, generating a respective relevance score for each of the candidate search queries, selecting, as a representative search query for the query image, a particular candidate search query based at least on the generated respective relevance scores and providing the representative search query for output in response to receiving the query image.

Abstract: Techniques are described herein for cross-device data synchronization based on simultaneous hotword triggers.