Abstract: A method for a smart device management resource picker includes receiving an authorization request from a third party. The authorization request requests access to a user resource managed by the device manager. The device manager manages access controls associated with a plurality of user devises, the access controls are configured by a user. The method also includes determining whether the third party is authorized to access the user resource managed by the device manager. When the third party is authorized to access the user resource managed by the device manager, the method includes determining whether the user has configured access controls at the device manager that governs the user resource subject to the authorization request. When the user has configured a respective access control that governs the user resource subject to the authorization request, the method includes communicating a response to the authorization request based on the respective access control.

Abstract: A method includes receiving a video comprising images representing an object, and determining, using a machine learning model, based on a first image of the images, and for each respective vertex of vertices of a bounding volume for the object, first two-dimensional (2D) coordinates of the respective vertex. The method also includes tracking, from the first image to a second image of the images, a position of each respective vertex along a plane underlying the bounding volume, and determining, for each respective vertex, second 2D coordinates of the respective vertex based on the position of the respective vertex along the plane. The method further includes determining, for each respective vertex, (i) first three-dimensional (3D) coordinates of the respective vertex based on the first 2D coordinates and (ii) second 3D coordinates of the respective vertex based on the second 2D coordinates.

Abstract: A method includes receiving sensor data representing a first object in an environment and generating, based on the sensor data, a first state vector that represents physical properties of the first object. The method also includes generating, by a first machine learning model and based on the first state vector and a second state vector that represents physical properties of a second object previously observed in the environment, a metric indicating a likelihood that the first object is the same as the second object. The method further includes determining, based on the metric, to update the second state vector and updating, by a second machine learning model configured to maintain the second state vector over time and based on the first state vector, the second state vector to incorporate into the second state vector information concerning physical properties of the second object as represented in the first state vector.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, that facilitate resource and space efficient analysis of large scale datasets. Methods include obtaining activity data for objects in a dataset. For each data item in the dataset, a hashed parameter having a binary representation is generated using an identifier for the object. A register is identified from among a set of registers based on the hashed parameter. A determination is made that the hashed parameter for the object contributes to an aggregation amount that specifies a number of occurrences of the object in the dataset. Based on this determination, an aggregation amount stored in the register is updated. Based on aggregation amounts stored in the set of registers, a reporting output is generated that provides an aggregate distribution of the objects in the dataset based on the activity data for the objects.

Abstract: A display configuration to extend the light emitting and touch sensitive portion of a display panel into areas normally reserved for other light devices is disclosed. In the configuration, light devices and the display panel are configured to operate together. The display panel may include an aperture through the display panel to provide an unobstructed light path to a light device. The display panel may also include a routing area to provide space for data lines feeding pixels to be rerouted around the aperture. The routing area is partially obstructed by data lines but otherwise transparent. Accordingly, some display devices may be positioned behind the routing area and still operate.

Abstract: Techniques are described herein to improve onboarding of third party entity data with existing knowledge graphs. In various implementation s, a computing system associated with an existing knowledge graph may receive a request from a third party to onboard, with the existing knowledge graph, a plurality of entities. Each entity may have associated identifier(s) and relationship(s) with other entities of the plurality of entities. First third party entity data that describes the plurality of entities and associated identifiers/relationships may be received from the third party. The first third entity party data may be analyzed to identify semantic fingerprint(s) matching respective subsets of the entities. Results related to the analyzing may be determined. The results may include a statistic representing success or failure of applying rule(s) to a respective subset of entities that match a given semantic fingerprint. Remedial action(s) may be triggered based on the failure statistic.

Abstract: This document describes techniques and apparatuses for managing scheduling requests in an unlicensed frequency band. These techniques include a user equipment receiving first and second configuration messages allocating radio resources for first and second logical channels, respectively, at a scheduling request (SR)-transmission occasion on first and second multiple-access (MA) channels, respectively, in the unlicensed frequency band (402, 404). In aspects, the user equipment detects that the first MA channel is busy and the second MA channel is idle (406). The user equipment transmits a scheduling request via the second radio resource (408) and initiates a timer for the second logical channel to prohibit transmission of an additional scheduling request via the second radio resource until expiration of the timer (410). The user equipment also determines whether to initiate another timer for the first logical channel based on a priority level of the first logical channel (414).

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for receiving a request from a client to process a computational graph; obtaining data representing the computational graph, the computational graph comprising a plurality of nodes and directed edges, wherein each node represents a respective operation, wherein each directed edge connects a respective first node to a respective second node that represents an operation that receives, as input, an output of an operation represented by the respective first node; identifying a plurality of available devices for performing the requested operation; partitioning the computational graph into a plurality of subgraphs, each subgraph comprising one or more nodes in the computational graph; and assigning, for each subgraph, the operations represented by the one or more nodes in the subgraph to a respective available device in the plurality of available devices for operation.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training acoustic models and using the trained acoustic models. A connectionist temporal classification (CTC) acoustic model is accessed, the CTC acoustic model having been trained using a context-dependent state inventory generated from approximate phonetic alignments determined by another CTC acoustic model trained without fixed alignment targets. Audio data for a portion of an utterance is received. Input data corresponding to the received audio data is provided to the accessed CTC acoustic model. Data indicating a transcription for the utterance is generated based on output that the accessed CTC acoustic model produced in response to the input data. The data indicating the transcription is provided as output of an automated speech recognition service.

Abstract: Hubs for social interaction via electronic devices are described. In one aspect, a data processing device includes a display screen displaying a social interaction hub, the social interaction hub including a collection of records. Each record includes a counterparty identifier identifying a counterparty of a past social interaction event, a mode indicium identifying a mode by which the past social interaction event with the counterparty occurred, and a collection of mode indicia each identifying a mode by which a future, outgoing social interaction event with the counterparty can occur. The counterparty identifier, the mode indicium, and the collection of mode indicia are associated with one another in the records of the social interaction hub.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for collecting data regarding resource views. In one aspect, a method includes generating a tag that includes instructions that, upon execution of the instructions, cause a device to perform operations including identifying content located in at least one particular portion of a structured resource. The particular portion can be specified by the instructions. The operations may include generating resource view data that includes parameters identifying the device and at least a portion of the identified content. The operations may include sending the resource view data to a server. A request for the tag may be received. The tag may be provided in response to the request.

Abstract: A method for generating emotionally intelligent responses to information seeking questions includes receiving audio data corresponding to a query spoken by a user and captured by an assistant-enabled device associated with the user, and processing, using a speech recognition model, the audio data to determine a transcription of the query. The method also includes performing query interpretation on the transcription of the query to identify an emotional state of the user that spoke the query, and an action to perform. The method also includes obtaining a response preamble based on the emotional state of the user and performing the identified action to obtain information responsive to the query. The method further includes generating a response including the obtained response preamble followed by the information responsive to the query.

Abstract: A method includes receiving audio data corresponding to an utterance spoken by the user and captured by the user device. The utterance includes a command for a digital assistant to perform an operation. The method also includes determining, using a hotphrase detector configured to detect each trigger word in a set of trigger words associated with a hotphrase, whether any of the trigger words in the set of trigger words are detected in the audio data during the corresponding fixed-duration time window. The method also includes determining identifying, in the audio corresponding to the utterance, the hotphrase when each other trigger word in the set of trigger words was also detected in the audio data. The method also includes triggering an automated speech recognizer to perform speech recognition on the audio data when the hotphrase is identified in the audio data corresponding to the utterance.

Abstract: A method for optimal time-to-event (TTE) modeling includes obtaining a forecast request requesting performance of a TTE forecast forecasting an amount of time an event will occur after a starting point in time. The method includes obtaining a cutoff value representing an amount of time after the starting point in time that the event has not occurred. The method also includes forecasting, using an uncertainty forecasting model, the amount of time the event will occur after the starting point in time and updating the forecasted amount of time based on the cutoff value. The method also includes returning the updated forecasted amount of time the event will occur after the starting point in time.

Abstract: A method includes generating, using an audio encoder, a higher-order feature representation for each acoustic frame in a sequence of acoustic frames; generating, using a decoder, based on the higher-order feature representation, a plurality of speech recognition hypotheses, each hypotheses corresponding to a candidate transcription of an utterance and having an associated first likelihood score; generating, using an external language model, for each speech recognition hypothesis, a second likelihood score; determining, using a learnable fusion module, for each speech recognition hypothesis, a set of fusion weights based on the higher-order feature representation and the speech recognition hypothesis; and generating, using the learnable fusion module, for each speech recognition hypothesis, a third likelihood score based on the first likelihood score, the second likelihood score, and the set of fusion weights, the audio encoder and decoder trained using minimum additive error rate training in the presence of t

Abstract: A method for detecting freeze words includes receiving audio data that corresponds to an utterance spoken by a user and captured by a user device associated with the user. The method also includes processing, using a speech recognizer, the audio data to determine that the utterance includes a query for a digital assistant to perform an operation. The speech recognizer is configured to trigger endpointing of the utterance after a predetermined duration of non-speech in the audio data. Before the predetermined duration of non-speech, the method includes detecting a freeze word in the audio data. In response to detecting the freeze word in the audio data, the method also includes triggering a hard microphone closing event at the user device. The hard microphone closing event prevents the user device from capturing any audio subsequent to the freeze word.

Abstract: A method for speech conversion includes obtaining a speech conversion model configured to convert input utterances of human speech directly into corresponding output utterances of synthesized speech. The method further includes receiving a speech conversion request including input audio data corresponding to an utterance spoken by a target speaker associated with atypical speech and a speaker identifier uniquely identifying the target speaker. The method includes activating, using the speaker identifier, a particular sub-model for biasing the speech conversion model to recognize a type of the atypical speech associated with the target speaker identified by the speaker identifier.

Abstract: A method includes receiving a set of training utterances each including a non-synthetic speech representation of a corresponding utterance, and for each training utterance, generating a corresponding synthetic speech representation by using a voice conversion model. The non-synthetic speech representation and the synthetic speech representation form a corresponding training utterance pair. At each of a plurality of output steps for each training utterance pair, the method also includes generating, for output by a speech recognition model, a first probability distribution over possible non-synthetic speech recognition hypotheses for the non-synthetic speech representation and a second probability distribution over possible synthetic speech recognition hypotheses for the synthetic speech representation.

Abstract: A method for training hotword detection includes receiving a training input audio sequence including a sequence of input frames that define a hotword that initiates a wake-up process on a device. The method also includes feeding the training input audio sequence into an encoder and a decoder of a memorized neural network. Each of the encoder and the decoder of the memorized neural network include sequentially-stacked single value decomposition filter (SVDF) layers. The method further includes generating a logit at each of the encoder and the decoder based on the training input audio sequence. For each of the encoder and the decoder, the method includes smoothing each respective logit generated from the training input audio sequence, determining a max pooling loss from a probability distribution based on each respective logit, and optimizing the encoder and the decoder based on all max pooling losses associated with the training input audio sequence.

Abstract: A method for training a generalized automatic speech recognition model for joint acoustic echo cancellation, speech enhancement, and voice separation includes receiving a plurality of training utterances paired with corresponding training contextual signals. The training contextual signals include a training contextual noise signal including noise prior to the corresponding training utterance, a training reference audio signal, and a training speaker vector including voice characteristics of a target speaker that spoke the corresponding training utterance. The operations also include training, using a contextual signal dropout strategy, a contextual frontend processing model on the training utterances to learn how to predict enhanced speech features. Here, the contextual signal dropout strategy uses a predetermined probability to drop out each of the training contextual signals during training of the contextual frontend processing model.