Abstract: The technology provides a rich attention mechanism for structured information extraction of web pages and other electronic documents. An input layer of a model obtains system, information associated with the document, including field tokens representing respective fields to be extracted from the document, structured document type tokens associated, and text tokens from a text sequence in the document. An encoder connects the field tokens, the S type tokens and the text tokens according to a set of different attention patterns. The encoder generates an overall token representation based on the set of different attention patterns. An output layer of the model extracts a final text span for the each of the respective fields from the set of text tokens. The extracted final text span for each of the respective fields is stored in memory, and can be produced in response to a search query, analytics evaluation or other request.

Abstract: Implementations relate to generating a proficiency measure, and utilizing the proficiency measure to adapt one or more automated assistant functionalities. The generated proficiency measure is for a particular class of automated assistant actions, and is specific to an assistant device and/or is specific to a particular user. A generated proficiency measure for a class can reflect a degree of proficiency, of a user and/or of an assistant device, for that class. Various automated assistant functionalities can be adapted, for a particular class, responsive to determining the proficiency measure satisfies a threshold, or fails to satisfy the threshold (or an alternate threshold). The adaptation(s) can make automated assistant processing more efficient and/or improve (e.g., shorten the duration of) user-assistant interaction(s).

Abstract: Methods, systems, and media for selectively presenting broadcast content based on user interests are provided. In some implementations a method for selectively presenting broadcast content is provided, the method comprising: receiving user information; associating one or more athletes, each on a roster of a team in a sports organization, with the user based on the user information; identifying broadcast programs that a user device can present; determining broadcast programs that depict a game between teams in the sports organization that are relevant to an athlete associated with the user based on program metadata; receiving event metadata for the relevant broadcast programs that is indicative of events in the game depicted therein; determining that a portion of the first broadcast program is relevant to a first entity based on the event metadata; and causing the user device to present the portion of the first broadcast.

Abstract: Systems, devices, and methods are described for mitigating or eliminating distortion patterns (such as those caused by one or more high-volume audio sources) in a display system such as a laser projection system. Frequency components of an incoming sound that correspond to one or more resonant frequencies of an optical reflector of the display system are determined to exceed a defined volume threshold. Responsive to that determination, a magnitude and phase of one or more harmonic motions of the optical reflector are measured. Sound waves are generated to destructively interfere with at least one frequency component corresponding to the resonant frequencies of the optical reflector.

Abstract: Features of various implementations are used to control operation of a second electronic device from a first electronic device using indirect control methods and optical character recognition. In some implementations, communication between a primary set-top box and a supplemental set-top box is configured in order to avoid the need for one or more provider-specific APIs. For example, one aspect of the disclosure is a method of identifying commands that can be used to control one or more features of a primary set-top box (e.g., DVR or VOD features) by a supplemental set-top box issuing commands to the primary set-top box via a wireless (e.g., IR) transmitter, and then using optical character recognition to identify the outcomes of those commands as displayed on a primary TV display or secondary display.

Abstract: A method, device, and computer-readable storage medium for predicting pronunciation of a text sample. The method includes selecting a predicted text sample corresponding to an audio sample, receiving a correction text sample corresponding to the audio sample, updating an encoding of allowable pronunciations of the correction text sample based on the predicted text sample and the audio sample, the updated encoding of allowable pronunciations of the correction text sample including a pronunciation of the predicted text sample, and predicting a pronunciation of the correction text sample based on the updated encoding of allowable pronunciations of the correction text sample.

Abstract: Systems and methods for the meta-estimation of data structures representing identifiers are disclosed. The system maintain one or more data records comprising one or more identifiers and one or more attributes. Using the data records, the system can generate a first data structure, such as a probabilistic data structure, that represents the plurality of data records. The first data structure can have a plurality of registers. The system can identify a subset of the plurality of registers that are equal to a predetermined value, and generate a second data structure that represents the subset of the plurality of registers. The system can then store the second data structure as a meta-estimation of the first, and can utilize the second data structure in further processing operations.

Abstract: Implementations relate to an automated assistant that provides augmented reality content, via a display interface of computerized glasses, resulting from post-processing of application content. The application content can be identified based on prior interactions between a user and one or more applications, and the application content can be processed to determine objects, and/or object classifications, that may be associated with the application content. When the user is wearing the computerized glasses, and the object is detected within a field of view of the computerized glasses, the automated assistant can cause certain content to be rendered at the display interface of the computerized glasses. In some implementations, the content can be generated to supplement, and/or be different from, existing content that the user may have already accessed, in furtherance of preventing duplicative usage of applications and/or preserving computational resources.

Abstract: Systems and methods for textual replacement can include the determination of a visual intent, which can trigger an interface for selecting an image to replace visual descriptors. The visually descriptive terms can be identified, and an indicator can be provided to indicate the text replacement option may be initiated. An image can then be selected by a user to replace the visually descriptive terms.

Abstract: Entropy coding a sequence of symbols is described. A first probability model for entropy coding is selected. At least one symbol of the sequence is coded using a probability determined using the first probability model. The probability according to the first probability model is updated with an estimation of a second probability model to entropy code a subsequent symbol. The combination may be a fixed or adaptive combination.

Abstract: Implementations can detect respective audio data that captures an acoustic event at multiple assistant devices in an ecosystem that includes a plurality of assistant devices, process the respective audio data locally at each of the multiple assistant devices to generate respective measures that are associated with the acoustic event using respective event detection models, process the respective measures to determine whether the detected acoustic event is an actual acoustic event, and cause an action associated with the actional acoustic event to be performed in response to determining that the detected acoustic event is the actual acoustic event. In some implementations, the multiple assistant devices that detected the respective audio data are anticipated to detect the respective audio data that captures the actual acoustic event based on a plurality of historical acoustic events being detected at each of the multiple assistant devices.

Abstract: The technology employs a patch-based multi-scale Transformer (300) that is usable with various imaging applications. This avoids constraints on image fixed input size and predicts the quality effectively on a native resolution image. A native resolution image (304) is transformed into a multi-scale representation (302), enabling the Transformer's self-attention mechanism to capture information on both fine-grained detailed patches and coarse-grained global patches. Spatial embedding (316) is employed to map patch positions to a fixed grid, in which patch locations at each scale are hashed to the same grid. A separate scale embedding (318) is employed to distinguish patches coming from different scales in the multiscale representation. Self-attention (508) is performed to create a final image representation. In some instances, prior to performing self-attention, the system may prepend a learnable classification token (322) to the set of input tokens.

Abstract: The present disclosure provides systems and methods for communication efficient distributed mean estimation. In particular, aspects of the present disclosure can be implemented by a system in which a number of vectors reside on a number of different clients, and a centralized server device seeks to estimate the mean of such vectors. According to one aspect of the present disclosure, a client computing device can rotate a vector by a random rotation matrix and then subsequently perform probabilistic quantization on the rotated vector. According to another aspect of the present disclosure, subsequent to quantization but prior to transmission, the client computing can encode the quantized vector according to a variable length coding scheme (e.g., by computing variable length codes).

Abstract: This specification is generally directed to techniques for generating interfacing source code between computing components based on natural language input. In various implementations, a natural language input that requests generation of interfacing source code to logically couple a first computing component with a second computing component may be processed to generate an interface request semantic embedding. The interface request semantic embedding may be processed based on one or more domain models associated with the first and second computing components to generate a pool(s) of candidate code snippets for logically coupling with first and second computing components. A plurality of candidate instances of interfacing source code may be generated between the first and second computing components. Each candidate software interface may include a different permutation of candidate code snippets from the pool(s) of candidate code snippets.

Abstract: A deep state space generative model is augmented with intervention prediction. The state space model provides a principled way to capture the interactions among observations, interventions, critical event occurrences, true states, and associated uncertainty. The state space model can include a discrete-time hazard rate model that provides flexible fitting of general survival time distributions. The state space model can output a joint prediction of event risk, observation and intervention trajectories based on patterns in temporal progressions, and correlations between past measurements and interventions.

Abstract: Methods and apparatus for providing query suggestions to a user based on one or more past queries submitted by the user. Candidate query suggestions responsive to a current query may be identified. A candidate query similarity measure may be determined for a given candidate query suggestion based on matching entities related to the given candidate query suggestion and the one or more past queries. In some implementations, the similarity measure of the given candidate query suggestion may be based on a comparison of current entities of the given candidate query suggestion that match entities of one or more past queries, to a group of the current entities that includes entities that do not match the entities of one or more past queries. In some implementations a ranking of the candidate query suggestions may be determined based on the similarity measure.

Abstract: Techniques are described herein for detecting and/or enrolling (or commissioning) new “hot commands” that are useable to cause an automated assistant to perform responsive action(s) without having to be first explicitly invoked. In various implementations, an automated assistant may be transitioned from a limited listening state into a full speech recognition state in response to a trigger event. While in the full speech recognition state, the automated assistant may receive and perform speech recognition processing on a spoken command from a user to generate a textual command. The textual command may be determined to satisfy a frequency threshold in a corpus of textual commands. Consequently, data indicative of the textual command may be enrolled as a hot command. Subsequent utterance of another textual command that is semantically consistent with the textual command may trigger performance of a responsive action by the automated assistant, without requiring explicit invocation.