Abstract: Implementations relate to dynamically, and in a context-sensitive manner, biasing voice to text conversion. In some implementations, the biasing of voice to text conversions is performed by a voice to text engine of a local agent, and the biasing is based at least in part on content provided to the local agent by a third-party (3P) agent that is in network communication with the local agent. In some of those implementations, the content includes contextual parameters that are provided by the 3P agent in combination with responsive content generated by the 3P agent during a dialog that: is between the 3P agent, and a user of a voice-enabled electronic device; and is facilitated by the local agent. The contextual parameters indicate potential feature(s) of further voice input that is to be provided in response to the responsive content generated by the 3P agent.

Abstract: Disclosed herein is methods and systems for providing different views to a viewer. One particular embodiment includes a method including providing, to a neural network, a plurality of 2D images of a 3D object. The neural network may include a signed distance function based sinusoidal representation network. The method may further include obtaining a neural model of a shape of the object by obtaining a zero-level set of the signed distance function; and modeling an appearance of the object using a spatially varying emission function. In some embodiments, the neural model may be converted into a triangular mesh representing the object which may be used to render multiple view-dependent images representative of the 3D object.

Abstract: Generating a location-specific three-dimensional model in response to a location query can provide users with a better understanding of a location through providing better interactivity, better perspective, and better understanding of dimensionality. Generation of the models can be enabled by leveraging a three-dimensional asset database and segmentation methods. The location-specific models can provide further utility by further including situation specific simulated effects, such as simulated weather or traffic.

Abstract: An electronic device places an augmented reality object in an image of a real environment based on a pose of the electronic device and based on image segmentation. The electronic device includes a camera that captures images of the real environment and sensors, such as an inertial measurement unit (IMU), that capture a pose of the electronic device. The electronic device selects an augmented reality (AR) object from a memory, segments a captured image of the real environment into foreground pixels and background pixels, and composites an image for display wherein the AR object is placed between the foreground pixels and the background pixels. As the pose of the electronic device changes, the electronic device maintains the relative position of the AR object with respect to the real environment in images for display.

Abstract: A set of user interface tools is described facilitating asynchronous adjudication of one or more regions-of-interest in a medical image by a group of two or more graders, each of which has access to the set of tools in a workstation environment. The set of tools include (1) a feature for enabling the graders to assess the medical image and manually delineate one or more specific regions-of-interest (ROI) in the medical image, (2) a feature for assessing the ROI(s) delineated by other graders, including display of the ROI delineated by other graders; (3) dialog features for explaining and discussing the assessments, including a text feature for discussing the assessments. The dialog features and the ROIs delineated by all the graders are visible to all the graders on the workstation display as they collectively adjudicate the medical image in a round-robin manner. The set of tools further include (4) a feature for manually verifying grader agreement with the other graders' assessments.

Abstract: This document describes a multi-function module for mmWave communication and user input using mechanical switches in an electronic device. The multi-function module may be located behind a non-conductive button (e.g., volume button, power button) of the electronic device. Further, the multi-function module includes an elongated antenna substrate with mechanical switches mounted thereon and distributed along a longitudinal length of the antenna substrate. At least one of the mechanical switches is implemented as a radiating element for a mmWave patch antenna. The multi-function module also includes one or more integrated circuit components mounted to the antenna substrate and configured to use the at least one mechanical switch as the radiating element for the mmWave patch antenna to provide the mmWave communication. In this way, the multi-function module enables the mechanical switches to coexist with the mmWave patch antennas.

Abstract: Systems and methods of invoking functions of agents via digital assistant applications are provided. Each action-inventory can have an address template for an action by an agent. The address template can include a portion having an input variable used to execute the action. A data processing system can parse an input audio signal from a client device to identify a request and a parameter to be executed by the agent. The data processing system can select an action-inventory for the action corresponding to the request. The data processing system can generate, using the address template, an address. The address can include a substring having the parameter used to control execution of the action. The data processing system can direct an action data structure including the address to the agent to cause the agent to execute the action and to provide output for presentation.

Abstract: This document describes techniques and apparatuses directed at detecting and preventing light-based injection attacks. In aspects, a computing device includes executable instructions of an input manager, an audio sensor having subtracting circuitry, and a light sensor. One or more processors executing instructions of the input manager is configured to receive and analyze signals generated by the audio sensor, the light sensor, and the subtracting circuit. Upon analysis, the input manager can detect and prevent light-based injection attacks.

Abstract: Methods and systems for a content server to select sets of video streams having different encoding parameters for transmitting the sets of video streams to a media device are disclosed herein. In some embodiments, a method for transmitting video streams for a media program from a server to a media device includes: selecting, by the server, first encoding parameters including a first bitrate for a first set of video streams for the media program based on a first estimated bandwidth capacity for a network linking the server and the media device, transmitting the first set of video streams from the server to the media device, determining, by the server, second encoding parameters including a second bitrate for a second set of video streams for the media program, and transmitting the second set of video streams from the server to the media device.

Abstract: A firewall intelligence system, includes a data storage storing a set of firewall rules for a network; a recommendation engine that receives, from a log service, traffic logs detailing traffic for the network and firewall logs detailing the usage of firewall rules in response to the traffic for the network, accesses, from the data storage, the set of firewall rules for the network; processes the set of firewall rules to evaluate the firewall rules against a set of quantitative evaluation rules to determine one or more firewall rule recommendations, wherein each firewall rule recommendation is a recommendation to change at least one of the firewall rules in the set of firewall rules; and a front end API that provides data describing the one or more firewall rule recommendations to a user device.

Abstract: A device for restoring a degraded frame resulting from reconstruction of a source frame includes a processor that is configured to receive a compressed bitstream. The compressed bitstream includes a first projection parameter ? a second projection parameter ?, first restoration parameters comprising a first radius value, and second restoration parameters comprising a second radius value. The processor is further configured to restore at least a portion of the degraded frame using a projection operation that uses the first projection parameter ?, the second projection parameter ?, and at least two guide tiles.

Abstract: Mapping-aware coding tools for 360 degree videos adapt conventional video coding tools for 360 degree video data using parameters related to a spherical projection of the 360 degree video data. The mapping-aware coding tools perform motion vector mapping techniques, adaptive motion search pattern techniques, adaptive interpolation filter selection techniques, and adaptive block partitioning techniques. Motion vector mapping includes calculating a motion vector for a pixel of a current block by mapping the location of the pixel within a two-dimensional plane (e.g., video frame) onto a sphere and mapping a predicted location of the pixel on the sphere determined based on rotation parameters back onto the plane. Adaptive motion searching, adaptive interpolation filter selection, and adaptive block partitioning operate according to density distortion based on locations along the sphere.

Abstract: Secure communication in mobile digital pages is provided. The system receives an electronic document and validates the electronic document for storage in a cache server. The system receives a request for the electronic document and provides it to a viewer component on a client computing device. The viewer component loads the electronic document in an iframe. The viewer component executes a runtime component to receive, via a secure communication channel, a tag from the electronic document. The system receives the tag and selects a data value for transmission to the viewer component. The viewer components provides the data value to cause the runtime component to execute an action with the data value.

Abstract: The various implementations described herein include a video camera assembly that includes: (1) a housing; (2) an image sensor positioned within the housing and having a field of view corresponding to a scene in the smart home environment; and (3) a concave-shaped front face positioned in front of the image sensor such that light from the scene passes through the front face prior to entering the image sensor; where the front face includes: (a) an inner section corresponding to the image sensor; and (b) an outer section between the housing and the inner section, the outer section having a concave shape that extends from an outer periphery of the outer section to an inner periphery of the outer section; and where the concave shape extends around an entirety of the outer periphery.

Abstract: A method for auralizing a multi-microphone device. Path information for one or more sound paths using dimensions and room reflection coefficients of a simulated room for one of a plurality of microphones included in a multi-microphone device is determined. An array-related transfer functions (ARTFs) for the one of the plurality of microphones is retrieved. The auralized impulse response for the one of the plurality of microphones is generated based at least on the retrieved ARTFs and the determined path information.

Abstract: A computer-implemented method for providing landmark-assisted navigation guidance by selectively utilizing database information includes receiving navigation requests from one or more mobile computing devices, each of the requests being associated with a respective starting point, destination, and travel mode. For each navigation request, a corresponding route to be depicted in a corresponding digital map is determined. For each navigation request associated with a first travel mode, corresponding points of interest (POIs) are determined from among a plurality of POIs stored in a database. The corresponding POIs are determined based at least on whether the corresponding POIs are associated with any landmark category. For each navigation request associated with a second travel mode, corresponding POIs are determined irrespective of whether the corresponding POIs are associated with any landmark categories.

Abstract: A system comprises data processing hardware and memory hardware. The memory hardware is in communication with the data processing hardware, and stores instructions that, when executed on the data processing hardware, cause the data processing hardware to perform a plurality of operations. In some examples, one of the operations may include receiving instance management configuration data for a single-tenant software-as-a-service (SaaS) application. Another operation may include further include receiving an image of the single-tenant SaaS application. Yet another operation can include generating, by the control plane manager, a control plane based on the instance management configuration data. The control plane is configured to create multiple instances of the single-tenant SaaS application based on the received image, and to manage the instances of the single-tenant SaaS application based on the received instance management configuration data.

Abstract: An example computer-implemented method includes receiving, from one or more vision components in an environment, vision data that captures features of the environment, including object features of an object that is located in the environment, and prior to a robot manipulating the object: (i) determining based on the vision data, at least one first adjustment to a programmed trajectory of movement of the robot operating in the environment to perform a task of transporting the object, and (ii) determining based on the object features of the object, at least one second adjustment to the programmed trajectory of movement of the robot operating in the environment to perform the task, and causing the robot to perform the task, in accordance with the at least one first adjustment and the at least one second adjustment to the programmed trajectory of movement of the robot.

Abstract: A method for accessing customer data includes receiving an access request requesting access to customer data stored on a storage abstraction. The access request includes a justification that specifies a purpose/reason for requesting access to the customer data. The method also includes validating the justification, and after validating the justification, transmitting the justification to an external key management service associated with a customer of the customer data. The external key management service is configured to grant or deny access to the customer data based on the justification. The method also includes receiving an approved access token from the external key management service when the external key management service grants access to the customer data and accessing the customer data stored on the storage abstraction using the approved access token received from the external key management service.

Abstract: Systems, devices, and methods for eyebox expansion in wearable heads-up displays (“WHUDs”) are described. The WHUDs described herein each include a projector and an optical waveguide positioned in an optical path between the projector and an eye of the user. For any given light signal from the projector, the optical waveguide receives the light signal at an input coupler and outputs multiple instances or copies of the light signal from multiple discrete, spatially-separated output couplers. The multiple instances or copies of the light signal may be converged by the optical waveguide directly to respective exit pupils at the user's eye or may be routed by the optical waveguide to a holographic combiner in the user's field of view from which the light signals may be converged to respective exit pupils at the user's eye. The optical waveguide employs exit pupil replication to expand the eyebox of the WHUD.

Abstract: A method includes receiving a series of writes that each requests respective data to be stored at a database. For each respective write, the method includes storing the respective data in a first buffer associated with a first data streaming application and a second buffer associated with a second data streaming application. The method includes transmitting the respective data to the database. The method includes receiving, from the database, confirmation that the respective data of the respective write has been committed to the database. The confirmation includes an order indicator indicating an order the respective data of the respective write was committed to the database relative to other writes in the series of writes. In response to receiving the confirmation, the method includes sending the respective data from the first buffer to the first data streaming application and from the second buffer to the second data streaming application.

Abstract: A method of providing a workspace graphical user interface (GUI) for a user of a cloud-based content management platform includes providing the workspace GUI for the user via the cloud-based content management platform. The workspace GUI presents visual representations of documents stored on the user's cloud storage of the cloud-based content management platform and visual representations of workspaces created by the user. Each workspace includes a set of documents previously added to a respective workspace by the user and stored on the user's cloud storage. The method further includes receiving, via the workspace GUI, a user input with respect to a corresponding document stored on the user's cloud storage. The user input indicates a request to add the corresponding document to a particular workspace.

Abstract: Interface and mode selection for digital action execution is provided. For example, a system loads a script library embedded in an electronic resource. The system determines a historic level of engagement between a client computing device and one or more digital assistants. The system selects, based on a first property of the client computing device and the historic level of engagement, a type of digital interface. The system generates, based on the type of digital interface, a digital interface with the call-to-action. The system determines, responsive to an instruction to execute the call-to-action, a mode of execution. The system selects a digital assistant and a second client device to execute the call-to-action. The system transmits the call-to-action to the second client device for execution.

Abstract: Image editing on a wearable device includes a system which obtains sensor data via the wearable device. The sensor data includes a representation of hand movement, head movement or voice command associated with a user. The system executes an application for editing an image based on the obtained sensor data. The system provides for display a list of image adjustment types associated with the application. The system selects an image adjustment type based on one or more of the hand movement, the head movement or the voice command. The system provides for display a prompt having options to adjust a property of the selected image adjustment type. The system selects one of the options included in the prompt. The system modifies an image based on the selected option. The system then provides the modified image for storage in a data structure of a memory unit in the wearable device.

Abstract: Embodiments provided herein relate to enforcing a device restriction policy. A device restriction policy may be stored that maps one or more portions of a household with particular household occupants of a plurality of household occupants. A request may be received to activate the device restriction policy on a household occupant. The device restriction policy may be activated against the household occupant based on the received request. One or more electronic devices may be disabled that are located in a portion of the household linked with the household occupant based on the received request and the device restriction policy.

Abstract: A computer-implemented method that includes monitoring execution of program code by first and second processor components. A computing system detects that a trigger condition is satisfied by: i) identifying an operand in a portion of the program code; or ii) determining that a current time of a clock of the computing system indicates a predefined time value. The operand and the predefined time value are used to initiate trace events. When the trigger condition is satisfied the system initiates trace events that generate trace data identifying respective hardware events occurring across the computing system. The system uses the trace data to generate a correlated set of trace data. The correlated trace data indicates a time ordered sequence of the respective hardware events. The system uses the correlated set of trace data to analyze performance of the executing program code.

Abstract: Interconnections for connecting flex circuit boards in classical and/or quantum computing systems can include a first flex circuit board having a removed portion that exposes one or more signal lines and a second flex circuit board having a removed portion that exposes one or more other signal lines. The flex circuit boards can be aligned at the removed portions to form a signal trace gap near the exposed signal lines. Exposed signal lines of the first flex circuit board can be coupled with exposed signal lines of the second flex circuit board. A ground support layer can be coupled to the first flex circuit board and the second flex circuit board along the same side. An isolation plate at least partially covering the signal trace gap can be coupled to the first flex circuit board and/or the second flex circuit board on a side opposite of the ground support layer.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for indicating location status. A computing device can receive a query from a user device, a current time, and a location for the user device. The computing device identifies results responsive to the query, including one or more business results that are each associated with a business location and operating hours. The computing device can select a subset of the business results as open results based on the operating hours of the business results, the current time, and travel times from the device location to the respective business locations. Data can be provided for a search engine results page that designates the subset of the business results as open results.

Abstract: Systems and methods of the present disclosure enable for a delayed, two-factor authentication to occur in networked devices. The system and methods can enable the immediate delivery of digital components, which results in fewer abandoned requests, and saves network resources. The system and methods can enable the authorization of data transmissions in networked computer devices that include limited user interfaces, such as voice-based interfaces.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for rollback resistant security are disclosed. In one aspect, a method, during a boot process of a computing device, includes the actions of obtaining a secret key derived from device-specific information for the computing device. The method further includes verifying that a signature for a software module is valid. The method further includes obtaining information indicating a current version of the software module. The method further includes using the secret key to generate a first encryption key corresponding to the current version of the software module and a second encryption key corresponding to a prior version of the software module. The method further includes preventing future access to the secret key until the computing device is rebooted. The method further includes providing the software module access to the first encryption key and the second encryption key.

Abstract: Aspects of the present disclosure enable humanly-specified relationships to contribute to a mapping that enables compression of the output structure of a machine-learned model. An exponential model such as a maximum entropy model can leverage a machine-learned embedding and the mapping to produce a classification output. In such fashion, the feature discovery capabilities of machine-learned models (e.g., deep networks) can be synergistically combined with relationships developed based on human understanding of the structural nature of the problem to be solved, thereby enabling compression of model output structures without significant loss of accuracy. These compressed models provide improved applicability to “on device” or other resource-constrained scenarios.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a machine learning model using teacher annealing.

Abstract: A neural network system is configured to receive an input image and to generate a classification output for the input image. The neural network system includes: a separable convolution subnetwork comprising a plurality of separable convolutional neural network layers arranged in a stack one after the other, in which each separable convolutional neural network layer is configured to: separately apply both a depthwise convolution and a pointwise convolution during processing of an input to the separable convolutional neural network layer to generate a layer output.

Abstract: Apparatus and methods related to image processing are provided. A computing device can determine a first image area of an image, such as an image captured by a camera. The computing device can determine a warping mesh for the image with a first portion of the warping mesh associated with the first image area. The computing device can determine a cost function for the warping mesh by: determining first costs associated with the first portion of the warping mesh that include costs associated with face-related transformations of the first image area to correct geometric distortions, and determining second costs associated with the warping mesh that include costs of edge-related transformations for preserving straightness of edges of the image. The computing device can determine an optimized mesh based on optimizing the cost function. The computing device can modify the first image area based on the optimized mesh.

Abstract: One or more servers can implement a method including generating an information resource comprising a content package including first content, metadata of a content extension, and a script for displaying the first content, and transmitting the information resource to a client device to cause the client device to: display the first content within an inline frame; identify an interaction with a user interface feature; transmit, responsive to identifying the interaction, a content extension request based on the metadata of the content extension; receive, responsive to the content extension request, a second content item; responsive to identifying the interaction, expand the inline frame from the first size to a second size; and display, in the expanded inline frame, the second content item, the second content item different from the first content item.

Abstract: Techniques are disclosed that enable processing of audio data to generate one or more refined versions of audio data, where each of the refined versions of audio data isolate one or more utterances of a single respective human speaker. Various implementations generate a refined version of audio data that isolates utterance(s) of a single human speaker by processing a spectrogram representation of the audio data (generated by processing the audio data with a frequency transformation) using a mask generated by processing the spectrogram of the audio data and a speaker embedding for the single human speaker using a trained voice filter model. Output generated over the trained voice filter model is processed using an inverse of the frequency transformation to generate the refined audio data.

Abstract: Implementations set forth herein relate to an automated assistant that can be customized by a user to provide custom assistant responses to certain assistant queries, which may originate from other users. The user can establish certain custom assistant responses by providing an assistant response request to the automated assistant and/or responding to a request from the automated assistant to establish a particular custom assistant response. In some instances, a user can elect to establish a custom assistant response when the user determines or acknowledges that certain common queries are being submitted to the automated assistant—but the automated assistant is unable to resolve the common query. Establishing such custom assistant responses can therefore condense interactions between other users and the automated assistant.

Abstract: An example method includes programming, based on image data of a frame of a plurality of frames and during a non-emission period of the frame, pixels of a plurality of pixels of a display of a computing device; causing pixels of the plurality of pixels to emit light during an emission period of the frame, wherein an amount of light emitted by the pixels during the emission period is based on the programming; and synchronizing operation of one or more sensors and operation of the plurality of pixels by at least causing the one or more sensors to emit, during a particular portion of the emission period of the frame, electromagnetic radiation through the display.

Abstract: Computing systems and methods are provided for detecting skin conditions of humans. A computing device can authenticate a user via a fingerprint scan of a first skin region of the user using a three-dimensional (3D) sonic sensor. The device can generate a three-dimensional (3D) volumetric sonic measurement of a second skin region of the user based at least in part on one or more sonic pulses of the three-dimensional sonic sensor. The device can input data indicative of the 3D volumetric sonic measurement into one or more machine-learning models, generate one or more skin cancer condition identifications associated with the second skin region of the user based on one or more outputs of the one or more machine-learned models, and provide one or more outputs including the one or more skin cancer condition identifications associated with the second skin region of the user.

Abstract: The various implementations described herein include methods and systems for synchronous audio playback. In one aspect, a method is performed at each of a plurality of electronic devices, each having an audio system, an internal clock, processors and memory storing programs for execution by the processors. Each device is configured for two-way communications with a server and associated with a user account. The device receives an identification of a first device as a common clock device that has a first internal clock being designated as a master clock. The device receives a synchronized audio playback command that includes audio data to be output and a future playback time. In response to receiving the audio data, the device determines a synchronized audio playback time. If the determined synchronized audio playback time has not yet occurred, the electronic device outputs the audio data based on the determined synchronized audio playback time.

Abstract: In one aspect, a method includes identifying (i) a computational problem that is a candidate for a quantum computation, and (ii) one or more numerical algorithms for solving the candidate computational problem; providing input task data identifying (i) the candidate computational problem, and (ii) the one or more numerical algorithms, to a numerical quantum experimentation system, wherein the numerical quantum experimentation system comprises multiple universal numerics workers, a universal numerics worker, of the multiple universal numerics workers being configured to solve the candidate computational problem using the one or more numerical algorithms; receiving, from the numerical quantum experimentation system, data representing results of the one or more numerical algorithms to solve the candidate computational problem; and determining whether the received data indicates that a quantum computation applied to the candidate computational problem has a greater efficacy at a solution than a classical computat

Abstract: Delusional bias can occur in function approximation Q-learning. Techniques for training and/or using a value network to mitigate delusional bias is disclosed herein, where the value network can be used to generate action(s) for an agent (e.g., a robot agent, a software agent, etc.). In various implementations, delusional bias can be mitigated by using a soft-consistency penalty. Additionally or alternatively, delusional bias can be mitigated by using a search framework over multiple Q-functions.

Abstract: Systems and methods are provided that train a machine-learned language encoding model through the use of a contrastive learning task. In particular, the present disclosure describes a contrastive learning task where the encoder learns to distinguish input tokens from plausible alternatives. In some implementations, on each training example the proposed method masks out some subset (e.g., 15%) of the original input tokens, replaces the masked tokens with samples from a “generator” (e.g., which may be a small masked language model), and then trains the encoder to predict whether each token comes from the original data or is a replacement produced by the generator.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for hotword detection on multiple devices are disclosed. In one aspect, a method includes the actions of receiving, by a first computing device, audio data that corresponds to an utterance. The actions further include determining a first value corresponding to a likelihood that the utterance includes a hotword. The actions further include receiving a second value corresponding to a likelihood that the utterance includes the hotword, the second value being determined by a second computing device. The actions further include comparing the first value and the second value. The actions further include based on comparing the first value to the second value, initiating speech recognition processing on the audio data.

Abstract: Techniques for presenting a content item with a media item are described. The techniques include receiving user input indicating a placement preference for a content item to be presented with a media item. The placement preference indicates a presentation preference of the content item relative to presentation of the media item. The placement preference is used to influence selection of a media item with which the content item is to be presented.

Abstract: A method includes receiving, from a camera disposed on a robotic device, a two-dimensional (2D) image of a body of an actor and determining, for each respective keypoint of a first subset of a plurality of keypoints, 2D coordinates of the respective keypoint within the 2D image. The plurality of keypoints represent body locations. Each respective keypoint of the first subset is visible in the 2D image. The method also includes determining a second subset of the plurality of keypoints. Each respective keypoint of the second subset is not visible in the 2D image. The method further includes determining, by way of a machine learning model, an extent of engagement of the actor with the robotic device based on (i) the 2D coordinates of keypoints of the first subset and (ii) for each respective keypoint of the second subset, an indicator that the respective keypoint is not visible.