Abstract: A computer-implemented method for upmixing audiovisual data can include obtaining audiovisual data including input audio data and video data accompanying the input audio data. Each frame of the video data can depict only a portion of a larger scene. The input audio data can have a first number of audio channels. The computer-implemented method can include providing the audiovisual data as input to a machine-learned audiovisual upmixing model. The audiovisual upmixing model can include a sequence-to-sequence model configured to model a respective location of one or more audio sources within the larger scene over multiple frames of the video data. The computer-implemented method can include receiving upmixed audio data from the audiovisual upmixing model. The upmixed audio data can have a second number of audio channels. The second number of audio channels can be greater than the first number of audio channels.

Abstract: Systems and methods for multi-image search can include obtaining two or more images and determining one or more search results that are based on the two or more images. The one or more search results can be determined based on determined shared attributes of the two or more images. The one or more search results may be based on feature embeddings associated with the two or more images. The two or more images may be obtained based on one or more user interactions with one or more databases.

Abstract: Aspects of the disclosure provide a hybrid smartwatch that incorporates digital technology with an analog timepiece in a wristwatch form factor. A digital display layer of a non-emissive material is configured to present notices, data, content and other information. An analog display layer includes one or more hands of the timepiece, and overlies the digital display layer. The hands may be controlled by a processor through micro-stepper motors or other actuators. Physical motion of the hands is simultaneously coupled with arrangement of content or other elements on the display layer, which results in numerous types of hybrid visualizations. This includes temporal presentations using hourly, daily, monthly or other time scales. Shape-shifting of the watch hands between 2D and 1D arrangements can linearly focus on certain information. And various content-aware layouts can highlight, bracket, occlude or otherwise emphasize or deemphasize displayed information.

Abstract: Systems and methods to reduce diffraction-angle effects, such as instances of double-bounces and bounce separation spacing effects in a laser projection system including an optical engine with laser diodes configured to emit light beams of different wavelengths and a beam combiner having a reflective surfaces each configured to receive one of the light beams from one of the laser diodes and to reflect the received light beam such that an edge of the reflected light beam lies on a tangent common to the other light beams reflected from the other reflective surfaces. The laser projection system may be implemented in head-mounted display (HMD) including a waveguide having an incoupler to receive the combined light beam, where an edge of the incoupler corresponds to the tangent on which the edges of the plurality of light beams are aligned.

Abstract: Adaptive filtering is used video stream for bitrate reduction. A first copy of the input video stream is encoded to a reference bitstream. Each of a number of candidate filters is applied to each frame of a second copy of the input video stream to produce a filtered second copy of the input video stream. The filtered second copy is encoded to a candidate bitstream. A cost value for the candidate filter is determined based on distortion value and bitrate differences between the candidate bitstream and the reference bitstream. The candidate bitstream corresponding to the candidate filter with a lowest one of the cost values is selected as the output bitstream, which is then output or stored. Processing the input video stream using the adaptive filter and before the encoding may result in bitrate reduction, thereby improving compression, decompression, and other performance.

Abstract: A method for active learning includes obtaining a set of unlabeled training samples and for each unlabeled training sample, perturbing the unlabeled training sample to generate an augmented training sample. The method includes generating, using a machine learning model, a predicted label for both samples and determining an inconsistency value for the unlabeled training sample that represents variance between the predicted labels for the unlabeled and augmented training samples. The method includes sorting the unlabeled training samples based on the inconsistency values and obtaining, for a threshold number of samples selected from the sorted unlabeled training samples, a ground truth label. The method includes selecting a current set of labeled training samples including each selected unlabeled training samples paired with the corresponding ground truth label. The method includes training, using the current set and a proper subset of unlabeled training samples, the machine learning model.

Abstract: Methods and systems for crowd source-based time marking of media items at a platform are provided herein. A media item is provided to first client devices associated with first users of a platform. An indication is received from each of the first client devices of an interesting content segment of the media item as selected by a first user of the platform. At least one content segment of the media item to be associated with a bookmark for a timeline of the media item is determined. The at least one content segment is determined in view of the indication received from each of the first client devices. The media item and the indication of the bookmark is provided to a second client device for presentation to a second user of the platform.

Abstract: Systems and methods provides for dialog management in real-time rather than turn taking. An example method included generating first candidate responses to triggering event. The triggering event may be receipt of a live stream chunk for the dialog or receipt of a backend response to a previous backend request for a dialog shema. The method also includes updating a list of candidate responses that are accepted or pending with at least on of the first candidate responses, and determining, for the triggering event, whether the list of candidate responses includes a candidate response that has a confidence score that meets a triggering threshold. The method also includes waiting for a next triggering event without providing a candidate response when the list does not include a candidate response that has a confidence score that meets the triggering threshold.

Abstract: Methods and apparatus related to identifying a video for completing a task and determining a plurality of video segments of the identified video based on one or more attributes of the task. A task and a plurality of how-to videos related to the task may be identified. A how-to video may be selected and a plurality of video segments of the selected how-to video may be determined. One or more video segments may be associated with one or more task attributes that relate to performing the task. The selected video may be provided to a user and segmented, indexed, and/or annotated based on the associated video segments. In some implementations a given object utilized in performing the task may be identified and one or more video segments corresponding to the given object may be identified and/or provided to the user.

Abstract: According an embodiment, a computing device can: identify, in a chat view associated with a video chat session, a first authorized participant and a second authorized participant of the video chat session; render, in the chat view, first visual data indicative of the first authorized participant and second visual data indicative of the second authorized participant based at least in part on identification of the first authorized participant and the second authorized participant, respectively; define, in the chat view, a chat zone indicative of a reference location of the first authorized participant; determine that the first authorized participant moved outside the chat zone; and/or conceal, in the chat view, the first visual data indicative of the first authorized participant based at least in part on determination that the first authorized participant moved outside the chat zone.

Abstract: Techniques are described herein for enabling the use of “dynamic” or “context-specific” hot words for an automated assistant. In various implementations, an automated assistant may be operated at least in part on a computing device. Audio data captured by a microphone may be monitored for default hot word(s). Detection of one or more of the default hot words may trigger transition of the automated assistant from a limited hot word listening state into a speech recognition state. Transition of the computing device into a given state may be detected, and in response, the audio data captured by the microphone may be monitored for context-specific hot word(s), in addition to or instead of the default hot word(s). Detection of the context-specific hot word(s) may trigger the automated assistant to perform a responsive action associated with the given state, without requiring detection of default hot word(s).

Abstract: A computing system and method can be used to implement a version of federated learning (FL) that incorporates adaptivity (e.g., leverages an adaptive learning rate). In particular, the present disclosure provides a general optimization framework in which (1) clients perform multiple epochs of training using a client optimizer to minimize loss on their local data and (2) a server system updates its global model by applying a gradient-based server optimizer to the average of the clients' model updates. This framework can seamlessly incorporate adaptivity by using adaptive optimizers as client and/or server optimizers. Building upon this general framework, the present disclosure also provides example specific adaptive optimization techniques for FL which use per-coordinate methods as server optimizers. By focusing on adaptive server optimization, the use of adaptive learning rates is enabled without increase in client storage or communication costs and compatibility with cross-device FL can be ensured.

Abstract: A method can include determining, by a correlation device, that a correlation event has occurred based on at least a first device and a second device having corresponding directional profiles, and responsive to occurrence of the correlation event, causing the first device to transfer a state of the first device by sending a state information message to the second device.

Abstract: Systems and methods for utilizing the dead space around the periphery of a chip for sealing a direct liquid cooled module are disclosed. One of the functions of a direct liquid cooled module is to provide cooling liquid to components located on a chip. A groove member for receiving a sealing member may be applied to the top surface of the chip. The groove member may be directly deposited to the top surface or coupled thereto via an adhesive and/or epoxy. The groove member may be in the form of opposing sidewalls or a u-shaped structure each of which form a partial enclosure for receipt of the sealing member. The groove member may be located entirely within the dead space or at least partially within the dead space and partially within a central area in which the chip components are located. The sealing member may be an O-ring or a gasket.

Abstract: A training dataset that includes a first dataset and a second dataset is received. The first dataset includes a first subset of first videos corresponding to a first context and respective first ground truth quality scores of the first videos, and the second dataset includes a second subset of second videos corresponding to a second context and respective second ground truth quality scores of the second videos. A machine learning model is trained to predict the respective first ground truth quality scores and the respective second ground truth quality scores. Training the model includes training it to obtain a global quality score for one of the videos; and training it to map the global quality score to context-dependent predicted quality scores. The context-dependent predicted quality scores include a first context-dependent predicted quality score corresponding to the first context and a second context-dependent predicted quality score corresponding to the second context.

Abstract: Processor(s) of a client device can: receive sensor data that captures environmental attributes of an environment of the client device; process the sensor data using a machine learning model to generate a predicted output that dictates whether one or more currently dormant automated assistant functions are activated; making a decision as to whether to trigger the one or more currently dormant automated assistant functions; subsequent to making the decision, determining that the decision was incorrect; and in response to determining that the determination was incorrect, generating a gradient based on comparing the predicted output to ground truth output. In some implementations, the generated gradient is used, by processor(s) of the client device, to update weights of the on-device speech recognition model. In some implementations, the generated gradient is additionally or alternatively transmitted to a remote system for use in remote updating of global weights of a global speech recognition model.

Abstract: Methods, systems, and media for presenting contextually relevant information are provided. In some implementations, the method includes: receiving information associated with a user of a user device from multiple data sources, where the user device comprises a display; identifying, without user intervention, a relevant document based on the received information associated with the user of the user device; determining that a new browser window or a new browser tab has been opened by a browser application being executed by the user device; and causing, without user intervention, the relevant document to be presented using the new browser window or new browser tab.

Abstract: In general, the subject matter described in this disclosure can be embodied in methods, systems, and program products for presenting display content on a display of a computing system. A method includes selecting, from a collection of luminance profiles that are each configured to reduce brightness of the display content in different manners, a first luminance profile based on the current display brightness setting, the first luminance profile specifying a first amount of brightness reduction to a peripheral portion of the display content and a first gradient of brightness reduction for a portion of the display content between the peripheral portion of the display content and a center portion of the display content; applying the first luminance profile to the display content to modify the display content by reducing a brightness of the display content according to the first luminance profile; and presenting the display content on the display.

Abstract: A method including receiving (S605) a request for a depth map, generating (S625) a hybrid depth map based on a device depth map (110) and downloaded depth information (105), and responding (S630) to the request for the depth map with the hybrid depth map (415). The device depth map (110) can be depth data captured on a user device (515) using sensors and/or software. The downloaded depth information (105) can be associated with depth data, map data, image data, and/or the like stored on a remote (to the user device) server (505).