Abstract: A video game system provides dialog responses based on a natural language model (NLM). The NLM is a language model that receives a language input, such as a dialog selection, audio recording, or natural language text input provided by a user of the video game system. In response to the language input, and based on a corpus of natural language candidate lines, the NLM identifies one or more potential responses. The video game system selects a final response from the identified potential responses and provides the selected response to the user via, for example, one or more display frames or via an audio output.

Abstract: Systems and methods of the present disclosure are directed to a method that can include obtaining a 3D mesh comprising polygons and texture/shading data. The method can include rasterizing the 3D mesh to obtain a 2D raster comprising pixels and coordinates respectively associated with a subset of pixels. The method can include determining an initial color value for the subset of pixels based on the coordinates of the pixel and the associated shading/texture data. The method can include constructing a splat at the coordinates of a respective pixel. The method can include determining an updated color value for a respective pixel based on a weighting of the subset of splats to generate a 2D rendering of the 3D mesh based on the coordinates of a pixel and a splat.

Abstract: A method, device, and computer-readable storage medium for predicting pronunciation of a text sample, including generating an encoding of allowable pronunciations of the text sample, selecting predicted text samples corresponding to an audio sample, the predicted text samples including the text sample and one or more co-emitted text samples, outputting the text sample, and updating the encoding of allowable pronunciations of the text sample based on pronunciations of the one or more co-emitted text samples.

Abstract: Implementations are directed to providing a voice bot development platform that enables a third-party developer to train a voice bot based on training instance(s). The training instance(s) can each include training input and training output. The training input can include a portion of a corresponding conversation and a prior context of the corresponding conversation. The training output can include a corresponding ground truth response to the portion of the corresponding conversation. Subsequent to training, the voice bot can be deployed for conducting conversations on behalf of a third-party. In some implementations, the voice bot is further trained based on a corresponding feature emphasis input that attentions the voice bot to a particular feature of the portion of the corresponding conversation. In some additional or alternative implementations, the voice bot is further trained to interact with third-party system(s) via remote procedure calls (RPCs).

Abstract: The various implementations described herein include methods, devices, and systems for interactive cloud gaming. In one aspect, an interface device with processors and memory storing an application program is used for cloud gaming. The interface device includes a communication module for receiving and/or transmitting messages and media streams between the interface device, a game controller, and a remote gaming server. The interface device further includes an output module configured to provide output of the application to an input of a display device coupled to or integrated with the interface device. The application is configured to implement a first mode in which the interface device processes and displays outputs associated with substantially all messages transmitted by the game controller, and a second mode in which the interface device processes and displays a predefined subset of messages from the game controller and displays gaming media streams from the gaming server.

Abstract: Speech processing techniques are disclosed that enable determining a text representation of alphanumeric sequences in captured audio data. Various implementations include determining a contextual biasing finite state transducer (FST) based on contextual information corresponding to the captured audio data. Additional or alternative implementations include modifying probabilities of one or more candidate recognitions of the alphanumeric sequence using the contextual biasing FST.

Abstract: A near-eye display system employs a low-amplitude, low-frequency micro-electromechanical system (MEMS) mirror in series with a higher-amplitude, higher-frequency resonant MEMS mirror to rotate at a reduced amplitude and frequency with respect to the resonant MEMS mirror redistribute illumination pulses or pixels at the extents of a sinusoidal angular scan pattern of the resonant MEMS mirror.

Abstract: A UE communicating in DC with an MN and an SN receives a configuration related to a C-SN and at least one condition for connecting to the C-SN, determines that a radio connection between the UE and the MN has failed, and transmits a first indication of the failed radio connection to the SN to initiate a recovery procedure. The UE also releases the configuration and/or the at least one condition, or suspending a procedure, for connecting to the C-SN\.

Abstract: A method for facilitating a Parkinson's Disease (“PD”) assessment of a patient includes capturing first video of a patient performing first test movements while holding the mobile device; capturing second video of the patient performing second test movements while maintaining the mobile device on their person; capturing third video of the patient performing third test movements including standing and walking; capturing one or more IMU readings using an IMU of the mobile device; processing the first video, the second video, and the third video according to (i) a hand landmark model to generate one or more hand biomarkers, (ii) a face landmark model to generate one or more face biomarkers, and (iii) a body landmark model to generate one or more body biomarkers; and determining an assessment score based on a standardized PD assessment by processing the biomarkers.

Abstract: The present disclosure provides systems and methods for on-device machine learning. In particular, the present disclosure is directed to an on-device machine learning platform and associated techniques that enable on-device prediction, training, example collection, and/or other machine learning tasks or functionality. The on-device machine learning platform can include a context provider that securely injects context features into collected training examples and/or client-provided input data used to generate predictions/inferences. Thus, the on-device machine learning platform can enable centralized training example collection, model training, and usage of machine-learned models as a service to applications or other clients.

Abstract: Systems, devices, and methods for accommodating multiple sets of laser light sources in an optical engine of a display system such as a laser projection system are described. Laser light beams may be combined via wavelength-, polarization-, and/or angular-separation-based techniques. First and second sets of laser light beams may be angularly separated such that different sets of partially overlapping pixels are projected by the system, thereby increasing the display pixel density and/or an expanding the field of view of the display. One or more laser die may be mounted to each submount of the optical engine. For embodiments with two laser dies on each submount, collimating lenses may introduce angular separation between laser light beams output by each pair of commonly mounted laser dies. A retroreflector prism may be disposed at a beam combiner to provide a compact extension of the optical path through the beam combiner.

Abstract: Methods, apparatus, systems, and computer-readable media are provided for generating and/or adapting automated assistant content according to a distance of a user relative to an automated assistant interface that renders the automated assistant content. For instance, the automated assistant can provide data for a client device to render. The client device can request additional data when the user relocates closer to, or further from, the client device. In some implementations, a request for additional data can identify a distance between the user and the client device. In this way, the additional data can be generated or selected according to the distance in the request. Other implementations can allow an automated assistant to determine an active user from a group of users in an environment, and determine a distance between the active user and the client device in order that any rendered content can be tailored for the active user.

Abstract: An audio playback system that provides intuitive audio playback of textual content responsive to user input actions, such as scrolling portions of textual content on a display. Playback of audio (e.g., text-to-speech audio) that includes textual content can begin based on a portion of textual content being positioned by a user input at a certain position on a device display. As one example, a user can simply scroll through a webpage or other content item to cause a text-to-speech system to perform audio playback of textual content displayed in one or more playback section(s) of the device's viewport (e.g., rather than requiring the user to perform additional tapping or gesturing to specifically select a certain portion of textual content).

Abstract: A user device operating in dual-mode connectivity with a master node and a secondary node detects a failure of a secondary cell group associated with the secondary node (1104). In response, the user device suspends an secondary cell group transmission (1106). The user device generates a message related to a status of the SCG for transmission to the MN (1108) and transmitting the message to the MN when the SCG transmission is suspended (1110).

Abstract: Systems and method for the structure, configuration, or arrangement of the input fields in a user interface. For example, a computer-implemented method includes displaying a structured input interface configured with a plurality of input fields. The structured input interface can be configured for rendering in a graphical user interface. A first input field can correspond to an initial notification element in the graphical user interface for rendering in association with the first input field. The method includes receiving user data. The method includes determining, based on the user data, an updated notification element for rendering in association with the first input field. The method includes updating the structured input interface comprising the first input field and the updated notification element.

Abstract: A multimodal search system is described. The system can receive image data from a user device. Additionally, the system can receive a prompt associated with the image data. Moreover, the system can determine, using a computer vision model, a first object in the image data that is associated with the prompt. Furthermore, the system can receive, from the user device, a user indication on whether the image data includes the first object. Subsequently, in response to receiving the user indication, the system can generate a response using a large language model.

Abstract: A head-mounted display (HMD) system includes a lens element supported by a support structure. The lens element includes a waveguide that includes an incoupler, an outcoupler, and an exit pupil expander. The incoupler is disposed within a first area of the waveguide. The outcoupler is disposed within a second area of the waveguide. The exit pupil expander is disposed within a third area of the waveguide. An anti-reflection coating is formed via fabrication used to form the incoupler, the outcoupler, and the exit pupil expander. The anti-reflection coating is disposed within a fourth area of the waveguide different than the first, second, and third areas of the waveguide.

Abstract: Implementations relate to managing multimedia content that is obtained by large language model(s) (LLM(s)) and/or generated by other generative model(s). Processor(s) of a system can: receive natural language (NL) based input that requests multimedia content, generate a response that is responsive to the NL based input, and cause the response to be rendered. In some implementations, and in generating the response, the processor(s) can process, using a LLM, LLM input to generate LLM output, and determine, based on the LLM output, at least multimedia content to be included in the response. Further, the processor(s) can evaluate the multimedia content to determine whether it should be included in the response. In response to determining that the multimedia content should not be included in the response, the processor(s) can cause the response, including alternative multimedia content or other textual content, to be rendered.

Abstract: Techniques are disclosed for evaluating manual driving data using an AV control system based on differences between data generated using the AV control system and the manual driving data. In many implementations, manual driving data captures action(s) of a vehicle controlled by a manual driver. Additionally or alternatively, an additional AV control system can be trained using the evaluated manual driving data.