Abstract: A visual programming platform can leverage a machine learning-based coding system to generate an initial set of programming-language code for further graphical editing by a human user. As an example, the visual programming platform can obtain a natural language description of a task to be performed by a computational pipeline. The visual programming platform can process the natural language description of the task with a machine learning coding system that includes one or more machine-learned models to generate, as an output of the machine learning coding system, a set of pseudocode that describes performance of the task. The platform can process the set of pseudocode that describes performance of the task with a compiler to generate a set of programming-language code that defines the computational pipeline for performing the task. The visual programming platform can generate a graphical visualization of the computational pipeline defined by the set of programming-language code.

Abstract: A method can include determining, by a head-mounted device mounted on a head of a user, a number of degrees of freedom of an input modality of an auxiliary device; determining an input modality of the head-mounted device based on the number of degrees of freedom of the input modality of the auxiliary device; and presenting the input modality of the head-mounted device to the user.

Abstract: A system including a computer vision model configured to perform a machine learning task is described. The computer vision model includes multiple wrapped convolutional layers, in which each wrapped convolutional layer includes a respective convolutional layer configured to receive, for each time step of multiple time steps, a layer input and to process the layer input to generate an initial output for the current time step, and a respective note-taking module configured to receive the initial output and to process the initial output to generate a feature vector for the current time step, the feature vector representing local information of the wrapped convolutional layer. The model includes a summarization module configured to receive the feature vectors and to process the feature vectors to generate a revision vector for the current time step, the revision vector representing global information of the plurality of wrapped convolutional layers.

Abstract: In described techniques, a depth of an object may be determined with respect to a head-mounted device (HMD). A gaze duration with respect to the object may be determined, relative to a gaze duration threshold. Then, based on the depth and on the gaze duration exceeding the gaze duration threshold, virtual content may be rendered at a virtual depth with respect to the HMD that is different than the depth.

Abstract: A visual programming platform can leverage a machine learning-based coding system to generate an initial set of programming-language code for further graphical editing by a human user. As an example, the visual programming platform can obtain a natural language description of a task to be performed by a computational pipeline. The visual programming platform can process the natural language description of the task with a machine learning coding system that includes one or more machine-learned models to generate, as an output of the machine learning coding system, a set of pseudocode that describes performance of the task. The platform can process the set of pseudocode that describes performance of the task with a compiler to generate a set of programming-language code that defines the computational pipeline for performing the task. The visual programming platform can generate a graphical visualization of the computational pipeline defined by the set of programming-language code.

Abstract: A method including determining a gaze direction of a user of a wearable device, capturing an image using a forward-looking camera of the wearable device, detecting a surroundings of the user based on the image, determining whether or not the user is distracted based on the gaze direction and the surroundings, and in response to determining the user is distracted, causing an operation to be performed on the wearable device, the operation configured to cause the user to change the user's attention.

Abstract: A method comprises selecting a two-dimensional image based on input to a computing device, generating multiple two-dimensional views of an object based on the two-dimensional image, and generating a three-dimensional virtual object based on the multiple two-dimensional views.

Abstract: A media stream comprising audio data and first lyric data associated with the audio data is received by a processing device. A set of user data associated with a user of a client device is identified. The first lyric data and the set of user data are provided as input to a generative machine learning model. An output of the generative machine learning model is obtained. The output comprises second lyric data. The second lyric data is a version of the first lyric data that is customized for the user. The second lyric data and the media stream are caused to be presented in a graphical user interface on the client device.

Abstract: Systems and methods are disclosed that address the need for adaptive exposure within high dynamic range (HDR) images. Solutions can leverage recent advances in the use of virtual reality (VR) headsets and Augmented Reality (AR) displays equipped with infrared (IR) eye tracking devices. A gaze vector determined by the eye tracking device identifies one or more fixation points on the image that corresponds to an area where there exists a faulty exposure. The exposure around the fixation point can be adaptively corrected using image processing techniques. Using spatial adaptive exposure, the resulting image, a type of foveated image, can be rendered on a low dynamic range (LDR) display with sufficient detail.

Abstract: According to an aspect, a method includes receiving, by a head-mounted display device, an authentication code associated with multi-factor authentication, receiving image data from an image camera on the head-mounted display device, detecting, by the head-mounted display device, that the image data includes an interface for receiving the authentication code, and displaying, by the head-mounted display device, the authentication code at a location that corresponds to the interface.

Abstract: Methods and devices are provided to allow for the transfer of a display of a visual representation between a head mounted device and a computing device during the display of a video. A video is displayed on a computing device display of a computing device, a visual representation of a speech for an audio component of the video is received, the visual representation is displayed on the computing device display, and the display of the visual representation is transferred to the head mounted device to display on a head mounted device display upon determining that a head mounted device is in use.

Abstract: Methods, a system, and a device are provided to allow co-watch devices to coordinate text interpretation services while co-watching a video or live event. A server receives an indication that a first co-watch device and a second co-watch device are preparing to co-watch a video or a live event while displaying a text interpretation of a speech component of the video or live event. An indication is sent to a first device of the first and second co-watch devices to operate as a text-processing device, generating the text interpretation, and transmitting the text interpretation to a second device of the first and second co-watch devices. The first device receives a portion of a video, processes a speech component of the portion of the video to generate a text interpretation, and sends the text interpretation to a second device.

Abstract: A method including receiving an image as a portion of a real-world space, placing an anchor on the image, determining a position of the anchor, determine a depth associated with the position of the anchor, applying an image editing algorithm based on the depth to the captured image, and rendering the edited image.

Abstract: A system and method for relighting of an augmented reality scene are provided. Image data and depth data corresponding to a physical environment may be captured by an image sensor of a computing device, for display in an AR scene generated by the computing device. The system may use the image data and depth data to generate a live depth map of the physical environment. A virtual light source may be placed in the AR scene of the physical environment in response to a user input. The system may relight the AR scene in response to the placement of the virtual light source based on the image data and the depth map.

Abstract: A method includes providing an interactive graphical user interface comprising a first menu providing one or more input options, a second menu providing one or more machine learning models, and a third menu providing one or more output formats. The method also includes generating a graph in a portion of the interactive graphical user interface by detecting one or more user selections of an input option, a machine learning model, and an output format, displaying nodes corresponding to the input option, the machine learning model, the output format, and displaying edges connecting the first node to the second node, and the second node to the third node. The method additionally includes applying the machine learning model to an input associated with the input option to generate an output in the output format. The method further includes providing, by the interactive graphical user interface, the output in the output format.

Abstract: Systems, methods, devices, and related techniques for visualizing machine learning models with hierarchical structure are described in this document. In some implementations, a visualization generator receives a computational graph for a machine learning model and outputs a visual layout of the computational graph. The visual layout includes visual elements organized to present the hierarchical structure of the computational graph The visual layout can include respective visual elements for each node aggregation layer defined by the hierarchical structure. The respective visual elements for each node aggregation layer are provided in lieu of respective visual elements for individual nodes of the multiple nodes assigned to the one or more node aggregation layers.

Abstract: According to an aspect, a method includes generating at least one surface for a subject based on image data, the image data comprising a first image from a first perspective and a second image from a second perspective. The method further includes identifying a distribution of light intensity over the at least one surface and identifying a view independent feature based on the distribution of light intensity. The method also provides generating a texture map for a sequence of images based on the view independent feature.

Abstract: Described techniques may be utilized to receive a transcription stream including transcribed text that has been transcribed from speech, and to receive a summary request for a summary to be provided on a display of a device. Extracted text may be identified from the transcribed text and in response to the summary request. The extracted text may be processed using a summarization machine learning (ML) model to obtain a summary of the extracted text, and the summary may be displayed on the display of the device. When an image is captured, an augmented summary may be generated that includes the image together with a visual indication of one or more of an emotion, an entity, or an intent associated with the image, the summary, or the extracted text.

Abstract: Systems and methods are disclosed that address the need for adaptive exposure within high dynamic range (HDR) images. Solutions can leverage recent advances in the use of virtual reality (VR) headsets and Augmented Reality (AR) displays equipped with infrared (IR) eye tracking devices. A gaze vector determined by the eye tracking device identifies one or more fixation points on the image that corresponds to an area where there exists a faulty exposure. The exposure around the fixation point can be adaptively corrected using image processing techniques. Using spatial adaptive exposure, the resulting image, a type of foveated image, can be rendered on a low dynamic range (LDR) display with sufficient detail.

Abstract: Described techniques including processing input text at a content type classifier machine learning (ML) model to obtain a content type of the input text. The input text and the content type may be processed at a content extractor ML model to obtain extracted content from the input text. The input text, the content type, and the extracted content may be processed at a summarizer ML model to obtain a summary of the input text.