Abstract: A virtual or augmented reality display system that controls power inputs to the display system as a function of image data. Image data itself is made of a plurality of image data frames, each with constituent color components of, and depth planes for displaying on, rendered content. Light sources or spatial light modulators to relay illumination from the light sources may receive signals from a display controller to adjust a power setting to the light source or spatial light modulator, and/or control depth of displayed image content, based on control information embedded in an image data frame.

Abstract: A computer implemented method for warping virtual content includes receiving rendered virtual content data, the rendered virtual content data including a far depth. The method also includes receiving movement data indicating a user movement in a direction orthogonal to an optical axis. The method further includes generating warped rendered virtual content data based on the rendered virtual content data, the far depth, and the movement data.

Abstract: A virtual or augmented reality display system that controls power inputs to the display system as a function of image data. Image data itself is made of a plurality of image data frames, each with constituent color components of, and depth planes for displaying on, rendered content. Light sources or spatial light modulators to relay illumination from the light sources may receive signals from a display controller to adjust a power setting to the light source or spatial light modulator based on control information embedded in an image data frame.

Abstract: A computer implemented method for warping virtual content includes receiving rendered virtual content data, the rendered virtual content data including a far depth. The method also includes receiving movement data indicating a user movement in a direction orthogonal to an optical axis. The method further includes generating warped rendered virtual content data based on the rendered virtual content data, the far depth, and the movement data.

Abstract: In one embodiment, a method includes a server generating first shading information for visible portions of objects relative to a first viewpoint, storing the first shading information in a texture atlas, and sending the texture atlas to a client device. The method also includes determining a first subset of the visible portions of the objects for which shading information is to be re-generated and a second subset for which elements of the first shading information are to be reused, generating second shading information for the first subset relative to a second viewpoint, updating the texture atlas to include the second shading information for the first subset and the elements of the first shading information for the second subset, and sending the updated texture atlas to the client device. The updated texture atlas is configured for rendering images of the visible portions of the objects from multiple viewpoints.

Abstract: In one embodiment, a method includes a step of receiving a geometric representation of a virtual object and a texture atlas, the geometric representation comprising multiple geometric primitives defining a shape of the virtual object, the texture atlas comprises regions each of which is allocated to include shading information of a respective geometric primitive of the plurality of geometric primitives, and the shading information of the respective geometric primitive being scaled down to be smaller than the allocated region so as to create a buffer between the allocated region and adjacent regions of the plurality of regions on the texture atlas. The method further includes steps of identifying, based on a first viewpoint from which to view the virtual object, visible geometric primitives from the plurality of geometric primitives and rendering images of the visible geometric primitives using corresponding shading information included in the texture atlas.

Abstract: Disclosed herein are related to a device and a method of remotely rendering an image. In one approach, a device divides an image of an artificial reality space into a plurality of slices. In one approach, the device encodes a first slice of the plurality of slices. In one approach, the device encodes a portion of a second slice of the plurality of slices, while the device encodes a portion of the first slice. In one approach, the device transmits the encoded first slice of the plurality of slices to a head wearable display. In one approach, the device transmits the encoded second slice of the plurality of slices to the head wearable display, while the device transmits a portion of the encoded first slice to the head wearable display.

Abstract: In one embodiment, a method includes the steps of receiving, from a client device, a first viewpoint from which to view a virtual object, the virtual object having a shape defined by multiple geometric primitives, identifying, relative to the first viewpoint, visible geometric primitives from multiple geometric primitives, allocating a region in a texture atlas for each of the visible geometric primitives, generating shading information for each of the visible geometric primitives, storing the shading information of each of the visible geometric primitives in a portion of the allocated region smaller than the allocated region to create a buffer around the portion of the allocated region where the shading information is stored, and sending, to the client device, the texture atlas and a list identifying the visible geometric primitives, the texture atlas being configured for rendering images of the visible geometric primitives from different viewpoints.

Abstract: Disclosed herein are related to systems and methods for providing artificial reality. In one aspect, a console receives feedback information indicative of a first completion time from a head wearable display (HWD). The HWD may complete generating a first image frame at the first completion time. The HWD may display the first image frame. In one aspect, the console compares a display time, at which the first image frame is displayed by the HWD, and the first completion time. In one aspect, the console adjusts, according to the comparison, image processing, to adjust a second completion time, at which the console completes generating a second image frame. In one aspect, the console generates, via the adjusted image processing, the second image frame to provide to the HWD.

Abstract: A virtual or augmented reality display system that controls power inputs to the display system as a function of image data. Image data itself is made of a plurality of image data frames, each with constituent color components of, and depth planes for displaying on, rendered content. Light sources or spatial light modulators to relay illumination from the light sources may receive signals from a display controlled to adjust a power setting to the light source or spatial light modulator based on control information embedded in an image data frame.

Abstract: Disclosed herein are related to a device and a method of remotely rendering an image. In one approach, a device divides an image of an artificial reality space into a plurality of slices. In one approach, the device encodes a first slice of the plurality of slices. In one approach, the device encodes a portion of a second slice of the plurality of slices, while the device encodes a portion of the first slice. In one approach, the device transmits the encoded first slice of the plurality of slices to a head wearable display. In one approach, the device transmits the encoded second slice of the plurality of slices to the head wearable display, while the device transmits a portion of the encoded first slice to the head wearable display.

Abstract: A virtual or augmented reality display system that controls power inputs to the display system as a function of image data. Image data itself is made of a plurality of image data frames, each with constituent color components of, and depth planes for displaying on, rendered content. Light sources or spatial light modulators to relay illumination from the light sources may receive signals from a display controlled to adjust a power setting to the light source or spatial light modulator based on control information embedded in an image data frame.

Abstract: A method includes the steps of receiving training data comprising images of an object and associated camera poses from which the images are captured, training, based on the training data, a machine-learning model to take as input a given viewpoint and synthesize an image of a virtual representation of the object viewed from the given viewpoint, generating, for each of predetermined viewpoints surrounding the virtual representation of the object, a view-dependent image of the object as viewed from that viewpoint using the trained machine-learning model, receiving, from a client device, a desired viewpoint from which to view the virtual representation of the object, selecting one or more of the predetermined viewpoints based on the desired viewpoint, and sending, to the client device, the view-dependent images associated with the selected one or more viewpoints for rendering an output image of the virtual representation of the object viewed from the desired viewpoint.

Abstract: In one embodiment, a method includes the steps of receiving, from a client device, a first viewpoint from which to view a virtual object, the virtual object having a shape defined by multiple geometric primitives, identifying, relative to the first viewpoint, visible geometric primitives from multiple geometric primitives, allocating a region in a texture atlas for each of the visible geometric primitives, generating shading information for each of the visible geometric primitives, storing the shading information of each of the visible geometric primitives in a portion of the allocated region smaller than the allocated region to create a buffer around the portion of the allocated region where the shading information is stored, and sending, to the client device, the texture atlas and a list identifying the visible geometric primitives, the texture atlas being configured for rendering images of the visible geometric primitives from different viewpoints.

Abstract: A method includes the steps of receiving training data comprising images of an object and associated camera poses from which the images are captured, training, based on the training data, a machine-learning model to take as input a given viewpoint and synthesize an image of a virtual representation of the object viewed from the given viewpoint, generating, for each of predetermined viewpoints surrounding the virtual representation of the object, a view-dependent image of the object as viewed from that viewpoint using the trained machine-learning model, receiving, from a client device, a desired viewpoint from which to view the virtual representation of the object, selecting one or more of the predetermined viewpoints based on the desired viewpoint, and sending, to the client device, the view-dependent images associated with the selected one or more viewpoints for rendering an output image of the virtual representation of the object viewed from the desired viewpoint.

Abstract: A computer implemented method for warping virtual content from two sources includes a first source generating first virtual content based on a first pose. The method also includes a second source generating second virtual content based on a second pose. The method further includes a compositor processing the first and second virtual content in a single pass. Processing the first and second virtual content includes generating warped first virtual content by warping the first virtual content based on a third pose, generating warped second virtual content by warping the second virtual content based on the third pose, and generating output content by compositing the warped first and second virtual content.

Abstract: A computer implemented method for warping virtual content includes generating warped virtual content by transforming source virtual content. The method also includes determining whether a memory location corresponding to an X, Y location of the warped virtual content in an output frame of reference is occupied by pre-existing virtual content. The method further includes storing the warped virtual content in the memory location if the memory location is not occupied. Moreover, the method includes comparing respective Z locations of the warped virtual content and the pre-existing virtual content to identify virtual content with a Z location closer to a viewing location if the memory location is occupied. The method also includes storing the warped virtual content in the memory location corresponding to the X, Y location if a Z location of warped virtual content is closer to the viewing location than a pre-existing Z location of pre-existing virtual content.

Abstract: A computer implemented method for warping virtual content includes receiving rendered virtual content data, the rendered virtual content data including a far depth. The method also includes receiving movement data indicating a user movement in a direction orthogonal to an optical axis. The method further includes generating warped rendered virtual content data based on the rendered virtual content data, the far depth, and the movement data.

Abstract: A computer implemented method for warping virtual content includes receiving rendered virtual content data, the rendered virtual content data including a far depth. The method also includes receiving movement data indicating a user movement in a direction orthogonal to an optical axis. The method further includes generating warped rendered virtual content data based on the rendered virtual content data, the far depth, and the movement data.

Abstract: Disclosed is an improved approach for generated recordings from augmented reality systems from the perspective of a camera within the system. Instead of re-using rendered virtual content from the perspective of the user's eyes for AR recordings, additional virtual content is rendered from an additional perspective specifically for the AR recording. That additional virtual content is combined with image frames generated by a camera to form the AR recording.