Abstract: Disclosed herein are related to a system and a method for providing an artificial reality. In one aspect, a system includes a shared physical memory and a first processor having access to the shared physical memory. In one aspect, the first processor performs, during a first time period, a first rendering process to generate a first image frame of a first view of an artificial reality. In one aspect, the first processor performs, during a second time period, a second rendering process to generate a second image frame of a second view of the artificial reality. In one aspect, the system includes a second processor including a neural network and having access to the shared physical memory. In one aspect, the second processor performs, during a third time period overlapping a portion of the second time period, an image enhancing process on the first image frame.

Abstract: In particular embodiments, a computing system may receive an image to be shown on a display having a plurality of backlight zones. The computing system may compute a backlight matrix for adjusting brightness levels of the backlight zones of the display by computing, for each of the backlight zones, a zone statistic to represent grayscale levels of a portion of the image within the backlight zone, mapping the zone statistic of each of the backlight zones to a brightness value using a particular mapping technique configured to adjust any of the backlight zones having a zone statistic that is below a predetermined threshold, and generating the backlight matrix by filtering the brightness values. The computing system may instruct the display to output the image and adjust the backlight zones based on the backlight matrix.

Abstract: In one embodiment, a method includes a server generating first shading information for visible portions of objects relative to a first viewpoint, storing the first shading information in a texture atlas, and sending the texture atlas to a client device. The method also includes determining a first subset of the visible portions of the objects for which shading information is to be re-generated and a second subset for which elements of the first shading information are to be reused, generating second shading information for the first subset relative to a second viewpoint, updating the texture atlas to include the second shading information for the first subset and the elements of the first shading information for the second subset, and sending the updated texture atlas to the client device. The updated texture atlas is configured for rendering images of the visible portions of the objects from multiple viewpoints.

Abstract: A method for providing imagery to a user on a display includes receiving eye tracking data. The method also includes determining a current gaze location and a relative distance between the current gaze location and an edge of the display using the eye tracking data. The method also includes defining a first tile centered at the current gaze location and multiple tiles that surround the first tile using the current gaze location and the relative distance between the current gaze location and the edge of the display. The method includes providing a foveated rendered image using the first tile and the multiple tiles.

Abstract: A method for providing imagery to a user on a display includes receiving eye tracking data. The method also includes determining a current gaze location and a relative distance between the current gaze location and an edge of the display using the eye tracking data. The method also includes defining a first tile centered at the current gaze location and multiple tiles that surround the first tile using the current gaze location and the relative distance between the current gaze location and the edge of the display. The method includes providing a foveated rendered image using the first tile and the multiple tiles.

Abstract: Embodiments of the present disclosure relate to a display assembly with color calibration for improving accuracy of displayed colors under thermal shifts. A data processing circuit of the display assembly determines color compensation coefficients corresponding to an operating temperature of a display panel, each color compensation coefficient accounting for a thermal shift of a respective color component at the operating temperature. The data processing circuit applies the color compensation coefficients to color transform elements corresponding to an initial temperature to determine a corrected version of the color transform elements corresponding to the operating temperature.

Abstract: Disclosed herein are related to a system and a method for providing an artificial reality. In one aspect, a system includes a shared physical memory and a first processor having access to the shared physical memory. In one aspect, the first processor performs, during a first time period, a first rendering process to generate a first image frame of a first view of an artificial reality. In one aspect, the first processor performs, during a second time period, a second rendering process to generate a second image frame of a second view of the artificial reality. In one aspect, the system includes a second processor including a neural network and having access to the shared physical memory. In one aspect, the second processor performs, during a third time period overlapping a portion of the second time period, an image enhancing process on the first image frame.

Abstract: Disclosed herein are related to a system and a method for providing an artificial reality. In one aspect, a system includes a shared physical memory and a first processor having access to the shared physical memory. In one aspect, the first processor performs, during a first time period, a first rendering process to generate a first image frame of a first view of an artificial reality. In one aspect, the first processor performs, during a second time period, a second rendering process to generate a second image frame of a second view of the artificial reality. In one aspect, the system includes a second processor including a neural network and having access to the shared physical memory. In one aspect, the second processor performs, during a third time period overlapping a portion of the second time period, an image enhancing process on the first image frame.

Abstract: A method for providing imagery to a user on a display includes receiving head tracking data and using the head tracking data to determine a change in an orientation of a user's head. The method also includes blending pixels of a previous frame with corresponding pixels of a current frame. The corresponding pixels of the current frame are identified using the change in the orientation of the user's head. The method also includes providing foveated imagery to the user on the display using the blended pixels and multiple tiles.

Abstract: A method for providing imagery to a user on a display includes receiving eye tracking data. The method also includes determining a current gaze location and a relative distance between the current gaze location and an edge of the display using the eye tracking data. The method also includes defining a first tile centered at the current gaze location and multiple tiles that surround the first tile using the current gaze location and the relative distance between the current gaze location and the edge of the display. The method includes providing a foveated rendered image using the first tile and the multiple tiles.

Abstract: A method for providing imagery to a user on a display includes receiving eye tracking data. The method also includes determining a gaze location on the display and at least one of a confidence factor of the gaze location, or a speed of the change of the gaze location using the eye tracking data. The method also includes establishing multiple tiles using the gaze location and at least one of the confidence factor or the speed of the change of the gaze location. The method also includes providing a foveated rendered image using the multiple tiles.

Abstract: A method for providing imagery to a user on a display includes receiving eye tracking data. The method also includes determining a gaze location on the display and at least one of a confidence factor of the gaze location, or a speed of the change of the gaze location using the eye tracking data. The method also includes establishing multiple tiles using the gaze location and at least one of the confidence factor or the speed of the change of the gaze location. The method also includes providing a foveated rendered image using the multiple tiles.

Abstract: Disclosed herein are related to a system and a method for providing an artificial reality. In one aspect, a system includes a shared physical memory and a first processor having access to the shared physical memory. In one aspect, the first processor performs, during a first time period, a first rendering process to generate a first image frame of a first view of an artificial reality. In one aspect, the first processor performs, during a second time period, a second rendering process to generate a second image frame of a second view of the artificial reality. In one aspect, the system includes a second processor including a neural network and having access to the shared physical memory. In one aspect, the second processor performs, during a third time period overlapping a portion of the second time period, an image enhancing process on the first image frame.

Abstract: A display device, such as a head mounted device (HMD), displays a virtual scene. The display device includes a motion tracker for detecting rotation and/or translation of the display device. The display device also includes a processor that is configured to determine, in response to the detected, an orientation of the display device relative to a plurality of world-aligned viewing frustums that are stationary relative to the virtual scene. The processor is also configured to identify a set of those world-aligned viewing frustums that overlap with an output field of view. The processor is further configured to render pixels of the set of those world-aligned viewing frustums that overlap with an output field of view and upsample the rendered pixels to generate values of display pixels for presentation by the display device.

Abstract: Systems and methods for performing foveated rendering are provided. An example system and method may warp a 3D scene based on a fixation point. The system and method may also render the warped 3D scene to generate a first image. The system and method may also unwarp the first image to generate a second image. For example, the first image may have fewer pixels than the second image.

Abstract: A processing unit is configured to render first pixels representative of a high-acuity region in the image and second pixels representative of a low-acuity region in the image. A shaper is configured to reorganize the first pixels based on at least one dimension of the low-acuity region. A multiplexer is configured to multiplex the reorganized first pixels and the second pixels to form a display stream. An encoder is configured to compress the display stream for transmission to a display. A decoder configured to decompress the display stream. A demultiplexer is configured to demultiplex the first pixels and the second pixels. Another processing unit is configured to blend the first pixels and the second pixels to form blended pixel values representative of the image for presentation on a screen.

Abstract: A display device, such as a head mounted device (HMD), displays a virtual scene. The display device includes a motion tracker for detecting rotation and/or translation of the display device. The display device also includes a processor that is configured to determine, in response to the detected, an orientation of the display device relative to a plurality of world-aligned viewing frustums that are stationary relative to the virtual scene. The processor is also configured to identify a set of those world-aligned viewing frustums that overlap with an output field of view. The processor is further configured to render pixels of the set of those world-aligned viewing frustums that overlap with an output field of view and upsample the rendered pixels to generate values of display pixels for presentation by the display device.

Abstract: A processing unit is configured to render first pixels representative of a high-acuity region in the image and second pixels representative of a low-acuity region in the image. A shaper is configured to reorganize the first pixels based on at least one dimension of the low-acuity region. A multiplexer is configured to multiplex the reorganized first pixels and the second pixels to form a display stream. An encoder is configured to compress the display stream for transmission to a display. A decoder configured to decompress the display stream. A demultiplexer is configured to demultiplex the first pixels and the second pixels. Another processing unit is configured to blend the first pixels and the second pixels to form blended pixel values representative of the image for presentation on a screen.

Abstract: Systems and methods for performing foveated rendering are provided. An example system and method may warp a 3D scene based on a fixation point. The system and method may also render the warped 3D scene to generate a first image. The system and method may also unwarp the first image to generate a second image. For example, the first image may have fewer pixels than the second image.