Abstract: A rendering workload for an individual frame can be split between a head-mounted display (HMD) and a host computer that is executing an application. To split a rendering workload for a frame, the HMD may send head tracking data to the host computer, and the head tracking data may be used by the host computer to generate pixel data associated with the frame and extra data in addition to the pixel data. The extra data can include, without limitation, pose data, depth data, motion vector data, and/or extra pixel data. The HMD may receive the pixel data and at least some of the extra data, determine an updated pose for the HMD, and apply re-projection adjustments to the pixel data based on the updated pose and the received extra data to obtain modified pixel data, which is used to present an image on the display panel(s) of the HMD.

Abstract: Controller visualization systems and methods for use in virtual/augmented reality environments such as walk-around virtual reality environments are described. The virtual representation of a physical control device may be altered based on the context of the virtual environment. Certain embodiments combine the virtual rendering of the tracked control device with a real-time video representation of part of the operating space. In certain embodiments, the display of additional information relating to the function of interactive elements may be displayed based on context, such as when a specific action is required from a user in the virtual space.

Abstract: A rendering workload for an individual frame can be split between a head-mounted display (HMD) and a host computer that is executing an application. To split a rendering workload for a frame, the HMD may send head tracking data to the host computer, and the head tracking data may be used by the host computer to generate pixel data associated with the frame and extra data in addition to the pixel data. The extra data can include, without limitation, pose data, depth data, motion vector data, and/or extra pixel data. The HMD may receive the pixel data and at least some of the extra data, determine an updated pose for the HMD, and apply re-projection adjustments to the pixel data based on the updated pose and the received extra data to obtain modified pixel data, which is used to present an image on the display panel(s) of the HMD.

Abstract: Techniques are described for controlling display of video data and/or other image data based at least in part on selective mapping of pixel values to pixels. Such techniques may include separating a display panel into multiple regions, with at least one primary region having a highest resolution of displayed image data and with one or more secondary regions having one or more lower resolutions of displayed image data (e.g., by using a 1-to-M mapping of image pixel values to display panel pixels for each such secondary region, where M is greater than 1, such that each such image pixel value controls the display of M such pixels). The image data may further be encoded and optionally decoded in accordance with such a display panel arrangement, such as to encode the image data per the display panel arrangement to reduce its size before transmission to the display panel.

Abstract: Techniques are described for controlling image display via compression of image data in some image regions while performing less or no compression in other (e.g., peripheral view) regions, with color-specific compression preserving chromatic aberration compensation. Such techniques may be used with display panel(s) of a head-mounted display device used for virtual reality display. A primary region of an image at which to encode and display data at a highest resolution level may be determined by tracking a gaze of a user, while other secondary regions may be selected to be surrounding or other outside the primary region. In the secondary regions, image data for a first (e.g., green) color channel may be encoded at a first compression level for a first resolution level lower higher than for other second color channels, and HDR data may be compressed at higher compression levels than the color-specific data.

Abstract: The disclosure relates generally to techniques for using information about a user's actual or predicted pupil location for correcting optical distortions that are specific to an optical lens and display assembly through which the user is viewing one or more images. The described techniques may include identifying and mapping optical distortions specific to an optical lens and display assembly, and using such mapped optical distortions to correct images displayed to a wearer or other user receiving images via the assembly, such as based at least in part on pupil location of the wearer or other user. As one example, the one or more optical lens may be mounted inside a head-mounted display (HMD) that also includes a display panel or other image source for an eye of a wearer, and if so one or more pupil tracking mechanisms may be integrated into the HMD.

Abstract: The disclosure relates generally to techniques for using information about a user's actual or predicted pupil location for correcting optical distortions that are specific to an optical lens and display assembly through which the user is viewing one or more images. The described techniques may include identifying and mapping optical distortions specific to an optical lens and display assembly, and using such mapped optical distortions to correct images displayed to a wearer or other user receiving images via the assembly, such as based at least in part on pupil location of the wearer or other user. As one example, the one or more optical lens may be mounted inside a head-mounted display (HMD) that also includes a display panel or other image source for an eye of a wearer, and if so one or more pupil tracking mechanisms may be integrated into the HMD.

Abstract: Techniques are described for controlling display of video data and/or other image data based at least in part on selective mapping of pixel values to pixels. Such techniques may include separating a display panel into multiple regions, with at least one primary region having a highest resolution of displayed image data and with one or more secondary regions having one or more lower resolutions of displayed image data (e.g., by using a 1-to-M mapping of image pixel values to display panel pixels for each such secondary region, where M is greater than 1, such that each such image pixel value controls the display of M such pixels). The image data may further be encoded and optionally decoded in accordance with such a display panel arrangement, such as to encode the image data per the display panel arrangement to reduce its size before transmission to the display panel.

Abstract: Techniques are described for controlling image display via compression of image data in some image regions while performing less or no compression in other (e.g., peripheral view) regions, with color-specific compression preserving chromatic aberration compensation. Such techniques may be used with display panel(s) of a head-mounted display device used for virtual reality display. A primary region of an image at which to encode and display data at a highest resolution level may be determined by tracking a gaze of a user, while other secondary regions may be selected to be surrounding or other outside the primary region. In the secondary regions, image data for a first (e.g., green) color channel may be encoded at a first compression level for a first resolution level lower higher than for other second color channels, and HDR data may be compressed at higher compression levels than the color-specific data.

Abstract: The disclosure relates generally to techniques for using information about a user's actual or predicted pupil location for correcting optical distortions that are specific to an optical lens and display assembly through which the user is viewing one or more images. The described techniques may include identifying and mapping optical distortions specific to an optical lens and display assembly, and using such mapped optical distortions to correct images displayed to a wearer or other user receiving images via the assembly, such as based at least in part on pupil location of the wearer or other user. As one example, the one or more optical lens may be mounted inside a head-mounted display (HMD) that also includes a display panel or other image source for an eye of a wearer, and if so one or more pupil tracking mechanisms may be integrated into the HMD.

Abstract: Controller visualization systems and methods for use in virtual/augmented reality environments such as walk-around virtual reality environments are described. The virtual representation of a physical control device may be altered based on the context of the virtual environment. Certain embodiments combine the virtual rendering of the tracked control device with a real-time video representation of part of the operating space. In certain embodiments, the display of additional information relating to the function of interactive elements may be displayed based on context, such as when a specific action is required from a user in the virtual space.

Abstract: The disclosure relates generally to techniques for using information about a user's actual or predicted pupil location for correcting optical distortions that are specific to an optical lens and display assembly through which the user is viewing one or more images. The described techniques may include identifying and mapping optical distortions specific to an optical lens and display assembly, and using such mapped optical distortions to correct images displayed to a wearer or other user receiving images via the assembly, such as based at least in part on pupil location of the wearer or other user. As one example, the one or more optical lens may be mounted inside a head-mounted display (HMD) that also includes a display panel or other image source for an eye of a wearer, and if so one or more pupil tracking mechanisms may be integrated into the HMD.

Abstract: Eye-tracking systems and methods for use in consumer-class virtual reality (VR)/augmented reality (AR) applications, among other uses, are described. Certain embodiments combine optical eye tracking that uses camera-based pupil and corneal reflection detection with optical flow hardware running at a higher frequency. This combination provides the accuracy that can be attained with the former and at the same time adds the desirable precision and latency characteristics of the latter, resulting in a higher performing overall system at a relatively reduced cost. By augmenting a camera tracker with an array of optical flow sensors pointed at different targets on the visual field, one can perform sensor fusion to improve precision. Since the camera image provides an overall picture of eye position, that information can be used to cull occluded optical flow sensors, thus mitigating drift and errors due to blinking and other similar phenomena.

Abstract: Controller visualization systems and methods for use in virtual/augmented reality environments such as walk-around virtual reality environments are described. The virtual representation of a physical control device may be altered based on the context of the virtual environment. Certain embodiments combine the virtual rendering of the tracked control device with a real-time video representation of part of the operating space. In certain embodiments, the display of additional information relating to the function of interactive elements may be displayed based on context, such as when a specific action is required from a user in the virtual space.