Abstract: Image data is received for rendering an image on an image display to a viewer (402). The image data specifies a pixel value of the image for a pixel of the image display to render. The pixel value for the pixel includes multiple component pixel values corresponding to multiple color components of a color space. A color gamut locational value of the pixel value is computed based on two or more component pixel values in the multiple component pixel values of the pixel value specified for the pixel (404). The color gamut locational value is used to determine whether bandwidth broadening is to be applied to image rendering light produced by the pixel of the image display to render the pixel value (406). The image rendering light is directed to the viewer.

Abstract: Information relating to gaze direction is used, e.g., with other information, to perform tone mapping, brightness adaptation, and/or sparkle generation/management for virtual or augmented reality displays. For example, speed of light adaptation may be increased as a user's gaze moves from a darker scene to a brighter scene and decreased in opposite direction to match human perception dynamics in real world. Sparkles from smaller objects may be adjusted depending on an object size, a brightness of the object's background, and/or the object's location relative to the user's gaze—that is, whether the object in the user's focus or in a periphery. Specularity of the objects may be obtained from a specularity map (e.g., metadata associated with each object) or from image processing a scene. Local sparkles may be shimmered as the user's head rotates deterministically or randomly to provide a sense of real-life glistening.

Abstract: A sequence of base layer images for a base reference image display and a set of one or more sequences of beta scale maps corresponding to one or more non-base reference image displays are generated. A subset of one or more sequences of beta scale maps is determined in the set of one or more sequences of beta scale maps based at least in part on display capabilities of a target image display. The sequence of base layer images, along with the subset of one or more sequences of beta scale maps, is encoded into a video signal to cause a recipient device of the video signal to generate a sequence of display images from the sequence of base layer images and the subset of one or more sequences of beta scale maps for rendering on the image display.

Abstract: Input texture images are received. Depth maps are determined for the input texture images. Each depth map in the depth maps includes depth values of pixels represented in a corresponding input texture image in the input texture images. Depth difference maps are generated from the depth maps. The depth difference maps include at least one depth difference map that is generated from two successive depth maps in the depth maps. A video signal encoded with a compressed version of the input texture images and the depth difference maps is outputted. The video signal causes a recipient device of the video signal to generate display images from the compressed version of the input texture images and the depth difference maps for rendering on an image display.

Abstract: A method for improved image stabilization may include shifting image data from a memory device integrated with a display device, where the image data represents motion of the display device and corresponds with a time of capture of the image data; analyzing the image data via a processor to determine a stabilization correction to apply to the image data, analyzing including, determining the stabilization correction to correct for a displacement of at least a portion of the image data; applying the determined stabilization correction to the image data stored in the memory device to achieve stabilized image data; and outputting the stabilized image data from the display device.

Abstract: A multi-view image stream encoded with primary and secondary image is accessed. Each primary image stream comprises groups of pictures (GOPs). Each secondary image stream comprises I-frames generated from a corresponding primary image stream. Viewpoint data collected in real time is received from a recipient decoding device to indicate that the viewer's viewpoint has changed from a specific time point. A camera is selected based on the viewer's changed viewpoint. It is determined whether the specific time point corresponds to a non-I-frame in a GOP of a primary image stream of the selected camera. If so, an I-frame from a secondary image stream corresponding to the primary image stream is transmitted to the recipient decoding device.

Abstract: The disclosed computer-implemented method may include receiving a pulse width value corresponding to a pulse for activating a pixel of a color channel for a frame and dividing the pulse into a plurality of sub-pulses. The method may also include activating the pixel for the frame based on the plurality of sub-pulses. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: Saliency regions are identified in a global scene depicted by volumetric video. Saliency video streams that track the saliency regions are generated. Each saliency video stream tracks a respective saliency region. A saliency stream based representation of the volumetric video is generated to include the saliency video streams. The saliency stream based representation of the volumetric video is transmitted to a video streaming client.

Abstract: Images of a user's head are acquired at a plurality of different orientational angles through image sensors operating in conjunction with a media consumption system. The acquired images of the user's head are used to select or predict a specific personalized head related transfer function for the user. Spatial audio rendered by audio speakers operating in conjunction with the media consumption system is adjusted or modified based at least in part on the specific personalized HRTF selected for the user.

Abstract: Images of an actual rendering environment are acquired through image sensors operating in conjunction with a media consumption system. The acquired images of the actual rendering environment are used to predict audio characteristics of objects present in the actual rendering environment. Spatial audio rendered, to a user in the actual rendering environment, by audio speakers operating in conjunction with the media consumption system is adjusted or modified based at least in part on the audio characteristics of the objects present in the actual rendering environment.

Abstract: Techniques disclosed herein relate generally to modifying images to reduce display power while maintaining visual fidelity of the displayed images. In one example, a computer-implemented method includes receiving an input image to be displayed by a type of display device, obtaining a machine learning model that is trained to edit display content to reduce power consumption of displaying the edited display content by the type of display device while maintaining a visual fidelity of the edited display content, applying the machine learning model to the input image to generate an output image, and displaying the output image via a display device of the type of display device.

Abstract: An eyewear device comprises a left lens assembly and a right lens assembly. The left lens assembly includes a left focus tunable lens and a left focus fixed lens. A right lens assembly includes a right focus tunable lens and a right focus fixed lens. The eyewear device may be used in 3D display applications, virtual reality applications, augmented reality applications, remote presence applications, etc. The eyewear device may also be used as vision correction glasses.

Abstract: An optical element includes a waveguide body that is configured to guide light by total internal reflection from an input end to an output end, an input coupling element located at the input end for coupling light into the waveguide body, and an output coupling element located at the output end for coupling light out of the waveguide body, where the waveguide body includes an organic solid crystal. The organic solid crystal may be a single crystal having principal axes rotated with respect to the dimensions of the waveguide body. Such an optical element may have low weight and exhibit good color uniformity while providing polarization scrambling of the guided light.

Abstract: A multi-view input image covering multiple sampled views is received. A multi-view layered image stack is generated from the multi-view input image. A target view of a viewer to an image space depicted by the multi-view input image is determined based on user pose data. The target view is used to select user pose selected sampled views from among the multiple sampled views. Layered images for the user pose selected sampled views, along with alpha maps and beta scale maps for the user pose selected sampled views are encoded into a video signal to cause a recipient device of the video signal to generate a display image for rendering on the image display.

Abstract: A sequence of base layer images for a base reference image display and a set of one or more sequences of beta scale maps corresponding to one or more non-base reference image displays are generated. A subset of one or more sequences of beta scale maps is determined in the set of one or more sequences of beta scale maps based at least in part on display capabilities of a target image display. The sequence of base layer images, along with the subset of one or more sequences of beta scale maps, is encoded into a video signal to cause a recipient device of the video signal to generate a sequence of display images from the sequence of base layer images and the subset of one or more sequences of beta scale maps for rendering on the image display.

Abstract: At a first time point, a first light capturing device at a first spatial location in a three-dimensional (3D) space captures first light rays from light sources located at designated spatial locations on a viewer device in the 3D space. At the first time point, a second light capturing device at a second spatial location in the 3D space captures second light rays from the light sources located at the designated spatial locations on the viewer device in the 3D space. Based on the first light rays captured by the first light capturing device and the second light rays captured by the second light capturing device, at least one of a spatial position and a spatial direction, at the first time point, of the viewer device is determined.

Abstract: Image data is received for rendering an image on an image display to a viewer (402). The image data specifies a pixel value of the image for a pixel of the image display to render. The pixel value for the pixel includes multiple component pixel values corresponding to multiple color components of a color space. A color gamut locational value of the pixel value is computed based on two or more component pixel values in the multiple component pixel values of the pixel value specified for the pixel (404). The color gamut locational value is used to determine whether bandwidth broadening is to be applied to image rendering light produced by the pixel of the image display to render the pixel value (406). The image rendering light is directed to the viewer.

Abstract: A communication client device operated by a first user in a communication session receives a viewing direction tracking data portion indicating a view direction of a second user in the communication session. It is determined that the view direction of the second user is towards a third user at a first time point in the communication session. The view direction of the second user is used to modify a pre-adapted visual depiction of the second user into an adapted visual depiction of the second user. The adapted visual depiction of the second user is rendered, to the first user, on an image display operating with the communication client device.

Abstract: Input texture images are received. Depth maps are determined for the input texture images. Each depth map in the depth maps includes depth values of pixels represented in a corresponding input texture image in the input texture images. Depth difference maps are generated from the depth maps. The depth difference maps include at least one depth difference map that is generated from two successive depth maps in the depth maps. A video signal encoded with a compressed version of the input texture images and the depth difference maps is outputted. The video signal causes a recipient device of the video signal to generate display images from the compressed version of the input texture images and the depth difference maps for rendering on an image display.

Abstract: Techniques for enhancing brightness and power efficiency in augmented reality (AR) display devices are described. To enhance brightness and power efficiency a near-eye display device's duty cycle may be adjusted based on a type of usage (e.g., world-locked or head-locked); a persistence or a display rate may be varied based on head movement speed; the brightness may be adjusted based on sensed ambient light; ambient brightness may be matched by utilizing camera-based scene understanding determining a user's gaze direction and then decreasing brightness of the gaze's peripheral; the display brightness may be controlled based on gaze and/or eye motion; and/or if the user is moving, world-locked rendering (WLR) targets and/or refresh rates may be altered, or WLR varied for peripheral content based on a comparison of gaze direction and a virtual object location.