Abstract: Video presence systems are described that detect an area of interest (e.g., facial region) within captured image data and analyze the area of interest using known color information of the content currently being presented by a display, along with measured ambient light color information from a color sensor, to determine whether the area of interest (e.g., face) is currently illuminated by the display or whether the AOI is not affected by the display and instead only illuminated by the ambient light. Upon determining that the display casts color shade on the AOI, the video presence system pre-processes the image data of the detected area of interest to correct the color back to skin color under ambient light prior to performing general white balance correction.

Abstract: A method for using cameras in an augmented reality headset is provided. The method includes receiving a signal from a sensor mounted on a headset worn by a user, the signal being indicative of a user intention for capturing an image. The method also includes identifying the user intention for capturing the image, based on a model to classify the signal from the sensor according to the user intention, selecting a first image capturing device in the headset based on a specification of the first image capturing device and the user intention for capturing the image, and capturing the image with the first image capturing device. An augmented reality headset, a memory storing instructions, and a processor to execute the instructions to cause the augmented reality headset as above are also provided.

Abstract: A display assembly generates environmentally matched virtual content for an electronic display. The display assembly includes a display controller and a display. The display controller is configured to estimate environmental matching information for a target area within a local area based in part on light information received from a light sensor. The target area is a region for placement of a virtual object. The light information describes light values. The display controller generates display instructions for the target area based in part on a human vision model, the estimated environmental matching information, and rendering information associated with the virtual object. The display is configured to present the virtual object as part of artificial reality content in accordance with the display instructions. The color and brightness of the virtual object is environmentally matched to the portion of the local area surrounding the target area.

Abstract: Systems and methods for an ambient light sensor (ALS) disposed under a display layer. In some implementations, an ALS reading is adjusted by compensating for light leakage in a display environment. In some aspects, a display leakage component may be determined based on image content driven onto a display pixel array during the ALS reading. An ambient light measurement may be generated by subtracting the display leakage component from the ALS reading and then adjusting a display brightness of the display pixel array in response to the ambient light measurement.

Abstract: A display assembly generates environmentally matched virtual content for an electronic display. The display assembly includes a display controller and a display. The display controller is configured to estimate environmental matching information for a target area within a local area based in part on light information received from a light sensor. The target area is a region for placement of a virtual object. The light information describes light values. The display controller generates display instructions for the target area based in part on a human vision model, the estimated environmental matching information, and rendering information associated with the virtual object. The display is configured to present the virtual object as part of artificial reality content in accordance with the display instructions. The color and brightness of the virtual object is environmentally matched to the portion of the local area surrounding the target area.

Abstract: A display assembly generates environmentally matched virtual content for an electronic display. The display assembly includes a display controller and a display. The display controller is configured to estimate environmental matching information for a target area within a local area based in part on light information received from a light sensor. The target area is a region for placement of a virtual object. The light information describes light values. The display controller generates display instructions for the target area based in part on a human vision model, the estimated environmental matching information, and rendering information associated with the virtual object. The display is configured to present the virtual object as part of artificial reality content in accordance with the display instructions. The color and brightness of the virtual object is environmentally matched to the portion of the local area surrounding the target area.

Abstract: A device may identify a first pixel array representing a high dynamic range (HDR) image of a scene. The device may determine one or more image statistics associated with the HDR image based on the first pixel array. The device may generate a corrected pixel array by applying an interpolative operation to the first pixel array, where the image statistics may be determined based on the corrected pixel array. The device may generate one or more tone-mapping curves for the HDR image based on the image statistics. The device may generate a compressed image of the scene by applying the one or more tone-mapping curves to the first pixel array. The device may output the compressed image of the scene (e.g., to a display of the device, to a system memory, etc.).

Abstract: According to various embodiments, the system and method disclosed herein process light-field image data so as to mitigate lens flare effects. A light-field image may be captured with a light-field image capture device with a microlens array and received in a data store. A plurality of flare-affected pixels may be identified in the light-field image. The flare-affected pixels may have flare-affected pixel values. Flare-corrected pixel values may be generated for the flare-affected pixels. Relative to the flare-affected pixel values, the flare-corrected pixel values may at least partially remove the lens flare effects. The flare-corrected pixel values may be used to generate a corrected light-field image in which the lens flare effects are at least partially corrected. The corrected light-field image may be displayed on a display screen.

Abstract: According to various embodiments, the system and method disclosed herein process light-field image data so as to mitigate lens flare effects. A light-field image may be captured with a light-field image capture device with a microlens array and received in a data store. A plurality of flare-affected pixels may be identified in the light-field image. The flare-affected pixels may have flare-affected pixel values. Flare-corrected pixel values may be generated for the flare-affected pixels. Relative to the flare-affected pixel values, the flare-corrected pixel values may at least partially remove the lens flare effects. The flare-corrected pixel values may be used to generate a corrected light-field image in which the lens flare effects are at least partially corrected. The corrected light-field image may be displayed on a display screen.

Abstract: According to various embodiments, the system and method disclosed herein process light-field image data so as to mitigate lens flare effects. A light-field image may be captured with a light-field image capture device with a microlens array and received in a data store. A plurality of flare-affected pixels may be identified in the light-field image. The flare-affected pixels may have flare-affected pixel values. Flare-corrected pixel values may be generated for the flare-affected pixels. Relative to the flare-affected pixel values, the flare-corrected pixel values may at least partially remove the lens flare effects. The flare-corrected pixel values may be used to generate a corrected light-field image in which the lens flare effects are at least partially corrected. The corrected light-field image may be displayed on a display screen.

Abstract: According to various embodiments, the system and method disclosed herein process light-field image data so as to mitigate lens flare effects. A light-field image may be captured with a light-field image capture device with a microlens array and received in a data store. A plurality of flare-affected pixels may be identified in the light-field image. The flare-affected pixels may have flare-affected pixel values. Flare-corrected pixel values may be generated for the flare-affected pixels. Relative to the flare-affected pixel values, the flare-corrected pixel values may at least partially remove the lens flare effects. The flare-corrected pixel values may be used to generate a corrected light-field image in which the lens flare effects are at least partially corrected. The corrected light-field image may be displayed on a display screen.

Abstract: A method determines a pixel value in a high dynamic range image from two images of different brightness by obtaining corresponding input pixel intensities from the two images, determining combination weights, and calculating the pixel value in the high dynamic range image as a weighted average of the input pixel intensities. Another method determines a pixel value in a high dynamic range image from more than two images by forming pairs of corresponding input pixel intensities, determining relative combination weights for the input pixels intensities for each pair, applying a normalization condition to determine absolute combination weights, and calculating the pixel value in the high dynamic range image as a weighted average of the input pixel intensities. Systems for generating high dynamic range image generation from two or more input images include a processor, a memory, a combination weight module, and a pixel value calculation module.

Abstract: A method of reading pixel data from a pixel array includes exposing each one of a plurality of regions of pixels a respective exposure time. Pixel data is read from the plurality of regions of pixels. The pixel data is interpolated from a first one of the plurality of regions of pixels to determine the pixel data of the regions of pixels other than the first one of the plurality of regions of pixels to generate a first image having the first exposure time. The pixel data is interpolated from the second one of the plurality of regions of pixels to determine the pixel data of the regions of pixels other than the second one of the plurality of regions to generate a second image having the second exposure time. The images are combined to produce a high dynamic range image.

Abstract: An image such as a light-field image may be displayed dynamically through the use of an automatically generated animation. The image may be received in a data store. One or more attributes of the image may be automatically evaluated. Based on the one or more attributes, a first animation parameter may be selected. An animation of the image may be automatically generated such that the animation possesses the first animation parameter. On a display device, the animation may be displayed. The one or more attributes may optionally include coloration of the image, presence of a computer-recognizable feature in the image, presence of a computer-recognizable human face in the image, a gaze direction of a person appearing in the image, and/or a depth, relative to the camera, of an object appearing in the image.

Abstract: An image sensor for three-dimensional (“3D”) imaging includes a first, a second, and a third pixel unit, where the second pixel unit is disposed between the first and third pixel units. Optical filters included in the pixel units are disposed on a light incident side of the image sensor to filter polarization-encoded light having a first polarization and a second polarization to photosensing regions of the pixel units. The first pixel unit includes a first optical filter having the first polarization, the second pixel unit includes a second optical filter having the second polarization, and the third pixel unit includes a third optical filter having the first polarization.

Abstract: An image data aggregating high dynamic range imaging system includes an image sensor for generating N image data sets from an array of photodiodes, where N is an integer greater than one. The image sensor is adapted to generate each of the N image data sets with a different respective exposure time duration of the array of photodiodes. The system further includes an image data aggregating module for aggregating the N image data sets to obtain a virtual long exposure image data set.

Abstract: An image data aggregating high dynamic range imaging system includes an image sensor for generating N image data sets from an array of photodiodes, where N is an integer greater than one. The image sensor is adapted to generate each of the N image data sets with a different respective exposure time duration of the array of photodiodes. The system further includes an image data aggregating module for aggregating the N image data sets to obtain a virtual long exposure image data set.

Abstract: A method determines a pixel value in a high dynamic range image from two images of different brightness by obtaining corresponding input pixel intensities from the two images, determining combination weights, and calculating the pixel value in the high dynamic range image as a weighted average of the input pixel intensities. Another method determines a pixel value in a high dynamic range image from more than two images by forming pairs of corresponding input pixel intensities, determining relative combination weights for the input pixels intensities for each pair, applying a normalization condition to determine absolute combination weights, and calculating the pixel value in the high dynamic range image as a weighted average of the input pixel intensities. Systems for generating high dynamic range image generation from two or more input images include a processor, a memory, a combination weight module, and a pixel value calculation module.

Abstract: A method of reading pixel data from a pixel array includes exposing each one of a plurality of regions of pixels a respective exposure time. Pixel data is read from the plurality of regions of pixels. The pixel data is interpolated from a first one of the plurality of regions of pixels to determine the pixel data of the regions of pixels other than the first one of the plurality of regions of pixels to generate a first image having the first exposure time. The pixel data is interpolated from the second one of the plurality of regions of pixels to determine the pixel data of the regions of pixels other than the second one of the plurality of regions to generate a second image having the second exposure time. The images are combined to produce a high dynamic range image.

Abstract: An image sensor includes an array of light sensitive elements and a filter array. Each filter element is in optical communication with a respective light sensitive element. The image sensor receives filtered light having a repeating pattern. Light sensitive elements in at least two successive rows alternately receive light having a first color and a second color, and light sensitive elements in common columns of the successive rows alternately receive light having the first color and the second color. Light sensitive elements in at least two additional successive rows alternately receive light having a third and a fourth color, and light sensitive elements in common columns of the additional successive rows alternately receive light having the third color and the fourth color. Output values of pairs of sampled light sensitive elements receiving light of a common color and from successive rows are combined to generate a down-sampled image.