Abstract: A processor accesses a depth map and a first image of a scene generated using one or more sensors of an artificial reality device. The processor generates, based on the first image, segmentation masks respectively associated with a plurality of object types. The segmentation masks segment the depth map into a plurality of segmented depth maps respectively associated with the object types. The processor generates meshes using, respectively, the segmented depth maps. For each eye of the user, the processor captures a second image and generates, based on the second image, segmentation information. The processor warps the plurality of meshes to generate warped meshes for the eye, and then generates an eye-specific mesh for the eye by compositing the warped meshes according to the segmentation information. The processor renders an output image for the eye using the second image and the eye-specific mesh.

Abstract: In particular embodiments, a computing system may receive a color image captured by a color camera and a monochrome image captured by a monochrome camera. The color camera and the monochrome camera are associated with an artificial reality system. The computing system may compute, for each of the color and monochrome images, histogram statistics and perform, based on the histogram statistics, tone map matching to normalize the monochrome image with respect to the color image. The computing system may perform local motion estimation to calculate motion vectors indicating pixel correspondence between the normalized monochrome image and the color image. The computing system may generate a mono-color merged image for display on the artificial reality system by adding, for each pixel in the normalized monochrome image, color information extracted from corresponding pixel in the color image using the motion vectors.

Abstract: In one embodiment, a system may capture one or more images of an object using one or more cameras. The system may detect features of the object depicted in the one or more images. The system may compute, based on the detected features in the one or more images, depth measurements for the object. The system may generate confidence scores associated with the depth measurements. The system may refine the computed depth measurements of the object by processing the depth measurements, the associated confidence scores, and at least one of the one or more images using a machine-learning model trained to refine depth measurements. The trained machine-learning model is configured to process one or more input images to refine the processed depth measurements in the input images.

Abstract: A depth camera assembly (DCA) determines depth information for a local area. The DCA includes a plurality of cameras and at least one illuminator. The DCA dynamically determines depth sensing modes (e.g., passive stereo, active stereo, structured stereo) based in part on the surrounding environment and/or user activity. The DCA uses the depth information to update a depth model describing the local area. The DCA may determine that a portion of the depth information associated with some of portion of the local area is not accurate. The DCA may then select a different depth sensing mode for the portion of the local area and update the depth model with the additional depth information. In some embodiments, the DCA may update the depth model by utilizing a machine learning model to generate a refined depth model.

Abstract: In one embodiment, a method includes accessing image-sensor data generated by the image sensor, where the image sensor has a color filter array with a pre-determined number of filter sets, and where each set of filters has a single color, splitting the image-sensor data into a pre-determined number of images, where each image corresponds to a portion of the image-sensor data associated with one of the sets of filters, compressing each of the images using an image compression algorithm, and sending the compressed images to a second computing device, where the second computing device is configured to create an output image based on the compressed images.

Abstract: In one embodiment, a method includes accessing image-sensor data generated by the image sensor, where the image sensor has a color filter array with a pre-determined number of filter sets, and where each set of filters has a single color, splitting the image-sensor data into a pre-determined number of images, where each image corresponds to a portion of the image-sensor data associated with one of the sets of filters, compressing each of the images using an image compression algorithm, and sending the compressed images to a second computing device, where the second computing device is configured to create an output image based on the compressed images.

Abstract: The disclosed display device may include (1) a display panel including pixel regions, (2) a backlight array coupled to the display panel that includes luminous elements, (3) a display housing configured to substantially prevent a user from referencing external brightness levels, (4) a display driver configured to receive an image including image blocks and scan the image to the display panel, and (5) a backlight driver configured to (a) determine an absolute brightness level of each of the image blocks, (b) derive, for each of the image blocks, a relative brightness level, (c) calculate, for each of the luminous elements, an illumination level based on the relative brightness level of a corresponding portion of the image blocks, and (d) illuminate, while the image is displayed via the display panel, each of the luminous elements according to the illumination level. Various other apparatus, systems, and methods are also disclosed.

Abstract: The disclosed display device may include (1) a display panel including pixel regions, (2) a backlight array coupled to the display panel that includes luminous elements, (3) a display housing configured to substantially prevent a user from referencing external brightness levels, (4) a display driver configured to receive an image including image blocks and scan the image to the display panel, and (5) a backlight driver configured to (a) determine an absolute brightness level of each of the image blocks, (b) derive, for each of the image blocks, a relative brightness level, (c) calculate, for each of the luminous elements, an illumination level based on the relative brightness level of a corresponding portion of the image blocks, and (d) illuminate, while the image is displayed via the display panel, each of the luminous elements according to the illumination level. Various other apparatus, systems, and methods are also disclosed.

Abstract: An imaging system is configured to identify an object represented by a plurality of preliminary images. The preliminary images are each associated with a different camera imaging channel, and include different image information. An object distance is determined based on the difference in preliminary image information. An object shift is determined based on the object distance and a pre-determined relationship between object shift and object distance. The object shift is applied to the portions of one or more preliminary images representing the object to form shifted preliminary images, and the shifted preliminary images are combined to form a final image.

Abstract: An imaging system is configured to identify an object represented by a plurality of preliminary images. The preliminary images are each associated with a different camera imaging channel, and include different image information. An object distance is determined based on the difference in preliminary image information. An object shift is determined based on the object distance and a pre-determined relationship between object shift and object distance. The object shift is applied to the portions of one or more preliminary images representing the object to form shifted preliminary images, and the shifted preliminary images are combined to form a final image.

Abstract: A method for performing white balancing of a digital image, the method includes: generating or receiving multiple candidates, each candidate represents a pixel area of the digital image; applying multiple white balancing estimation processes on the candidates to provide multiple digital image imbalance estimations; wherein a first white balancing estimation process differs from a second white balancing estimation process by its inherent reliability; calculating at least one white balancing compensation parameter in response to at least two digital image imbalance estimations; and providing a compensated digital image by applying at least one white balancing compensation image.

Abstract: A system for demosaicing an image that is acquired by a color sensor, the system includes: (i) an interface for receiving intensity values for pixels of the image; wherein the intensity value of each pixel is indicative of an amount of light that was detected by a chromatically filtered sensing element of the color sensor; and (ii) a processor configured to: (a) calculate at least one chrominance value for a pixel in response to intensity values of neighboring pixels; and (b) compute a luminance value for the pixel in response to the at least one chrominance value of the pixel and to an intensity value of the pixel; wherein the processor is further configured to carry out the calculating and the computing for each pixel of a group of pixels of the image.

Abstract: The subject matter discloses an apparatus, comprising a first resistive sheet and a second resistive sheet disposed in proximity to the first resistive sheet, such that pressure applied at a first touch point and at a second touch point on the first resistive sheet causes flow of electrical current between the first resistive sheet and the second resistive sheet. The apparatus further comprises a control unit coupled to a first terminal and to a second terminal, and configured to measure a first resistance between the first terminal and the second terminal; and configured to estimate a distance between the first touch point and the second touch point. The apparatus further estimates the location of the first touch point and the second touch point.