Abstract: Techniques for generating an aggregated set of features from multiple images generated by a camera are disclosed. A first image generated by the camera is accessed, where the first image was generated at a first time. A first set of features are identified from within the first image. A second image generated by the camera is accessed, where the second image is generated at a subsequent, second time. A second set of features are identified from within the second image. Movement data is obtained. This movement data details a movement of the camera between the first and second times. The movement data is used to reproject a pose embodied in the first image to correspond to a pose embodied in the second image. The embodiments aggregate the two sets of features to generate the aggregated set of features. The aggregated set of features for the camera are then cached.

Abstract: Techniques for performing foveated rendering are disclosed. A service accesses an original image of a scene. The service creates a binned image by down-sampling the original image. The service creates an un-binned image by cropping the original image. The service performs an image processing operation on the binned and/or the un-binned images. The service creates an up-sampled image by up-sampling the binned image. The service creates a cropped image by cropping the up-sampled image. Based on a light parameter of the scene, the service imposes a biasing function that biases an incorporation of pixels from one or both of the un-binned image and the cropped image, resulting in generation of a biased image. The service generates a foveated image by overlaying and aligning the biased image onto the up-sampled image.

Abstract: A method for data compression includes dividing a dataset of data elements into a plurality of blocks, such that each block includes a subset of data elements, and each block has a predetermined size. Compressed floating-point representations are calculated for the subset of data elements of each block. The compressed floating-point representations are calculated by, for each data element in a block, selecting a selected exponent from a set of two or more exponents, wherein two or more data elements of the block share the selected exponent; and for each data element in the block, calculating a mantissa value, such that the mantissa value and the selected exponent for the data element together comprise a compressed floating-point representation of the data element. The compressed floating-point representations for the subset of data elements of each block are stored in a compressed representation of the dataset.

Abstract: A system for regulating temporal filtering strength is configurable to: (i) obtain a light level indicator indicating light level associated with a real-world environment; (ii) determine a motion compensation confidence indicator using a current image and a previous image; (iii) determine a filter weight by processing the light level indicator and the motion compensation confidence indicator using a filter strength regulation module; and (iv) generate an output image by using at least the filter weight to filter a current frame with a previous frame.

Abstract: A system for generating a dark current residual image is configurable to generate a weighted average image by: (i) determining a region-based weight value for each pixel of an input image based upon a light level of a region in which the pixel lies; and (ii) combining the input image with a previous image using the region-based weight values for each pixel of the input image. The system is also configurable to generate a dark current residual image based upon the weighted average image.

Abstract: A system for providing zoomed representations of scene content is configurable to: (i) capture one or more images using one or more image sensors; (ii) generate display output for presentation on one or more displays, wherein the display output comprises at least a first region and a second region, wherein the first region depicts first scene content represented in the one or more images with a first zoom level, and wherein the second region depicts second scene content represented in the one or more images with a second zoom level, wherein the second zoom level is different from the first zoom level; and (iii) present the display output using the one or more displays, wherein, upon capturing the one or more images, generating the display output and presenting the display output occurs in real time or near real time.

Abstract: A system for generating foveated imagery is configurable to: (i) access an image; (ii) identify an ROI of the image; (iii) generate a downsampled image by performing downsampling on the image; (iv) generate a transformed downsampled image by performing one or more image processing operations on the downsampled image; (v) generate a transformed image by performing upsampling on the transformed downsampled image; and (vi) generate a foveated image by using the ROI of the image to define a corresponding ROI of the transformed image.

Abstract: Techniques for aligning images generated by an integrated camera physically mounted to an HMD with images generated by a detached camera physically unmounted from the HMD are disclosed. A 3D feature map is generated and shared with the detached camera. Both the integrated camera and the detached camera use the 3D feature map to relocalize themselves and to determine their respective 6 DOF poses. The HMD receives the detached camera's image of the environment and the 6 DOF pose of the detached camera. A depth map of the environment is accessed. An overlaid image is generated by reprojecting a perspective of the detached camera's image to align with a perspective of the integrated camera and by overlaying the reprojected detached camera's image onto the integrated camera's image.

Abstract: Techniques for correcting an overlay misalignment between an external camera image and a system camera image are disclosed. A first system camera image and a first external camera image are acquired. A first visual alignment is performed between those two images to produce an overlaid image. Some of the content in the overlaid image is surrounded by a bounding element. A position of the bounding element is modified based on movements of the system camera and/or the external camera. In response to performing a second visual alignment using new images, an update vector is computed. Relative movement between the two cameras is determined. Based on the movement and based on the update vector, the bounding element is progressively transitioned to a corrected position in the overlaid image. A speed by which the bounding element is progressively transitioned is proportional to the amount of movement.

Abstract: A system for generating depth information from low-resolution images is configured to access a plurality of image frames capturing an environment, identify a first group of image frames from the plurality of image frames, and generate a first image comprising a first composite image of the environment using the first group of image frames as input. The first composite image has an image resolution that is higher than an image resolution of the image frames of the first group of image frames. The system is also configured to obtain a second image of the environment, where parallax exists between a capture perspective associated with the first image and a capture perspective associated with the second image. The system is also configured to generate depth information for the environment based on the first image and the second image.

Abstract: A system for modifying a dark current image is configurable to receive an input image depicting a dark current state for one or more pixels of the input image, The dark current state for one or more pixels of the input image comprises one of: a faulty state or a non-faulty state. The system is configurable to partition the input image into a plurality of partitions and generate an updated input image by imposing at least one quantity constraint or at least one severity constraint to the plurality of partitions in association with at least one type of dark current state.

Abstract: A system for facilitating dark current compensation by weighted filtering is configurable to (i) receive an input dark current image; (ii) generate a corrected dark current image at least by scaling pixel values of the input dark current image based upon ambient light conditions; and generate a weight map comprising a weight value for each pixel of the corrected dark current image. For each pixel of the corrected dark current image, the weight value of the weight map may be based upon a light level of the pixel of the corrected dark current image. The system is also configurable to generate an output image by utilizing the weight map to filter an input image.

Abstract: An HMD is disclosed. The HMD includes a first pair of stereoscopic cameras of a first modality. The first pair of stereoscopic cameras includes a first camera and a second camera. The HMD includes a second pair of stereoscopic cameras of a second modality. The second pair of stereoscopic cameras includes a third camera and a fourth camera. The HMD includes a fifth camera of the first modality and a sixth camera of the second modality. A first separation distance between the first camera and the second camera is set to a user's IPD. A second separation distance between the third camera and the fourth camera is set to the user's IPD. The fifth camera and the sixth camera are positioned on the HMD between the first camera and the second camera and between the third camera and the fourth camera.

Abstract: Techniques for performing image alignment between a first image generated by a first camera and a second image generated by a second camera are disclosed. The image alignment is performed using staggered feature extraction in which a set of features are reused to align the second image with the first image. A first set of features are identified from within the first image, and a second set of features, which were previously detected within the second image, are accessed. The second set of features were previously used at least once to perform a previous image alignment operation. A current image alignment operation is performed by using the first set of features and by reusing the second set of features to align the first image with the second image.

Abstract: A system for facilitating dark current compensation by weighted filtering is configurable to (i) receive an input dark current image; (ii) generate a corrected dark current image at least by scaling pixel values of the input dark current image based upon ambient light conditions; and generate a weight map comprising a weight value for each pixel of the corrected dark current image. For each pixel of the corrected dark current image, the weight value of the weight map may be based upon a light level of the pixel of the corrected dark current image. The system is also configurable to generate an output image by utilizing the weight map to filter an input image.

Abstract: A system for modifying a dark current image is configurable to receive an input image depicting a dark current state for one or more pixels of the input image, The dark current state for one or more pixels of the input image comprises one of: a faulty state or a non-faulty state. The system is configurable to partition the input image into a plurality of partitions and generate an updated input image by imposing at least one quantity constraint or at least one severity constraint to the plurality of partitions in association with at least one type of dark current state.

Abstract: Disclosed herein are techniques for providing an illumination system that emits illumination into an environment while also enabling that system to be undetectable to certain types of external light detection systems. The system includes a single photon avalanche diode (SPAD) low light (LL) detection device and a light emitting device. The light emitting device provides illumination having a wavelength of at least 950 nanometers (nm). An intensity of the illumination is set to a level that causes the illumination to be undetectable from a determined distance away based on the roll off rate of the light. While the light emitting device is providing the illumination, the SPAD LL detection device generates an image of an environment in which the illumination is being provided.

Abstract: A system for power efficient image acquisition is configurable to capture, using an image sensor, a plurality of partial image frames including at least a first partial image frame and a second partial image frame. The first partial image frame is captured at a first timepoint using a first subset of image sensing pixels of the plurality of image sensing pixels of the image sensor. The second partial image frame is captured at a second timepoint using a second subset of image sensing pixels of the plurality of image sensing pixels of the image sensor. The second subset of image sensing pixels includes different image sensing pixels than the first subset of image sensing pixels, and the second timepoint is temporally subsequent to the first timepoint. The system is configurable to generate a composite image frame based on the plurality of partial image frames.

Abstract: A system for updating continuous image alignment of separate cameras identifies a previous alignment matrix associated with a previous frame pair captured at one or more previous timepoints by a reference camera and a match camera. The previous alignment matrix is based on visual correspondences in the previous frame pair. The system also identifies a current matrix associated with a current frame pair captured at one or more current timepoints by the reference camera and the match camera. The current matrix is based on visual correspondences in the current frame pair. The system also identifies a difference value associated with the reference camera or the match camera relative to the one or more previous timepoints and the one or more current timepoints. The system also generates an updated alignment matrix by using the previous alignment matrix, the current matrix, and the difference value as inputs.

Abstract: A system for generating foveated imagery is configurable to: (i) access an image; (ii) identify an ROI of the image; (iii) generate a downsampled image by performing downsampling on the image; (iv) generate a transformed downsampled image by performing one or more image processing operations on the downsampled image; (v) generate a transformed image by performing upsampling on the transformed downsampled image; and (vi) generate a foveated image by using the ROI of the image to define a corresponding ROI of the transformed image.