Abstract: Techniques for generating a temporally filtered image designed to compensate for global motions of a camera and to compensate for local motions of an object are disclosed. A history frame and a current frame are acquired. A global motion compensation operation is performed on the history frame to reproject a pose of the history frame to match the pose of the current frame. The history frame is compared against the current frame to identify pixels that represent moving objects. For each of those pixels, an optical flow vector is computed. The optical flow vectors are then applied to those pixels to shift those pixels to new locations. These new positions, which are in the history frame, correspond to positions that were identified in the current frame. Afterwards, the current frame is temporally filtered with the history frame.

Abstract: A system for regulating temporal filtering strength is configurable to: (i) obtain a light level indicator indicating light level associated with a real-world environment; (ii) determine a motion compensation confidence indicator using a current image and a previous image; (iii) determine a filter weight by processing the light level indicator and the motion compensation confidence indicator using a filter strength regulation module; and (iv) generate an output image by using at least the filter weight to filter a current frame with a previous frame.

Abstract: A system for adding persistence to SPAD imagery is configurable to capture, using a SPAD array, a plurality of image frames. The system is configurable to capture, using an IMU, pose data associated with the plurality of image frames. The pose data includes at least respective pose data associated with each of the plurality of image frames. The system is configurable to determine a persistence term based on the pose data. The system is also configurable to generate a composite image based on the plurality of image frames, the respective pose data associated with each of the plurality of image frames, and the persistence term. The persistence term defines a contribution of each of the plurality of image frames to the composite image.

Abstract: Techniques for aligning image content are disclosed. The alignment occurs by concurrently identifying corner and line features and by aligning the images based on those features. As a result, relaxed constraints can be used during the alignment process as compared to constraints that are required when aligning images using only corner features. To achieve these benefits, first and second images are acquired. A first set of pixels that correspond to corner features are detected within the images. In parallel, a second set of pixels that correspond to line features are detected within the images. Image content is then aligned from the two images. The aligning process is performed using at least some of the pixels from the first and second sets.

Abstract: A system for obtaining color imagery using SPADs includes a SPAD array that has a plurality of SPAD pixels. Each of the plurality of SPAD pixels includes a respective color filter positioned thereover. The system is configurable to capture an image frame using the SPAD array and generate a filtered image by performing a temporal filtering operation using the image frame and at least one preceding image frame. The at least one preceding image frame is captured by the SPAD array at a timepoint that temporally precedes a timepoint associated with the image frame. The system is also configurable to, after performing the temporal filtering operation, generate a color image by demosaicing the filtered image.

Abstract: A system for facilitating intensity image capture and time of flight capture. The system includes an image sensor array comprising a plurality of image sensor pixels, one or more processors, and one or more hardware storage devices storing instructions that are executable by the one or more processors to configure the system to facilitate intensity image capture and time of flight capture by configuring the system to perform interleaved intensity image capture and time of flight capture operations using the image sensor array.

Abstract: A system for generating high-resolution video from low-resolution images is configured to access a first video stream and a second video stream capturing an environment. The first video stream is captured by a first video capture device. The second video stream is captured by a second video capture device. Image frames of the first video stream are temporally synchronized with corresponding image frames of the second video stream. The system is also configured to generate a composite video stream with a higher resolution than the first or second video streams. Each composite image frame of the composite video stream is generated using a respective image frame of the first video stream and a temporally synchronized corresponding image frame of the second video stream as input.

Abstract: A system for facilitating tone mapping is configurable to (i) generate an input image histogram based on one or more input images, the input image histogram comprising a plurality of bins, wherein each bin of the plurality of bins is associated with one or more pixel values and indicates a quantity of pixels that comprise the one or more pixel values; (ii) generate a target histogram based at least on the input image histogram; and (iii) generate an output image by using the target histogram to map pixel values of at least one of the one or more input images to corresponding pixel values in the output image.

Abstract: A system for facilitating motion compensation is configurable to access an affine transformation-compensated image. The affine transformation-compensated image is generated by applying affine transformation-based motion compensation to a previous image. The system is further configurable to generate a motion-compensated image by applying optical flow-based motion compensation to the affine transformation-compensated image. The optical flow-based motion compensation utilizes the affine transformation-compensated image and a current image as inputs.

Abstract: A system for facilitating dark current compensation by weighted filtering is configurable to (i) receive an input dark current image; (ii) generate a corrected dark current image at least by scaling pixel values of the input dark current image based upon ambient light conditions; and generate a weight map comprising a weight value for each pixel of the corrected dark current image. For each pixel of the corrected dark current image, the weight value of the weight map may be based upon a light level of the pixel of the corrected dark current image. The system is also configurable to generate an output image by utilizing the weight map to filter an input image.

Abstract: A system for modifying a dark current image is configurable to receive an input image depicting a dark current state for one or more pixels of the input image, The dark current state for one or more pixels of the input image comprises one of: a faulty state or a non-faulty state. The system is configurable to partition the input image into a plurality of partitions and generate an updated input image by imposing at least one quantity constraint or at least one severity constraint to the plurality of partitions in association with at least one type of dark current state.

Abstract: A system for generating a dark current residual image is configurable to generate a weighted average image by: (i) determining a region-based weight value for each pixel of an input image based upon a light level of a region in which the pixel lies; and (ii) combining the input image with a previous image using the region-based weight values for each pixel of the input image. The system is also configurable to generate a dark current residual image based upon the weighted average image.

Abstract: A system for low compute high-resolution depth map generation using low-resolution cameras is configured to obtain a stereo pair of images and generate a depth map by performing stereo matching on the stereo pair of images. The system is also configured to obtain a first image comprising first texture information for the environment that has a first image resolution that is higher than an image resolution of images of the stereo pair of images. The system is further configured to generate a reprojected first image by reprojecting the first image to correspond to an image capture perspective associated with the depth map. The reprojection of the first image is based on depth information from the depth map and includes reprojected first texture information for the environment. The system is also configured to generate an upsampled depth map based on the depth map.

Abstract: A system for power efficient image acquisition is configurable to capture, using an image sensor, a plurality of partial image frames including at least a first partial image frame and a second partial image frame. The first partial image frame is captured at a first timepoint using a first subset of image sensing pixels of the plurality of image sensing pixels of the image sensor. The second partial image frame is captured at a second timepoint using a second subset of image sensing pixels of the plurality of image sensing pixels of the image sensor. The second subset of image sensing pixels includes different image sensing pixels than the first subset of image sensing pixels, and the second timepoint is temporally subsequent to the first timepoint. The system is configurable to generate a composite image frame based on the plurality of partial image frames.

Abstract: A computing system includes an illumination light source configured to emit illumination light into an external environment and an orientation sensor configured to estimate an orientation of the illumination light source relative to the external environment. The computing system includes a logic subsystem and a storage subsystem holding instructions executable by the logic subsystem to define a light restriction zone within the external environment. Based at least in part on the orientation of the illumination light source, the illumination light source is dynamically controlled to direct the illumination light toward at least a portion of the external environment outside the light restriction zone, while mitigating emission of the illumination light into the light restriction zone.

Abstract: Examples are disclosed that relate to providing image data to a user in a defined space of a surrounding environment from a perspective of the user. One example provides a computing system, comprising a logic subsystem, and a storage subsystem comprising instructions executable by the logic subsystem to obtain information regarding a pose of a user within a defined space, based upon the pose of the user, determine a portion of an environment surrounding the defined space toward which the user is looking, obtain image data representing the portion of the environment from a perspective of the user, and provide the image data for display via a display device within the defined space.

Abstract: A system for structured light depth computation using single photon avalanche diodes (SPADs) is configurable to, over a frame capture time period, selectively activate the illuminator to perform interleaved structured light illumination operations. The interleaved structured light illumination operations comprise alternately emitting at least a first structured light pattern from the illuminator and emitting at least a second structured light pattern from the illuminator. The system is also configurable to, over the frame capture time period, perform a plurality of sequential shutter operations to configure each SPAD pixel of the SPAD array to enable photon detection. The plurality of sequential shutter operations generates, for each SPAD pixel of the SPAD array, a plurality of binary counts indicating whether a photon was detected during each of the plurality of sequential shutter operations.

Abstract: Techniques for generating an enhanced image. A first image is generated using a camera of a first modality, and a second image is generated using a camera of a second modality. Pixels that are common between the two images are identified. An alpha map is generated. The alpha map reflects edge detection weights that are computed for the common pixels based on saliency values. A determination is made as to how much texture from the images to use to generate an enhanced image. This determination is based on the edge detection weights included within the alpha map. Based on the edge detection weights, textures are merged from the common pixels to generate the enhanced image. Color is also added to the enhanced image, where the color reflects an additional property (e.g., the texture source for the pixel) that is associated with one or both of the images.

Abstract: A system for low compute high-resolution depth map generation using low-resolution cameras is configured to obtain a stereo pair of images and generate a depth map by performing stereo matching on the stereo pair of images. The system is also configured to obtain a first image comprising first texture information for the environment that has a first image resolution that is higher than an image resolution of images of the stereo pair of images. The system is further configured to generate a reprojected first image by reprojecting the first image to correspond to an image capture perspective associated with the depth map. The reprojection of the first image is based on depth information from the depth map and includes reprojected first texture information for the environment. The system is also configured to generate an upsampled depth map based on the depth map.

Abstract: Techniques for using motion data to generate a high resolution output color image from multiple images having sparse color information are disclosed. A camera generates multiple images. The camera's sensor is configured to have a sparse Bayer pattern. While the camera is generating the images, IMU data for each image is acquired. The IMU data indicates a corresponding pose the camera was in while the camera generated each image. The images and the IMU data are fed as input into a motion model. The motion model performs temporal filtering on the images and uses the IMU data to generate a red-only image, a green-only image, and a blue-only image. A high resolution output color image is generated by combining the red-only image, the green-only image, and the blue-only image.