Abstract: A system for facilitating intensity image capture and time of flight capture. The system includes a single photon avalanche diode (SPAD) array comprising a plurality of SPAD pixels, one or more processors, and one or more hardware storage devices storing instructions that are executable by the one or more processors to configure the system to facilitate intensity image capture and time of flight capture by configuring the system to perform interleaved intensity image capture and time of flight capture operations using the SPAD array.

Abstract: Examples are described that relate to correcting line bias in images. One example provides a method comprising receiving, from an imaging device, a plurality of images each comprising a plurality of lines of pixels. The method further comprises, for each image of the plurality of images, for each line of pixels of the plurality of lines of pixels, based at least on one or more pixel values of one or more pixels in the line of pixels, determining a line bias correction for the line, and applying the line bias correction to each pixel in the line, the line bias correction comprising an offset applied to each pixel value in the line of pixels.

Abstract: A computing system includes an illumination light source configured to emit illumination light into an external environment and an orientation sensor configured to estimate an orientation of the illumination light source relative to the external environment. The computing system includes a logic subsystem and a storage subsystem holding instructions executable by the logic subsystem to define a light restriction zone within the external environment. Based at least in part on the orientation of the illumination light source, the illumination light source is dynamically controlled to direct the illumination light toward at least a portion of the external environment outside the light restriction zone, while mitigating emission of the illumination light into the light restriction zone.

Abstract: Examples are disclosed that relate to providing image data to a user in a defined space of a surrounding environment from a perspective of the user. One example provides a computing system, comprising a logic subsystem, and a storage subsystem comprising instructions executable by the logic subsystem to obtain information regarding a pose of a user within a defined space, based upon the pose of the user, determine a portion of an environment surrounding the defined space toward which the user is looking, obtain image data representing the portion of the environment from a perspective of the user, and provide the image data for display via a display device within the defined space.

Abstract: A system for obtaining color imagery using SPADs includes a SPAD array that has a plurality of SPAD pixels. Each of the plurality of SPAD pixels includes a respective color filter positioned thereover. The system is configurable to capture an image frame using the SPAD array and generate a filtered image by performing a temporal filtering operation using the image frame and at least one preceding image frame. The at least one preceding image frame is captured by the SPAD array at a timepoint that temporally precedes a timepoint associated with the image frame. The system is also configurable to, after performing the temporal filtering operation, generate a color image by demosaicing the filtered image.

Abstract: A system for obtaining dark current images includes one or more processors and one or more hardware storage devices storing instructions that are executable by the one or more processors to configure the system to perform various acts. The acts include obtaining a first image frame, generating a first low-pass filtered image by applying a low-pass filter to the first image frame, and generating a first estimated dark current image by subtracting the first low-pass filtered image from the first image frame.

Abstract: A system for dark current compensation in SPAD imagery is configurable to capture an image frame with the SPAD array and generate a temporally filtered image by performing a temporal filtering operation using the image frame and at least one preceding image frame. The at least one preceding image frame is captured by the SPAD array at a timepoint that temporally precedes a timepoint associated with the image frame. The system is also configurable to obtain a dark current image frame. The dark current image frame includes data indicating one or more SPAD pixels of the plurality of SPAD pixels that detect an avalanche event without detecting a corresponding photon. The system is also configurable to generate a dark current compensated image by performing a subtraction operation on the temporally filtered image or the image frame based on the dark current image frame.

Abstract: A system is configurable to, over a frame capture time period, selectively activate an illuminator to alternately emit light from the illuminator and refrain from emitting light from the illuminator. The system is configurable to, over the frame capture time period, perform a plurality of sequential shutter operations to configure each image sensor pixel of an image sensor array to enable photon detection. The plurality of sequential shutter operations generates, for each image sensor pixel of the image sensor array, a plurality of binary counts indicating whether a photon was detected during each of the plurality of sequential shutter operations. The system is configurable to, based on a first set of binary counts of the plurality of binary counts, generate an ambient light image. The system is configurable to, based on a second set of binary counts of the plurality of binary counts, generate an illuminated image.

Abstract: Techniques to temporally filter images via a filtering weight computation are disclosed. A first image having a first timestamp and a second image having a second timestamp are acquired. These images are generated by a camera, and the first timestamp is before the second timestamp. A motion compensation (MC) operation is performed on the first image to produce an MC image. A difference image is generated using the MC image and the second image. The difference image reflects differences in intensities that exist between the two images. A local weight map is generated based on those differences. A global weight map is generated based on certain IMU data. A final weight map is generated based on a combination of the local weight map and the global weight map. The final weight map is used to generate a filtered image.

Abstract: Techniques for aligning image content are disclosed. The alignment occurs by concurrently identifying corner and line features and by aligning the images based on those features. As a result, relaxed constraints can be used during the alignment process as compared to constraints that are required when aligning images using only corner features. To achieve these benefits, first and second images are acquired. A first set of pixels that correspond to corner features are detected within the images. In parallel, a second set of pixels that correspond to line features are detected within the images. Image content is then aligned from the two images. The aligning process is performed using at least some of the pixels from the first and second sets.

Abstract: An image sensor configured to capture imagery with mitigated noise includes a plurality of image sensing pixels arranged to form a sensor array. Each image sensing pixel of the plurality of image sensing pixels comprises an active area configured to receive photons to facilitate image capture. Each active area comprises a length and a width. For at least one image sensing pixel of the plurality of image sensing pixels, the length or the width of the active area is smaller than about 80% of a pixel pitch measurement between the at least one image sensing pixel and an adjacent image sensing pixel. A size of the active area relative to the pixel pitch measurement contributes to mitigating sensor noise for the image sensor.

Abstract: Techniques for generating a temporally filtered image designed to compensate for global motions of a camera and to compensate for local motions of an object are disclosed. A history frame and a current frame are acquired. A global motion compensation operation is performed on the history frame to reproject a pose of the history frame to match the pose of the current frame. The history frame is compared against the current frame to identify pixels that represent moving objects. For each of those pixels, an optical flow vector is computed. The optical flow vectors are then applied to those pixels to shift those pixels to new locations. These new positions, which are in the history frame, correspond to positions that were identified in the current frame. Afterwards, the current frame is temporally filtered with the history frame.

Abstract: A system for low compute high-resolution depth map generation using low-resolution cameras is configured to obtain a stereo pair of images and generate a depth map by performing stereo matching on the stereo pair of images. The system is also configured to obtain a first image comprising first texture information for the environment that has a first image resolution that is higher than an image resolution of images of the stereo pair of images. The system is further configured to generate a reprojected first image by reprojecting the first image to correspond to an image capture perspective associated with the depth map. The reprojection of the first image is based on depth information from the depth map and includes reprojected first texture information for the environment. The system is also configured to generate an upsampled depth map based on the depth map.

Abstract: Techniques for using motion data to generate a high resolution output color image from multiple images having sparse color information are disclosed. A camera generates multiple images. The camera's sensor is configured to have a sparse Bayer pattern. While the camera is generating the images, IMU data for each image is acquired. The IMU data indicates a corresponding pose the camera was in while the camera generated each image. The images and the IMU data are fed as input into a motion model. The motion model performs temporal filtering on the images and uses the IMU data to generate a red-only image, a green-only image, and a blue-only image. A high resolution output color image is generated by combining the red-only image, the green-only image, and the blue-only image.

Abstract: Techniques for aligning images generated by two cameras are disclosed. This alignment is performed by computing a relative 3D orientation between the two cameras. A first gravity vector for a first camera and a second gravity vector for a second camera are determined. A first camera image is obtained from the first camera, and a second camera image is obtained from the second camera. A first alignment process is performed to partially align the first camera’s orientation with the second camera’s orientation. This process is performed by aligning the gravity vectors, thereby resulting in two degrees of freedom of the relative 3D orientation being eliminated. Visual correspondences between the two images are identified. A second alignment process is performed to fully align the orientations. This process is performed by using the identified visual correspondences to identify and eliminate a third degree of freedom of the relative 3D orientation.

Abstract: A system for structured light depth computation using single photon avalanche diodes (SPADs) is configurable to, over a frame capture time period, selectively activate the illuminator to perform interleaved structured light illumination operations. The interleaved structured light illumination operations comprise alternately emitting at least a first structured light pattern from the illuminator and emitting at least a second structured light pattern from the illuminator. The system is also configurable to, over the frame capture time period, perform a plurality of sequential shutter operations to configure each SPAD pixel of the SPAD array to enable photon detection. The plurality of sequential shutter operations generates, for each SPAD pixel of the SPAD array, a plurality of binary counts indicating whether a photon was detected during each of the plurality of sequential shutter operations.

Abstract: The described implementations relate to reducing distance errors in depth maps. One example can detect blooming in a depth map. This example can identify a depth range affected by the blooming and take an action relating to the identified depth range to address the blooming.

Abstract: A system for updating continuous image alignment of separate cameras identifies a previous alignment matrix associated with a previous frame pair captured at one or more previous timepoints by a reference camera and a match camera. The previous alignment matrix is based on visual correspondences in the previous frame pair. The system also identifies a current matrix associated with a current frame pair captured at one or more current timepoints by the reference camera and the match camera. The current matrix is based on visual correspondences in the current frame pair. The system also identifies a difference value associated with the reference camera or the match camera relative to the one or more previous timepoints and the one or more current timepoints. The system also generates an updated alignment matrix by using the previous alignment matrix, the current matrix, and the difference value as inputs.

Abstract: A method for estimating human body temperature includes receiving, via a thermal camera, a thermal image captured of a real-world environment, the thermal image including thermal intensity values for each of a plurality of pixels of the thermal image. A position of a first human face is identified within the thermal image, the first human face corresponding to a first human subject. A skin tone of the first human face is identified. Based on the identified skin tone, a skin tone correction factor is applied to one or more thermal intensity values of one or more pixels corresponding to the first human face to give one or more tone-corrected thermal intensity values. Based on the one or more tone-corrected thermal intensity values an indication of a body temperature of the first human subject is reported.

Abstract: A system for obtaining color imagery using SPADs includes a SPAD array that has a plurality of SPAD pixels. Each of the plurality of SPAD pixels includes a respective color filter positioned thereover. The system is configurable to capture an image frame using the SPAD array and generate a filtered image by performing a temporal filtering operation using the image frame and at least one preceding image frame. The at least one preceding image frame is captured by the SPAD array at a timepoint that temporally precedes a timepoint associated with the image frame. The system is also configurable to, after performing the temporal filtering operation, generate a color image by demosaicing the filtered image.