Abstract: An image capturing system includes a light source configured to emit light toward an object or scene that is to be imaged. The system also includes a time-of-flight image sensor configured to receive light signals based on reflected light from the object or scene. The system also includes a processor operatively coupled to the light source and the time-of-flight image sensor. The processor is configured to perform compressive sensing of the received light signals. The processor is also configured to generate an image of the object or scene based at least in part on the compressive sensing of the received light signals.

Abstract: A camera assembly for determining depth information for a local area includes a light source assembly, a camera assembly, and a controller. The light source assembly projects pulses of light into the local area. The camera assembly images a portion of the local area illuminated with the pulses. The camera assembly includes augmented pixels, each augmented pixel having a plurality of gates and at least some of the gates have a respective local storage location. An exposure interval of each augmented pixel is divided into intervals associated with the gates, and each local storage location stores image data during a respective interval. The controller reads out, after the exposure interval of each augmented pixel, the image data stored in the respective local storage locations of each augmented pixel to generate image data frames. The controller determines depth information for the local area based in part on the image data frames.

Abstract: A camera assembly for generating high dynamic range images. The camera assembly includes a sensor that images a portion of a local area, and a controller. The sensor includes a plurality of augmented pixels, each augmented pixel having a plurality of gates and at least some of the gates have a respective local storage location. An exposure interval of each augmented pixel is divided into intervals associated with the gates, and each local storage location stores image data during a respective interval. The controller reads out, after the exposure interval of each augmented pixel, the image data stored in the respective local storage locations of each augmented pixel to form intermediate images that each have a dynamic range. The controller then generates an image for the portion of the local area using the intermediate images, the image having a higher dynamic range than each of the intermediate images.

Abstract: A camera assembly for depth sensing of a local area. The camera assembly includes a projector, a sensor and a controller. The projector emits, into the local area, a structured light (SL) pattern at multiple phases. The sensor images the local area using multiple augmented pixels. Each augmented pixel has multiple gates and at least some of the gates have a respective local storage location. Each capture phase of the augmented pixel is divided into a respective plurality of time bins associated with a respective subset of the gates. Each local storage location of the subset of gates stores image data during a respective time bin, the image data based on portions of the SL pattern emitted at a respective phase and reflected from the local area. The controller determines depth information for the local area based on the image data stored during at least one capture phase.

Abstract: An example system includes a patterned light projector operable to direct first and second portions of patterned light toward first and second surfaces, respectively, in an environment. The first and second surfaces may be at first and second distances, respectively, from the structured light projector. A graduated optical filter may be situated along an optical path of the patterned light. The graduated optical filter includes first and second regions to attenuate an intensity of the first and second portions of the patterned light, respectively, by first and second amounts, respectively. The first amount is greater than the second amount. The system additionally includes an image sensor operable to generate image data based on at least the first and second portions of the patterned light and a processor configured to determine first and second values indicative of an estimate of the first and second distances, respectively, based on the image data.

Abstract: An example system includes a patterned light projector operable to direct first and second portions of patterned light toward first and second surfaces, respectively, in an environment. The first and second surfaces may be at first and second distances, respectively, from the structured light projector. A graduated optical filter may be situated along an optical path of the patterned light. The graduated optical filter includes first and second regions to attenuate an intensity of the first and second portions of the patterned light, respectively, by first and second amounts, respectively. The first amount is greater than the second amount. The system additionally includes an image sensor operable to generate image data based on at least the first and second portions of the patterned light and a processor configured to determine first and second values indicative of an estimate of the first and second distances, respectively, based on the image data.

Abstract: A depth camera assembly (DCA) optimizes illumination and image capture of a local area to generate depth information of the local area. The DCA determines depth information for a first portion of the local area viewable at a first pose. The DCA is moved from the first pose to a second pose, where a second portion of the local area is viewable and overlaps with the first portion. The overlapping region is not illuminated by the DCA. A non-overlapping portion of the second portion is illuminated, captured, and depth information determined.

Abstract: A camera captures image data at a target frame rate. The camera includes a sensor and a controller. The sensor is configured to detect light from a local area and includes a plurality of augmented pixels. Each augmented pixel comprises at least a first and a second gate. The first gates are configured to store a first plurality of image frames as first image data according to a first activation pattern. The second gates are configured to store a second plurality of image frames as second image data according to a second activation pattern. The controller reads out the image data to generate a first image from the first image data and a second image from the second image data. The first and second images may be used to reconstruct a combined set of image frames at the target frame rate with a reconstruction algorithm.

Abstract: A system configured to generate images includes one or more narrowband electromagnetic sources configured to emit a first radiation and a second radiation. The system also includes a detector configured to detect first reflected radiation off of an object and second reflected radiation off of the object, where the first reflected radiation results from the first radiation and the second reflected radiation results from the second radiation. The system further includes a processor operatively coupled to the detector and configured to generate an image of the object based at least in part on a first amplitude of the first reflected radiation and a second amplitude of the second reflected radiation. The processor is also configured to determine a depth profile of the object based at least in part on the first reflected radiation and the second reflected radiation detected by the detector.

Abstract: An example system includes a patterned light projector operable to direct first and second portions of patterned light toward first and second surfaces, respectively, in an environment. The first and second surfaces may be at first and second distances, respectively, from the structured light projector. A graduated optical filter may be situated along an optical path of the patterned light. The graduated optical filter includes first and second regions to attenuate an intensity of the first and second portions of the patterned light, respectively, by first and second amounts, respectively. The first amount is greater than the second amount. The system additionally includes an image sensor operable to generate image data based on at least the first and second portions of the patterned light and a processor configured to determine first and second values indicative of an estimate of the first and second distances, respectively, based on the image data.

Abstract: A calibration assembly for calibrating a depth sensor. The calibration assembly includes a pathway and a controller coupled to a depth sensor. The pathway is coupled to a light source, and a pulse of light from the light source is directed through the pathway to the depth sensor at a first time. The controller determines a second time at which a measurement of the light pulse detected at an augmented pixel of the depth sensor is read out. The controller computes a readout delay for the augmented pixel based on the first time, the second time, and an estimated travel time for the light pulse from the light source to the augmented pixel. The controller stores the readout delay, and a depth camera that includes the depth sensor is configured to calibrate out the readout delay relative to other augmented pixels of the depth sensor.

Abstract: A camera assembly for depth sensing of a local area. The camera assembly includes a projector, a sensor and a controller. The projector emits, into the local area, a structured light (SL) pattern at multiple phases. The sensor images the local area using multiple augmented pixels. Each augmented pixel has multiple gates and at least some of the gates have a respective local storage location. Each capture phase of the augmented pixel is divided into a respective plurality of time bins associated with a respective subset of the gates. Each local storage location of the subset of gates stores image data during a respective time bin, the image data based on portions of the SL pattern emitted at a respective phase and reflected from the local area. The controller determines depth information for the local area based on the image data stored during at least one capture phase.

Abstract: A camera assembly for generating high dynamic range images. The camera assembly includes a sensor that images a portion of a local area, and a controller. The sensor includes a plurality of augmented pixels, each augmented pixel having a plurality of gates and at least some of the gates have a respective local storage location. An exposure interval of each augmented pixel is divided into intervals associated with the gates, and each local storage location stores image data during a respective interval. The controller reads out, after the exposure interval of each augmented pixel, the image data stored in the respective local storage locations of each augmented pixel to form intermediate images that each have a dynamic range. The controller then generates an image for the portion of the local area using the intermediate images, the image having a higher dynamic range than each of the intermediate images.

Abstract: An example system includes a patterned light projector operable to direct first and second portions of patterned light toward first and second surfaces, respectively, in an environment. The first and second surfaces may be at first and second distances, respectively, from the structured light projector. A graduated optical filter may be situated along an optical path of the patterned light. The graduated optical filter includes first and second regions to attenuate an intensity of the first and second portions of the patterned light, respectively, by first and second amounts, respectively. The first amount is greater than the second amount. The system additionally includes an image sensor operable to generate image data based on at least the first and second portions of the patterned light and a processor configured to determine first and second values indicative of an estimate of the first and second distances, respectively, based on the image data.

Abstract: A camera assembly for determining depth information for a local area includes a light source assembly, a camera assembly, and a controller. The light source assembly projects pulses of light into the local area. The camera assembly images a portion of the local area illuminated with the pulses. The camera assembly includes augmented pixels, each augmented pixel having a plurality of gates and at least some of the gates have a respective local storage location. An exposure interval of each augmented pixel is divided into intervals associated with the gates, and each local storage location stores image data during a respective interval. The controller reads out, after the exposure interval of each augmented pixel, the image data stored in the respective local storage locations of each augmented pixel to generate image data frames. The controller determines depth information for the local area based in part on the image data frames.

Abstract: A camera assembly for generating differential images for a local area. The camera assembly includes a sensor and a controller. The sensor images at least a portion of a local area and includes a plurality of augmented pixels. Each augmented pixel has at least a first gate with a first local storage location and a second gate with a second local storage location. An exposure interval of the sensor is divided into intervals. The first local storage location stores first image data during a first of the intervals, and the second local storage location stores second image data during a second of the intervals. The controller determines, for each augmented pixel, a difference between the first image data and the second image data. The controller then reads out the difference for each augmented pixel as a differential image of the portion of the local area.

Abstract: A camera assembly for generating high dynamic range images. The camera assembly includes a sensor that images a portion of a local area, and a controller. The sensor includes a plurality of augmented pixels, each augmented pixel having a plurality of gates and at least some of the gates have a respective local storage location. An exposure interval of each augmented pixel is divided into intervals associated with the gates, and each local storage location stores image data during a respective interval. The controller reads out, after the exposure interval of each augmented pixel, the image data stored in the respective local storage locations of each augmented pixel to form intermediate images that each have a dynamic range. The controller then generates an image for the portion of the local area using the intermediate images, the image having a higher dynamic range than each of the intermediate images.

Abstract: A camera assembly for generating differential images for a local area. The camera assembly includes a sensor and a controller. The sensor images at least a portion of a local area and includes a plurality of augmented pixels. Each augmented pixel has at least a first gate with a first local storage location and a second gate with a second local storage location. An exposure interval of the sensor is divided into intervals. The first local storage location stores first image data during a first of the intervals, and the second local storage location stores second image data during a second of the intervals. The controller determines, for each augmented pixel, a difference between the first image data and the second image data. The controller then reads out the difference for each augmented pixel as a differential image of the portion of the local area.

Abstract: A camera assembly for generating high dynamic range images. The camera assembly includes a sensor that images a portion of a local area, and a controller. The sensor includes a plurality of augmented pixels, each augmented pixel having a plurality of gates and at least some of the gates have a respective local storage location. An exposure interval of each augmented pixel is divided into intervals associated with the gates, and each local storage location stores image data during a respective interval. The controller reads out, after the exposure interval of each augmented pixel, the image data stored in the respective local storage locations of each augmented pixel to form intermediate images that each have a dynamic range. The controller then generates an image for the portion of the local area using the intermediate images, the image having a higher dynamic range than each of the intermediate images.

Abstract: A depth camera assembly (DCA) for depth sensing of a local area. The DCA includes a light source assembly, a camera assembly, and a controller. The light source assembly projects pulses of light into the local area, each pulse having a respective polarization type. The camera assembly images a portion of the local area illuminated with the pulses of light. The camera assembly includes a plurality of augmented pixels, each augmented pixel having a plurality of gates and local storage locations. An exposure interval of the camera assembly is divided into intervals and some of the intervals are synchronized to the projected pulses of light such that each respective local storage location stores image data associated with a different polarization type. The controller determines depth information for the local area based in part on the polarization types associated with the image data stored in respective local storage locations.