Abstract: In one example, an apparatus comprises: multiple distinct sets of photodiodes, wherein each set of photodiodes includes one or more photodiodes, one or more charge sensing units, and a controller. The controller is configured to: transfer charge generated by the one or more photodiodes in response to a different component of incident light to the one or more charge sensing units in order to convert the charge to voltages; perform one or more quantization processes of a plurality of quantization processes corresponding to a plurality of intensity ranges, wherein the one or more quantization processes quantizes the voltages from the one or more charge sensing units to digital values representing components of a pixel of different wavelength ranges; and generate a pixel value based on the at least some of the digital values.

Abstract: In some examples, a sensor apparatus comprises: a pixel cell configured to generate a voltage, the pixel cell including one or more photodiodes configured to generate a charge in response to light and a charge storage device to convert the charge to a voltage; an integrated circuit comprising a plurality of integrated memory circuits and configured to: generate, based on a first voltage obtained from the charge storage device of the pixel cell, a first voltage value during a first time period; and generate, based on a second voltage generated by fixed pattern noise from the pixel cell and the integrated circuit, a second voltage value occurring a second time period; and one or more analog-to-digital converters (ADC) configured the convert the first voltage value to a first digital pixel value and the second voltage value to a second digital pixel value; and a processor configured to generate a first altered digital pixel value based on the first digital pixel value and the second digital pixel value.

Abstract: In one example, an apparatus comprises: a semiconductor substrate comprising a first photodiode and a second photodiode, the first photodiode being positioned adjacent to the second photodiode along a first axis; a birefringent crystal positioned over the first photodiode and the second photodiode along a second axis perpendicular to the first axis; and a microlens positioned over the birefringent crystal along the second axis, the microlens having an asymmetric curvature along the first axis, an apex point of the curvature being positioned over the first photodiode along the second axis.

Abstract: A depth camera assembly (DCA) for depth sensing of a local area. The DCA includes a transmitter, a receiver, and a controller. The transmitter illuminates a local area with outgoing light in accordance with emission instructions. The transmitter includes a fine steering element and a coarse steering element. The fine steering element deflects one or more optical beams at a first deflection angle to generate one or more first order deflected scanning beams. The coarse steering element deflects the one or more first order deflected scanning beams at a second deflection angle to generate the outgoing light projected into the local area. The receiver captures one or more images of the local area including portions of the outgoing light reflected from the local area. The controller determines depth information for one or more objects in the local area based in part on the captured one or more images.

Abstract: A sensor assembly for determining one or more features of a local area is presented herein. The sensor assembly includes a plurality of stacked sensor layers. A first sensor layer of the plurality of stacked sensor layers located on top of the sensor assembly includes an array of pixels. The top sensor layer can be configured to capture one or more images of light reflected from one or more objects in the local area. The sensor assembly further includes one or more sensor layers located beneath the top sensor layer. The one or more sensor layers can be configured to process data related to the captured one or more images. A plurality of sensor assemblies can be integrated into an artificial reality system, e.g., a head-mounted display.

Abstract: In some examples, a sensor apparatus comprises: a pixel cell configured to generate a voltages, the pixel cell including a photodiode configured to generate charge in response to incoming light, and a charge storage device to convert the charge to a voltage; an integrated circuit configured to: determine a first captured voltage converted by the charge storage device during a first time period; compare the first captured voltage to a threshold voltage value; and in response to determining that the first captured voltage meets or exceeds the threshold voltage value: determine first time data corresponding to the first time period; and prevent the charge storage device from further generating a charge; and an analog-to-digital converter (ADC) configured to generate a digital pixel value based on the first captured voltage, and a memory to store the digital pixel value and the first time data.

Abstract: An apparatus comprises a first photodiode, a second photodiode, a quantizer, a memory, and a controller configured to: set a first exposure period in which the first photodiode generates a first charge; set a second exposure period in which the second photodiode generates a second charge, the second exposure period being set based on the first exposure period and at least one of: a first time associated with a read out operation of the memory to a second apparatus, or a second time associated with a quantization operation by the quantizer; perform, using the quantizer, the first quantization operation of the first charge to generate a first digital output; and perform, using the quantizer, a second quantization operation of the second charge to generate a second digital output.

Abstract: A depth camera assembly (DCA) for depth sensing of a local area. The DCA includes a transmitter, a receiver, and a controller. The transmitter illuminates a local area with outgoing light in accordance with emission instructions. The transmitter includes a fine steering element and a coarse steering element. The fine steering element deflects one or more optical beams at a first deflection angle to generate one or more first order deflected scanning beams. The coarse steering element deflects the one or more first order deflected scanning beams at a second deflection angle to generate the outgoing light projected into the local area. The receiver captures one or more images of the local area including portions of the outgoing light reflected from the local area. The controller determines depth information for one or more objects in the local area based in part on the captured one or more images.

Abstract: A depth camera assembly (DCA) determines depth information. The DCA projects a dynamic structured light pattern into a local area and captures images including a portion of the dynamic structured light pattern. The DCA determines regions of interest in which it may be beneficial to increase or decrease an amount of texture added to the region of interest using the dynamic structured light pattern. For example, the DCA may identify the regions of interest based on contrast values calculated using a contrast algorithm, or based on the parameters received from a mapping server including a virtual model of the local area. The DCA may selectively increase or decrease an amount of texture added by the dynamic structured light pattern in portions of the local area. By selectively controlling portions of the dynamic structured light pattern, the DCA may decrease power consumption and/or increase the accuracy of depth sensing measurements.

Abstract: A depth camera assembly (DCA) includes a light generator emitting a beam of light into a local area and a detector. The detector captures light from the beam reflected by objects in the local area to a portion of an array of pixels that each include a single photon avalanche diode (SPAD). The location of the portion of the array is based in part on the angle of the beam emitted from the projector. The DCA identifies a set of pixels of the array corresponding to the portion and selectively retrieves current generated from the reflected light by the pixels in the portion of the array without retrieving current generated by pixels in other portions of the array.

Abstract: A depth estimation system is described capable of determining depth information using two images from two cameras. A first camera captures a first image and a second camera captures a second image, both images including a plurality of light channels. A scan direction is selected from a plurality of scan directions. For the selected scan direction, along each of a plurality of scanlines, the system compares pixels from the first image to pixels from the second image. The comparison is based on calculating a census transform for each pixel in the first image and a census transform for each pixel in the second image. This comparison is used to determine a stereo correspondence between the pixels in the first image and the pixels in the second image. The system generates a depth map based on the stereo correspondence.

Abstract: An apparatus is provided. The apparatus includes a semiconductor substrate including a first photodiode to generate a first charge, a second photodiode to generate a second charge, a barrier layer between the first photodiode and the second photodiode, wherein the first photodiode, the barrier layer, and the second photodiode form a stack. The apparatus further includes a floating drain and one or more gates including: a first gate portion on the semiconductor substrate and a second gate portion extending from the front side surface through the first photodiode and reaching the barrier layer. The first gate portion is configured to conduct a first signal to control flow of charge from the first photodiode to the floating drain, and the second gate portion is configured to conduct a second signal to control the barrier layer to control the flow of the second charge.

Abstract: An apparatus comprises a first photodiode, a second photodiode, a charge sensing unit, a quantizer, a memory, and a controller configured to: set a first exposure period in which the first photodiode generates a first charge; set a second exposure period in which the second photodiode generates a second charge, the second exposure period being set based on the first exposure period and at least one of: a first time associated with a read out operation of the memory to a second apparatus, or a second time associated with a quantization operation by the quantizer; perform, using the quantizer, the first quantization operation of the first charge to generate a first digital output; store the first digital output in the memory; perform, using the quantizer, a second quantization operation of the second charge to generate a second digital output; and store the second digital output in the memory.

Abstract: A depth camera assembly (DCA) optimizes illumination and image capture of a local area to generate depth information of the local area. The DCA determines depth information for a first portion of the local area viewable at a first pose. The DCA is moved from the first pose to a second pose, where a second portion of the local area is viewable and overlaps with the first portion. The overlapping region is not illuminated by the DCA. A non-overlapping portion of the second portion is illuminated, captured, and depth information determined.

Abstract: A depth camera assembly (DCA) determines depth information. The DCA projects a dynamic structured light pattern into a local area and captures images including a portion of the dynamic structured light pattern. The DCA determines regions of interest in which it may be beneficial to increase or decrease an amount of texture added to the region of interest using the dynamic structured light pattern. For example, the DCA may identify the regions of interest based on contrast values calculated using a contrast algorithm, or based on the parameters received from a mapping server including a virtual model of the local area. The DCA may selectively increase or decrease an amount of texture added by the dynamic structured light pattern in portions of the local area. By selectively controlling portions of the dynamic structured light pattern, the DCA may decrease power consumption and/or increase the accuracy of depth sensing measurements.

Abstract: A photo sensor includes a plurality of pixel blocks, each including one or more anchor pixels and one or more non-anchor pixels. The anchor pixels produce first sensor signals and the non-anchor pixels produce second sensor signals. An amplifier circuit amplifies the first and second sensor signals. A variable bit-depth analog to digital converter (ADC) circuit quantizes amplified versions of the first sensor signals into first digitized sensor signals with a first bit-depth. The ADC circuit also quantizes amplified versions of the second sensor signals into second digitized sensor signals with a second bit-depth that is lower than the first bit depth. The second bit-depth may be selected based on anchor pixel statistics derived from the one or more first digitized sensor signals.

Abstract: In one example, an apparatus comprises a plurality of sub-pixels arranged sideway, a shared optical element positioned over the plurality of sub-pixels, the shared optical element being configured to direct light originating from a same location in a scene to each sub-pixel in the plurality of sub-pixels; one or more polarizers positioned between the shared optical element and one or more first sub-pixels of the plurality of sub-pixels and configured to selectively pass one or more components of the light having one or more pre-determined polarization states, to enable the photodiodes of each of the one or more first sub-pixels to generate signals based on intensities of the one or more components; and one or more processors configured to generate polarimetric measurements of the received light based on signals obtained from the photodiodes of the one or more first sub-pixels and polarization properties of the one or more polarizers.

Abstract: Examples of an apparatus are disclosed. In some example, an apparatus may include an array of digital pixel cells, each digital pixel cell including a photodiode and a memory device to store a digital output generated based on charge generated by the photodiode in an exposure period. The apparatus may also include an image processor configured to: receive first digital outputs from the memory devices of a first set of digital pixel cells of the array of digital pixel cells; determine, from the first set of digital pixel cells, a second set of digital pixel cells of which the first digital outputs satisfy one or more pre-determined conditions; identify, based on the second set of digital pixel cells, a third set of digital pixel cells; receive the second digital outputs generated by the third set of digital pixel cells; and perform image processing operations based on the second digital outputs.

Abstract: In one example, a method comprises: within an integration period: enabling a photodiode of a pixel cell to accumulate charge responsive to incident light, and transferring the charge from the photodiode to a charge storage device of the pixel cell. The method further comprises: performing, using a sampling capacitor, a sample-and-hold operation to convert the charge stored in the charge storage device into a voltage; and generating a digital output based on the voltage to represent an intensity of the incident light received by the photodiode.

Abstract: A sensor assembly for determining one or more features of a local area is presented herein. The sensor assembly includes a plurality of stacked sensor layers. A first sensor layer of the plurality of stacked sensor layers located on top of the sensor assembly includes an array of pixels. The top sensor layer can be configured to capture one or more images of light reflected from one or more objects in the local area. The sensor assembly further includes one or more sensor layers located beneath the top sensor layer. The one or more sensor layers can be configured to process data related to the captured one or more images. Different sensor architectures featuring various arrangements of memory and computing devices are described, some of which feature in-memory computing. A plurality of sensor assemblies can be integrated into an artificial reality system, e.g., a head-mounted display.