Abstract: An image sensor includes imaging pixels and a patterned liquid crystal polarizer (LCP). The imaging pixel include subpixels. The patterned LCP is disposed over the subpixels and configured to direct a particular polarized portion of imaging light to particular subpixels.

Abstract: A stacked camera-image-sensor circuit may include (i) a first layer that includes a plurality of image sensing elements, (ii) a second layer that includes components that interface with the image sensing elements, and (iii) at least one additional layer that includes image-processing components. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: In one example, an apparatus comprises: a memory to store input data and weights, the input data comprising groups of data elements, each group being associated with a channel of channels, the weights comprising weight tensors, each weight tensor being associated with a channel of the channels; a data sparsity map generation circuit configured to generate, based on the input data, a channel sparsity map and a spatial sparsity map, the channel sparsity map indicating channels associated with first weights tensors to be selected, the spatial sparsity map indicating spatial locations of first data elements; a gating circuit configured to: fetch, based on the channel sparsity map and the sparsity map, the first weights tensors and the first data elements from the memory; and a processing circuit configured to perform neural network computations on the first data elements and the first weights tensors to generate a processing result.

Abstract: In one example, an apparatus comprises: a plurality of photodiodes, one or more charge sensing units, one or more analog-to-digital converters (ADCs), and a controller. The controller is configured to: enable the each photodiode to generate charge in response to a different component of the incident light; transfer the charge from the plurality of photodiodes to the one or more charge sensing units to convert to voltages; receive a selection of one or more quantization processes of a plurality of quantization processes corresponding to a plurality of intensity ranges; based on the selection, control the one or more ADCs to perform the selected one or more quantization processes to quantize the voltages from the one or more charge sensing units to digital values representing components of a pixel of different wavelength ranges; and generate a pixel value based on the digital values.

Abstract: In some examples, an apparatus comprises: a first photodiode to sense a first component of light associated with a first wavelength, and a second photodiode configured to sense a second component of the light associated with a second wavelength, the first component and the second component being associated with, respectively, a first wavelength and a second wavelength. The apparatus further comprises a first optical structure and a second optical structure positioned over, respectively, the first photodiode and the second photodiode. The first optical structure is configured to increase a propagation path of the first component of the light within the first photodiode and has a first optical property based on the first wavelength. The second optical structure is configured to increase a propagation path of the second component of the light within the second photodiode, and has a second optical property based on the second wavelength.

Abstract: A sensor assembly for determining one or more features of a local area is presented herein. The sensor assembly includes a plurality of stacked sensor layers. A first sensor layer of the plurality of stacked sensor layers located on top of the sensor assembly includes an array of pixels. The top sensor layer can be configured to capture one or more images of light reflected from one or more objects in the local area. The sensor assembly further includes one or more sensor layers located beneath the top sensor layer. The one or more sensor layers can be configured to process data related to the captured one or more images. A plurality of sensor assemblies can be integrated into an artificial reality system, e.g., a head-mounted display.

Abstract: Methods and systems for image sensing are provided. In one example, an apparatus comprises a semiconductor substrate comprising a light incident surface to receive light, a first pinned photodiode, and a second pinned photodiode, the first pinned photodiode and the second pinned photodiode forming a stack structure in the semiconductor substrate along an axis perpendicular to the light incident surface, the stack structure enabling the first pinned photodiode and the second pinned photodiode to, respectively, convert a first component of the light and a second component of the light to first charge and second charge. The apparatus further comprises one or more capacitors formed in the semiconductor substrate and configured to generate a first voltage and a second voltage based on, respectively, the first charge and the second charge.

Abstract: One example described herein includes the digital image sensor having a pixel cell with a photodiode and a quantizer circuit coupled to the pixel cell. The quantizer circuit includes a charge storage device that generates a voltage based on electric charge from the photodiode. The quantizer circuit also includes a single-input comparator that can switch from a first output state to a second output state in response to a ramp signal provided by a ramp generator. The quantizer circuit further includes a memory switch that can cause a counter value from a digital counter to be stored in a digital memory in response to the single-input comparator switching from the first output state to the second output state. The counter value can serve as a digital pixel value associated with the pixel cell.

Abstract: In one example, an apparatus is provided. The apparatus comprises an image sensor configured to generate a first raw output to represent a first intensity of incident light based on a first relationship, and to generate a second raw output to represent a second intensity of incident light based on a second relationship. The apparatus further comprises a post processor configured to: generate a first post-processed output based on the first raw output and based on the first relationship such that the first post-processed output is linearly related to the first intensity based on a third relationship, and to generate a second post-processed output based on the second raw output and based on the second relationship such that the second post-processed output is linearly related to the second intensity based on the third relationship.

Abstract: In one example, an apparatus comprises: multiple distinct sets of photodiodes, wherein each set of photodiodes includes one or more photodiodes, one or more charge sensing units, and a controller. The controller is configured to: transfer charge generated by the one or more photodiodes in response to a different component of incident light to the one or more charge sensing units in order to convert the charge to voltages; perform one or more quantization processes of a plurality of quantization processes corresponding to a plurality of intensity ranges, wherein the one or more quantization processes quantizes the voltages from the one or more charge sensing units to digital values representing components of a pixel of different wavelength ranges; and generate a pixel value based on the at least some of the digital values.

Abstract: In some examples, a sensor apparatus comprises: a pixel cell configured to generate a voltage, the pixel cell including one or more photodiodes configured to generate a charge in response to light and a charge storage device to convert the charge to a voltage; an integrated circuit comprising a plurality of integrated memory circuits and configured to: generate, based on a first voltage obtained from the charge storage device of the pixel cell, a first voltage value during a first time period; and generate, based on a second voltage generated by fixed pattern noise from the pixel cell and the integrated circuit, a second voltage value occurring a second time period; and one or more analog-to-digital converters (ADC) configured the convert the first voltage value to a first digital pixel value and the second voltage value to a second digital pixel value; and a processor configured to generate a first altered digital pixel value based on the first digital pixel value and the second digital pixel value.

Abstract: A depth camera assembly (DCA) for depth sensing of a local area. The DCA includes a transmitter, a receiver, and a controller. The transmitter illuminates a local area with outgoing light in accordance with emission instructions. The transmitter includes a fine steering element and a coarse steering element. The fine steering element deflects one or more optical beams at a first deflection angle to generate one or more first order deflected scanning beams. The coarse steering element deflects the one or more first order deflected scanning beams at a second deflection angle to generate the outgoing light projected into the local area. The receiver captures one or more images of the local area including portions of the outgoing light reflected from the local area. The controller determines depth information for one or more objects in the local area based in part on the captured one or more images.

Abstract: In one example, an apparatus comprises: a semiconductor substrate comprising a first photodiode and a second photodiode, the first photodiode being positioned adjacent to the second photodiode along a first axis; a birefringent crystal positioned over the first photodiode and the second photodiode along a second axis perpendicular to the first axis; and a microlens positioned over the birefringent crystal along the second axis, the microlens having an asymmetric curvature along the first axis, an apex point of the curvature being positioned over the first photodiode along the second axis.

Abstract: A sensor assembly for determining one or more features of a local area is presented herein. The sensor assembly includes a plurality of stacked sensor layers. A first sensor layer of the plurality of stacked sensor layers located on top of the sensor assembly includes an array of pixels. The top sensor layer can be configured to capture one or more images of light reflected from one or more objects in the local area. The sensor assembly further includes one or more sensor layers located beneath the top sensor layer. The one or more sensor layers can be configured to process data related to the captured one or more images. A plurality of sensor assemblies can be integrated into an artificial reality system, e.g., a head-mounted display.

Abstract: In some examples, a sensor apparatus comprises: a pixel cell configured to generate a voltages, the pixel cell including a photodiode configured to generate charge in response to incoming light, and a charge storage device to convert the charge to a voltage; an integrated circuit configured to: determine a first captured voltage converted by the charge storage device during a first time period; compare the first captured voltage to a threshold voltage value; and in response to determining that the first captured voltage meets or exceeds the threshold voltage value: determine first time data corresponding to the first time period; and prevent the charge storage device from further generating a charge; and an analog-to-digital converter (ADC) configured to generate a digital pixel value based on the first captured voltage, and a memory to store the digital pixel value and the first time data.

Abstract: An apparatus comprises a first photodiode, a second photodiode, a quantizer, a memory, and a controller configured to: set a first exposure period in which the first photodiode generates a first charge; set a second exposure period in which the second photodiode generates a second charge, the second exposure period being set based on the first exposure period and at least one of: a first time associated with a read out operation of the memory to a second apparatus, or a second time associated with a quantization operation by the quantizer; perform, using the quantizer, the first quantization operation of the first charge to generate a first digital output; and perform, using the quantizer, a second quantization operation of the second charge to generate a second digital output.

Abstract: A depth camera assembly (DCA) for depth sensing of a local area. The DCA includes a transmitter, a receiver, and a controller. The transmitter illuminates a local area with outgoing light in accordance with emission instructions. The transmitter includes a fine steering element and a coarse steering element. The fine steering element deflects one or more optical beams at a first deflection angle to generate one or more first order deflected scanning beams. The coarse steering element deflects the one or more first order deflected scanning beams at a second deflection angle to generate the outgoing light projected into the local area. The receiver captures one or more images of the local area including portions of the outgoing light reflected from the local area. The controller determines depth information for one or more objects in the local area based in part on the captured one or more images.

Abstract: A depth camera assembly (DCA) determines depth information. The DCA projects a dynamic structured light pattern into a local area and captures images including a portion of the dynamic structured light pattern. The DCA determines regions of interest in which it may be beneficial to increase or decrease an amount of texture added to the region of interest using the dynamic structured light pattern. For example, the DCA may identify the regions of interest based on contrast values calculated using a contrast algorithm, or based on the parameters received from a mapping server including a virtual model of the local area. The DCA may selectively increase or decrease an amount of texture added by the dynamic structured light pattern in portions of the local area. By selectively controlling portions of the dynamic structured light pattern, the DCA may decrease power consumption and/or increase the accuracy of depth sensing measurements.

Abstract: A depth camera assembly (DCA) includes a light generator emitting a beam of light into a local area and a detector. The detector captures light from the beam reflected by objects in the local area to a portion of an array of pixels that each include a single photon avalanche diode (SPAD). The location of the portion of the array is based in part on the angle of the beam emitted from the projector. The DCA identifies a set of pixels of the array corresponding to the portion and selectively retrieves current generated from the reflected light by the pixels in the portion of the array without retrieving current generated by pixels in other portions of the array.

Abstract: A depth estimation system is described capable of determining depth information using two images from two cameras. A first camera captures a first image and a second camera captures a second image, both images including a plurality of light channels. A scan direction is selected from a plurality of scan directions. For the selected scan direction, along each of a plurality of scanlines, the system compares pixels from the first image to pixels from the second image. The comparison is based on calculating a census transform for each pixel in the first image and a census transform for each pixel in the second image. This comparison is used to determine a stereo correspondence between the pixels in the first image and the pixels in the second image. The system generates a depth map based on the stereo correspondence.