Abstract: The disclosed computer-implemented method may include (1) receiving a present frame of a video stream, the present frame comprising a present depiction of a multi-segment articulated body system, (2) identifying a previous frame of the video stream that comprises a previous depiction of the multi-segment articulated body system, (3) analyzing the present frame and the previous frame to determine whether the multi-segment articulated body system remained substantially rigid between the previous frame and the present frame, and (4) estimating a pose of the multi-segment articulated body system in the present frame using a first pose estimation computation that treats the multi-segment articulated body system as rigid and that is selected in contrast to a second pose estimation computation based on determining that the multi-segment articulated body system remained substantially rigid between the previous frame and the present frame. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: In one example, an apparatus comprises a plurality of sub-pixels arranged sideway, a shared optical element positioned over the plurality of sub-pixels, the shared optical element being configured to direct light originating from a same location in a scene to each sub-pixel in the plurality of sub-pixels; one or more polarizers positioned between the shared optical element and one or more first sub-pixels of the plurality of sub-pixels and configured to selectively pass one or more components of the light having one or more pre-determined polarization states, to enable the photodiodes of each of the one or more first sub-pixels to generate signals based on intensities of the one or more components; and one or more processors configured to generate polarimetric measurements of the received light based on signals obtained from the photodiodes of the one or more first sub-pixels and polarization properties of the one or more polarizers.

Abstract: In some examples, an apparatus comprises: a first photodiode to sense a first component of light associated with a first wavelength, and a second photodiode configured to sense a second component of the light associated with a second wavelength, the first component and the second component being associated with, respectively, a first wavelength and a second wavelength. The apparatus further comprises a first optical structure and a second optical structure positioned over, respectively, the first photodiode and the second photodiode. The first optical structure is configured to increase a propagation path of the first component of the light within the first photodiode and has a first optical property based on the first wavelength. The second optical structure is configured to increase a propagation path of the second component of the light within the second photodiode, and has a second optical property based on the second wavelength.

Abstract: A depth camera assembly (DCA) determines depth information. The DCA projects a dynamic structured light pattern into a local area and captures images including a portion of the dynamic structured light pattern. The DCA determines regions of interest in which it may be beneficial to increase or decrease an amount of texture added to the region of interest using the dynamic structured light pattern. For example, the DCA may identify the regions of interest based on contrast values calculated using a contrast algorithm, or based on the parameters received from a mapping server including a virtual model of the local area. The DCA may selectively increase or decrease an amount of texture added by the dynamic structured light pattern in portions of the local area. By selectively controlling portions of the dynamic structured light pattern, the DCA may decrease power consumption and/or increase the accuracy of depth sensing measurements.

Abstract: Examples of an apparatus are disclosed. In some example, an apparatus may include an array of digital pixel cells, each digital pixel cell including a photodiode and a memory device to store a digital output generated based on charge generated by the photodiode in an exposure period. The apparatus may also include an image processor configured to: receive first digital outputs from the memory devices of a first set of digital pixel cells of the array of digital pixel cells; determine, from the first set of digital pixel cells, a second set of digital pixel cells of which the first digital outputs satisfy one or more pre-determined conditions; identify, based on the second set of digital pixel cells, a third set of digital pixel cells; receive the second digital outputs generated by the third set of digital pixel cells; and perform image processing operations based on the second digital outputs.

Abstract: In some examples, a sensor apparatus comprises: an array of pixel cells each including one or more photodiodes configured to generate a charge in response to light, and a charge storage device to convert the charge to output a voltage of an array of voltages, one or more an analog-to-digital converter (ADC) configured the convert the array of voltages to first pixel data, and an on-sensor controller configured to input the first pixel data into a machine-learning model to generate output data comprising prediction data associated with one or more features of the first pixel data, generate, based on the prediction data, second pixel data, the second pixel data associated with one or more transformed features of the first pixel data, and send, from the sensor apparatus to a separate receiving apparatus, the second pixel data.

Abstract: A depth estimation system is described capable of determining depth information using two images from two cameras. A first camera captures a first image and a second camera captures a second image, both images including a plurality of light channels. A scan direction is selected from a plurality of scan directions. For the selected scan direction, along each of a plurality of scanlines, the system compares pixels from the first image to pixels from the second image. The comparison is based on calculating a census transform for each pixel in the first image and a census transform for each pixel in the second image. This comparison is used to determine a stereo correspondence between the pixels in the first image and the pixels in the second image. The system generates a depth map based on the stereo correspondence.

Abstract: An eye-tracking system includes a substrate transparent to visible light and configured to be placed in a field of view of an eye of a user, a plurality of waveguides on the substrate, a light source optically coupled to the plurality of waveguides, and a plurality of polarization volume holograms (PVHs) in the field of view of the eye of the user. Each PVH of the plurality of PVHs is optically coupled to a respective waveguide of the plurality of waveguides and is configured to couple a respective light beam out of the respective waveguide towards the eye of the user.

Abstract: Methods, systems, and media for partitioning neural networks are provided. In some embodiments, a method comprises obtaining a training set. The method comprises training a plurality of neural networks using the training set, wherein neural networks differ based on dimensions of one or more layers of the neural networks and a location of a compression block positioned between a first set of layers of a neural network and a second set of layers of the neural network. The method comprises selecting a neural network based on hardware constraints of a system on which the neural network is to be implemented, wherein the first set of layers of the selected neural network are executed by one or more sensor devices of the system, and wherein the second set of layers of the selected neural network are executed by an aggregator computing device of the system.

Abstract: In one example, an apparatus comprises a first photodiode, a second photodiode, a first floating diffusion, a second floating diffusion, a quantizer, and a controller. The controller can enable the first photodiode and the second photodiode to generate and accumulate photo charge within an exposure period, and use the quantizer to quantize reset voltages at the first floating diffusion and at the second floating diffusion to generate a first digital reset value and a second digital reset value. After the exposure period ends, the controller can transfer the photo charge from the first photodiode and the second photodiode to, respectively, the first floating diffusion and the second floating diffusion to generate a first signal voltage and a second signal voltage, and quantize the signal voltages into digital signal values using the quantizer. Digital representations can be generated based on the digital reset values and the digital signal values.

Abstract: In one example, a mobile device comprises: a physical link; a plurality of image sensors, each image sensor being configured to transmit image data via the physical link; and a controller coupled to the physical link, whereby the physical link, the plurality of image sensors, and the controller form a multi-drop network. The controller is configured to: transmit a control signal to configure image sensing operations at the plurality of image sensors; receive, via the physical link, image data from at least a subset of the plurality of image sensors; combine the image data from the at least a subset of the plurality of image sensors to obtain an extended field of view (FOV); determine information of a surrounding environment of the mobile device captured within the extended FOV; and provide the information to an application to generate content based on the information.

Abstract: Methods and systems for image sensing are provided. In one example, an apparatus comprises a semiconductor substrate comprising a light incident surface to receive light, a first pinned photodiode, and a second pinned photodiode, the first pinned photodiode and the second pinned photodiode forming a stack structure in the semiconductor substrate along an axis perpendicular to the light incident surface, the stack structure enabling the first pinned photodiode and the second pinned photodiode to, respectively, convert a first component of the light and a second component of the light to first charge and second charge. The apparatus further comprises one or more capacitors formed in the semiconductor substrate and configured to generate a first voltage and a second voltage based on, respectively, the first charge and the second charge.

Abstract: Methods and systems for sensing physical quantities are provided. In one example, an apparatus comprises: a plurality of sensing elements, each sensing element being configured to measure a physical quantity and to generate a measurement result; and a controller configured to: receive first measurement results from at least a subset of the plurality of sensing elements, each of the first measurement results corresponding to an event based at least in part on satisfying a relationship with a threshold; populate an event matrix based in part on the first measurement results; populate a change matrix based in part on the threshold; and generate information related to a distribution of the measurement results among the plurality of sensing elements based at least in part on the event matrix and the change matrix.

Abstract: Methods and systems for quantizing a physical quantity, such as light, are provided. In one example, an apparatus comprises an analog-to-digital (A/D) converter configured to generate raw digital outputs based on performing at least one of: (1) a first quantization operation to quantize a physical stimulus within a first intensity range based on a first A/D conversion relationship, or (2) a second quantization operation to quantize the physical stimulus within a second intensity range based on a second A/D conversion relationship; and a raw output conversion circuit configured generate a refined digital output based on a raw digital output obtained from the A/D converter and at least one predetermined conversion parameter. The at least one conversion parameter compensates for a discontinuity between the first A/D conversion relationship and the second A/D conversion relationship.

Abstract: In one example, an apparatus is provided. The apparatus includes a photodiode, a charge sensing unit, an analog-to-digital converter (ADC), and a controller. The controller is configured to: enable the photodiode to generate charge in response to incident light, accumulate at least a portion of the charge as residual charge until the photodiode becomes saturated by the residual charge, and transfer the remaining portion of the charge to the charge sensing unit as overflow charge if the photodiode becomes saturated by the residual charge. The controller is further configured to: generate, using the ADC, a first digital output based on the residual charge; after generating the first digital output, generate, using the ADC, a second digital output based on the overflow charge; and generate a digital representation of an intensity of the incident light based on at least one of the first digital output or the second digital output.

Abstract: The disclosed computer-implemented method may include (i) conditionally operating, at a first frequency, a first stage of an eye-tracking system processing pipeline that detects a region of interest and (ii) operating, at a second frequency that is substantially greater than the first frequency, a second stage of the eye-tracking system processing pipeline that predicts a gaze orientation based at least in part on the detected region of interest. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: Imaging systems, cameras, and image sensors of this disclosure include imaging pixels that include subpixels. Diffractive optical elements such as a metasurface lens layers or a liquid crystal polarization hologram (LCPH) are configured to focus image light to the subpixels of the imaging pixels.

Abstract: The disclosed system may include a first layer that includes multiple digital pixel sensors configured to detect light. The system may also include a second layer that includes various image processing components configured to process the light detected by the digital pixel sensors. Still further, the system may include a third layer that includes machine learning (ML) hardware processing components. The image processing components of the second layer may be communicatively connected to the ML hardware processing components of the third layer via multiple micro through-silicon vias (uTSVs). Various other methods of manufacturing, apparatuses, and computer-readable media are also disclosed.

Abstract: In one example, an apparatus comprises: a first sensor layer, including an array of pixel cells configured to generate pixel data; and one or more semiconductor layers located beneath the first sensor layer with the one or more semiconductor layers being electrically connected to the first sensor layer via interconnects. The one or more semiconductor layers comprises on-chip compute circuits configured to receive the pixel data via the interconnects and process the pixel data, the on-chip compute circuits comprising: a machine learning (ML) model accelerator configured to implement a convolutional neural network (CNN) model to process the pixel data; a first memory to store coefficients of the CNN model and instruction codes; a second memory to store the pixel data of a frame; and a controller configured to execute the codes to control operations of the ML model accelerator, the first memory, and the second memory.

Abstract: An eye monitoring system is included in a headset of a virtual reality system or of an augmented reality system. The eye monitoring system determines distances between the eye monitoring system and portions of a user's eye enclosed by the headset. The eye monitoring system projects a temporally periodic pattern of light onto the user's eye via a sensor. The eye monitoring system determines a distance between the eye monitoring system and locations of the user's eye based on a phase shift of the periodic pattern of light captured by each pixel of the sensor. From the determined distances, the eye monitoring system determines features of the user's eye.