Abstract: Methods and systems for quantizing a physical quantity, such as light, are provided. In one example, an apparatus comprises an analog-to-digital (A/D) converter configured to generate raw digital outputs based on performing at least one of: (1) a first quantization operation to quantize a physical stimulus within a first intensity range based on a first A/D conversion relationship, or (2) a second quantization operation to quantize the physical stimulus within a second intensity range based on a second A/D conversion relationship; and a raw output conversion circuit configured generate a refined digital output based on a raw digital output obtained from the A/D converter and at least one predetermined conversion parameter. The at least one conversion parameter compensates for a discontinuity between the first A/D conversion relationship and the second A/D conversion relationship.

Abstract: The disclosed system may include a first layer that includes multiple digital pixel sensors configured to detect light. The system may also include a second layer that includes various image processing components configured to process the light detected by the digital pixel sensors. Still further, the system may include a third layer that includes machine learning (ML) hardware processing components. The image processing components of the second layer may be communicatively connected to the ML hardware processing components of the third layer via multiple micro through-silicon vias (uTSVs). Various other methods of manufacturing, apparatuses, and computer-readable media are also disclosed.

Abstract: In one embodiment, a method includes instructing, at a first time, a camera having a plurality of pixel sensors to capture a first image of an environment comprising an object to determine a first object pose; determining, based on the first object pose, a predicted object pose of the object at a second time; generating pixel-activation instructions based on a buffer region around a projection of a 3D model of the object having the predicted object pose onto a virtual image plane associated with a predicted camera pose, where the size of the buffer region may be dependent on predicted dynamics for the object; instructing, at the second time, the camera to use a subset of the plurality of pixel sensors to capture a second image of the environment according to the pixel-activation instructions, and; determining, based on the second image, a second object pose of the object.

Abstract: A stacked camera-image-sensor circuit may include (i) a first layer that includes a plurality of image sensing elements, (ii) a second layer that includes components that interface with the image sensing elements, and (iii) at least one additional layer that includes image-processing components. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: In one example, an apparatus comprises: a plurality of photodiodes, one or more charge sensing units, one or more analog-to-digital converters (ADCs), and a controller. The controller is configured to: enable the each photodiode to generate charge in response to a different component of the incident light; transfer the charge from the plurality of photodiodes to the one or more charge sensing units to convert to voltages; receive a selection of one or more quantization processes of a plurality of quantization processes corresponding to a plurality of intensity ranges; based on the selection, control the one or more ADCs to perform the selected one or more quantization processes to quantize the voltages from the one or more charge sensing units to digital values representing components of a pixel of different wavelength ranges; and generate a pixel value based on the digital values.

Abstract: One example apparatus for image sub-sampling with a color grid array includes a super-pixel comprising an array of pixels, each pixel comprising a photodiode configured to generate a charge in response to incoming light, a filter positioned to filter the incoming light, a charge storage device to convert the charge to a voltage, a row-select switch, and a column-select switch; an analog-to-digital converter (“ADC”) connected to each of the charge storage devices of the super-pixel via the respective row-select and column-select switches and configured to selectively convert each respective stored voltage into a pixel value in response to a control signal; and wherein each row-select and column-select switch for a pixel is configured to selectively allow the charge or the voltage to propagate to the respective ADC, the row-select and column-select switches arranged in series.

Abstract: In one example, an apparatus is provided. The apparatus comprises an image sensor configured to generate a first raw output to represent a first intensity of incident light based on a first relationship, and to generate a second raw output to represent a second intensity of incident light based on a second relationship. The apparatus further comprises a post processor configured to: generate a first post-processed output based on the first raw output and based on the first relationship such that the first post-processed output is linearly related to the first intensity based on a third relationship, and to generate a second post-processed output based on the second raw output and based on the second relationship such that the second post-processed output is linearly related to the second intensity based on the third relationship.

Abstract: In one example, an apparatus comprises: multiple distinct sets of photodiodes, wherein each set of photodiodes includes one or more photodiodes, one or more charge sensing units, and a controller. The controller is configured to: transfer charge generated by the one or more photodiodes in response to a different component of incident light to the one or more charge sensing units in order to convert the charge to voltages; perform one or more quantization processes of a plurality of quantization processes corresponding to a plurality of intensity ranges, wherein the one or more quantization processes quantizes the voltages from the one or more charge sensing units to digital values representing components of a pixel of different wavelength ranges; and generate a pixel value based on the at least some of the digital values.

Abstract: In one embodiment, a method includes instructing, at a first time, a camera having a plurality of pixel sensors to capture a first image of an environment comprising an object to determine a first object pose; determining, based on the first object pose, a predicted object pose of the object at a second time; generating pixel-activation instructions based on a buffer region around a projection of a 3D model of the object having the predicted object pose onto a virtual image plane associated with a predicted camera pose, where the size of the buffer region may be dependent on predicted dynamics for the object; instructing, at the second time, the camera to use a subset of the plurality of pixel sensors to capture a second image of the environment according to the pixel-activation instructions, and; determining, based on the second image, a second object pose of the object.

Abstract: In some examples, a sensor apparatus comprises: a pixel cell configured to generate a voltage, the pixel cell including one or more photodiodes configured to generate a charge in response to light and a charge storage device to convert the charge to a voltage; an integrated circuit comprising a plurality of integrated memory circuits and configured to: generate, based on a first voltage obtained from the charge storage device of the pixel cell, a first voltage value during a first time period; and generate, based on a second voltage generated by fixed pattern noise from the pixel cell and the integrated circuit, a second voltage value occurring a second time period; and one or more analog-to-digital converters (ADC) configured the convert the first voltage value to a first digital pixel value and the second voltage value to a second digital pixel value; and a processor configured to generate a first altered digital pixel value based on the first digital pixel value and the second digital pixel value.

Abstract: In one embodiment, a computing system accesses a set of 3D locations associated with features in an environment previously captured by a camera from a previous camera pose. The computing system determines a predicted camera pose using the previous camera pose and motion measurements generated using a motion sensor associated with the camera. The computing system projects the set of 3D locations toward the predicted camera pose and onto a 2D image plane associated with the camera. The computing system generates, based on the projected set of 3D locations on the 2D image plane, an activation map specifying a subset of the pixel sensors of the camera that are to be activated. The computing system instructs, using the activation map, the camera to activate the subset of pixel sensors to capture a new image of the environment. The computing system reads pixel values of the new image.

Abstract: In some examples, an apparatus comprises: an array of pixel cells, each pixel cell of the array of pixel cells configured to output a voltage; first analog-to-digital converters (ADCs), each first ADC being associated with a pixel cell or a block of pixel cells; second ADCs, each second ADC being associated with a column of pixel cells of the array of pixel cells; and a controller configured to: in a first mode, enable at least a subset of the first ADCs to perform a quantization operation of the voltages output by at least a subset of the pixel cells in parallel to generate a first image frame; and in a second mode, enable at least a subset of the second ADCs to perform quantization operations sequentially of the voltages output by pixel cells within at least a subset of the columns of pixel cells to generate a second image frame.

Abstract: In one embodiment, a method includes instructing, at a first time, a camera with multiple pixel sensors to capture a first image of an environment comprising an object to determine a first object pose of the object. Based on the first object pose, the method determines a predicted object pose of the object at a second time. The method determines a predicted camera pose of the camera at the second time. The method generates pixel-activation instructions based on a projection of a 3D model of the object having the predicted object pose onto a virtual image plane associated with the predicted camera pose. The method instructs, at the second time, the camera to use a subset of the plurality of pixel sensors to capture a second image of the environment according to the pixel-activation instructions. The method determines, based on the second image, a second object pose of the object.

Abstract: In some examples, an apparatus comprises an array of pixel cells, and processing circuits associated with blocks of pixel cells of the array of pixel cells and associated with first hierarchical power domains. The apparatus further includes banks of memory devices, each bank of memory devices being associated with a block of pixel cells, to store the quantization results of the associated block of pixel cells, the banks of memory devices further being associated with second hierarchical power domains. The apparatus further includes a processing circuits power state control circuit configured to control a power state of the processing circuits based on programming data targeted at each block of pixel cells and global processing circuits power state control signals, and a memory power state control circuit configured to control a power state of the banks of memory devices based on the programming data and global memory power state control signals.

Abstract: In one example, a method comprises: receiving programming data; determining, based on the programming data, at least one of: an integration period in which a charge storage unit including a floating drain accumulates charge received from a photodiode, or a number of times of sampling the charge; enabling the photodiode to accumulate residual charge, and to transmit overflow charge to the charge storage unit after the photodiode saturates; controlling the charge storage unit to accumulate at least a part of the overflow charge received from the photodiode within the integration period; controlling a quantizer to sample the at least a part of the overflow charge or the residual charge for the number of times to obtain the number of samples; and controlling the quantizer to quantize the number of samples to generate the number of quantization results.

Abstract: In one example, an apparatus is provided. The apparatus includes a photodiode, a charge sensing unit, an analog-to-digital converter (ADC), and a controller. The controller is configured to: enable the photodiode to generate charge in response to incident light, accumulate at least a portion of the charge as residual charge until the photodiode becomes saturated by the residual charge, and transfer the remaining portion of the charge to the charge sensing unit as overflow charge if the photodiode becomes saturated by the residual charge. The controller is further configured to: generate, using the ADC, a first digital output based on the residual charge; after generating the first digital output, generate, using the ADC, a second digital output based on the overflow charge; and generate a digital representation of an intensity of the incident light based on at least one of the first digital output or the second digital output.

Abstract: In one example, a method comprises: exposing a first photodiode to incident light to generate first charge; exposing a second photodiode to the incident light to generate second charge; converting, by a first charge sensing unit, the first charge to a first voltage; converting, by a second charge sensing unit, the second charge to a second voltage; controlling an ADC to detect, based on the first voltage, that a quantity of the first charge reaches a saturation threshold, and to measure a saturation time when the quantity of the first charge reaches the saturation threshold; stopping the exposure of the first photodiode and the second photodiode to the incident light based on detecting that the quantity of the first charge reaches the saturation threshold; and controlling the ADC to measure, based on the second voltage, a quantity of the second charge generated by the second photodiode before the exposure ends.

Abstract: In one example, an apparatus comprises: an image sensor comprising a plurality of pixel cells; a frame buffer; and a sensor compute circuit configured to: receive, from the frame buffer, a first image frame comprising first active pixels and first inactive pixels, the first active pixels being generated by a first subset of the pixel cells selected based on first programming data; perform an image-processing operation on a first subset of pixels of the first image frame, whereby a second subset of pixels of the first image frame are excluded from the image-processing operation, to generate a processing output; based on the processing output, generate second programming data; and transmit the second programming data to the image sensor to select a second subset of the pixel cells to generate second active pixels for a second image frame.

Abstract: In one example, a mobile device comprises: a physical link; a plurality of image sensors, each image sensor being configured to transmit image data via the physical link; and a controller coupled to the physical link, whereby the physical link, the plurality of image sensors, and the controller form a multi-drop network. The controller is configured to: transmit a control signal to configure image sensing operations at the plurality of image sensors; receive, via the physical link, image data from at least a subset of the plurality of image sensors; combine the image data from the at least a subset of the plurality of image sensors to obtain an extended field of view (FOV); determine information of a surrounding environment of the mobile device captured within the extended FOV; and provide the information to an application to generate content based on the information.

Abstract: Methods and systems for performing analog-to-digital conversion are proposed. In one example, An analog-to-digital converter (ADC) comprising a quantizer, the quantizer having a first quantization resolution for a first quantization operation subrange and a second quantization resolution for a second quantization operation subrange. At least one of the first quantization resolution or the first quantization operation subrange is programmable. At least one of the second quantization resolution or the second quantization operation subrange is programmable. The quantizer is configured to: receive an input voltage; and based on whether the input voltage belongs to the first quantization operation subrange or to the second quantization operation subrange, quantize the input voltage at the first quantization resolution or at the second quantization resolution to generate a digital output.