Patents by Inventor Xinqiao Liu

Xinqiao Liu has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20210264679
    Abstract: A sensor assembly for determining one or more features of a local area is presented herein. The sensor assembly includes a plurality of stacked sensor layers. A first sensor layer of the plurality of stacked sensor layers located on top of the sensor assembly includes an array of pixels. The top sensor layer can be configured to capture one or more images of light reflected from one or more objects in the local area. The sensor assembly further includes one or more sensor layers located beneath the top sensor layer. The one or more sensor layers can be configured to process data related to the captured one or more images. Different sensor architectures featuring various arrangements of memory and computing devices are described, some of which feature in-memory computing. A plurality of sensor assemblies can be integrated into an artificial reality system, e.g., a head-mounted display.
    Type: Application
    Filed: May 6, 2021
    Publication date: August 26, 2021
    Inventors: Xinqiao LIU, Barbara DE SALVO, Hans REYSERHOVE, Ziyun LI, Asif Imtiaz KHAN, Syed Shakib SARWAR
  • Publication number: 20210265401
    Abstract: In one example, a method comprises: within an integration period: enabling a photodiode of a pixel cell to accumulate charge responsive to incident light, and transferring the charge from the photodiode to a charge storage device of the pixel cell. The method further comprises: performing, using a sampling capacitor, a sample-and-hold operation to convert the charge stored in the charge storage device into a voltage; and generating a digital output based on the voltage to represent an intensity of the incident light received by the photodiode.
    Type: Application
    Filed: April 21, 2021
    Publication date: August 26, 2021
    Inventors: Xinqiao LIU, Song CHEN
  • Patent number: 11102430
    Abstract: In one example, a pixel cell comprises a first photodiode to generate a first charge and a second photodiode to generate a second charge. The pixel cell may include a charge sensing unit shared between the first photodiode and the second photodiode. The charge sensing unit may include a charge storage device to temporarily store a charge and convert the charge to a voltage. The pixel cell may include a quantizer to quantize the voltage output by the charge sensing unit, and a memory to store the quantization output. Depending on an operation mode, the first charge and the second charge can be controlled to flow simultaneously to the charge sensing unit for read out, or can be controlled to flow separately to the charge sensing unit for read out. The pixel cell further includes a memory to store a quantization result of the first charge and the second charge.
    Type: Grant
    Filed: December 9, 2019
    Date of Patent: August 24, 2021
    Assignee: Facebook Technologies, LLC
    Inventors: Song Chen, Xinqiao Liu
  • Patent number: 11089241
    Abstract: In one example, a method comprises: transmitting a first signal to transfer a first portion of a first charge from a first photodiode to a first charge sensing unit to obtain a first measurement result and transmitting a second signal to transfer a second portion of the first charge from the first photodiode to a second charge sensing unit to obtain a second measurement result. The timing of transmission of the first signal and the second signal are based on the indirect time-of-flight measurement operation. The method further comprises performing the indirect time-of-flight measurement operation based on the first measurement result and the second measurement result; transmitting a third signal to transfer a second charge from a second photodiode to the second charge sensing unit via the first photodiode to obtain a third measurement result; and performing a D sensing operation based on the third measurement result.
    Type: Grant
    Filed: June 10, 2019
    Date of Patent: August 10, 2021
    Assignee: Facebook Technologies, LLC
    Inventors: Song Chen, Xinqiao Liu
  • Patent number: 11089210
    Abstract: In one example, a method comprises: receiving programming data; determining, based on the programming data, at least one of: an integration period in which a charge storage unit including a floating drain accumulates charge received from a photodiode, or a number of times of sampling the charge; enabling the photodiode to accumulate residual charge, and to transmit overflow charge to the charge storage unit after the photodiode saturates; controlling the charge storage unit to accumulate at least a part of the overflow charge received from the photodiode within the integration period; controlling a quantizer to sample the at least a part of the overflow charge or the residual charge for the number of times to obtain the number of samples; and controlling the quantizer to quantize the number of samples to generate the number of quantization results.
    Type: Grant
    Filed: June 10, 2019
    Date of Patent: August 10, 2021
    Assignee: Facebook Technologies, LLC
    Inventors: Andrew Samuel Berkovich, Xinqiao Liu
  • Publication number: 20210243390
    Abstract: In one example, an apparatus is provided. The apparatus includes a photodiode, a charge sensing unit, an analog-to-digital converter (ADC), and a controller. The controller is configured to: enable the photodiode to generate charge in response to incident light, accumulate at least a portion of the charge as residual charge until the photodiode becomes saturated by the residual charge, and transfer the remaining portion of the charge to the charge sensing unit as overflow charge if the photodiode becomes saturated by the residual charge. The controller is further configured to: generate, using the ADC, a first digital output based on the residual charge; after generating the first digital output, generate, using the ADC, a second digital output based on the overflow charge; and generate a digital representation of an intensity of the incident light based on at least one of the first digital output or the second digital output.
    Type: Application
    Filed: January 15, 2021
    Publication date: August 5, 2021
    Inventors: Xinqiao LIU, Song CHEN, Andrew Samuel BERKOVICH
  • Patent number: 11057581
    Abstract: Examples of an apparatus are disclosed. In some example, an apparatus may include an array of digital pixel cells, each digital pixel cell including a photodiode and a memory device to store a digital output generated based on charge generated by the photodiode in an exposure period. The apparatus may also include an image processor configured to: receive first digital outputs from the memory devices of a first set of digital pixel cells of the array of digital pixel cells; determine, from the first set of digital pixel cells, a second set of digital pixel cells of which the first digital outputs satisfy one or more pre-determined conditions; identify, based on the second set of digital pixel cells, a third set of digital pixel cells; receive the second digital outputs generated by the third set of digital pixel cells; and perform image processing operations based on the second digital outputs.
    Type: Grant
    Filed: January 23, 2019
    Date of Patent: July 6, 2021
    Assignee: Facebook Technologies, LLC
    Inventor: Xinqiao Liu
  • Publication number: 20210203830
    Abstract: In one example, a method comprises: exposing a first photodiode to incident light to generate first charge; exposing a second photodiode to the incident light to generate second charge; converting, by a first charge sensing unit, the first charge to a first voltage; converting, by a second charge sensing unit, the second charge to a second voltage; controlling an ADC to detect, based on the first voltage, that a quantity of the first charge reaches a saturation threshold, and to measure a saturation time when the quantity of the first charge reaches the saturation threshold; stopping the exposure of the first photodiode and the second photodiode to the incident light based on detecting that the quantity of the first charge reaches the saturation threshold; and controlling the ADC to measure, based on the second voltage, a quantity of the second charge generated by the second photodiode before the exposure ends.
    Type: Application
    Filed: February 19, 2021
    Publication date: July 1, 2021
    Inventors: Xinqiao LIU, Andrew Samuel BERKOVICH, Song CHEN
  • Patent number: 11010911
    Abstract: A depth estimation system is described capable of determining depth information using two images from two cameras. A first camera captures a first image and a second camera captures a second image, both images including a plurality of light channels. In a first light channel of the plurality of light channels, the system calculates a census transform for each pixel of the first image and a census transform for each pixel of the second image. In a second light channel of the plurality of light channels, the system calculates a census transform for each pixel of the first image and a census transform for each pixel of the second image. The system generates a depth map based in part on the census transforms for each pixel of the first image and the second image in the first light channel and in the second light channel.
    Type: Grant
    Filed: May 21, 2019
    Date of Patent: May 18, 2021
    Assignee: Facebook Technologies, LLC
    Inventors: Michael Hall, Xinqiao Liu, Zhaoming Zhu, Rajesh Lachhmandas Chhabria, Huixuan Tang, Shuochen Su
  • Patent number: 11004881
    Abstract: Examples of image sensors are provided. In one example, a pixel cell comprises a first semiconductor die, a sampling capacitor, and a second semiconductor die which may include the sampling capacitor. The first semiconductor die includes a photodiode and a charge sensing device. The second semiconductor die forms a stack with the first semiconductor die, the second semiconductor die including an interface circuit coupled with the photodiode, the charge sensing device, and the sampling capacitor. The interface circuit is configured to: enable the photodiode to accumulate charge responsive to incident light within a integration period; transfer the charge from the photodiode to the charge sensing device; perform, using the sampling capacitor, a sample-and-hold operation to convert the charge in the charge sensing device into a voltage; and generate a digital output based on the voltage to represent an intensity of the incident light received by the photodiode.
    Type: Grant
    Filed: March 29, 2019
    Date of Patent: May 11, 2021
    Assignee: Facebook Technologies, LLC
    Inventors: Xinqiao Liu, Song Chen
  • Patent number: 11004222
    Abstract: A depth measurement assembly (DMA) includes a structured light emitter, an augmented camera, and a controller. The structured light emitter projects structured light into a local area under instructions from the controller. The augmented camera generates image data of an object illuminated with the structured light pattern projected by the structured light emitter in accordance with camera instructions generated by the controller. The augmented camera includes a high speed computation tracking sensor that comprises a plurality of augmented photodetectors. Each augmented photodetector converts light to data and stores the data in its own memory unit. The controller receives the image data and determines depth information of the object in the local area based in part on the image data. The depth measurement unit can be incorporated into a head-mounted display (HMD).
    Type: Grant
    Filed: May 4, 2020
    Date of Patent: May 11, 2021
    Assignee: Facebook Technologies, LLC
    Inventors: Xinqiao Liu, Richard Andrew Newcombe, Steven John Lovegrove, Renzo De Nardi
  • Publication number: 20210133452
    Abstract: In one example, a mobile device comprises: a physical link; a plurality of image sensors, each image sensor being configured to transmit image data via the physical link; and a controller coupled to the physical link, whereby the physical link, the plurality of image sensors, and the controller form a multi-drop network. The controller is configured to: transmit a control signal to configure image sensing operations at the plurality of image sensors; receive, via the physical link, image data from at least a subset of the plurality of image sensors; combine the image data from the at least a subset of the plurality of image sensors to obtain an extended field of view (FOV); determine information of a surrounding environment of the mobile device captured within the extended FOV; and provide the information to an application to generate content based on the information.
    Type: Application
    Filed: October 29, 2020
    Publication date: May 6, 2021
    Inventors: Andrew Samuel Berkovich, Xinqiao Liu, Hans Reyserhove
  • Patent number: 10969273
    Abstract: Methods and systems for performing analog-to-digital conversion are proposed. In one example, An analog-to-digital converter (ADC) comprising a quantizer, the quantizer having a first quantization resolution for a first quantization operation subrange and a second quantization resolution for a second quantization operation subrange. At least one of the first quantization resolution or the first quantization operation subrange is programmable. At least one of the second quantization resolution or the second quantization operation subrange is programmable. The quantizer is configured to: receive an input voltage; and based on whether the input voltage belongs to the first quantization operation subrange or to the second quantization operation subrange, quantize the input voltage at the first quantization resolution or at the second quantization resolution to generate a digital output.
    Type: Grant
    Filed: January 16, 2019
    Date of Patent: April 6, 2021
    Assignee: Facebook Technologies, LLC
    Inventors: Andrew Samuel Berkovich, Xinqiao Liu
  • Patent number: 10972715
    Abstract: A depth camera assembly (DCA) determines depth information within a local area. The DCA may selectively process a subset of data captured by an imaging sensor and obtained from the imaging sensor, such as pixels corresponding to a region of interest, for depth information. Alternatively, the DCA may limit retrieval of data from the imaging sensor to pixels corresponding to the region of interest from the imaging sensor for processing for depth information. The depth processing may include a semi-global match (SGM) algorithm, and the DCA adjusts a number of neighboring pixels used for determining depth information for a specific pixel based on one or more criteria. In some embodiments, the DCA performs the depth processing by analyzing images from different image sensors using left to right and right to left correspondence checks that are performed in parallel.
    Type: Grant
    Filed: May 21, 2019
    Date of Patent: April 6, 2021
    Assignee: Facebook Technologies, LLC
    Inventors: Michael Hall, Xinqiao Liu, Zhaoming Zhu, Rajesh Lachhmandas Chhabria, Huixuan Tang, Shuochen Su
  • Patent number: 10951849
    Abstract: Disclosed herein are techniques for digital imaging. A digital pixel image sensor includes a digitizer in each pixel of a plurality of pixels, where the digitizer digitizes analog output signals from a photodiode of the pixel using a comparator, a global reference ramp signal, and a clock counter. In some embodiments, the comparator includes a pre-charging circuit, rather than a constant biasing circuit, to reduce the power consumption of each pixel. In some embodiments, each pixel includes a digital or analog correlated double sampling (CDS) circuit to reduce noise and provide a higher dynamic range.
    Type: Grant
    Filed: September 10, 2019
    Date of Patent: March 16, 2021
    Assignee: Facebook Technologies, LLC
    Inventor: Xinqiao Liu
  • Patent number: 10929997
    Abstract: A depth camera assembly (DCA) determines depth information within a local area by capturing images of the local area including a local region using a plurality of imaging sensors. The local region is represented by a first set of pixels in each captured image. For each image, the DCA identifies the first set of pixels corresponding to the surface in the local region and determines a depth measurement from the DCA to the local region by comparing the first set of pixels from images captured by different imaging sensors. To determine depth measurements for second sets of pixels neighboring the first set of pixels, the DCA selectively propagates depth information from the first set of pixels to second sets of pixels satisfying one or more criteria (e.g., satisfying a threshold saturation measurement or a threshold contrast measurement).
    Type: Grant
    Filed: May 21, 2019
    Date of Patent: February 23, 2021
    Assignee: Facebook Technologies, LLC
    Inventors: Michael Hall, Xinqiao Liu, Zhaoming Zhu, Rajesh Lachhmandas Chhabria, Huixuan Tang, Shuochen Su, Zihe Gao
  • Patent number: 10931884
    Abstract: In one example, a method comprises: exposing a first photodiode to incident light to generate first charge; exposing a second photodiode to the incident light to generate second charge; converting, by a first charge sensing unit, the first charge to a first voltage; converting, by a second charge sensing unit, the second charge to a second voltage; controlling an ADC to detect, based on the first voltage, that a quantity of the first charge reaches a saturation threshold, and to measure a saturation time when the quantity of the first charge reaches the saturation threshold; stopping the exposure of the first photodiode and the second photodiode to the incident light based on detecting that the quantity of the first charge reaches the saturation threshold; and controlling the ADC to measure, based on the second voltage, a quantity of the second charge generated by the second photodiode before the exposure ends.
    Type: Grant
    Filed: August 19, 2019
    Date of Patent: February 23, 2021
    Assignee: Facebook Technologies, LLC
    Inventors: Xinqiao Liu, Andrew Samuel Berkovich, Song Chen
  • Patent number: 10923523
    Abstract: In one example, an apparatus comprises: a first photodiode configured to convert a first component of light to a first charge, second photodiode configured to convert a second component of the light to a second charge; and an interface circuit configured to: perform a first quantization and a second quantization of the first charge to generate, respectively, a first result and a second result, the first quantization and the second quantization being associated with different light intensity ranges; provide one of the first result or the second result to represent an intensity of the first component of a pixel; perform the first quantization and the second quantization of the second charge to generate, respectively, a third result and a fourth result; and provide one of the third result or the fourth result to represent an intensity of the second component of the pixel.
    Type: Grant
    Filed: April 15, 2019
    Date of Patent: February 16, 2021
    Assignee: Facebook Technologies, LLC
    Inventors: Xinqiao Liu, Song Chen
  • Patent number: 10917625
    Abstract: A dual-band pixel includes a backside passivation layer, a corresponding input terminal coupled to the backside passivation layer, a first n-type layer covering the backside passivation layer, a p-type layer covering the first n-type layer, a second n-type layer within the p-type layer, and a pinning layer covering the second n-type layer. A first or second voltage is applied to the corresponding input terminal to operate the dual-band pixel in a visible or infrared (IR) mode. A depth camera assembly (DCA) may include a sensor pixel array comprising a plurality of dual-band pixels. The DCA may take visible or IR images in a time multiplexed manner using the sensor pixel array and determine depth information based on the captured IR images.
    Type: Grant
    Filed: August 3, 2017
    Date of Patent: February 9, 2021
    Assignee: Facebook Technologies, LLC
    Inventor: Xinqiao Liu
  • Patent number: 10917589
    Abstract: Examples of a pixel cell are disclosed. In one example, a pixel cell may include a first semiconductor layer including a photodiode and one or more transistor devices configured to convert charges generated by the photodiode into an analog signal. The pixel cell may also include a second semiconductor layer including one or more transistor devices configured to convert the analog signal to one or more digital signals. The first semiconductor layer and the second semiconductor layer may form a stack structure. In another example, a pixel cell may include a photodiode and a capacitor. The pixel cell may be operated, in a first mode of measurement, to measure the charges stored at the capacitor when the capacitor is electrically coupled with the photodiode, and in a second mode of measurement, to measure the charges stored at the capacitor when the capacitor is electrically isolated from the photodiode.
    Type: Grant
    Filed: September 28, 2017
    Date of Patent: February 9, 2021
    Assignee: Facebook Technologies, LLC
    Inventor: Xinqiao Liu