Abstract: In one embodiment, a method includes accessing a sequence of raw images comprising at least a first raw image, a second raw image, and a third raw image sequentially, wherein the second raw image comprises image noise, warping the first and third raw images with respect to the second raw image based on an optical flow associated with the sequence of raw images, generating an input tensor based on the first warped raw image, the second raw image, and the third warped raw image, and generating a denoised raw image based on one or more machine-learning models for the second raw image based on the input tensor.

Abstract: An illumination module and a depth camera on a near-eye-display (NED) device used for depth tracking may be subject to strict power consumption budgets. To reduce power consumption of depth tracking, the illumination power of the illumination module is controllably varied. Such variation entails using a previous frame, or previously recorded data, to inform the illumination power used to generate a current frame. Once the NED determines the next minimum illumination power, the illumination module activates at that power level. The illumination module emits electromagnetic (EM) radiation (e.g. IR light), the EM radiation reflects off surfaces in the scene, and the reflected light is captured by the depth camera. The method repeats for subsequent frames, using contextual information from each of the previous frames to dynamically control the illumination power. Thus, the method reduces the overall power consumption of the depth camera assembly of the NED to a minimum level.

Abstract: A stereoscopic 3D imaging system includes multiple imaging sensors with adjustable optics. The adjustable optics are variable to alter the FOV of each of the multiple imaging sensors to improve angular resolution of the imaging system.

Abstract: Disclosed is a technique for 3D reconstruction through a combination of time-of-flight (ToF) and stereoscopy. Use of a single modulation frequency in ToF computation provides a set of ambiguous distances. To determine a single accurate distance out of this candidate set, a stereoscopic comparison of an image pair provides a disambiguating distance. The stereoscopic comparison uses a stored virtual image of at least part of the emitted light, and the detected image of light reflected from an object. The stereoscopic distance is used to determine which of the multiple accurate distances is correct based on proximity. The closest of the multiple distances to the disambiguating distance is taken as the actual distance.

Abstract: A method for active eye-tracking comprises pulsing on and off a plurality of infrared optical sources configured to emit infrared light with a narrow spectral linewidth toward an eye of a user, such that a pulse-on duration is less than a duration needed to fully thermalize each optical source. One or more shuttered optical sensors are configured to receive infrared light reflected off the eye of the user. The shuttered optical sensors are opened for a detection duration based on the pulse-on duration, the shuttered optical sensors. A conformation of the user's eye is indicated based on infrared light received at the shuttered optical sensor during the detection duration.

Abstract: Disclosed are a device and a method of depth sensing that handle light leakage issues. In some embodiments, the depth sensing device includes a light emitter that illuminates an environment of the depth sensing device. The device identifies a first portion of the emitted light that is prevented from reaching the environment of the device due to being redirected by an optical component located in proximity to the light emitter. An imaging sensor of the device detects a second portion of the emitted light that reaches and is reflected by a surface in the environment of the device other than a surface of the optical component. The device generates, based on the second portion of the emitted light, a depth map that includes a plurality of values corresponding to distances relative to the device, wherein said generating excludes from consideration the identified first portion of the emitted light.

Abstract: An illumination source is operated to illuminate an operating environment and an optical sensor is periodically operated for a detection period to detect illumination reflected from one or more subjects within the operating environment. Upon recognizing a source of interfering pulsed illumination within the operating environment, the timing of a subsequent detection period may be varied. In this way, sensing of the interfering pulsed illumination may be averted.

Abstract: A method for three-dimensional imaging includes emitting an output light with a structured light illuminator in a structured light pattern, receiving a trigger command, changing a field of illumination of the illuminator, and changing a field of view of an imaging sensor. The field of view and the field of illumination are linked, such that the field of view of the imaging sensor is the same as the field of illumination of the illuminator at a short throw field of view and a long throw field of view. The method further includes detecting a reflected light with the imaging sensor and measuring a depth value by calculating a distortion of the structured light pattern.

Abstract: A stereoscopic 3D imaging system includes multiple imaging sensors with adjustable optics. The adjustable optics are variable to alter the FOV of each of the multiple imaging sensors to improve angular resolution of the imaging system.

Abstract: A device and method use multiple light emitters with a single, multi-spectrum imaging sensor to perform multi-modal infrared light based depth sensing and visible light based Simultaneous Localization and Mapping (SLAM). The multi-modal infrared based depth sensing may include, for example, any combination of infrared-based spatial mapping, infrared based hand tracking and/or infrared based semantic labeling. The visible light based SLAM may include head tracking, for example.

Abstract: A three dimensional imaging system includes a hybrid imaging sensor with infrared wavelength photoreceptors and visible wavelength photoreceptors integrated in one photoreceptor array. The three dimensional imaging system includes a bandpass filter to filter incoming light before the light is received at the hybrid imaging sensor to reduce crosstalk between the infrared wavelength photoreceptors and visible wavelength photoreceptors. The infrared photoreceptors receive infrared light provided by a structured light source and the visible wavelength photoreceptors receive ambient visible light. The hybrid imaging sensor collects infrared image data and visible light image data concurrently and/or simultaneously.

Abstract: A method for three-dimensional imaging includes emitting an output light with a structured light illuminator in a structured light pattern, receiving a trigger command, changing a field of illumination of the illuminator, and changing a field of view of an imaging sensor. The field of view and the field of illumination are linked, such that the field of view of the imaging sensor is the same as the field of illumination of the illuminator at a short throw field of view and a long throw field of view. The method further includes detecting a reflected light with the imaging sensor and measuring a depth value by calculating a distortion of the structured light pattern.

Abstract: Disclosed are an apparatus and a method for detecting deformation of a structured light depth imaging system and automatic re-calibration for the depth imaging system. In one embodiment, a depth imaging device includes a light projector, a camera and a processor. The light projector emits light corresponding to a projected pattern image having a plurality of features with known locations in the projected pattern image. The camera captures the light reflected by an environment of the depth imaging device and generates a reflected pattern image as a two-dimensional (2D) projection of the environment. The reflected pattern image includes a plurality of features that correspond to the features of the projected pattern image. The processor detects a misalignment for a distance or an orientation between the light projector and the camera based on a relationship between the features in the projected pattern image and the features in the reflected pattern image.

Abstract: A device for detecting light includes a photoreceptor array that is sensitive to the visible light spectrum and has a first wavelength sensor, a second wavelength sensor, and a third wavelength sensor. The first wavelength sensor has a first peak sensitivity at a first wavelength. The second wavelength sensor has a second peak sensitivity at a second wavelength. The third wavelength sensor has a third peak sensitivity at a third wavelength. The first wavelength sensor has a first base sensitivity at least 30% of the first peak sensitivity at the second wavelength and third wavelength. The second wavelength sensor has a second base sensitivity at least 30% of the second peak sensitivity at the first wavelength and third wavelength. The third wavelength sensor has a third base sensitivity at least 30% of the third peak sensitivity at the first wavelength and second wavelength.

Abstract: A stereoscopic 3D imaging system includes multiple imaging sensors with adjustable optics. The adjustable optics are variable to alter the FOV of each of the multiple imaging sensors to improve angular resolution of the imaging system.

Abstract: A method for active eye-tracking comprises pulsing on and off a plurality of infrared optical sources configured to emit infrared light with a narrow spectral linewidth toward an eye of a user, such that a pulse-on duration is less than a duration needed to fully thermalize each optical source. One or more shuttered optical sensors are configured to receive infrared light reflected off the eye of the user. The shuttered optical sensors are opened for a detection duration based on the pulse-on duration, the shuttered optical sensors. A conformation of the user's eye is indicated based on infrared light received at the shuttered optical sensor during the detection duration.

Abstract: Disclosed are an apparatus and a method of compensating temperature shifts of a structured light pattern for a depth imaging system. In some embodiments, a depth imaging device includes a light source, an imaging sensor and a processor. The light source emits light corresponding to a pattern. A temperature drift of the light source can cause a shift of the pattern. The imaging sensor receives the light reflected by environment in front of the depth imaging device and generates a depth map including a plurality of pixel values corresponding to depths of the environment relative to the depth imaging device. The processor estimates the shift of the pattern based on a polynomial model depending on the temperature drift of the light source. The processor further adjusts the depth map based on the shift of the pattern.

Abstract: A sensor device is provided that includes an array of light sensitive pixels configured in a predetermined pattern of light sensitive pixels that alternates between broadband visible light sensitive pixels and infrared light sensitive pixels, each broadband visible light sensitive pixel being configured to have increased sensitivity to light of the visible light spectrum and to have reduced sensitivity to light of the infrared light spectrum compared to each infrared light sensitive pixel, and each infrared light sensitive pixel being configured to have increased sensitivity to light of the infrared light spectrum and to have reduced sensitivity to light of the visible light spectrum compared to each broadband visible light sensitive pixel.

Abstract: A device for three-dimensional imaging includes a structured light illuminator and an imaging sensor. The structured light illuminator has one or more movable illuminator lenses positioned proximate an output of the illuminator that are configured to vary a field of illumination of the illuminator. The imaging sensor has one or more movable imaging lenses positioned proximate an input of the imaging sensor that are configured to vary a field of view of the imaging sensor.

Abstract: A system for three-dimensional imaging includes a structured light illuminator, an imaging sensor, and a time-of-flight controller in data communication with the structured light illuminator and the imaging sensor. The structured light illuminator provides an output light in a structured light pattern and the imaging sensor receives a reflected portion of the output light. The time-of-flight controller coordinates the structured light illuminator and imaging sensor and calculates a time-of-flight of the output light and the reflected portion.