Abstract: Indirect time-of-flight camera systems for operating in multiple optical channels using active modulated light and accompanying methods of operation are provided. In one aspect, the indirect time-of-flight camera system includes first and second modulatable laser sources outputting light of different wavelengths for illuminating a target environment. The camera system further includes a wavelength-selective reflective element designed to reflect the light of a first wavelength and to transmit the light of a second wavelength. The camera system further includes a controller comprising instructions executable to control the camera system to, in a first time period, activate the first modulatable laser source and deactivate the second modulatable laser source, and in a second time period, deactivate the first modulatable laser source and activate the second modulatable laser source.

Abstract: Examples are provided relating to recovering depth data from noisy phase data of low-signal pixels. One example provides a computing system, comprising a logic machine, and a storage machine holding instructions executable by the logic machine to process depth data by obtaining depth image data and active brightness image data for a plurality of pixels, the depth image data comprising phase data for a plurality of frequencies, and identifying low-signal pixels based at least on the active brightness image data. The instructions are further executable to apply a denoising filter to phase data of the low-signal pixels to obtain denoised phase data and not applying the denoising filter to phase data of other pixels. The instructions are further executable to, after applying the denoising filter, perform phase unwrapping on the phase data for the plurality of frequencies to obtain a depth image, and output the depth image.

Abstract: One example provides a computing system comprising a depth sensor comprising a plurality of pixels, and a storage machine holding instructions executable by a logic machine to, for each pixel, make K phase measurements to form a set of noisy phase measurements, determine a location at which a projection line that passes through the set of noisy phase measurements in a K-dimensional phase space passes through a lower dimensional plane, the projection line being parallel to a noise free phase evolution line, compare the location to a plurality of independent terms of a predetermined matrix of points in the lower dimensional plane, locate a corresponding set of noiseless phase orders by using a selected set of independent terms to reference a look-up table, determine a distance value for the pixel based upon the corresponding set of noiseless phase orders, and output the distance value for the pixel.

Abstract: Examples are disclosed that relate to motion blur corrections for time-of-flight (ToF) depth imaging. One example provides a depth camera comprising a ToF image sensor, a logic machine, and a storage machine storing instructions executable by the logic machine to receive depth image data from the ToF image sensor, the depth image data comprising phase data and active brightness (AB) data, determine a first two-dimensional (2D) AB image corresponding to a first modulation frequency, and determine a second 2D AB image corresponding to a second modulation frequency. The instructions are further executable to determine a 2D translation based upon a comparison between the first 2D AB image and the second 2D AB image, determine corrected phase data based on the 2D translation to form corrected phase data, perform phase unwrapping on the corrected phase data to obtain a three-dimensional (3D) depth image, and output the 3D depth image.

Abstract: One example provides a computing system comprising a depth sensor comprising a plurality of pixels, and a storage machine holding instructions executable by a logic machine to, for each pixel, make K phase measurements to form a set of noisy phase measurements, determine a location at which a projection line that passes through the set of noisy phase measurements in a K-dimensional phase space passes through a lower dimensional plane, the projection line being parallel to a noise free phase evolution line, compare the location to a plurality of independent terms of a predetermined matrix of points in the lower dimensional plane, locate a corresponding set of noiseless phase orders by using a selected set of independent terms to reference a look-up table, determine a distance value for the pixel based upon the corresponding set of noiseless phase orders, and output the distance value for the pixel.

Abstract: Examples are provided that relate to processing depth camera data over a distributed computing system, where phase unwrapping is performed prior to denoising. One example provides a time-of-flight camera comprising a time-of-flight depth image sensor, a logic machine, a communication subsystem, and a storage machine holding instructions executable by the logic machine to process time-of-flight image data acquired by the time-of-flight depth image sensor by, prior to denoising, performing phase unwrapping pixel-wise on the time-of-flight image data to obtain coarse depth image data comprising depth values; and send the coarse depth image data and active brightness image data to a remote computing system via the communication subsystem for denoising.

Abstract: Examples are disclosed that relate to methods and systems for determining a depth of a pixel of a depth image. In one example, a method comprises illuminating a scene with amplitude-modulated light of a plurality K of different frequencies. A sensor detects the amplitude-modulated light as reflected from the scene. A plurality K of complex signals k is formed by determining, for each frequency k of the plurality of different frequencies K, a complex signal k comprising a real portion k and an imaginary portion k. A comparison function is used to compare the plurality K of complex signals to a corresponding plurality K of modeled signals, each modeled signal k comprising a modeled real portion k and a modeled imaginary portion k. The method further comprises determining the depth of the pixel based at least on the comparison function and outputting the depth of the pixel.

Abstract: Examples are disclosed herein relating to signal processing in a time of flight (ToF) system. One example provides, a method comprising emitting, via a light source, amplitude-modulated light toward an object, acquiring, via an image sensor comprising a plurality of pixels, a plurality of image frames capturing light emitted from the light source that is reflected by the object, wherein the plurality of image frames are acquired at two or more different frequencies of the amplitude-modulated light and collectively form a multifrequency frame, and for each pixel of the multifrequency frame, determining a brightness level, applying an adaptive denoising process by setting a kernel size based on the brightness level, and performing a phase unwrapping process to determine a depth value for the pixel.

Abstract: A method to correct a digital image to reverse the effect of signal diffusion among pixels of the digital image. For a target pixel j of the digital image, a set of signal values and a set of signal amplitudes are received, each corresponding to a set of kernel pixels i surrounding and including the target pixel j. For each kernel pixel i, a weighting coefficient is computed based on the signal amplitude of that kernel pixel i and on the signal amplitude of the target pixel j. A linear combination of signal values corresponding to the set of kernel pixels i is computed, wherein the signal value for each pixel i is weighted by the weighting coefficient corresponding to that pixel i. The linear combination is stored in volatile memory of an electronic device as a corrected signal value for the target pixel j.

Abstract: One example provides a computing device comprising a logic machine and a storage machine holding instructions executable by the logic machine to implement a depth image processing pipeline comprising a neural network, the neural network comprising an edge detecting layer. The neural network is configured to receive input of an active brightness image and receive input of one or more of real data or imaginary data of a complex depth image, the complex depth image corresponding to the active brightness image. The neural network is further configured to, at the edge detecting layer, apply one or more convolutional processes to the active brightness image to identify one or more edge pixels in the active brightness image, and at a second layer, denoise one or more of the real data or the imaginary data of the complex depth image based on the one or more edge pixels identified.

Abstract: Examples are disclosed relating to performing denoising and adaptive precision control on time-of-flight sensor data using noise metrics. One example provides a computing system, comprising, a logic machine, and a storage machine holding instructions executable by the logic machine to obtain time-of-flight (ToF) image data comprising a plurality of pixels, for each pixel of the ToF image data, determine one or more noise metrics by applying a spatial kernel, segment the ToF image data based on the one or more noise metrics to obtain differently classified pixels, during a denoising phase, process pixels of a first classification differently than pixels of a second classification, after the denoising phase, determine a depth image, and output the depth image.

Abstract: Examples are provided relating to recovering depth data from noisy phase data of low-signal pixels. One example provides a computing system, comprising a logic machine, and a storage machine holding instructions executable by the logic machine to process depth data by obtaining depth image data and active brightness image data for a plurality of pixels, the depth image data comprising phase data for a plurality of frequencies, and identifying low-signal pixels based at least on the active brightness image data. The instructions are further executable to apply a denoising filter to phase data of the low-signal pixels to obtain denoised phase data and not applying the denoising filter to phase data of other pixels. The instructions are further executable to, after applying the denoising filter, perform phase unwrapping on the phase data for the plurality of frequencies to obtain a depth image, and output the depth image.

Abstract: Examples are disclosed relating to performing signal processing on time-of-flight sensor data using pixelwise temporal metrics. One example provides a computing system comprising a logic machine, and a storage machine holding instructions executable by the logic machine to obtain temporal phase data for a plurality of pixels as acquired by a time-of-flight image sensor, the temporal phase data comprising phase data for a plurality of light modulation frequencies, determine temporal active brightness data for the pixel, and, for each pixel of the plurality of pixels, determine a statistical metric for the temporal active brightness data. The instructions are further executable to perform phase unwrapping on the temporal phase data for the plurality of pixels to obtain a depth image, based on the statistical metric for the temporal active brightness data, perform a denoising operation on at least some pixels of the depth image, and output the depth image.

Abstract: A method to correct a digital image to reverse the effect of signal diffusion among pixels of the digital image. For a target pixel j of the digital image, a set of signal values and a set of signal amplitudes are received, each corresponding to a set of kernel pixels i surrounding and including the target pixel j. For each kernel pixel i, a weighting coefficient is computed based on the signal amplitude of that kernel pixel i and on the signal amplitude of the target pixel j. A linear combination of signal values corresponding to the set of kernel pixels i is computed, wherein the signal value for each pixel i is weighted by the weighting coefficient corresponding to that pixel i. The linear combination is stored in volatile memory of an electronic device as a corrected signal value for the target pixel j.

Abstract: Examples are disclosed herein relating to signal processing in a time-of-flight (ToF) system. One example provides, a method comprising emitting, via a light source, amplitude-modulated light toward an object, acquiring, via an image sensor comprising a plurality of pixels, a plurality of image frames capturing light emitted from the light source that is reflected by the object, wherein the plurality of image frames are acquired at two or more different frequencies of the amplitude-modulated light and collectively form a multifrequency frame, and for each pixel of the multifrequency frame, determining a brightness level, applying an adaptive denoising process by setting a kernel size based on the brightness level, and performing a phase unwrapping process to determine a depth value for the pixel.

Abstract: Examples are provided that relate to processing depth camera data over a distributed computing system, where phase unwrapping is performed prior to denoising. One example provides a time-of-flight camera comprising a time-of-flight depth image sensor, a logic machine, a communication subsystem, and a storage machine holding instructions executable by the logic machine to process time-of-flight image data acquired by the time-of-flight depth image sensor by, prior to denoising, performing phase unwrapping pixel-wise on the time-of-flight image data to obtain coarse depth image data comprising depth values; and send the coarse depth image data and the active brightness image data to a remote computing system via the communication subsystem for denoising.

Abstract: One example provides a computing system comprising a depth sensor comprising a plurality of pixels, and a storage machine holding instructions executable by a logic machine to, for each pixel, make K phase measurements to form a set of noisy phase measurements, determine a location at which a projection line that passes through the set of noisy phase measurements in a K-dimensional phase space passes through a lower dimensional plane, the projection line being parallel to a noise free phase evolution line, compare the location to a plurality of independent terms of a predetermined matrix of points in the lower dimensional plane, locate a corresponding set of noiseless phase orders by using a selected set of independent terms to reference a look-up table, determine a distance value for the pixel based upon the corresponding set of noiseless phase orders, and output the distance value for the pixel.

Abstract: A method to correct a digital image to reverse the effect of signal diffusion among pixels of the digital image. For a target pixel j of the digital image, a set of signal values and a set of signal amplitudes are received, each corresponding to a set of kernel pixels i surrounding and including the target pixel j. For each kernel pixel i, a weighting coefficient is computed based on the signal amplitude of that kernel pixel i and on the signal amplitude of the target pixel j. A linear combination of signal values corresponding to the set of kernel pixels i is computed, wherein the signal value for each pixel i is weighted by the weighting coefficient corresponding to that pixel i. The linear combination is stored in volatile memory of an electronic device as a corrected signal value for the target pixel j.

Abstract: Examples are disclosed that relate to determining phase orders for phase data in a time-of-flight camera sensor, for use in unwrapping the phase data. One example provides a computing system comprising a depth sensor comprising a plurality of pixels, an illumination source, and a storage machine holding instructions executable by a logic machine to control the illumination source to output amplitude-modulated light at two or more modulation frequencies. The instructions are further executable to, for each pixel, make two or more phase measurements corresponding to different modulation frequencies, based at least on the two or more phase measurements, determine a series of phase order sets, determine a most likely phase order set by comparing the series of phase order sets to a line representing an evolution of phase with distance, and based on the most likely phase order set, determine a distance value associated with the pixel.

Abstract: A camera system. The camera system includes a sensor array; an infrared (IR) illumination system configured to emit active IR light in an IR light sub-band; a spectral illumination system configured to emit active spectral light in a spectral light sub-band; one or more logic machines; and one or more storage machines. The storage machines hold instructions executable by the one or more logic machines to address the sensor array to acquire an ambient image; based on at least the ambient image, activate the IR illumination system and address the sensor array to acquire an actively-IR-illuminated depth image; and based on at least the actively-IR-illuminated depth image, activate the spectral illumination system and address the sensor array to acquire an actively-spectrally-illuminated spectral image.