Abstract: Systems and methods for error correction in structured light are disclosed. In one aspect, a method includes receiving, via a receiver sensor, a structured light image of at least a portion of a composite code mask encoding a plurality of codewords, the image including an invalid codeword. The method further includes detecting the invalid codeword. The method further includes generating a plurality of candidate codewords based on the invalid codeword. The method further includes selecting one of the plurality of candidate codewords to replace the invalid codeword. The method further includes generating a depth map for an image of the scene based on the selected candidate codeword. The method further includes generating a digital representation of a scene based on the depth map. The method further includes outputting the digital representation of the scene to an output device.

Abstract: Systems, apparatus, and methods for generating a fused depth map from one or more individual depth maps, wherein the fused depth map is configured to provide robust depth estimation for points within the depth map. The methods, apparatus, or systems may comprise components that identify a field of view (FOV) of an imaging device configured to capture an image of the FOV and select a first depth sensing method. The system or method may sense a depth of the FOV with respect to the imaging device using the first selected depth sensing method and generate a first depth map of the FOV based on the sensed depth of the first selected depth sensing method. The system or method may also identify a region of one or more points of the first depth map having one or more inaccurate depth measurements and determine if additional depth sensing is needed.

Abstract: Systems and methods of triggering an event based on meeting a certain depth criteria in an image. One innovation of a method includes a identifying at least one object in a field of view of an imaging device, the imaging device configured to capture at least one image of the field of view, determining a threshold depth level, measuring a depth of the at least one object within the field of view with respect to the imaging device, comparing the measured depth of the at least one object to the threshold depth level, and capturing an image of the object when the depth of the object within the field of view exceeds the threshold depth level.

Abstract: Methods and apparatus for sharing a bus between multiple imaging sensors, include, in some aspects, a device having at least two imaging sensors, an electronic hardware processor, and an imaging sensor controller. The imaging sensor controller includes first clock and data lines, operably coupling the electronic hardware processor to the imaging sensor controller, and a second clock line, operably coupling the imaging sensor controller to the first imaging sensor and the second imaging sensor. A second data line operably couples the imaging sensor controller to the first imaging sensor. A third data line operably couples the sensor controller to the second imaging sensor. The imaging sensor controller is configured to use the second clock line, and second data line to send a first command to the first imaging sensor, and, use the second clock line, and third data line to send a second command to the second imaging sensor.

Abstract: Described are methods and apparatus for adjusting images of a stereoscopic image pair. The methods and apparatus may capture a first and second image with first and second imaging sensors. The two imaging sensors have intrinsic and extrinsic parameters. A normalized focal distance of a reference imaging sensor may also be determined based on intrinsic and extrinsic parameters. A calibration matrix is then adjusted based on the normalized focal distance. The calibration matrix may be applied to an image captured by an image sensor.

Abstract: Systems and methods for depth enhanced and content aware video stabilization are disclosed. In one aspect, the method identifies keypoints in images, each keypoint corresponding to a feature. The method then estimates the depth of each keypoint, where depth is the distance from the feature to the camera. The method selects keypoints of within a depth tolerance. The method determines camera positions based on the selected keypoints, each camera position representing the position of the camera when the camera captured one of the images. The method determines a first trajectory of camera positions based on the camera positions, and generates a second trajectory of camera positions based on the first trajectory and adjusted camera positions. The method generates adjusted images by adjusting the images based on the second trajectory of camera positions.

Abstract: Aspects relate to a method of generating a high-resolution image containing depth information of an object. In one aspect, the method includes downsampling a first reference image and a second reference image from a first resolution to a second resolution, wherein the first resolution is higher than the second resolution, and wherein the first reference image and the second reference image comprising a stereo image pair. The method further includes generating a depth map at the second resolution based on global minimization techniques, using the downsampled stereo image pair. The method also includes upsampling the depth map from the second resolution to the first resolution and using a guided filter to align contours of the upsampled depth map to contours of the first reference image.

Abstract: Aspects relate to methods of generating a high-resolution image containing object depth information. A method may include capturing a first image of an object using a first camera, the first image including light projected in a known pattern on the object, extracting depth information at a first resolution from the first image, the depth information extracted based on a pattern of the projected light in the first image and capturing a second image of the scene using a second camera, the second image captured at a second resolution which is higher than the first resolution. The method also includes aligning geometries of the first and second image based on a known difference in location of the first camera and the second camera and using the second image to up-sample the depth information from the first resolution to a third resolution, the third resolution is higher than the first resolution.

Abstract: Certain aspects relate to systems and techniques for performing local intensity equalization on images in a set of images exhibiting local intensity variations. For example, the local intensity equalization can be used to perform accurate region matching and alignment of the images. The images can be partitioned into regions of pixel blocks, for instance based on location, shape, and size of identified keypoints in the images. Regions depicting the same feature in the images can be equalized with respect to intensity. Region matching based on the keypoints in the intensity-equalized regions can be performed with accuracy even in images captured by asymmetric sensors or exhibiting spatially varying intensity.

Abstract: A method includes identifying one or more codewords of a bit sequence that fail to satisfy at least one codeword constraint. The method also includes removing the one or more codewords from the bit sequence to generate a punctured bit sequence. The method further includes determining whether the punctured bit sequence is symmetric. The method includes, in response to determining that the punctured bit sequence is symmetric, generating a hermitian symmetric codebook primitive based at least in part on the punctured bit sequence, where the hermitian symmetric codebook primitive is useable to form a diffractive optical element (DOE) of a structured light depth sensing system.

Abstract: A structured light three-dimensional (3D) depth map based on content filtering is disclosed. In a particular embodiment, a method includes receiving, at a receiver device, image data that corresponds to a structured light image. The method further includes processing the image data to decode depth information based on a pattern of projected coded light. The depth information corresponds to a depth map. The method also includes performing one or more filtering operations on the image data. An output of the one or more filtering operations includes filtered image data. The method further includes performing a comparison of the depth information to the filtered image data and modifying the depth information based on the comparison to generate a modified depth map.

Abstract: Aspects relate to an depth sensing system for capturing an image containing depth information of an object. In one embodiment, a depth sensing device for use in conjunction with multiple depth sensing devices for capturing an image containing depth information of an object comprises a near-infrared transmitter comprising a laser capable of producing a near infra-red light beam, a diffractive optical element positioned to receive a light beam emitted from the laser, the diffractive optical element, and a collimating lens, and a near-infrared receiver coupled to the transmitter in a relative position, the receiver comprising a sensor assembly capable of producing an image of the received light, the depth sensing device being configured to transmit and receive near infra-red light beams during a time period that is different than any of the other of two or more transmitter-receiver pairs of devices in communication with the depth sensing device.

Abstract: A method includes identifying one or more codewords of a bit sequence that fail to satisfy at least one codeword constraint. The method also includes removing the one or more codewords from the bit sequence to generate a punctured bit sequence. The method includes, in response to determining that the punctured bit sequence is symmetric, generating a hermitian symmetric codebook primitive based at least in part on the punctured bit sequence, where the hermitian symmetric codebook primitive is useable to form a diffractive optical element (DOE) of a structured light depth sensing system.

Abstract: A method operational on a transmitter device is provided for projecting a composite code mask. A composite code mask on a tangible medium is obtained, where the composite code mask includes a code layer combined with a carrier layer. The code layer may include uniquely identifiable spatially-coded codewords defined by a plurality of symbols. The carrier layer may be independently ascertainable and distinct from the code layer and includes a plurality of reference objects that are robust to distortion upon projection. At least one of the code layer and carrier layer may be pre-shaped by a synthetic point spread function prior to projection. At least a portion of the composite code mask is projected, by the transmitter device, onto a target object to help a receiver ascertain depth information for the target object with a single projection of the composite code mask.

Abstract: A method for generating codes for a code mask is provided. A plurality of symbols may be arranged into an n1 by n2 symbol structure, where n1 and n2 are integer values. A plurality of codewords may be defined from different overlapping k1 by k2 windows within the symbol structure, wherein co-linear and spatially overlapping windows define unique codewords, and the codewords are unique in a first direction of the symbol structure but are repeated in a second direction that is perpendicular to the first direction. A plurality of the symbol structures as a code mask, wherein symbols in two adjacent k1 by k2 windows are selected so as to avoid codeword aliasing of codewords in the two adjacent k1 by k2 windows.

Abstract: Methods, devices, and computer program products for multi-frame termporal de-noising using image alignment are describe. In one aspect, a method of capturing an image using a multi-frame temporal de-noising is described. The method includes capturing a plurality of frames and aligning the captured plurality of frames with each other. The method further includes determining a subset of frames of the captured plurality of frames, the subset determined based upon a focus quality of each frame of the plurality of frames. Finally, the method includes combining the subset of frames into a single image using a motion filter to reduce blurriness and ghosting.

Abstract: An interactive display, including a cover glass having a front surface that includes a viewing area provides an input/output (I/O) interface for a user of an electronic device. An arrangement includes a processor, a light source, and a camera disposed outside the periphery of the viewing area coplanar with or behind the cover glass. The camera receives scattered light resulting from interaction, with an object, of light outputted from the interactive display, the outputted light being received by the cover glass from the object and directed toward the camera. The processor determines, from image data output by the camera, an azimuthal angle of the object with respect to an optical axis of the camera and/or a distance of the object from the camera.

Abstract: In photography, High Dynamic Range (HDR) technology typically consists of (1) acquiring a wide dynamic range image and (2) adapting the wide dynamic range image to fit to the display range of the device. The first part can be achieved by using a special sensor or by combining two or more images with same or different exposures, and is optional. The second part, contrast adaptation, locally adapts the tone mapping function thus effectively re-using the available range. Described is a system and method that enables a user to create a new image by selectively combining contrast adapted and non-contrast adapted versions of the same image. The new image can retain the natural quality of the well illuminated areas and enhance salient features as selected by the user.

Abstract: Systems and methods for correcting stereo yaw of a stereoscopic image sensor pair using autofocus feedback are disclosed. A stereo depth of an object in an image is estimated from the disparity of the object between the images captured by each sensor of the image sensor pair. An autofocus depth to the object is found from the autofocus lens position. If the difference between the stereo depth and the autofocus depth is non zero, one of the images is warped and the disparity is recalculated until the stereo depth and the autofocus depth to the object is substantially the same.

Abstract: Described are systems and methods for measuring objects using stereoscopic imaging. After determining keypoints within a set of stereoscopic images, a user may select a desired object within an imaged scene to be measured. Using depth map information and information about the boundary of the selected object, the desired measurement may be calculated and displayed to the user on a display device. Tracking of the object in three dimensions and continuous updating of the measurement of a selected object may also be performed as the object or the imaging device is moved.