Abstract: The present disclosure overcomes the limitations of the prior art by providing 3D compression (i.e., compression that also accounts for changes in depth), including 3D compression of video sequences. In one aspect, depth information is obtained by processing captured images from a multi-aperture imaging device. The availability of IR images and depth information facilitates additional processing compared to when only color images are available.

Abstract: A multi-aperture imaging system determines depth map information. A series of image frames of a scene are captured. The frames include a normal image frame and at least one structured image frame. The multi-aperture imaging system determines edge information of an object in the scene using a deblur technique and the normal image frame. The multi-aperture imaging system determines fill depth information for the object based in part on the at least one structured image frame. The multi-aperture imaging system generates a depth map of the scene using the edge depth information and the fill depth information.

Abstract: The present disclosure overcomes the limitations of the prior art by providing 3D compression (i.e., compression that also accounts for changes in depth), including 3D compression of video sequences. In one aspect, depth information is obtained by processing captured images from a multi-aperture imaging device. The availability of IR images and depth information facilitates additional processing compared to when only color images are available.

Abstract: An image sensor includes both visible pixels and infrared pixels. The visible pixels produce signals indicative of light within a visible band received by the visible pixels, and the infrared pixels produce signals indicative of light within an infrared band received by the infrared pixels. A selective infrared (SIR) filter array is integrated on the image sensor. The SIR filter array includes SIR pixel filters that are positioned to filter out light within the infrared band propagating to the visible pixels. In this way, infrared crosstalk to the visible pixels can be reduced.

Abstract: An application programming interface (API) provides access to a multi-aperture imaging system. By using the API, higher level applications can access and/or specify the image capture by the multi-aperture imaging device. For example, the API can support the specification of various image capture parameters: resolution, frame rate, exposure, flash, filtering, and additional processing to be applied. These could be specified for all frames and all channels (Red, Green, Blue and Infrared), or could be specified for specific subframes within a frame and/or for specific channels. The API can also specify output data from the multi-aperture imaging device. This can include raw pixel data, image data (color and Infrared), depth maps, and other processed image data. It can also include metadata and data about the status of the multi-aperture imaging device.

Abstract: A multi-aperture imaging system determines depth map information. First raw image data associated with a first image of a scene is captured using a first imaging system characterized by a first point spread function (PSF). Second raw image data associated with a second image of the scene is captured using a second imaging system characterized by a second PSF that varies as a function of depth differently than the first point spread function. High-frequency image data is generated using the first raw image data and the second raw image data. Edges are identified using normalized derivative values of the high-frequency image data, and edge depth information is determined for the identified edges using a bank of blur kernels. Fill depth information is determined for image components other than the identified edges, and a depth map of the scene is generated using the edge depth information and the fill depth information.