Abstract: A method for estimating a pose of a deformable object includes: receiving, by a processor, a plurality of images depicting the deformable object from multiple viewpoints; computing, by the processor, one or more object-level correspondences and a class of the deformable object depicted in the images; loading, by the processor, a 3-D model corresponding to the class of the deformable object; aligning, by the processor, the 3-D model to the deformable object depicted in the plurality of images to compute a six-degree of freedom (6-DoF) pose of the object; and outputting, by the processor, the 3-D model and the 6-DoF pose of the object.

Abstract: An optical system includes: a beam splitter system configured to split an input beam into a plurality of output beams including a first output beam, a second output beam, and a third output beam; a first polarizing filter having a first polarization angle and configured to filter the first output beam to produce a first filtered output beam; a second polarizing filter having a second angle of polarization and configured to filter the second output beam to produce a second filtered output beam; and a third polarizing filter having a third angle of polarization and configured to filter the third output beam to produce a third filtered output beam, the first, second, and third angles of polarization being different from one another.

Abstract: Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.

Abstract: In an embodiment, a 3D facial modeling system includes a plurality of cameras configured to capture images from different viewpoints, a processor, and a memory containing a 3D facial modeling application and parameters defining a face detector, wherein the 3D facial modeling application directs the processor to obtain a plurality of images of a face captured from different viewpoints using the plurality of cameras, locate a face within each of the plurality of images using the face detector, wherein the face detector labels key feature points on the located face within each of the plurality of images, determine disparity between corresponding key feature points of located faces within the plurality of images, and generate a 3D model of the face using the depth of the key feature points.

Abstract: Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. An imaging device in accordance with one embodiment of the invention includes at least one imager array, and each imager in the array comprises a plurality of light sensing elements and a lens stack including at least one lens surface, where the lens stack is configured to form an image on the light sensing elements, control circuitry configured to capture images formed on the light sensing elements of each of the imagers, and a super-resolution processing module configured to generate at least one higher resolution super-resolved image using a plurality of the captured images.

Abstract: Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. An imaging device in accordance with one embodiment of the invention includes at least one imager array, and each imager in the array comprises a plurality of light sensing elements and a lens stack including at least one lens surface, where the lens stack is configured to form an image on the light sensing elements, control circuitry configured to capture images formed on the light sensing elements of each of the imagers, and a super-resolution processing module configured to generate at least one higher resolution super-resolved image using a plurality of the captured images.

Abstract: Systems and methods in accordance with embodiments of the invention are disclosed that use super-resolution (SR) processes to use information from a plurality of low resolution (LR) images captured by an array camera to produce a synthesized higher resolution image. One embodiment includes obtaining input images using the plurality of imagers, using a microprocessor to determine an initial estimate of at least a portion of a high resolution image using a plurality of pixels from the input images, and using a microprocessor to determine a high resolution image that when mapped through the forward imaging transformation matches the input images to within at least one predetermined criterion using the initial estimate of at least a portion of the high resolution image.

Abstract: Systems and methods in accordance with embodiments of the invention are configured to render images using light field image files containing an image synthesized from light field image data and metadata describing the image that includes a depth map. One embodiment of the invention includes a processor and memory containing a rendering application and a light field image file including an encoded image, a set of low resolution images, and metadata describing the encoded image, where the metadata comprises a depth map that specifies depths from the reference viewpoint for pixels in the encoded image. In addition, the rendering application configures the processor to: locate the encoded image within the light field image file; decode the encoded image; locate the metadata within the light field image file; and post process the decoded image by modifying the pixels based on the depths indicated within the depth map and the set of low resolution images to create a rendered image.

Abstract: A computer-implemented method for computing a prediction on images of a scene includes: receiving one or more polarization raw frames of a scene, the polarization raw frames being captured with a polarizing filter at a different linear polarization angle; extracting one or more first tensors in one or more polarization representation spaces from the polarization raw frames; and computing a prediction regarding one or more optically challenging objects in the scene based on the one or more first tensors in the one or more polarization representation spaces.

Abstract: In an embodiment, a 3D facial modeling system includes a plurality of cameras configured to capture images from different viewpoints, a processor, and a memory containing a 3D facial modeling application and parameters defining a face detector, wherein the 3D facial modeling application directs the processor to obtain a plurality of images of a face captured from different viewpoints using the plurality of cameras, locate a face within each of the plurality of images using the face detector, wherein the face detector labels key feature points on the located face within each of the plurality of images, determine disparity between corresponding key feature points of located faces within the plurality of images, and generate a 3D model of the face using the depth of the key feature points.

Abstract: Systems and methods for depth estimation in accordance with embodiments of the invention are illustrated. One embodiment includes a method for estimating depth from images. The method includes steps for receiving a plurality of source images captured from a plurality of different viewpoints using a processing system configured by an image processing application, generating a target image from a target viewpoint that is different to the viewpoints of the plurality of source images based upon a set of generative model parameters using the processing system configured by the image processing application, and identifying depth information of at least one output image based on the predicted target image using the processing system configured by the image processing application.

Abstract: A method of generating synthetic images of virtual scenes includes: placing, by a synthetic data generator implemented by a processor and memory, three-dimensional (3-D) models of objects in a 3-D virtual scene; adding, by the synthetic data generator, lighting to the 3-D virtual scene, the lighting including one or more illumination sources; applying, by the synthetic data generator, imaging modality-specific materials to the 3-D models of objects in the 3-D virtual scene in accordance with a selected multimodal imaging modality, each of the imaging modality-specific materials including an empirical model; setting a scene background in accordance with the selected multimodal imaging modality; and rendering, by the synthetic data generator, a two-dimensional image of the 3-D virtual scene based on the selected multimodal imaging modality to generate a synthetic image in accordance with the selected multimodal imaging modality.

Abstract: Systems and methods for hybrid depth regularization in accordance with various embodiments of the invention are disclosed. In one embodiment of the invention, a depth sensing system comprises a plurality of cameras; a processor; and a memory containing an image processing application. The image processing application may direct the processor to obtain image data for a plurality of images from multiple viewpoints, the image data comprising a reference image and at least one alternate view image; generate a raw depth map using a first depth estimation process, and a confidence map; and generate a regularized depth map. The regularized depth map may be generated by computing a secondary depth map using a second different depth estimation process; and computing a composite depth map by selecting depth estimates from the raw depth map and the secondary depth map based on the confidence map.

Abstract: Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. An imaging device in accordance with one embodiment of the invention includes at least one imager array, and each imager in the array comprises a plurality of light sensing elements and a lens stack including at least one lens surface, where the lens stack is configured to form an image on the light sensing elements, control circuitry configured to capture images formed on the light sensing elements of each of the imagers, and a super-resolution processing module configured to generate at least one higher resolution super-resolved image using a plurality of the captured images.

Abstract: A sensor system includes: a radar system configured to emit a radar beam and receive reflected radar signals from in a field of view of the radar system; a camera system including one or more cameras, at least one camera including a linear polarization filter in an optical axis of the camera, a field of view of the camera system overlapping the field of view of the radar system; and a processing system including a processor and memory, the memory storing instructions that, when executed by the processor, cause the processor to: receive radar data based on the reflected radar signals captured by the radar system; receive polarization raw frames captured by the camera system; and compute a track of a target in the field of view of the camera system and the field of view of the radar system based on the radar data and the polarization raw frames.

Abstract: Systems and methods in accordance with embodiments of the invention are configured to render images using light field image files containing an image synthesized from light field image data and metadata describing the image that includes a depth map. One embodiment of the invention includes a processor and memory containing a rendering application and a light field image file including an encoded image, a set of low resolution images, and metadata describing the encoded image, where the metadata comprises a depth map that specifies depths from the reference viewpoint for pixels in the encoded image. In addition, the rendering application configures the processor to: locate the encoded image within the light field image file; decode the encoded image; locate the metadata within the light field image file; and post process the decoded image by modifying the pixels based on the depths indicated within the depth map and the set of low resolution images to create a rendered image.

Abstract: A computer-implemented method for surface modeling includes: receiving one or more polarization raw frames of a surface of a physical object, the polarization raw frames being captured with a polarizing filter at different linear polarization angles; extracting one or more first tensors in one or more polarization representation spaces from the polarization raw frames; and detecting a surface characteristic of the surface of the physical object based on the one or more first tensors in the one or more polarization representation spaces.

Abstract: According to one embodiment of the present disclosure, an imaging system includes: an image sensor including a plurality of subpixels grouped into a plurality of pixels; a polarization system including: a rotatable linear polarizer; and a polarizer mask including a plurality of polarizer filters, the polarizer filters being aligned with corresponding ones of the subpixels, the subpixels of a pixel of the plurality of pixels being located behind polarizer filters at different angles of linear polarization; and imaging optics configured to focus light from a scene onto the image sensor.

Abstract: Systems and methods for adjusting an exposure parameter of an imaging device are disclosed. A first exposure level of the imaging device is identified, and a first image of a scene is captured via the imaging device at the first exposure level. The first image of the scene comprises a plurality of polarization images corresponding to different degrees and angles of polarization. Each of the polarization images comprise a plurality of color channels. A gradient for the first image is computed based on the plurality of the polarization images, and a second exposure level is computed based on the gradient. A second image of the scene is captured based on the second exposure level, where the gradient of the second image is greater than a gradient for the first image.

Abstract: A multi-modal sensor system includes: an underlying sensor system; a polarization camera system configured to capture polarization raw frames corresponding to a plurality of different polarization states; and a processing system including a processor and memory, the processing system being configured to control the underlying sensor system and the polarization camera system, the memory storing instructions that, when executed by the processor, cause the processor to: control the underlying sensor system to perform sensing on a scene and the polarization camera system to capture a plurality of polarization raw frames of the scene; extract first tensors in polarization representation spaces based on the plurality of polarization raw frames; and compute a characterization output based on an output of the underlying sensor system and the first tensors in polarization representation spaces.