Abstract: A polarized event camera system includes: an event camera having a field of view centered around an optical axis, the event camera including an image sensor including a plurality of pixels, each pixel of the event camera operating independently and asynchronously and being configured to generate change events based on changes in intensity of light received by the pixel; and a rotatable linear polarizer aligned with the optical axis of the event camera, the rotatable linear polarizer having a polarization axis, the polarization axis of the rotatable linear polarizer being rotatable about the optical axis of the event camera.

Abstract: A method of performing surface profilometry includes receiving one or more polarization raw frames of a printed layer of a physical object undergoing additive manufacturing, the one or more polarization raw frames being captured at different polarizations by one or more polarization cameras, extracting one or more polarization feature maps in one or more polarization representation spaces from the one or more polarization raw frames, obtaining a coarse layer depth map of the printed layer, generating one or more surface-normal images based on the coarse layer depth map and the one or more polarization feature maps, and generating a 3D reconstruction of the printed layer based on the one or more surface-normal images.

Abstract: A method for estimating a pose of an object includes: receiving, by a processor, an observed image depicting the object from a viewpoint; computing, by the processor, an instance segmentation map identifying a class of the object depicted in the observed image; loading, by the processor, a 3-D model corresponding to the class of the object; computing, by the processor, a rendered image of the 3-D model in accordance with an initial pose estimate of the object and the viewpoint of the observed image; computing, by the processor, a plurality of dense image-to-object correspondences between the observed image of the object and the 3-D model based on the observed image and the rendered image; and computing, by the processor, the pose of the object based on the dense image-to-object correspondences.

Abstract: A stereo camera array system includes: a first camera array at a first viewpoint including: a first camera configured to capture images in a first modality, the first modality being viewpoint-independent; and a second camera configured to capture images in a second modality different from the first modality; and a second camera array at a second viewpoint spaced apart along a first baseline from the first camera array at the first viewpoint, the second camera array including: a first camera configured to capture images in the first modality; and a second camera configured to capture images in the second modality.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for detection of pixel correspondences in stereo imaging through the use of polarization data. One of the methods include: obtaining a first image from a first viewpoint of a stereo pair and a second image from a second viewpoint of the stereo pair; obtaining an angle of linear polarization (AOLP) map and a degree of linear polarization (DOLP) map having a field of view overlapping the first image and the second image; computing a surface normal map based on the AOLP map and the DOLP map; and detecting corresponding pixels between the first image and the second image by computing a disparity map that minimizes an energy function comprising a pixel matching cost term and a polarized smoothness regularization term computed based on pixels in a local neighborhood and based on the surface normal map.

Abstract: A method for estimating a pose of an object includes: receiving a plurality of images of the object captured from multiple viewpoints with respect to the object; initializing a current pose of the object based on computing an initial estimated pose of the object from at least one of the plurality of images; predicting a plurality of 2-D keypoints associated with the object from each of the plurality of images; and computing an updated pose that minimizes a cost function based on a plurality of differences between the 2-D keypoints and a plurality of 3-D keypoints associated with a 3-D model of the object as arranged in accordance with the current pose, and as projected to each of the viewpoints.

Abstract: A method of tracking a pose of an object includes determining an initial pose of the object at a first position, receiving position data and velocity data corresponding to movement of the object to a second position by a moving device, determining an expected pose of the object at the second position based on the position and velocity data and the initial pose, receiving second image data corresponding to the object at the second position from a camera, and determining a refined pose of the object at the second position based on the second image data and the expected pose.

Abstract: A multi-modal sensor system includes: an underlying sensor system; a polarization camera system configured to capture polarization raw frames corresponding to a plurality of different polarization states; and a processing system including a processor and memory, the processing system being configured to control the underlying sensor system and the polarization camera system, the memory storing instructions that, when executed by the processor, cause the processor to: control the underlying sensor system to perform sensing on a scene and the polarization camera system to capture a plurality of polarization raw frames of the scene; extract first tensors in polarization representation spaces based on the plurality of polarization raw frames; and compute a characterization output based on an output of the underlying sensor system and the first tensors in polarization representation spaces.

Abstract: An optical system includes: a beam splitter system configured to split an input beam into a plurality of output beams including a first output beam, a second output beam, and a third output beam; a first polarizing filter having a first polarization angle and configured to filter the first output beam to produce a first filtered output beam; a second polarizing filter having a second angle of polarization and configured to filter the second output beam to produce a second filtered output beam; and a third polarizing filter having a third angle of polarization and configured to filter the third output beam to produce a third filtered output beam, the first, second, and third angles of polarization being different from one another.

Abstract: A method for estimating a pose of a deformable object includes: receiving, by a processor, a plurality of images depicting the deformable object from multiple viewpoints; computing, by the processor, one or more object-level correspondences and a class of the deformable object depicted in the images; loading, by the processor, a 3-D model corresponding to the class of the deformable object; aligning, by the processor, the 3-D model to the deformable object depicted in the plurality of images to compute a six-degree of freedom (6-DoF) pose of the object; and outputting, by the processor, the 3-D model and the 6-DoF pose of the object.

Abstract: A computer-implemented method for computing a prediction on images of a scene includes: receiving one or more polarization raw frames of a scene, the polarization raw frames being captured with a polarizing filter at a different linear polarization angle; extracting one or more first tensors in one or more polarization representation spaces from the polarization raw frames; and computing a prediction regarding one or more optically challenging objects in the scene based on the one or more first tensors in the one or more polarization representation spaces.

Abstract: A method of generating synthetic images of virtual scenes includes: placing, by a synthetic data generator implemented by a processor and memory, three-dimensional (3-D) models of objects in a 3-D virtual scene; adding, by the synthetic data generator, lighting to the 3-D virtual scene, the lighting including one or more illumination sources; applying, by the synthetic data generator, imaging modality-specific materials to the 3-D models of objects in the 3-D virtual scene in accordance with a selected multimodal imaging modality, each of the imaging modality-specific materials including an empirical model; setting a scene background in accordance with the selected multimodal imaging modality; and rendering, by the synthetic data generator, a two-dimensional image of the 3-D virtual scene based on the selected multimodal imaging modality to generate a synthetic image in accordance with the selected multimodal imaging modality.

Abstract: A sensor system includes: a radar system configured to emit a radar beam and receive reflected radar signals from in a field of view of the radar system; a camera system including one or more cameras, at least one camera including a linear polarization filter in an optical axis of the camera, a field of view of the camera system overlapping the field of view of the radar system; and a processing system including a processor and memory, the memory storing instructions that, when executed by the processor, cause the processor to: receive radar data based on the reflected radar signals captured by the radar system; receive polarization raw frames captured by the camera system; and compute a track of a target in the field of view of the camera system and the field of view of the radar system based on the radar data and the polarization raw frames.

Abstract: A computer-implemented method for surface modeling includes: receiving one or more polarization raw frames of a surface of a physical object, the polarization raw frames being captured with a polarizing filter at different linear polarization angles; extracting one or more first tensors in one or more polarization representation spaces from the polarization raw frames; and detecting a surface characteristic of the surface of the physical object based on the one or more first tensors in the one or more polarization representation spaces.

Abstract: Systems and methods for adjusting an exposure parameter of an imaging device are disclosed. A first exposure level of the imaging device is identified, and a first image of a scene is captured via the imaging device at the first exposure level. The first image of the scene comprises a plurality of polarization images corresponding to different degrees and angles of polarization. Each of the polarization images comprise a plurality of color channels. A gradient for the first image is computed based on the plurality of the polarization images, and a second exposure level is computed based on the gradient. A second image of the scene is captured based on the second exposure level, where the gradient of the second image is greater than a gradient for the first image.

Abstract: A multi-modal sensor system includes: an underlying sensor system; a polarization camera system configured to capture polarization raw frames corresponding to a plurality of different polarization states; and a processing system including a processor and memory, the processing system being configured to control the underlying sensor system and the polarization camera system, the memory storing instructions that, when executed by the processor, cause the processor to: control the underlying sensor system to perform sensing on a scene and the polarization camera system to capture a plurality of polarization raw frames of the scene; extract first tensors in polarization representation spaces based on the plurality of polarization raw frames; and compute a characterization output based on an output of the underlying sensor system and the first tensors in polarization representation spaces.

Abstract: Systems and methods for calibrating an imaging device. A processing circuit identifies first timestamps of one or more events in one or more first image frames of a first imaging device, and identifies one or more corresponding events in one or more second image frames. Second timestamps may be determined for the corresponding events. A calibration value may be calculated based on a difference between the first and second timestamps. In one embodiment, the processing circuit also identifies a first plurality of events in a first sequence of image frames transmitted by a first imaging device, and selects a second plurality of events in a second sequence of image frames by a second imaging device that minimizes discrepancy between the first plurality of events and the second plurality of events. A calibration value is computed based on the timestamps of the matched events.

Abstract: A polarized event camera system includes: an event camera having a field of view centered around an optical axis, the event camera including an image sensor including a plurality of pixels, each pixel of the event camera operating independently and asynchronously and being configured to generate change events based on changes in intensity of light received by the pixel; and a rotatable linear polarizer aligned with the optical axis of the event camera, the rotatable linear polarizer having a polarization axis, the polarization axis of the rotatable linear polarizer being rotatable about the optical axis of the event camera.

Abstract: Aspects of embodiments of the present disclosure relate to systems and methods for performing three-dimensional reconstruction (or depth reconstruction) and for generating segmentation masks using data captured by one or more event cameras.

Abstract: A method of tracking a pose of an object includes determining an initial pose of the object at a first position, receiving position data and velocity data corresponding to movement of the object to a second position by a moving device, determining an expected pose of the object at the second position based on the position and velocity data and the initial pose, receiving second image data corresponding to the object at the second position from a camera, and determining a refined pose of the object at the second position based on the second image data and the expected pose.