Abstract: A method to perform hiearchical video segmentation includes: defining voxels over a spatio-temporal video; grouping into segments contiguous voxels that display similar characteristics including similar appearance or motion; determining a trajectory-based feature that complements color and optical flow cues, wherein trajectory cues are probabilistically meaningful histograms combinable for use in a graph-based framework; and applying a max-margin module for cue combination that learns a supervised distance metric for region dissimilarity that combines color, flow and trajectory features.

Abstract: A method to reconstruct 3D model of an object includes receiving with a processor a set of training data including images of the object from various viewpoints; learning a prior comprised of a mean shape describing a commonality of shapes across a category and a set of weighted anchor points encoding similarities between instances in appearance and spatial consistency; matching anchor points across instances to enable learning a mean shape for the category; and modeling the shape of an object instance as a warped version of a category mean, along with instance-specific details.

Abstract: Methods and systems for moving object localization include estimating a ground plane in a video frame based on a detected object within the video frame and monocular structure-from-motion (SFM) information; computing object pose for objects in the frame based on the SFM information using dense feature tracking; and determining a three-dimensional location for the detected object based on the estimated ground plane and the computed object pose.

Abstract: Systems and methods are disclosed for road scene understanding of vehicles in traffic by capturing images of traffic with a camera coupled to a vehicle; generating a continuous model of occlusions with a continuous occlusion mode for traffic participants to enhance point track association accuracy without distinguishing between moving and static objects; applying the continuous occlusion model to handle visibility constraints in object tracks; and combining point track association and soft object track modeling to improve 3D localization accuracy.

Abstract: Systems and methods are disclosed to provide an Advanced Warning System (AWS) for a driver of a vehicle, by capturing traffic scene types from a single camera video; generating real-time monocular SFM and 2D object detection from the single camera video; detecting a ground plane from the real-time monocular SFM and the 2D object detection; performing dense 3D estimation from the real-time monocular SFM and the 2D object detection; generating a joint 3D object localization from the ground plane and dense 3D estimation; and communicating a situation that requires caution to the driver.

Abstract: Systems and methods are disclosed for determining three dimensional (3D) shape by capturing with a camera a plurality of images of an object in differential motion; derive a general relation that relates spatial and temporal image derivatives to BRDF derivatives; exploiting rank deficiency to eliminate BRDF terms and recover depth or normal for directional lighting; and using depth-normal-BRDF relation to recover depth or normal for unknown arbitrary lightings.

Abstract: A computer vision method that includes deriving a relationship of spatial and temporal image derivatives of an object to bidirectional reflectance distribution function (BRDF) derivatives under camera motion, and deriving with a processor a quasilinear partial differential equation for solving surfaced depth for orthographic projections using the relationship of spatial and temporal image derivatives without requiring knowledge of the BRDF. The method may further recover surface depth for an object with unknown BRDF under perspective projection.

Abstract: A method for performing three-dimensional (3D) localization requiring only a single camera including capturing images from only one camera; generating a cue combination from sparse features, dense stereo and object bounding boxes; correcting for scale in monocular structure from motion (SFM) using the cue combination for estimating a ground plane; and performing localization by combining SFM, ground plane and object bounding boxes to produce a 3D object localization.

Abstract: Systems and methods are disclosed for multithreaded visual odometry by acquired with a single camera on-board a vehicle; using 2D-3D correspondences for continuous pose estimation; and combining the pose estimation with 2D-2D epipolar search to replenish 3D points.

Abstract: Methods and systems for moving object localization include estimating a ground plane in a video frame based on a detected object within the video frame and monocular structure-from-motion (SFM) information; computing object pose for objects in the frame based on the SFM information using dense feature tracking; and determining a three-dimensional location for the detected object based on the estimated ground plane and the computed object pose.

Abstract: Systems and methods are disclosed for autonomous driving with only a single camera by moving object localization in 3D with a real-time framework that harnesses object detection and monocular structure from motion (SFM) through the ground plane estimation; tracking feature points on moving cars a real-time framework to and use the feature points for 3D orientation estimation; and correcting scale drift with ground plane estimation that combines cues from sparse features and dense stereo visual data.

Abstract: A method to perform hiearchical video segmentation includes: defining voxels over a spatio-temporal video; grouping into segments contiguous voxels that display similar characteristics including similar appearance or motion; determining a trajectory-based feature that complements color and optical flow cues, wherein trajectory cues are probabilistically meaningful histograms combinable for use in a graph-based framework; and applying a max-margin module for cue combination that learns a supervised distance metric for region dissimilarity that combines color, flow and trajectory features.

Abstract: A computer vision method that includes deriving a relationship of spatial and temporal image derivatives of an object to bidirectional reflectance distribution function (BRDF) derivatives under camera motion, and deriving with a processor a quasilinear partial differential equation for solving surfaced depth for orthographic projections using the relationship of spatial and temporal image derivatives without requiring knowledge of the BRDF. The method may further recover surface depth for an object with unknown BRDF under perspective projection.

Abstract: A method for performing three-dimensional (3D) localization requiring only a single camera including capturing images from only one camera; generating a cue combination from sparse features, dense stereo and object bounding boxes; correcting for scale in monocular structure from motion (SFM) using the cue combination for estimating a ground plane; and performing localization by combining SFM, ground plane and object bounding boxes to produce a 3D object localization.

Abstract: A computer implemented method for determining shape from differential motion with unknown reflectance includes deriving a general relation that relates spatial and temporal image derivatives to bidirectional reflectance distribution function BRDF derivatives, responsive to 3D points and relative camera poses from images and feature tracks of an object in motion under colocated and unknown directional light conditions, employing a rank deficiency in image sequences from the deriving for shape determinations, under predetermined multiple camera and lighting conditions, to eliminate BDRF terms; and recovering a surface depth for determining a shape of the object.

Abstract: Systems and methods are disclosed for autonomous driving with only a single camera by moving object localization in 3D with a real-time framework that harnesses object detection and monocular structure from motion (SFM) through the ground plane estimation; tracking feature points on moving cars a real-time framework to and use the feature points for 3D orientation estimation; and correcting scale drift with ground plane estimation that combines cues from sparse features and dense stereo visual data.

Abstract: Systems and methods are disclosed for determining three dimensional (3D) shape by capturing with a camera a plurality of images of an object in differential motion; derive a general relation that relates spatial and temporal image derivatives to BRDF derivatives; exploiting rank deficiency to eliminate BRDF terms and recover depth or normal for directional lighting; and using depth-normal-BRDF relation to recover depth or normal for unknown arbitrary lightings.

Abstract: A method to reconstruct 3D model of an object includes receiving with a processor a set of training data including images of the object from various viewpoints; learning a prior comprised of a mean shape describing a commonality of shapes across a category and a set of weighted anchor points encoding similarities between instances in appearance and spatial consistency; matching anchor points across instances to enable learning a mean shape for the category; and modeling the shape of an object instance as a warped version of a category mean, along with instance-specific details.

Abstract: Systems and methods are disclosed for multithreaded visual odometry by acquired with a single camera on-board a vehicle; using 2D-3D correspondences for continuous pose estimation; and combining the pose estimation with 2D-2D epipolar search to replenish 3D points.

Abstract: A system and method are disclosed for estimating camera motion and structure reconstruction of a scene using lines. The system includes a line detection module, a line correspondence module, a temporal line tracking module and structure and motion module. The line detection module is configured to detect lines in visual input data comprising a plurality of image frames. The line correspondence module is configured to find line correspondence between detected lines in the visual input data. The temporal line tracking module is configured to track the detected lines temporally across the plurality of the image frames. The structure and motion module is configured to estimate the camera motion using the detected lines in the visual input data and to reconstruct three-dimensional lines from the estimated camera motion.