Abstract: A method and system for coordinated tracking of objects is disclosed. A plurality of images is received from a plurality of nodes, each node comprising at least one image capturing device. At least one target in the plurality of images is identified to produce at least one local track corresponding to each of the plurality of nodes having the at least one target in its field of view. The at least one local track corresponding to each of the plurality of nodes is fused according to a multi-hypothesis tracking method to produce at least one fused track corresponding to the at least one target. At least one of the plurality of nodes is assigned to track the at least one target based on minimizing at least one cost function comprising a cost matrix using the k-best algorithm for tracking at least one target for each of the plurality of nodes. The at least one fused track is sent to the at least one of the plurality of nodes assigned to track the at least one target based on the at least one fused track.

Abstract: A computer implemented method for determining a vehicle type of a vehicle detected in an image is disclosed. An image having a detected vehicle is received. A number of vehicle models having salient feature points is projected on the detected vehicle. A first set of features derived from each of the salient feature locations of the vehicle models is compared to a second set of features derived from corresponding salient feature locations of the detected vehicle to form a set of positive match scores (p-scores) and a set of negative match scores (n-scores). The detected vehicle is classified as one of the vehicle models models based at least in part on the set of p-scores and the set of n-scores.

Abstract: A method and system for creating a histogram of oriented occurrences (HO2) is disclosed. A plurality of entities in at least one image are detected and tracked. One of the plurality of entities is designated as a reference entity. A local 2-dimensional ground plane coordinate system centered on and oriented with respect to the reference entity is defined. The 2-dimensional ground plane is partitioned into a plurality of non-overlapping bins, the bins forming a histogram, a bin tracking a number of occurrences of an entity class. An occurrence of at least one other entity of the plurality of entities located in the at least one image may be associated with one of the plurality of non-overlapping bins. A number of occurrences of entities of at least one entity class in at least one bin may be into a vector to define an HO2 feature.

Abstract: The present invention relates to a method and apparatus for detecting and tracking vehicles. One embodiment of a system for detecting and tracking an object (e.g., vehicle) in a field of view includes a moving object indication stage for detecting a candidate object in a series of input video frames depicting the field of view and a track association stage that uses a joint probabilistic graph matching framework to associate an existing track with the candidate object.

Abstract: The present invention provides a computer implemented process for detecting multi-view multi-pose objects. The process comprises training of a classifier for each intra-class exemplar, training of a strong classifier and combining the individual exemplar-based classifiers with a single objective function. This function is optimized using the two nested AdaBoost loops. The first loop is the outer loop that selects discriminative candidate exemplars. The second loop, the inner loop selects the discriminative candidate features on the selected exemplars to compute all weak classifiers for a specific position such as a view/pose. Then all the computed weak classifiers are automatically combined into a final classifier (strong classifier) which is the object to be detected.

Abstract: A computer implemented method for detecting the presence of one or more pedestrians in the vicinity of the vehicle is disclosed. Imagery of a scene is received from at least one image capturing device. A depth map is derived from the imagery. A plurality of pedestrian candidate regions of interest (ROIs) is detected from the depth map by matching each of the plurality of ROIs with a 3D human shape model. At least a portion of the candidate ROIs is classified by employing a cascade of classifiers tuned for a plurality of depth bands and trained on a filtered representation of data within the portion of candidate ROIs to determine whether at least one pedestrian is proximal to the vehicle.

Abstract: A computer-implemented method for estimating a volume of at least one food item on a food plate is disclosed. A first and second plurality of images are received from different positions above a food plate, wherein angular spacing between the positions of the first plurality of images is greater than angular spacing between the positions of the second plurality of images. A first set of poses of each of the first plurality of images is estimated. A second set of poses of each of the second plurality of images is estimated based on at least the first set of poses. A pair of images taken from each of the first and second plurality of images is rectified based on at least the first and second set of poses. A 3D point cloud is reconstructed based on at least the rectified pair of images. At least one surface of the at least one food item above the food plate is estimated based on at least the reconstructed 3D point cloud. The volume of the at least one food item is estimated based on the at least one surface.

Abstract: The present invention is a system and a method for processing stereo images utilizing a real time, robust, and accurate stereo matching system and method based on a coarse-to-fine architecture. At each image pyramid level, non-centered windows for matching and adaptive upsampling of coarse-level disparities are performed to generate estimated disparity maps using the ACTF approach. In order to minimize propagation of disparity errors from coarser to finer levels, the present invention performs an iterative optimization, at each level, that minimizes a cost function to generate smooth disparity maps with crisp occlusion boundaries.

Abstract: The present invention provides a computer implemented process for detecting multi-view multi-pose objects. The process comprises training of a classifier for each intra-class exemplar, training of a strong classifier and combining the individual exemplar-based classifiers with a single objective function. This function is optimized using the two nested AdaBoost loops. The first loop is the outer loop that selects discriminative candidate exemplars. The second loop, the inner loop selects the discriminative candidate features on the selected exemplars to compute all weak classifiers for a specific position such as a view/pose. Then all the computed weak classifiers are automatically combined into a final classifier (strong classifier) which is the object to be detected.

Abstract: A method for estimating pose from a sequence of images, which includes the steps of detecting at least three feature points in both the left image and right image of a first pair of stereo images at a first point in time; matching the at least three feature points in the left image to the at least three feature points in the right image to obtain at least three two-dimensional feature correspondences; calculating the three-dimensional coordinates of the at least three two-dimensional feature correspondences to obtain at least three three-dimensional reference feature points; tracking the at least three feature points in one of the left image and right image of a second pair of stereo images at a second point in time different from the first point in time to obtain at least three two-dimensional reference feature points; and calculating a pose based on the at least three three-dimensional reference feature points and its corresponding two-dimensional reference feature points in the stereo images.

Abstract: A computer-implemented method for matching objects is disclosed. At least two images where one of the at least two images has a first target object and a second of the at least two images has a second target object are received. At least one first patch from the first target object and at least one second patch from the second target object are extracted. A distance-based part encoding between each of the at least one first patch and the at least one second patch based upon a corresponding codebook of image parts including at least one of part type and pose is constructed. A viewpoint of one of the at least one first patch is warped to a viewpoint of the at least one second patch. A parts level similarity measure based on the view-invariant distance measure for each of the at least one first patch and the at least one second patch is applied to determine whether the first target object and the second target object are the same or different objects.

Abstract: A computer implemented method for automatically detecting and classifying acoustic signatures across a set of recording conditions is disclosed. A first acoustic signature is received. The first acoustic signature is projected into a space of a minimal set of exemplars of acoustic signature types derived from a larger set of exemplars using a wrapper method. At least one vector distance is calculated between the projected acoustic signature and each exemplar of the minimal set of exemplars. An exemplar is selected from the minimal set of exemplars having the smallest vector distance to the projected acoustic signature as a class corresponding to and classifying the first acoustic signature. The first acoustic signature and the plurality of acoustic signatures may correspond to one of gunshots, musical instruments, songs, and speech. The minimal set of exemplars may correspond to a hierarchy of acoustic signature types.

Abstract: A method and system for coordinated tracking of objects is disclosed. A plurality of images is received from a plurality of nodes, each node comprising at least one image capturing device. At least one target in the plurality of images is identified to produce at least one local track corresponding to each of the plurality of nodes having the at least one target in its field of view. The at least one local track corresponding to each of the plurality of nodes is fused according to a multi-hypothesis tracking method to produce at least one fused track corresponding to the at least one target. At least one of the plurality of nodes is assigned to track the at least one target based on minimizing at least one cost function comprising a cost matrix using the k-best algorithm for tracking at least one target for each of the plurality of nodes. The at least one fused track is sent to the at least one of the plurality of nodes assigned to track the at least one target based on the at least one fused track.

Abstract: Method for tracking an object recorded within a selected frame of a sequence of frames of video data, using a plurality of layers, where at least one object layer of the plurality of layers represents the object includes initializing layer ownership probabilities for pixels of the selected frame using a non-parametric motion model, estimating a set of motion parameters of the plurality of layers for the selected frame using a parametric maximization algorithm and tracking the object. The non-parametric motion model is optical flow and includes warping the mixing probabilities, the appearances of the plurality of layers, and the observed pixel data from the pixels of the preceding frame to the pixels of the selected frame to initialize the layer ownership probabilities for the pixels of the selected frame.

Abstract: A computer implemented method computer implemented method for deriving a fingerprint from video data is disclosed, comprising the steps of receiving a plurality of frames from the video data; selecting at least one key frame from the plurality of frames, the at least one key frame being selected from two consecutive frames of the plurality of frames that exhibiting a maximal cumulative difference in at least one spatial feature of the two consecutive frames; detecting at least one 3D spatio-temporal feature within the at least one key frame; and encoding a spatio-temporal fingerprint based on mean luminance of the at least one 3D spatio-temporal feature. The least one spatial feature can be intensity. The at least one 3D spatio-temporal feature can be at least one Maximally Stable Volume (MSV).

Abstract: A method and system for creating a histogram of oriented occurrences (HO2) is disclosed. A plurality of entities in at least one image are detected and tracked. One of the plurality of entities is designated as a reference entity. A local 2-dimensional ground plane coordinate system centered on and oriented with respect to the reference entity is defined. The 2-dimensional ground plane is partitioned into a plurality of non-overlapping bins, the bins forming a histogram, a bin tracking a number of occurrences of an entity class. An occurrence of at least one other entity of the plurality of entities located in the at least one image may be associated with one of the plurality of non-overlapping bins. A number of occurrences of entities of at least one entity class in at least one bin may be into a vector to define an HO2 feature.

Abstract: The present invention relates to a method and system for creating a strong classifier based on motion patterns wherein the strong classifier may be used to determine an action being performed by a body in motion. When creating the strong classifier, action classification is performed by measuring similarities between features within motion patterns. Embodiments of the present invention may utilize candidate part-based action sets and training samples to train one or more weak classifiers that are then used to create a strong classifier.

Abstract: A method for extracting a 3D terrain model for identifying at least buildings and terrain from LIDAR data is disclosed, comprising the steps of generating a point cloud representing terrain and buildings mapped by LIDAR; classifying points in the point cloud, the point cloud having ground and non-ground points, the non-ground points representing buildings and clutter; segmenting the non-ground points into buildings and clutter; and calculating a fit between at least one building segment and at least one rectilinear structure, wherein the fit yields the rectilinear structure with the fewest number of vertices.

Abstract: A method and apparatus for providing immersive surveillance wherein a remote security guard may monitor a scene using a variety of imagery sources that are rendered upon a model to provide a three-dimensional conceptual view of the scene. Using a view selector, the security guard may dynamically select a camera view to be displayed on his conceptual model, perform a walk through of the scene, identify moving objects and select the best view of those moving objects and so on.

Abstract: A method and apparatus for providing immersive surveillance wherein a remote security guard may monitor a scene using a variety of imagery sources that are rendered upon a model to provide a three-dimensional conceptual view of the scene. Using a view selector, the security guard may dynamically select a camera view to be displayed on his conceptual model, perform a walk through of the scene, identify moving objects and select the best view of those moving objects and so on.