Abstract: Methods and systems are described for three-dimensional pose estimation. A training module determines a mapping function between a training image sequence and pose representations of a subject in the training image sequence. The training image sequence is represented by a set of appearance and motion patches. A set of filters are applied to the appearance and motion patches to extract features of the training images. Based on the extracted features, the training module learns a multidimensional mapping function that maps the motion and appearance patches to the pose representations of the subject. A testing module outputs a fast human pose estimation by applying the learned mapping function to a test image sequence.

Abstract: A system and method recognizes and tracks human motion from different motion classes. In a learning stage, a discriminative model is learned to project motion data from a high dimensional space to a low dimensional space while enforcing discriminance between motions of different motion classes in the low dimensional space. Additionally, low dimensional data may be clustered into motion segments and motion dynamics learned for each motion segment. In a tracking stage, a representation of human motion is received comprising at least one class of motion. The tracker recognizes and tracks the motion based on the learned discriminative model and the learned dynamics.

Abstract: Visual tracking over a sequence of images is formulated by defining an object class and one or more background classes. The most discriminant features available in the images are then used to select a portion of each image as belonging to the object class. Fisher's linear discriminant method is used to project high-dimensional image data onto a lower-dimensional space, e.g., a line, and perform classification in the lower-dimensional space. The projection function is incrementally updated.

Abstract: A system and a method model the motion of a non-rigid object using a thin plate spline (TPS) transform. A first image of a video sequence is received, and a region of interest, referred to as a template, is chosen manually or automatically. A set of arbitrarily-chosen fixed reference points is positioned on the template. A target image of the video sequence is chosen for motion estimation relative to the template. A set of pixels in the target image corresponding to the pixels of the template is determined, and this set of pixels is back-warped to match the template using a thin-plate-spline-based technique. The error between the template and the back-warped image is determined and iteratively minimized using a gradient descent technique. The TPS parameters can then be used to estimate the relative motion between the template and the corresponding region of the target image.

Abstract: A visual tracker tracks an object in a sequence of input images. A tracking module detects a location of the object based on a set of weighted blocks representing the object's shape. The tracking module then refines a segmentation of the object from the background image at the detected location. Based on the refined segmentation, the set of weighted blocks are updated. By adaptively encoding appearance and shape into the block configuration, the present invention is able to efficiently and accurately track an object even in the presence of rapid motion that causes large variations in appearance and shape of the object.

Abstract: An online sparse matrix Gaussian process (OSMGP) uses online updates to provide an accurate and efficient regression for applications such as pose estimation and object tracking. A regression calculation module calculates a regression on a sequence of input images to generate output predictions based on a learned regression model. The regression model is efficiently updated by representing a covariance matrix of the regression model using a sparse matrix factor (e.g., a Cholesky factor). The sparse matrix factor is maintained and updated in real-time based on the output predictions. Hyperparameter optimization, variable reordering, and matrix downdating techniques can also be applied to further improve the accuracy and/or efficiency of the regression process.

Abstract: A method and system efficiently and accurately detects humans in a test image and classifies their pose. In a training stage, a probabilistic model is derived in an unsupervised or semi-supervised manner such that at least some poses are not manually labeled. The model provides two sets of model parameters to describe the statistics of images containing humans and images of background scenes. In a testing stage, the probabilistic model is used to determine if a human is present in the image, and classify the human's pose based on the poses in the training images. A solution is efficiently provided to both human detection and pose classification by using the same probabilistic model to solve the problems.

Abstract: The present invention meets these needs by providing temporal coherency to recognition systems. One embodiment of the present invention comprises a manifold recognition module to use a sequence of images for recognition. A manifold training module receives a plurality of training image sequences (e.g. from a video camera), each training image sequence including an individual in a plurality of poses, and establishes relationships between the images of a training image sequence. A probabilistic identity module receives a sequence of recognition images including a target individual for recognition, and identifies the target individual based on the relationship of training images corresponding to the recognition images. An occlusion module masks occluded portions of an individual's face to prevent distorted identifications.

Abstract: The present invention meets these needs by providing temporal coherency to recognition systems. One embodiment of the present invention comprises a manifold recognition module to use a sequence of images for recognition. A manifold training module receives a plurality of training image sequences (e.g. from a video camera), each training image sequence including an individual in a plurality of poses, and establishes relationships between the images of a training image sequence. A probabilistic identity module receives a sequence of recognition images including a target individual for recognition, and identifies the target individual based on the relationship of training images corresponding to the recognition images. An occlusion module masks occluded portions of an individual's face to prevent distorted identifications.

Abstract: A system and a method are disclosed for adaptive probabilistic tracking of an object within a motion video. The method utilizes a time-varying Eigenbasis and dynamic, observation and inference models. The Eigenbasis serves as a model of the target object. The dynamic model represents the motion of the object and defines possible locations of the target based upon previous locations. The observation model provides a measure of the distance of an observation of the object relative to the current Eigenbasis. The inference model predicts the most likely location of the object based upon past and present observations. The method is effective with or without training samples. A computer-based system provides a means for implementing the method. The effectiveness of the system and method are demonstrated through simulation.

Abstract: Disclosed is a method and system for efficiently and accurately tracking three-dimensional (3D) human motion from a two-dimensional (2D) video sequence, even when self-occlusion, motion blur and large limb movements occur. In an offline learning stage, 3D motion capture data is acquired and a prediction model is generated based on the learned motions. A mixture of factor analyzers acts as local dimensionality reducers. Clusters of factor analyzers formed within a globally coordinated low-dimensional space makes it possible to perform multiple hypothesis tracking based on the distribution modes. In the online tracking stage, 3D tracking is performed without requiring any special equipment, clothing, or markers. Instead, motion is tracked in the dimensionality reduced state based on a monocular video sequence.

Abstract: The face detection system and method attempts classification of a test image before performing all of the kernel evaluations. Many subimages are not faces and should be relatively easy to identify as such. Thus, the SVM classifier try to discard non-face images using as few kernel evaluations as possible using a cascade SVM classification. In the first stage, a score is computed for the first two support vectors, and the score is compared to a threshold. If the score is below the threshold value, the subimage is classified as not a face. If the score is above the threshold value, the cascade SVM classification function continues to apply more complicated decision rules, each time doubling the number of kernel evaluations, classifying the image as a non-face (and thus terminating the process) as soon as the test image fails to satisfy one of the decision rules.

Abstract: Methods and systems are described for three-dimensional pose estimation. A training module determines a mapping function between a training image sequence and pose representations of a subject in the training image sequence. The training image sequence is represented by a set of appearance and motion patches. A set of filters are applied to the appearance and motion patches to extract features of the training images. Based on the extracted features, the training module learns a multidimensional mapping function that maps the motion and appearance patches to the pose representations of the subject. A testing module outputs a fast human pose estimation by applying the learned mapping function to a test image sequence.

Abstract: A method for representing images for pattern classification extends the conventional Isomap method with Fisher Linear Discriminant (FLD) or Kernel Fisher Linear Discriminant (KFLD) for classification. The extended Isomap method estimates the geodesic distance of data points corresponding to images for pattern classification, and uses pairwise geodesic distances as feature vectors. The method applies FLD to the feature vectors to find an optimal projection direction to maximize the distances between cluster centers of the feature vectors. The method may apply KFLD to the feature vectors instead of FLD.

Abstract: A system and a method are disclosed for an adaptive discriminative generative model with a probabilistic interpretation. As applied to visual tracking, the discriminative generative model separates the target object from the background more accurately and efficiently than conventional methods. A computationally efficient algorithm constantly updates the discriminative model over time. The discriminative generative model adapts to accommodate dynamic appearance variations of the target and background. Experiments show that the discriminative generative model effectively tracks target objects undergoing large pose and lighting changes.

Abstract: A system and a method are disclosed for clustering images of objects seen from different viewpoints. That is, given an unlabelled set of images of n objects, an unsupervised algorithm groups the images into N disjoint subsets such that each subset only contains images of a single object. The clustering method makes use of a broad geometric framework that exploits the interplay between the geometry of appearance manifolds and the symmetry of the 2D affine group.

Abstract: The advantage of the present invention is to appropriately detect the object. The object detection apparatus in the present invention has a plurality of cameras to determine the distance to the objects, a distance determination unit to determine the distance therein, a histogram generation unit to specify the frequency of the pixels against the distances to the pixels, an object distance determination unit that determines the most likely distance, a probability mapping unit that provides the probabilities of the pixels based on the difference of the distance, a kernel detection unit that determines a kernel region as a group of the pixels, a periphery detection unit that determines a peripheral region as a group of the pixels, selected from the pixels being close to the kernel region and an object specifying unit that specifies the object region where the object is present with a predetermined probability.

Abstract: Disclosed is a method and system for efficiently and accurately tracking three-dimensional (3D) human motion from a two-dimensional (2D) video sequence, even when self-occlusion, motion blur and large limb movements occur. In an offline learning stage, 3D motion capture data is acquired and a prediction model is generated based on the learned motions. A mixture of factor analyzers acts as local dimensionality reducers. Clusters of factor analyzers formed within a globally coordinated low-dimensional space makes it possible to perform multiple hypothesis tracking based on the distribution modes. In the online tracking stage, 3D tracking is performed without requiring any special equipment, clothing, or markers. Instead, motion is tracked in the dimensionality reduced state based on a monocular video sequence.

Abstract: A system and method recognizes and tracks human motion from different motion classes. In a learning stage, a discriminative model is learned to project motion data from a high dimensional space to a low dimensional space while enforcing discriminance between motions of different motion classes in the low dimensional space. Additionally, low dimensional data may be clustered into motion segments and motion dynamics learned for each motion segment. In a tracking stage, a representation of human motion is received comprising at least one class of motion. The tracker recognizes and tracks the motion based on the learned discriminative model and the learned dynamics.

Abstract: A method and system efficiently and accurately detects humans in a test image and classifies their pose. In a training stage, a probabilistic model is derived in an unsupervised or semi-supervised manner such that at least some poses are not manually labeled. The model provides two sets of model parameters to describe the statistics of images containing humans and images of background scenes. In a testing stage, the probabilistic model is used to determine if a human is present in the image, and classify the human's pose based on the poses in the training images. A solution is efficiently provided to both human detection and pose classification by using the same probabilistic model to solve the problems.