Abstract: There are provided systems and methods for performing object classification through semantic mapping. Such an object classification system includes a system processor, a system memory, and an object categorizing unit stored in the system memory. The system processor is configured to execute the object categorizing unit to receive image data corresponding to an object, and to transform the image data into a directed quantity expressed at least in part in terms of semantic parameters. The system processor is further configured to determine a projection of the directed quantity onto an object representation map including multiple object categories, and to associate the object with a category from among the multiple object categories based on the projection.

Abstract: A system and method of estimating the body shape of an individual from input data such as images or range maps. The body may appear in one or more poses captured at different times and a consistent body shape is computed for all poses. The body may appear in minimal tight-fitting clothing or in normal clothing wherein the described method produces an estimate of the body shape under the clothing. Clothed or bare regions of the body are detected via image classification and the fitting method is adapted to treat each region differently. Body shapes are represented parametrically and are matched to other bodies based on shape similarity and other features. Standard measurements are extracted using parametric or non-parametric functions of body shape. The system components support many applications in body scanning, advertising, social networking, collaborative filtering and Internet clothing shopping.

Abstract: The disclosure provides an approach for estimating a state-space controller from a set of video frames depicting a motion of an entity. The approach includes incrementally optimizing parameters of the state-space controller and changing a structure of the state-space controller based on expanding subsets of the set of video frames. In one embodiment, a controller-estimation application greedily selects, at every stage of the incremental optimization, structure and parameters of the controller which minimize an objective function. In another embodiment, the controller-estimation application re-optimizes, after the incremental optimization, all parameters of the state-space controller based on all of the video frames.

Abstract: There are provided systems and methods of incremental category embedding for categorization. One method including selecting one or more input categories from a plurality of input categories to be added to learned categories, determining at least one representative category from the learned categories for each input category from the one or more input categories, the at least one representative category representing the input category, and approximating the input category using the at least one representative category.

Abstract: Embodiments presented herein describe techniques for generating a story graph using a collection of digital media, such as images and video. The story graph presents a structure for activities, events, and locales commonly occurring in sets of photographs taken by different individuals across a given location (e.g., a theme park, tourist attraction, convention, etc.). To build a story graph, streams from sets of digital media are generated. Each stream corresponds to media (e.g., images or video) taken in sequence at the location by an individual (or related group of individuals) over a period of time. For each stream, features from each media are extracted relative to the stream. Clusters of media are generated and are connected by directed edges. The connections indicate a path observed to have occurred in the streams from one cluster to another cluster.

Abstract: There are provided systems and methods of incremental category embedding for categorization. One method including selecting one or more input categories from a plurality of input categories to be added to learned categories, determining at least one representative category from the learned categories for each input category from the one or more input categories, the at least one representative category representing the input category, and approximating the input category using the at least one representative category.

Abstract: Systems and method for modeling hair in real-time with user interactive controls are presented. One embodiment may take the form of a method of hair motion modeling including representing hair with hair guides, each hair guide comprising a plurality of hair points and reducing a dimensionality of the hair guides to achieve a reduced sub-space. Additionally, the method includes generating a data tensor for multiple factors related to the hair guides and decomposing the tensor to create a model characterizing the multiple factors in a multi-linear hair framework. The hair may be hair, such as human hair, animal fur, or clothing fibers.

Abstract: A system and method of estimating the body shape of an individual from input data such as images or range maps. The body may appear in one or more poses captured at different times and a consistent body shape is computed for all poses. The body may appear in minimal tight-fitting clothing or in normal clothing wherein the described method produces an estimate of the body shape under the clothing. Clothed or bare regions of the body are detected via image classification and the fitting method is adapted to treat each region differently. Body shapes are represented parametrically and are matched to other bodies based on shape similarity and other features. Standard measurements are extracted using parametric or non-parametric functions of body shape. The system components support many applications in body scanning, advertising, social networking, collaborative filtering and Internet clothing shopping.

Abstract: The disclosure provides an approach for detecting objects in images. An object detection application receives a set of training images with object annotations. Given these training images, the object detection application generates semantic labeling for object detections, where the labeling includes lower-level subcategories and higher-level visual composites. In one embodiment, the object detection application identifies subcategories using an exemplar support vector machine (SVM) based clustering approach. Identified subcategories are used to initialize mixture components in mixture models which the object detection application trains in a latent SVM framework, thereby learning a number of subcategory classifiers that produce, for any given image, a set of candidate windows and associated subcategory labels.

Abstract: Techniques are disclosed for the automatic recovery of two dimensional (2D) and three dimensional (3D) poses of multiple subjects interacting with one another, as depicted in a sequence of 2D images. As part of recovering 2D and 3D pose estimates, a pose recovery tool may account for constraints on positions of body parts of the first and second person resulting from the correlated activity. That is, individual subjects in the video are treated as mutual context for one another.

Abstract: Techniques for generating a photo album are described. Embodiments analyze a plurality of photo albums to determine photo album characteristics, where each of the plurality of photo albums comprises a respective plurality of digital photographs arranged in a respective order. Here, the plurality of photo albums relate to one or more digital photograph collections. Additionally, a digital photograph collection is processed to determine characteristics of each digital photograph within the digital photograph collection. Embodiments also generate a photo album using digital photographs from the digital photograph collection, based on the determined photograph album characteristics, where the generated photo album comprises a second plurality of digital photographs arranged in a determined order.

Abstract: Methods and systems for video action recognition using poselet keyframes are disclosed. An action recognition model may be implemented to spatially and temporally model discriminative action components as a set of discriminative keyframes. One method of action recognition may include the operations of selecting a plurality of poselets that are components of an action, encoding each of a plurality of video frames as a summary of the detection confidence of each of the plurality of poselets for the video frame, and encoding correlations between poselets in the encoded video frames.

Abstract: An input device for tracking three-dimensional movements. The input device includes a tracking structure and is in electronic communication with a computing device. The tracking structure is detectable by a tracking device. The tracking structure is configured so that as the input device is deformed, the tracking structure deforms correspondingly.

Abstract: Body-mounted cameras are used to accurately reconstruct the motion of a subject. Outward-looking cameras are attached to the limbs of the subject, and the joint angles and root pose that define the subject's configuration are estimated through a non-linear optimization, which can incorporate image matching error and temporal continuity of motion. Instrumentation of the environment is not required, allowing for motion capture over extended areas and in outdoor settings.

Abstract: Systems and method for modeling hair in real-time with user interactive controls are presented. One embodiment may take the form of a method of hair motion modeling including representing hair with hair guides, each hair guide comprising a plurality of hair points and reducing a dimensionality of the hair guides to achieve a reduced sub-space. Additionally, the method includes generating a data tensor for multiple factors related to the hair guides and decomposing the tensor to create a model characterizing the multiple factors in a multi-linear hair framework. The hair may be hair, such as human hair, animal fur, or clothing fibers.

Abstract: The disclosure provides an approach for estimating a state-space controller from a set of video frames depicting a motion of an entity. The approach includes incrementally optimizing parameters of the state-space controller and changing a structure of the state-space controller based on expanding subsets of the set of video frames. In one embodiment, a controller-estimation application greedily selects, at every stage of the incremental optimization, structure and parameters of the controller which minimize an objective function. In another embodiment, the controller-estimation application re-optimizes, after the incremental optimization, all parameters of the state-space controller based on all of the video frames.

Abstract: The disclosure provides an approach for recognizing and analyzing activities. In one embodiment, a learning application trains parameters of a hierarchical model which represents human (or object) activity at multiple levels of detail. Higher levels of detail may consider more context, and vice versa. Further, learning may be optimized for a user-preferred type of inference by adjusting a learning criterion. An inference application may use the trained model to answer queries about variable(s) at any level of detail. In one embodiment, the inference application may determine scores for each possible value of the query variable by finding the best hierarchical event representation that maximizes a scoring function while fixing the value of the query variable to its possible values. Here, the inference application may approximately determine the best hierarchical event representation by iteratively optimizing one level-of-detail variable at a time while fixing other level-of-detail variables, until convergence.

Abstract: Techniques are disclosed for the automatic recovery of two dimensional (2D) and three dimensional (3D) poses of multiple subjects interacting with one another, as depicted in a sequence of 2D images. As part of recovering 2D and 3D pose estimates, a pose recovery tool may account for constraints on positions of body parts of the first and second person resulting from the correlated activity. That is, individual subjects in the video are treated as mutual context for one another.

Abstract: Body-mounted cameras are used to accurately reconstruct the motion of a subject. Outward-looking cameras are attached to the limbs of the subject, and the joint angles and root pose that define the subject's configuration are estimated through a non-linear optimization, which can incorporate image matching error and temporal continuity of motion. Instrumentation of the environment is not required, allowing for motion capture over extended areas and in outdoor settings.

Abstract: An input device for tracking three-dimensional movements. The input device includes a tracking structure and is in electronic communication with a computing device. The tracking structure is detectable by a tracking device. The tracking structure is configured so that as the input device is deformed, the tracking structure deforms correspondingly.