Abstract: There is provided a system comprising a memory and a processor configured to receive a plurality of images, train a plurality of independent detectors for identifying a first plurality of objects in the plurality of images based on individual attributes including a first attribute and a second attribute, train a plurality of joint detectors for identifying the first plurality of objects in the plurality of images based on composite attributes including a plurality of composite attributes each including the first attribute and the second attribute, analyze features of the plurality of images to determine a difference between a first training performance of the independent detectors and a second training performance of the joint detectors, and select, based on the analyzing, between using the independent detectors and using the joint detectors for identifying a second plurality of objects in the plurality of images using a first new attribute and a second new attribute in the attribute database.

Abstract: Techniques are disclosed for identifying and filtering outliers from a sample set of data prior to training a classifier on an object using the sample set. A data set including a plurality of samples used to train a classification model is retrieved. The samples in the data set have a feature dimensionality. A graph of the data set is built. Each node in the graph corresponds to a sample in the data set and edges connecting the nodes correspond to a measure of similarity between the nodes. The feature dimensionality of the sample data set is reduced based on a topology of the graph. One or more outliers in the data set are identified based on the reduced feature dimensionality.

Abstract: Systems, methods, and computer program products to perform an operation comprising assigning each of a plurality of images in a blog post and each of a plurality of images in a collection of images to a respective node in a graph, computing an adjacency matrix for the graph, wherein the adjacency matrix defines relationships between images in the blog post and images in the collections of images, and determining a first subset of the images in the collection of images that summarize the images in the image collection, wherein the subset is determined based on the adjacency matrix, wherein the adjacency matrix is computed based on the subset of the images in the collection of images.

Abstract: The disclosure provides an approach for recognizing and analyzing activities. In one embodiment, a learning application trains parameters of a hierarchical model which represents human (or object) activity at multiple levels of detail. Higher levels of detail may consider more context, and vice versa. Further, learning may be optimized for a user-preferred type of inference by adjusting a learning criterion. An inference application may use the trained model to answer queries about variable(s) at any level of detail. In one embodiment, the inference application may determine scores for each possible value of the query variable by finding the best hierarchical event representation that maximizes a scoring function while fixing the value of the query variable to its possible values. Here, the inference application may approximately determine the best hierarchical event representation by iteratively optimizing one level-of-detail variable at a time while fixing other level-of-detail variables, until convergence.

Abstract: Systems, methods and articles of manufacture for modeling a joint language-visual space. A textual query to be evaluated relative to a video library is received from a requesting entity. The video library contains a plurality of instances of video content. One or more instances of video content from the video library that correspond to the textual query are determined, by analyzing the textual query using a data model that includes a soft-attention neural network module that is jointly trained with a language Long Short-term Memory (LSTM) neural network module and a video LSTM neural network module. At least an indication of the one or more instances of video content is returned to the requesting entity.

Abstract: Techniques disclose an incrementally expanding object detection model. An object detection tool identifies, based on an object detection model, one or more objects in a sequence of video frames. The object detection model provides an object space including a plurality of object classes. Each object class includes one or more prototypes. Each object is classified as being an instance of one of the object classes. Each identified object is tracked across at least one of the frames. The object detection tool generates a measure of confidence for that object based on the tracking. Upon determining that the measure of confidence exceeds a threshold, the object detection tool adds a prototype of the instance to the object detection model.

Abstract: There is provided a system comprising a label database including a plurality of label, a non-transitory memory storing an executable code, and a hardware processor executing the executable code to receive a media content including a plurality of segments, each segment including a plurality of frames, extract a first plurality of features from a segment, extract a second plurality of features from each frame of the segment, determine an attention weight for each frame of the segment based on the first plurality of features extracted from the segment and the second plurality of features extracted from the segment, and determine that the segment depicts one of the plurality of labels in a label database based on the first plurality of features, the second plurality of features, and the attention weight of each frame of the plurality of frames of the segment.

Abstract: There is provided a system including a non-transitory memory storing an executable code and a hardware processor executing the executable code to receive a plurality of training contents depicting a plurality of activities, extract training object data from the plurality of training contents including a first training object data corresponding to a first activity, extract training scene data from the plurality of training contents including a first training scene data corresponding to the first activity, determine that a probability of the first activity is maximized when the first training object data and the first training scene data both exist in a sample media content.

Abstract: There is provided a system comprising a memory and a processor configured to receive a media content depicting an activity, extract a first plurality of features from a first segment of the media content, make a first prediction that the media content depicts a first activity based on the first plurality of features, wherein the first prediction has a first confidence level, extract a second plurality of features from a second segment of the media content, the second segment temporally following the first segment in the media content, make a second prediction that the media content depicts the first activity based on the second plurality of features, wherein the second prediction has a second confidence level, determine that the media content depicts the first activity based on the first prediction and the second prediction, wherein the second confidence level is at least as high as the first confidence level.

Abstract: Items of interest within digital information may be detected and associated with a label that provides context to the item of interest. The label may describe an item category of the item of interest. The knowledge base of item categories may be limited. Additional item categories may be learned by accessing sets of vocabulary that may relate to the known item categories.

Abstract: One or more image parameters of an image may be analyzed using a hierarchical set of models. Executing individual models in the set of models may generate outputs from analysis of different image parameters of the image. Inputs of one or more of the models may be conditioned on a set of outputs derived from one or more preceding model in the hierarchy.

Abstract: Systems, methods, and computer program products to perform an operation comprising assigning each of a plurality of images in a blog post and each of a plurality of images in a collection of images to a respective node in a graph, computing an adjacency matrix for the graph, wherein the adjacency matrix defines relationships between images in the blog post and images in the collections of images, and determining a first subset of the images in the collection of images that summarize the images in the image collection, wherein the subset is determined based on the adjacency matrix, wherein the adjacency matrix is computed based on the subset of the images in the collection of images.

Abstract: There are provided systems and methods for performing object classification through semantic mapping. Such an object classification system includes a system processor, a system memory, and an object categorizing unit stored in the system memory. The system processor is configured to execute the object categorizing unit to receive image data corresponding to an object, and to transform the image data into a directed quantity expressed at least in part in terms of semantic parameters. The system processor is further configured to determine a projection of the directed quantity onto an object representation map including multiple object categories, and to associate the object with a category from among the multiple object categories based on the projection.

Abstract: Techniques are disclosed for providing a learning-based clothing model that enables the simultaneous animation of multiple detailed garments in real-time. A simple conditional model learns and preserves key dynamic properties of cloth motions and folding details. Such a conditional model may be generated for each garment worn by a given character. Once generated, the conditional model may be used to determine complex body/cloth interactions in order to render the character and garment from frame-to-frame. The clothing model may be used for a variety of garments worn by male and female human characters (as well as non-human characters) while performing a varied set of motions typically used in video games (e.g., walking, running, jumping, turning, etc.).

Abstract: Embodiments presented herein describe techniques for generating a story graph using a collection of digital media, such as images and video. The story graph presents a structure for activities, events, and locales commonly occurring in sets of photographs taken by different individuals across a given location (e.g., a theme park, tourist attraction, convention, etc.). To build a story graph, streams from sets of digital media are generated. Each stream corresponds to media (e.g., images or video) taken in sequence at the location by an individual (or related group of individuals) over a period of time. For each stream, features from each media are extracted relative to the stream. Clusters of media are generated and are connected by directed edges. The connections indicate a path observed to have occurred in the streams from one cluster to another cluster.

Abstract: Techniques disclose an incrementally expanding object detection model. An object detection tool identifies, based on an object detection model, one or more objects in a sequence of video frames. The object detection model provides an object space including a plurality of object classes. Each object class includes one or more prototypes. Each object is classified as being an instance of one of the object classes. Each identified object is tracked across at least one of the frames. The object detection tool generates a measure of confidence for that object based on the tracking. Upon determining that the measure of confidence exceeds a threshold, the object detection tool adds a prototype of the instance to the object detection model.

Abstract: There are provided systems and methods for performing metadata extraction and management. Such a system includes a computing platform having a hardware processor, a system memory, and metadata extraction and management unit stored in the system memory. The system is configured to extract multiple metadata types from a media asset, and to aggregate the multiple metadata types to produce an aggregated metadata describing the media asset. The system is further configured to transform the aggregated metadata into at least one database entry identifying the media asset, and to map the at least one database entry into a graphical database so as to relate the media asset to at least one other media asset represented in the graphical database.

Abstract: Systems, methods, and computer program products to perform an operation comprising assigning each of a plurality of nodes of a graph to a distinct image, of a plurality of images, wherein the graph represents a story, wherein each node corresponds to a respective element of the story, wherein each node comprises: (i) an attribute and (ii) a text of the respective element of the story, wherein the plurality of graph nodes are assigned based on a set of attributes of each of the plurality of images and the attribute of each node, and generating a visual depiction of the story, wherein the visual depiction comprises an ordered representation of each of the distinct images and the text of each respective element of the story.

Abstract: The disclosure provides an approach for detecting and localizing action in video. In one embodiment, an action detection application receives training video sequences and associated eye gaze fixation data collected from a sample of human viewers. Using the training video sequences and eye gaze data, the action detection application learns a model which includes a latent regions potential term that measures the compatibility of latent spatio-temporal regions with the model, as well as a context potential term that accounts for contextual information that is not directly produced by the appearance and motion of the actor. The action detection application may train this model in, e.g., the latent structural SVM framework by minimizing a cost function which encodes the cost of an incorrect action label prediction and a mislocalization of the eye gaze. During training and thereafter, inferences using the model may be made using an efficient dynamic programming algorithm.

Abstract: The disclosure provides an approach for detecting objects in images. An object detection application receives a set of training images with object annotations. Given these training images, the object detection application generates semantic labeling for object detections, where the labeling includes lower-level subcategories and higher-level visual composites. In one embodiment, the object detection application identifies subcategories using an exemplar support vector machine (SVM) based clustering approach. Identified subcategories are used to initialize mixture components in mixture models which the object detection application trains in a latent SVM framework, thereby learning a number of subcategory classifiers that produce, for any given image, a set of candidate windows and associated subcategory labels.