Abstract: Systems and methods for learning topic models from unstructured data and applying the learned topic models to recognize semantics for new data items are described herein. In at least one embodiment, a corpus of multimedia data items associated with a set of labels may be processed to generate a refined corpus of multimedia data items associated with the set of labels. Such processing may include arranging the multimedia data items in clusters based on similarities of extracted multimedia features and generating intra-cluster and inter-cluster features. The intra-cluster and the inter-cluster features may be used for removing multimedia data items from the corpus to generate the refined corpus. The refined corpus may be used for training topic models for identifying labels. The resulting models may be stored and subsequently used for identifying semantics of a multimedia data item input by a user.

Abstract: A similarity of a first video to a second video may be identified automatically. Images are received from the videos, and divided into sub-images. The sub-images are evaluated based on a feature common to each of the sub-images. Binary representations of the images may be created based on the evaluation of the sub-images. A similarity of the first video to the second video may be determined based on a number of occurrences of a binary representation in the first video and the second video.

Abstract: Described is a technology for computing visual and textual summaries for tagged image collections. Heterogeneous affinity propagation is used to together identify both visual and textual exemplars. The heterogeneous affinity propagation finds the exemplars for relational heterogeneous data (e.g., images and words) by considering the relationships (e.g., similarities) within pairs of images, pairs of words, and relationships of words to images (affinity) in an integrated manner.

Abstract: Architecture that performs image page index selection. A learning-based framework learns a statistical model based on the hyperlink (URL-uniform resource locator) previous click information obtained from the image search users. The learned model can combine the features of a newly discovered URL to predict the possibility of the newly-discovered URL being clicked in the future image search. In addition to existing web index selection features, image clicks are added as features, and the image clicks are aggregated over different URL segments, as well as the site modeling pattern trees to reduce the sparse problem of the image click information.

Abstract: A computing device configured to determine that one or more regions of an image are associated with a tag of the image is described herein. The computing device is further configured to determine one or more attribute tags describing at least one of the content or context of the one or more regions. Upon determining the attribute tags, the computing device associates the attribute tags with the tag to enable image searching based on the tag and attribute tags.

Abstract: This document describes techniques that utilize a learning method to generate a ranking model for use in image search systems. The techniques leverage textual information and visual information simultaneously when generating the ranking model. The tools are further configured to apply the ranking model responsive to receiving an image search query.

Abstract: Techniques for intelligent image search results summarization and browsing scheme are described. Images having visual attributes are evaluated for similarities based in part on their visual attributes. At least one preference score indicating a probability of an image to be selected into a summary is calculated for each image. Images are selected based on the similarity of the selected images to the other images and the preference scores of the selected images. A summary of the plurality of images is generated including the selected one individual image.

Abstract: Dominant color names may be extracted from an image by analyzing spatial-context of pixels contained in the image. A dominant color region may be defined by taking a double-threshold approach that addresses ambiguous color regions and a degree of confidence that each pixel belongs in the dominant color region. Affiliation maps and binary maps may be used to generate the dominant color region. Images may be converted to a saliency map, from which a region of interest may be assigned a dominant color name. Image search results may be filtered by the dominant color name associated with the image.

Abstract: Events may be determined based on an image and context data associated with the image. An event type associated with the image may be determined based on a concept of the image. A list of events may be retrieved from an event database based on the context data. The retrieved list of events may then be ranked based on the determined event type and the context data. Through this event determination, a user may obtain information of one or more events happening at a specific location simply by capturing an image of that specific location, thereby saving the user from searching and browsing the Internet or brochure to locate the information of the one or more events at the specific location.

Abstract: Attractiveness of an image may be estimated by integrating extracted visual features with contextual cues pertaining to the image. Image attractiveness may be defined by the visual features (e.g., perceptual quality, aesthetic sensitivity, and/or affective tone) of elements contained within the image. Images may be indexed based on the estimated attractiveness, search results may be presented based on image attractiveness, and/or a user may elect, after receiving image search results, to re-rank the image search results by attractiveness.

Abstract: Methods and systems for active image tagging are usable to build large datasets of tagged images by combining manual tagging by a user and automatic tagging by a computing device based on the manual tagging. Such tags may be used to effectively sort, organize, link, and search for images within large datasets of images. Additionally, the active image tagging may be configured to utilize a tagging game where multiple users manually tag images by playing a game on a computing device.

Abstract: Visual concepts contained within a video clip are classified based upon a set of target concepts. The clip is segmented into shots and a multi-layer multi-instance (MLMI) structured metadata representation of each shot is constructed. A set of pre-generated trained models of the target concepts is validated using a set of training shots. An MLMI kernel is recursively generated which models the MLMI structured metadata representation of each shot by comparing prescribed pairs of shots. The MLMI kernel is subsequently utilized to generate a learned objective decision function which learns a classifier for determining if a particular shot (that is not in the set of training shots) contains instances of the target concepts. A regularization framework can also be utilized in conjunction with the MLMI kernel to generate modified learned objective decision functions. The regularization framework introduces explicit constraints which serve to maximize the precision of the classifier.

Abstract: Techniques for intelligent image search results summarization and browsing scheme are described. Images having visual attributes are evaluated for similarities based in part on their visual attributes. At least one preference score indicating a probability of an image to be selected into a summary is calculated for each image. Images are selected based on the similarity of the selected images to the other images and the preference scores of the selected images. A summary of the plurality of images is generated including the selected one individual image.

Abstract: Architecture that includes a junk (unwanted) image detection algorithm which performs junk image detection of unwanted images before the images are actually downloaded for indexing. Features are employed related to image location information and host websites, such as image path descriptor (e.g., URL-uniform resource locator) pattern features, webpage content features, click features, and image aggregated information in a machine learning based framework to predict the probability that an image is unwanted (or wanted) before the images are downloaded. The framework is then applied to build a statistical model and predict junk scores. By removing image URLs marked as “junk” from the work list of an automated indexer (e.g., crawler), the indexer bandwidth is significantly improved with a corresponding improvement in the publish rate.

Abstract: The concept-structured image search technique described herein pertains to a technique for enabling a user to indicate their semantic intention and then retrieve and rank images from a database or other image set according to this intention. The concept-structured image search technique described herein includes a new interface for image search. With this interface, a user can freely type several key textual words in arbitrary positions on a blank image, and also describe a region for each keyword that indicates its influence scope, which is called concept structure herein. The concept-structured image search technique will return and rank images that are in accordance with the concept structure indicated by the user. One embodiment of the technique can be used to create a synthesized image without actually using the synthesized image to perform a search of an image set.

Abstract: Technologies for recommending relevant tags for the tagging of media based on one or more initial tags provided for the media and based on a large quantity of other tagged media. Sample media as candidates for recommendation are provided by a set of weak rankers based on corresponding relevance measures in semantic and visual domains. The various samples provided by the weak rankers are then ranked based on relative order to provide a list of recommended tags for the media. The weak rankers provide sample tags based on relevance measures including tag co-occurrence, tag content correlation, and image-conditioned tag correlation.

Abstract: Architecture that performs image page index selection. A learning-based framework learns a statistical model based on the hyperlink (URL-uniform resource locator) previous click information obtained from the image search users. The learned model can combine the features of a newly discovered URL to predict the possibility of the newly-discovered URL being clicked in the future image search. In addition to existing web index selection features, image clicks are added as features, and the image clicks are aggregated over different URL segments, as well as the site modeling pattern trees to reduce the sparse problem of the image click information.

Abstract: This disclosure describes various exemplary systems, computer program products, and methods for feature distance metric learning with feature decomposition (DMLFD). The disclosure describes decomposing a high-dimensional feature space into one or more low-dimensional feature spaces according to minimum dependence. Furthermore, the disclosure describes how the sub-metrics are constructed and combined to form a global metric.

Abstract: Described herein is technology for, among other things, selecting a representative thumbnail from a video clip. The technology involves analyzing frames of the video clip to determine which frames are stable, the result of the analysis being a number of segments of stable frames. From the stable segments, a number of candidate segments are selected, where candidate segments are those segments determined to a degree of certainty to be program content. The representative thumbnail is then selected from among the frames of the candidate segments.

Abstract: Systems and methods for determining insertion points in a first video stream are described. The insertions points being configured for inserting at least one second video into the first video. In accordance with one embodiment, a method for determining the insertion points includes parsing the first video into a plurality of shots. The plurality of shots includes one or more shot boundaries. The method then determines one or more insertion points by balancing a discontinuity metric and an attractiveness metric of each shot boundary.