Abstract: Systems and methods to present web image search results for effective image browsing are described. In one aspect, task-based attention objects for each of multiple images associated with image search results are generated. Thumbnail images from respective ones of the images are created as a function of at least the task-based attention objects. The thumbnail images emphasize image region(s) of greater priority to a user in view of a keyword or expanded keyword associated with the search results.

Abstract: Word correlations are estimated using a content-based method, which uses visual features of image representations of the words. The image representations of the subject words may be generated by retrieving images from data sources (such as the Internet) using image search with the subject words as query words. One aspect of the techniques is based on calculating the visual distance or visual similarity between the sets of retrieved images corresponding to each query word. The other is based on calculating the visual consistence among the set of the retrieved images corresponding to a conjunctive query word. The combination of the content-based method and a text-based method may produce even better result.

Abstract: A dual cross-media relevance model (DCMRM) is used for automatic image annotation. In contrast to the traditional relevance models which calculate the joint probability of words and images over a training image database, the DCMRM model estimates the joint probability by calculating the expectation over words in a predefined lexicon. The DCMRM model may be advantageous because a predefined lexicon potentially has better behavior than a training image database. The DCMRM model also takes advantage of content-based techniques and image search techniques to define the word-to-image and word-to-word relations involved in image annotation. Both relations can be estimated by using image search techniques on the web data as well as available training data.

Abstract: A method and system for determining the importance of news events based on news reports published by news sources on news home pages is provided. A news system identifies news home pages that contain references to news reports on detail pages. The news system calculates the importance of a news event based on the importance of news reports reporting that news event. The news system may determine the importance of a news report based on the assumption that the importance of a news report is based on the credibility of news home pages and the importance of similar news reports. The news system may recursively define the importance of a news report based on the credibility of news home pages and the similarity to other news pages and the importance of a news home page based on the importance of its news reports and similar news reports.

Abstract: Systems and methods for bipartite graph reinforcement modeling to annotate web images are described. In one aspect the systems and methods implement bipartite graph reinforcement modeling operations to identify a set of annotations that are relevant to a Web image. The systems and methods annotate the Web image with the identified annotations. The systems and methods then index the annotated Web image. Responsive to receiving an image search query from a user, wherein the image search query comprises information relevant to at least a subset of the identified annotations, the image search engine service presents the annotated Web image to the user.

Abstract: Systems and methods for visual language modeling for image classification are described. In one aspect the systems and methods model training images corresponding to multiple image categories as matrices of visual words. Visual language models are generated from the matrices. In view of a given image, for example, provided by a user or from the Web, the systems and methods determine an image category corresponding to the given image. This image categorization is accomplished by maximizing the posterior probability of visual words associated with the given image over the visual language models. The image category, or a result corresponding to the image category, is presented to the user.

Abstract: A method and system for generating a search request from a multimodal query that includes a query image and query text is provided. The multimodal query system identifies images of a collection that are textually related to the query image based on similarity between words associated with each image and the query text. The multimodal query system then selects those images of the identified images that are visually related to the query image. The multimodal query system may formulate a search request based on keywords of web pages that contain the selected images and submit that search request to a search engine service.

Abstract: An image advertisement system of a computing device displays as part of a display page an advertisement-infused thumbnail of an image prior to displaying the image. The image advertisement system initially receives a display page with an indication of an image to be displayed as part of the display page. The image advertisement system generates an advertisement-infused thumbnail of the image by combining advertisement content with a thumbnail of the image. The image advertisement system then displays the display page with the advertisement-infused thumbnail of the image in place of the image. The image advertisement system then replaces the displayed advertisement-infused thumbnail with the image.

Abstract: An advertisement image classification system trains a binary classifier to classify images as advertisement images or non-advertisement images and then uses the binary classifier to classify images of web pages as advertisement images or non-advertisement images. During a training phase, the classification system generates training data of feature vectors representing the images and labels indicating whether an image is an advertisement image or a non-advertisement image. The classification system trains a binary classifier to classify images using training data. During a classification phase, the classification system inputs a web page with an image and generates a feature vector for the image. The classification system then applies the trained binary classifier to the feature vector to generate a score indicating whether the image is an advertisement image or a non-advertisement image.

Abstract: A method and system for determining dominance of the media elements of display pages is provided. The dominance system provides a scoring mechanism for scoring the dominance of media elements of display pages based on features of each media element of the display page. To generate the scores for the media elements of the display page, the dominance system first identifies the media elements and then identifies the features of the media elements. The dominance system then scores the identified media elements using the provided scoring mechanism and the identified features.

Abstract: A method and system for generating a search request from a multimodal query that includes a query image and query text is provided. The multimodal query system identifies images of a collection that are textually related to the query image based on similarity between words associated with each image and the query text. The multimodal query system then selects those images of the identified images that are visually related to the query image. The multimodal query system may formulate a search request based on keywords of web pages that contain the selected images and submit that search request to a search engine service.

Abstract: An object search system generates a hierarchical clustering of objects of a collection based on similarity of the objects. The object search system generates a separate hierarchical clustering of objects for multiple features of the objects. To identify objects similar to a target object, the object search system first generates a feature vector for the target object. For each feature of the feature vector, the object search system uses the hierarchical clustering of objects to identify the cluster of objects that is most “feature similar” to that feature of the target object. The object search system indicates the similarity of each candidate object based on the features for which the candidate object is similar.

Abstract: A method and system for determining dominance of the media elements of display pages is provided. The dominance system provides a scoring mechanism for scoring the dominance of media elements of display pages based on features of each media element of the display page. To generate the scores for the media elements of the display page, the dominance system first identifies the media elements and then identifies the features of the media elements. The dominance system then scores the identified media elements using the provided scoring mechanism and the identified features.

Abstract: A method and system for determining relevance of a document having text and images to a text string is provided. A scoring system identifies image text associated with an image of the document. The scoring system calculates an image score indicating relevance of the image text to the text string. The image score may be used in many applications, such as searching, summary generation, and document classification, image search, and image classification.

Abstract: A method and system for generating a detector to detect a dominant color of an image is provided. A dominant color system trains a detector to classify colors as being dominant colors of images. The dominant color system trains the detector using a collection of training images. To train the detector, the dominant color system first identifies candidate dominant colors of the training images. The dominant color system then extracts features of the candidate dominant colors. The dominant color system also inputs an indication of dominance of each of the candidate dominant colors. The dominant color system then trains a detector to detect the dominant color of images using the extracted features and indications of dominance of the candidate dominant colors as training data.

Abstract: A duplicate image detection system generates an image table that maps hash codes of images to their corresponding images. The image table may group images according to their group identifiers generated from the most significant elements of the hash codes based on significance of the elements in representing an image. The image table thus segregates images by their group identifiers. To detect a duplicate image of a target image, the detection system generates a target hash code for the target image. The detection system then identifies the group of the target image based on the group identifier of the target hash code. After identifying the group identifier, the detection system searches the corresponding group table to identify hash codes that have values that are similar to the target hash code. The detection system then selects the images associated with those similar hash codes as being duplicates of the target image.

Abstract: A method and system for propagating the relevance of labeled documents to a query to unlabeled documents is provided. The propagation system provides training data that includes queries, documents labeled with their relevance to the queries, and unlabeled documents. The propagation system then calculates the similarity between pairs of documents in the training data. The propagation system then propagates the relevance of the labeled documents to similar, but unlabeled, documents. The propagation system may iteratively propagate labels of the documents until the labels converge on a solution. The training data with the propagated relevances can then be used to train a ranking function.

Abstract: A method and system for propagating the relevance of labeled documents to a query to unlabeled documents is provided. The propagation system provides training data that includes queries, documents labeled with their relevance to the queries, and unlabeled documents. The propagation system then calculates the similarity between pairs of documents in the training data. The propagation system then propagates the relevance of the labeled documents to similar, but unlabeled, documents. The propagation system may iteratively propagate labels of the documents until the labels converge on a solution. The training data with the propagated relevances can then be used to train a ranking function.

Abstract: A method and system for classifying an image as a photograph or a graphic based on a ranked prevalent color histogram feature or a ranked region size feature is provided. The prevalent color histogram feature contains counts of the colors that are most prevalent in the image sorted in descending order. The region size feature contains counts of the largest regions of the image sorted in descending order. The classification system then classifies the image based on the ranked prevalent color histogram feature and/or the ranked region size feature using a previously trained classifier.

Abstract: A content object indexing process including creating a content object knowledge index, calculating a description vector of a target content object, and indexing the target content object by searching for the description vector in the content object knowledge database. It may be difficult to search for an exact content object such as a music file or academic researcher as a conventional search index may not include related hierarchical information. A content object indexing process may add hierarchical information taken from a content object knowledge index and incorporate the hierarchical information to the index entry for a specific content object. An application of such a content object indexing process may be a world wide web search engine.