Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for creating an image similarity model. In one aspect, a method includes obtaining feature vectors for images in a set of images, and determining first similarity measures for unlabeled images relative to a reference image. The first similarity measures are independent of first similarity feedback between the unlabeled images and the reference image. The unlabeled images are ranked based on the first similarity measures, and a weighted feature vector is generated based, in part, on the ranking. Second similarity measures are determined, independent of second similarity feedback, for labeled images and a second reference image. The labeled images are ranked based on the second similarity measures. The weighted feature vector is adjusted based, in part, on a comparison of the ranking to a second ranking of the labeled images that is based on the second similarity feedback.

Abstract: Methods, systems and apparatus for identifying modified images based on seed images that are known to be modified images. In an aspect, a method includes accessing data identifying a set of first seed images; for each first seed image, determining a respective first set of similar images from images in an image corpus, each similar image having a visual similarity score that is a measure of visual similarity of the similar image to the first seed image based on the image content of the similar image and the first seed image that satisfies a first seed image similarity threshold; and for each similar image in each respective first set of similar images, attributing to the similar image signal data of each first seed image for which the similar image has a respective visual similarity score satisfying the first seed image similarity threshold.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a query image model. In one aspect, a method includes receiving a set of images determined to be responsive to a query and ranked according to a first order; determining a positive image signature from a first subset of images selected from images ranked highest in the first order, determining a negative image signature from a second subset of images selected from images ranked lowest in the first order, determining a query image signature for the query based on a difference of the positive image signature and the negative image signature; and applying the query image signature to each image in the set of images to rank the images according to a second order that is different from the first order.

Abstract: Methods, systems and apparatus for choosing image labels. In one aspect, a method includes receiving data specifying a first image, receiving text labels for the first image, receiving search results in response to a web search performed using at least some of the text labels as queries, ranking the text labels, at least in part, based on a number of resources referenced by the received search results, wherein at least some of the resources each include an image matching the first image, and selecting an image label for the image from the ranked text labels, the image label being selected based on the ranking.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing queries made up of images. In one aspect, a method includes indexing images by image descriptors. The method further includes associating descriptive n-grams with the images. In another aspect, a method includes receiving a query, identifying text describing the query, and performing a search according to the text identified for the query.

Abstract: Methods, systems and apparatus for identifying modified images based on visual dissimilarity to a first image. In an aspect, a method includes determining, for each of a first image and a second image, a respective set of local image feature descriptions; determining one or more unmatched regions of the images that include unmatched image features and that correspond to one or more same respective regions in both the first image and the second image; determining, for each of the one or more unmatched regions of the images, a modification measure based on the image data corresponding to the unmatched region in the first image and the image data corresponding to the unmatched region in the second image; and determining that the second image is a modification of the first image when one of the modification measures meets a modification measure threshold.

Abstract: An image classification system trains an image classification model to classify images relative to text appearing with the images. Training images are iteratively selected and classified by the image classification model according to feature vectors of the training images. An independent model is trained for unique n-grams of text. The image classification system obtains text appearing with an image and parses the text into candidate labels for the image. The image classification system determines whether an image classification model has been trained for the candidate labels. When an image classification model corresponding to a candidate label has been trained, the image classification subsystem classifies the image relative to the candidate label. The image is labeled based on candidate labels for which the image is classified as a positive image.

Abstract: Methods, systems and apparatus for refining image relevance models. In general, one aspect of the subject matter described in this specification can be implemented in methods that include re-training an image relevance model by generating a first re-trained model based on content feature values of first images of a first portion of training images in a set of training images, receiving, from the first re-trained model, image relevance scores for second images of a second portion of the set of training images, removing, from the set of training images, some of the second images identified as outlier images for which the image relevance score received from the first re-trained model is below a threshold score, and generating a second re-trained model based on content feature values of the first images of the first portion and the second images of the second portion that remain following removal of the outlier images.

Abstract: Methods, systems, and apparatus, including computer program products, for identifying images relevant to a query are disclosed. An image search subsystem selects images to reference in image search results that are responsive to a query based on an image relevance model that is trained for the query. An independent image relevance model is trained for each unique query that is identified by the image search subsystem. The image relevance models can be applied to images to order image search results obtained for the query. Each relevance model is trained based on content feature values of images that are identified as being relevant to the query (e.g., frequently selected from the image search results) and images that are identified as being relevant to another unique query. The trained model is applied to the content feature values of all known images to generate an image relevance score that can be used to order search results for the query.

Abstract: A method includes receiving a search query comprising one or more query terms, receiving image relevance models that each generate relevance measures of content feature values of images to a query, each image relevance model being a predictive model that has been trained based on content feature values of a set of training images, and each of the queries being a unique set of one or more query terms received by a search system as a query input, identifying an image relevance model for a different query that has been identified as similar to the received search query, and calculating a fractional adjustment multiplier for search results responsive to the received search query, the fractional adjustment multiplier being based on a relevance measure generated by the identified image relevance model for the different query and based on a degree of similarity between the different query and the received search query.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for identifying images responsive to a search phrase are disclosed. In one aspect, a method includes identifying a set of responsive images for a search phrase that includes two or more terms. Interaction rankings are determined for images in the set of responsive images. Two or more sub-queries are created based on the search phrase. Sub-query model rankings are determined for images in the set of responsive images. A search phrase score is determined for the image relevance model. Based on the search phrase scores for the sub-queries, one of the sub-query models is selected as a model for the search phrase.

Abstract: Systems, method, and apparatus including computer program products for providing image search results. In some implementations, a method is provided. The method includes receiving from a user a query for images including static images, moving images, and images within multimedia content, identifying at least one of a language attribute and a locale attribute of the user, generating multiple search results, each result corresponding to an image content item that satisfies the query, ordering the search results based at least on click data for image content items that satisfy the query, the click data gathered from users having at least one of the language attribute and the locale attribute, and presenting the ordered search results to the user, including presenting representations of the corresponding image content items.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for determining image search results. One of the methods includes generating a plurality of feature vectors for each image in a collection of images, wherein each feature vector is associated with an image tile of an image, wherein each feature vector corresponds to one of a plurality of predetermined visual words. All images in the collection of images that share at least a threshold number of matching visual words associated with matching image tiles are classified as near-duplicate images.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for creating an image similarity model. In one aspect, a method includes obtaining feature vectors for images in a set of images, and determining first similarity measures for unlabeled images relative to a reference image. The first similarity measures are independent of first similarity feedback between the unlabeled images and the reference image. The unlabeled images are ranked based on the first similarity measures, and a weighted feature vector is generated based, in part, on the ranking. Second similarity measures are determined, independent of second similarity feedback, for labeled images and a second reference image. The labeled images are ranked based on the second similarity measures. The weighted feature vector is adjusted based, in part, on a comparison of the ranking to a second ranking of the labeled images that is based on the second similarity feedback.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for identifying images responsive to a search phrase are disclosed. In one aspect, a method includes identifying a set of responsive images for a search phrase that includes two or more terms. Interaction rankings are determined for images in the set of responsive images. Two or more sub-queries are created based on the search phrase. Sub-query model rankings are determined for images in the set of responsive images. A search phrase score is determined for the image relevance model. Based on the search phrase scores for the sub-queries, one of the sub-query models is selected as a model for the search phrase.

Abstract: Methods, systems and apparatus for refining image relevance models. In general, one aspect includes receiving a trained image relevance model that generates relevance measures of content feature values of images to a query, identifying a first threshold number of common content feature values for the set of training images, the common content feature values being identified as a set of content feature values that are each shared by at least a portion of the training images, identifying a subset of the set of training images having a quantity of the common content feature values greater than a second threshold number of content features values, and generating a re-trained image relevance model based on content feature values of the set of training images, wherein content feature values of the subset of training images are weighted higher than content feature values of the training images not in the subset.

Abstract: Methods, systems and apparatus for refining image annotations. In one aspect, a method includes receiving, for each image in a set of images, a corresponding set of labels determined to be indicative of subject matter of the image. For each label, one or more confidence values are determined. Each confidence value is a measure of confidence that the label accurately describes the subject matter of a threshold number of respective images to which it corresponds. Labels for which each of the one or more confidence values meets a respective confidence threshold are identified as high confidence labels. For each image in the set of images, labels in its corresponding set of labels that are high confidence labels are identified. Images having a corresponding set of labels that include at least a respective threshold number of high confidence labels are identified as high confidence images.

Abstract: Methods, systems and apparatus for choosing image labels. In one aspect, a method includes receiving data specifying a first image, receiving text labels for the first image, receiving search results in response to a web search performed using at least some of the text labels as queries, ranking the text labels, at least in part, based on a number of resources referenced by the received search results, wherein at least some of the resources each include an image matching the first image, and selecting an image label for the image from the ranked text labels, the image label being selected based on the ranking.

Abstract: A method includes receiving a search query comprising one or more query terms, receiving image relevance models that each generate relevance measures of content feature values of images to a query, each image relevance model being a predictive model that has been trained based on content feature values of a set of training images, and each of the queries being a unique set of one or more query terms received by a search system as a query input, identifying an image relevance model for a different query that has been identified as similar to the received search query, and calculating a fractional adjustment multiplier for search results responsive to the received search query, the fractional adjustment multiplier being based on a relevance measure generated by the identified image relevance model for the different query and based on a degree of similarity between the different query and the received search query.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for analyzing image search result relevance. In one aspect, a method includes receiving result data specifying a search query and responsive image search results that reference images that are responsive to the search query. A determination is made that the search query matches an indexed query. An image relevance model is identified for the indexed query. The image relevance model can output a relevance score adjustment factor for an image search result based on image feature values of the image that is referenced by the search result. A relevance score adjustment factor is determined for each image search result using the identified image relevance model. A relevance score for each image search result is adjusted using the image's image relevance score adjustment factor. The images are ranked based on the adjusted relevance scores.