Abstract: This document describes techniques that utilize a learning method to generate a ranking model for use in image search systems. The techniques leverage textual information and visual information simultaneously when generating the ranking model. The tools are further configured to apply the ranking model responsive to receiving an image search query.

Abstract: An ImageWiki architecture is used to generate an image-based web page for an image on the Web. An ImageWiki page may be created automatically or individually, by a user of the Web. Additionally, a user may revise existing ImageWiki pages to update a particular page or correct an incorrect or misleading previous entry. The ImageWiki application indexes images located on the Web. Once the images are indexed, the information related to the images is mined and extracted from various sources of web data. Finally, an ImageWiki page or web page is generated for each image. The resulting ImageWiki page contains the image as well as the aggregated information relating to the image.

Abstract: A prototype-based re-ranking method may re-rank search results to provide a re-ranked set of search results. In response to receiving one or more queries, a set of search results may be generated whereby each of the search results may be associated with a rank position. Based at least in part on the search results, one or more prototypes may be generated that visually represent the one or more queries or the search results. The one or more prototypes may be used to construct one or more meta re-rankers that may generate re-ranking scores for the search results. The re-ranking scores may be aggregated to produce a final relevance score for each search result included in the set of search results. Based at least in part on the relevance score of each search result and/or a learned re-ranking model, a set of re-ranked search results may be provided.

Abstract: Attractiveness of an image may be estimated by integrating extracted visual features with contextual cues pertaining to the image. Image attractiveness may be defined by the visual features (e.g., perceptual quality, aesthetic sensitivity, and/or affective tone) of elements contained within the image. Images may be indexed based on the estimated attractiveness, search results may be presented based on image attractiveness, and/or a user may elect, after receiving image search results, to re-rank the image search results by attractiveness.

Abstract: Technologies for recommending relevant tags for the tagging of media based on one or more initial tags provided for the media and based on a large quantity of other tagged media. Sample media as candidates for recommendation are provided by a set of weak rankers based on corresponding relevance measures in semantic and visual domains. The various samples provided by the weak rankers are then ranked based on relative order to provide a list of recommended tags for the media. The weak rankers provide sample tags based on relevance measures including tag co-occurrence, tag content correlation, and image-conditioned tag correlation.

Abstract: Systems and methods for determining insertion points in a first video stream are described. The insertions points being configured for inserting at least one second video into the first video. In accordance with one embodiment, a method for determining the insertion points includes parsing the first video into a plurality of shots. The plurality of shots includes one or more shot boundaries. The method then determines one or more insertion points by balancing a discontinuity metric and an attractiveness metric of each shot boundary.

Abstract: Some implementations provide techniques and arrangements to perform image retrieval. For example, some implementations identify an object of interest and a visual context in a first image. In some implementations, a second image that includes a second object of interest and a second visual context may be compared to the object of interest and the visual content, respectively, to determine whether the second image matches the first image.

Abstract: Colors of images and videos are modified to make differences in the colors more perceptible to colorblind users. An exemplary recoloring process utilizes a color space transformation, a local color rotation and a global color rotation to transform colors of visual objects from colors which may not be distinguishable by the colorblind user to colors which may be distinguishable by the colorblind user.

Abstract: Supervised re-ranking for visual search may include re-ordering images that are returned in response to a text-based image search by exploiting visual information included in the images. In one example, supervised re-ranking for visual search may include receiving a textual query, obtaining an initial ranking result including a plurality of images corresponding to the textual query, and representing the textual query by a visual context of the plurality of images. A query-independent re-ranking model may be trained based on visual re-ranking features of the plurality of images of the textual query in accordance with a supervised training algorithm.

Abstract: A hierarchical sparse codebook allows efficient search and comparison of images in image retrieval. The hierarchical sparse codebook includes multiple levels and allows a gradual determination/classification of an image feature of an image into one or more groups or nodes by traversing the image feature through one or more paths to the one or more groups or nodes of the codebook. The image feature is compared with a subset of nodes at each level of the codebook, thereby reducing processing time.

Abstract: Techniques described herein enable better understanding of the intent of a user that submits a particular search query. These techniques receive a search request for images associated with a particular query. In response, the techniques determine images that are associated with the query, as well as other keywords that are associated with these images. The techniques then cluster, for each set of images associated with one of these keywords, the set of images into multiple groups. The techniques then rank the images and determine a representative image of each cluster. Finally, the tools suggest, to the user that submitted the query, to refine the search based on user selection of a keyword and a representative image. Thus, the techniques better understand the user's intent by allowing the user to refine the search based on another keyword and based on an image on which the user wishes to focus the search.

Abstract: Techniques for construction of a visual codebook are described herein. Feature points may be extracted from large numbers of images. In one example, images providing N feature points may be used to construct a codebook of K words. The centers of each of K clusters of feature points may be initialized. In a looping or iterative manner, an assignment step assigns each feature point to a cluster and an update step locates a center of each cluster. The feature points may be assigned to a cluster based on a lesser of a distance to a center of a previously assigned cluster and a distance to a center derived by operation of an approximate nearest neighbor algorithm having aspects of randomization. The loop terminates when the feature points have sufficiently converged to their respective clusters. Centers of the clusters represent visual words, which may be used to construct the visual codebook.

Abstract: Colorblind accessible image search technique embodiments are presented that re-rank the results of a relevance-ranked image search to account for the accessibility of the images to a colorblind person. This is accomplished by first computing a colorblind accessibility quantity for each image of interest in the search results. A colorblind accessibility quantity quantizes the degree to which color information is preserved when an image is perceived by a colorblind person viewing the image. It is computed by generating a colorblind version of an image that simulates how the image would appear to the colorblind person. An amount quantifying the loss of color information between the image and the colorblind version of the image is then estimated. This estimate is used to compute the colorblind accessibility quantity for the image. Once the colorblind accessibility quantities have been computed, the image search results are re-ranked based on these quantities.

Abstract: Technologies for a human computation framework suitable for answering common sense questions that are difficult for computers to answer but easy for humans to answer. The technologies support solving general common sense problems without a priori knowledge of the problems; support for determining whether an answer is from a bot or human so as to screen out spurious answers from bots; support for distilling answers collected from human users to ensure high quality solutions to the questions asked; and support for preventing malicious elements in or out of the system from attacking other system elements or contaminating the solutions produced by the system, and preventing users from being compensated without contributing answers.

Abstract: Some implementations provide techniques and arrangements to perform image retrieval. For example, some implementations identify an object of interest and a visual context in a first image. In some implementations, a second image that includes a second object of interest and a second visual context may be compared to the object of interest and the visual content, respectively, to determine whether the second image matches the first image.

Abstract: Computer program products, devices, and methods for generating in-text embedded advertising are described. Embedded advertising is “hidden” or embedded into a message by matching an advertisement to the message and identifying a place in the message to insert the advertisement. For textual messages, statistical analysis of individual sentences is performed to determine where it would be most natural to insert an advertisement. Statistical rules of grammar derived from a language model may be used choose a natural and grammatical place in the sentence for inserting the advertisement. Insertion of the advertisement creates a modified sentence without degrading a meaning of the original sentence, yet also includes the advertisement as a part of a new sentence.

Abstract: An informative priors image search result summarization system and method that summarizes image search results based on the image relevance (as determined by a search engine's initial ranking) and the image quality. Embodiments of the system and method cluster the image search results, rank images within each cluster based on a computed image score, and then select a summary image for the cluster. Each cluster is analyzed and an image in the cluster having the maximum image score is included in a selected summary collection. The image score is computed using the image relevance and the image quality, as well as a cluster coherence, a density, and a diversity. The selection of images from a collection of candidate images generates an image search result summarization, which is presented to a user. The summaries are presented to the user in a ranked order based on their image scores.

Abstract: A similarity of a first video to a second video may be identified automatically. Images are received from the videos, and divided into sub-images. The sub-images are evaluated based on a feature common to each of the sub-images. Binary representations of the images may be created based on the evaluation of the sub-images. A similarity of the first video to the second video may be determined based on a number of occurrences of a binary representation in the first video and the second video.

Abstract: Techniques describe submitting a video clip as a query by a user. A process retrieves images and information associated with the images in response to the query. The process decomposes the video clip into a sequence of frames to extract the features in a frame and to quantize the extracted features into descriptive words. The process further tracks the extracted features as points in the frame, a first set of points to correspond to a second set of points in consecutive frames to construct a sequence of points. Then the process identifies the points that satisfy criteria of being stable points and being centrally located in the frame to represent the video clip as a bag of descriptive words for searching for images and information related to the video clip.

Abstract: Technologies for a human computation framework suitable for answering common sense questions that are difficult for computers to answer but easy for humans to answer. The technologies support solving general common sense problems without a priori knowledge of the problems; support for determining whether an answer is from a bot or human so as to screen out spurious answers from bots; support for distilling answers collected from human users to ensure high quality solutions to the questions asked; and support for preventing malicious elements in or out of the system from attacking other system elements or contaminating the solutions produced by the system, and preventing users from being compensated without contributing answers.