Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for automatically extracting logos from images. Methods include generating a query list including a plurality of logo search queries, for each logo search query of the plurality of logo search queries: generating a plurality of image search results, each image search result including image data, and clustering the plurality of image search results into a plurality of clusters, each cluster including a plurality of images of the plurality of image search results, extracting, for each cluster of the plurality of clusters, a representative image to provide a plurality of representative images, and a name corresponding to the representative image to provide a plurality of names, and providing the plurality of representative images and the plurality of names to a logo index, the logo index being accessible to identify one or more logo images in a query image.

Abstract: A method and apparatus for enabling semantic association of images with augmentation data is described. The method may include receiving digital image data captured by a mobile device. The method may also include performing image recognition analysis on an object within the digital image data to identify the object. The method may also include querying a semantic associations database based on a determined identify of the object to determine a secondary meaning associated with the object, and transmitting augmentation data to the mobile device that is semantically relevant to the object based on the secondary meaning.

Abstract: A facial recognition search system identifies one or more likely names (or other personal identifiers) corresponding to the facial image(s) in a query as follows. After receiving the visual query with one or more facial images, the system identifies images that potentially match the respective facial image in accordance with visual similarity criteria. Then one or more persons associated with the potential images are identified. For each identified person, person-specific data comprising metrics of social connectivity to the requester are retrieved from a plurality of applications such as communications applications, social networking applications, calendar applications, and collaborative applications. An ordered list of persons is then generated by ranking the identified persons in accordance with at least metrics of visual similarity between the respective facial image and the potential image matches and with the social connection metrics.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for adjusting facial image search results. In one aspect, a method includes detecting, in images that are responsive to a query, faces in the images and building facial templates. The images are then clustered according to similarity measures of the facial templates. The cluster with the highest quantity is selected, and each image in the set of returned images is compared to each image in the selected cluster. The similarity of each face with the faces in the largest cluster, based on the facial templates, is determined as an inlier score for the image in which the face is depicted. The system then increases the rank of images with high inlier scores and decreases the rank of images with low inlier scores.

Abstract: A method and apparatus for enabling dynamic product and vendor identification and the display of relevant purchase information are described herein. According to embodiments of the invention, a recognition process is executed on sensor data captured via a mobile computing device to identify one or more items, and to identify at least one product associated with the one or more items. Product and vendor information for the at least one product is retrieved and displayed via the mobile computing device. In the event a user gesture is detected in response to displaying the product and vendor information data, processing logic may submit a purchase order for the product (e.g., for an online vendor) or contact the vendor (e.g., for an in-store vendor).

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for selecting training images. One of the methods includes determining, for each of a plurality of labels that each designate a respective food class of a plurality of food classes, a respective measure of importance. A respective sample size is determined for the label based on the respective measure of importance of the label. A number of training images are selected for each respective label according to the determined sample size for the label. A predictive model is trained using the selected training images as training data.

Abstract: A facial recognition search system identifies one or more likely names (or other personal identifiers) corresponding to the facial image(s) in a query as follows. After receiving the visual query with one or more facial images, the system identifies images that potentially match the respective facial image in accordance with visual similarity criteria. Then one or more persons associated with the potential images are identified. For each identified person, person-specific data comprising metrics of social connectivity to the requester are retrieved from a plurality of applications such as communications applications, social networking applications, calendar applications, and collaborative applications. An ordered list of persons is then generated by ranking the identified persons in accordance with at least metrics of visual similarity between the respective facial image and the potential image matches and with the social connection metrics.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing queries made up of images. In one aspect, a method includes indexing images by image descriptors. The method further includes associating descriptive n-grams with the images. In another aspect, a method includes receiving a query, identifying text describing the query, and performing a search according to the text identified for the query.

Abstract: Systems and methods for generating an augmented reality interface for generics activities are disclosed. The systems and methods may be directed to creating an augmented reality display for an activity performed on a surface. Given an image of the activity, an activity solver library and associated configuration information for the activity may be selected. The surface of the activity from the image may be rectified, forming a rectified image, from which activity state information may be extracted using the configuration information. The activity state information may be provided to the activity solver library to generate solution information, and elements indicating the solution information may be rendered in a perspective of the original image. By providing the configuration information associated with an activity solver library, an augmented reality interface can be generated for an activity by capturing an image of the activity.

Abstract: Techniques for providing image search templates are provided. An image search template may be associated with an image search query to aid the user in capturing an image that will be appropriate for processing the search query. The template may be displayed as an overlay during an image capturing process to indicate an appropriate image capturing pose, range, angle, or other view characteristics that may provide more accurate search results. The template may also be used in the image search query to segment the image and identify features relevant to the search query. Images in an image database may be clustered using characteristics of the images or metadata associated with the images in order to establish groups of images from which templates may be derived. The generated templates may be provided to users to assist in capturing images to be used as search engine queries.

Abstract: A system and computer-implemented method for associating images with semantic entities and providing search results using the semantic entities. An image database contains one or more source images associated with one or more images labels. A computer may generate one or more documents containing the labels associated with each image. Analysis may be performed on the one or more documents to associate the source images with semantic entities. The semantic entities may be used to provide search results. In response to receiving a target image as a search query, the target image may be compared with the source images to identify similar images. The semantic entities associated with the similar images may be used to determine a semantic entity for the target image. The semantic entity for the target image may be used to provide search results in response to the search initiated by the target image.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for adjusting facial image search results. In one aspect, a method includes detecting, in images that are responsive to a query, faces in the images and building facial templates. The images are then clustered according to similarity measures of the facial templates. The cluster with the highest quantity is selected, and each image in the set of returned images is compared to each image in the selected cluster. The similarity of each face with the faces in the largest cluster, based on the facial templates, is determined as an inlier score for the image in which the face is depicted. The system then increases the rank of images with high inlier scores and decreases the rank of images with low inlier scores.

Abstract: A method and apparatus for enabling media data capture based on a real-world user experience is described herein. According to embodiments of the invention, media data is captured by a mobile computing device and transmitted to a server computer system in response to identifying one or more events from sensor data. The server computer system adds the captured media data, along with data identifying the event that initiated the capture of the media data, to a history of real-world experiences of a user of the mobile computing device. The server computer system may further perform one or more recognition processes on the captured media data and, when the one or more recognition processes find a match, associate the recognized items to the captured media data.

Abstract: Systems and methods for modeling the occurrence of common image components (e.g., sub-regions) in order to improve visual object recognition are disclosed. In one example, a query image may be matched to a training image of an object. A matched region within the training image to which the query image matches may be determined and a determination may be made whether the matched region is located within an annotated image component of the training image. When the matched region matches only to the image component, an annotation associated with the component may be identified. In another example, sub-regions within a plurality of training image corpora may be annotated as common image components including associated information (e.g., metadata). Matching sub-regions appearing in many training images of objects may be down-weighted in the matching process to reduce possible false matches to query images including common image components.

Abstract: Implementations of the present disclosure include actions of receiving image data, the image data being provided from a camera and corresponding to a scene viewed by the camera, receiving one or more annotations, the one or more annotations being provided based on one or more entities determined from the scene, each annotation being associated with at least one entity, determining one or more actions based on the one or more annotations, and providing instructions to display an action interface including one or more action elements, each action element being selectable to induce execution of a respective action, the action interface being displayed in a viewfinder.

Abstract: Embodiments of this invention relate to detecting and blurring images. In an embodiment, a system detects objects in a photographic image. The system includes an object detector module configured to detect regions of the photographic image that include objects of a particular type at least based on the content of the photographic image. The system further includes a false positive detector module configured to determine whether each region detected by the object detector module includes an object of the particular type at least based on information about the context in which the photographic image was taken.

Abstract: Methods, systems, and apparatus for scoring images related to entities. In one aspect, a method includes identifying images associated with a person, each image being included in one or more resources; obtaining, for each resource that includes one of the images, a quality score that represents a quality of the resource; for each of the images: generating an image resource quality score from the quality scores of the resources that include the image; identifying a set of similar images from the images, each similar image having a measure of similarity to the image that meets a similarity measure threshold; generating an image score based on image resource quality scores of the resources that include the similar images relative to image resource quality scores of the resources that include each of the images; and generating an image authority score based on the image resource quality score and the image score.

Abstract: Methods and systems for automatic detection of landmarks in digital images and annotation of those images are disclosed. A method for detecting and annotating landmarks in digital images includes the steps of automatically assigning a tag descriptive of a landmark to one or more images in a plurality of text-associated digital images to generate a set of landmark-tagged images, learning an appearance model for the landmark from the set of landmark-tagged images, and detecting the landmark in a new digital image using the appearance model. The method can also include a step of annotating the new image with the tag descriptive of the landmark.

Abstract: In one embodiment the present invention is a method for populating and updating a database of images of landmarks including geo-clustering geo-tagged images according to geographic proximity to generate one or more geo-clusters, and visual-clustering the one or more geo-clusters according to image similarity to generate one or more visual clusters. In another embodiment, the present invention is a system for identifying landmarks from digital images, including the following components: a database of geo-tagged images; a landmark database; a geo-clustering module; and a visual clustering module. In other embodiments the present invention may be a method of enhancing user queries to retrieve images of landmarks, or a method of automatically tagging a new digital image with text labels.

Abstract: Techniques for providing image search templates are provided. An image search template may be associated with an image search query to aid the user in capturing an image that will be appropriate for processing the search query. The template may be displayed as an overlay during an image capturing process to indicate an appropriate image capturing pose, range, angle, or other view characteristics that may provide more accurate search results. The template may also be used in the image search query to segment the image and identify features relevant to the search query. Images in an image database may be clustered using characteristics of the images or metadata associated with the images in order to establish groups of images from which templates may be derived. The generated templates may be provided to users to assist in capturing images to be used as search engine queries.