Abstract: A method includes identifying a named entity, retrieving images associated with the named entity, and using a face detection algorithm to perform face detection on the retrieved images to detect faces in the retrieved images. At least one representative face image from the retrieved images is identified, and the representative face image is used to identify one or more additional images representing the at least one named entity.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for classification using a neural network. One of the methods for processing an input through each of multiple layers of a neural network to generate an output, wherein each of the multiple layers of the neural network includes a respective multiple nodes includes for a particular layer of the multiple layers: receiving, by a classification system, an activation vector as input for the particular layer, selecting one or more nodes in the particular layer using the activation vector and a hash table that maps numeric values to nodes in the particular layer, and processing the activation vector using the selected nodes to generate an output for the particular layer.

Abstract: A motion manifold system analyzes a set of videos, identifying image patches within those videos corresponding to regions of interest and identifying patch trajectories by tracking the movement of the regions over time in the videos. Based on the patch identification and tracking, the system produces a motion manifold data structure that captures the way in which the same semantic region can have different visual representations over time. The motion manifold can then be applied to determine the semantic similarity between different patches, or between higher-level constructs such as images or video segments, including detecting semantic similarity between patches or other constructs that are visually dissimilar.

Abstract: A system identifies a set of initial segments of a time-based data item, such as audio. The segments can be defined at regular time intervals within the time-based data item. The initial segments are short segments. The system computes a short-timescale vectorial representation for each initial segment and compares the short-timescale vectorial representation for each initial segment with other short-timescale vectorial representations of the segments in a time duration within the time-based data item (e.g., audio) immediately preceding or immediately following the initial segment. The system generates a representation of long-timescale information for the time-based data item based on a comparison of the short-timescale vectorial representations of the initial segments and the short-timescale vectorial representations of immediate segments. The representation of long-timescale information identifies an underlying repetition structure of the time-based data item, such as rhythm or phrasing in an audio item.

Abstract: A video demographics analysis system selects a training set of videos to use to correlate viewer demographics and video content data. The video demographics analysis system extracts demographic data from viewer profiles related to videos in the training set and creates a set of demographic distributions, and also extracts video data from videos in the training set. The video demographics analysis system correlates the viewer demographics with the video data of videos viewed by that viewer. Using the prediction model produced by the machine learning process, a new video about which there is no a priori knowledge can be associated with a predicted demographic distribution specifying probabilities of the video appealing to different types of people within a given demographic category, such as people of different ages within an age demographic category.

Abstract: Techniques are disclosed for producing a collaborative recording of an audio event. An online server or service identifies participating mobile devices with recording capabilities that are available for recording at least a portion of the audio event. The online server or service determines the locations of the potential participating mobile devices, and identifies ranges of frequencies to be recorded by each of the participating mobile devices. The individual recordings are then compiled into a final collaborative recording.

Abstract: A volume identification system identifies a set of unlabeled spatio-temporal volumes within each of a set of videos, each volume representing a distinct object or action. The volume identification system further determines, for each of the videos, a set of volume-level features characterizing the volume as a whole. In one embodiment, the features are based on a codebook and describe the temporal and spatial relationships of different codebook entries of the volume. The volume identification system uses the volume-level features, in conjunction with existing labels assigned to the videos as a whole, to label with high confidence some subset of the identified volumes, e.g., by employing consistency learning or training and application of weak volume classifiers. The labeled volumes may be used for a number of applications, such as training strong volume classifiers, improving video search (including locating individual volumes), and creating composite videos based on identified volumes.

Abstract: Sound representations and winner-take-all codes of auditory spectra are used in the identification of audio content. A transformation component converts a set of sound frames from audio content into a set of spectral slices. A spectral encoder component encodes the spectral slices of auditory spectra into winner-take-all codes with a winner-take-all hash function. An identification component identifies which spectral dimension of a subset of spectral dimensions within a spectral slice has highest spectral value according to the winner-take-all codes. Reference audio content is determined to be similar or matching to the audio content based on the winner-take-all codes.

Abstract: This disclosure relates to transformation invariant media matching. A fingerprinting component can generate a transformation invariant identifier for media content by adaptively encoding the relative ordering of signal markers in media content. The signal markers can be adaptively encoded via reference point geometry, or ratio histograms. An identification component compares the identifier against a set of identifiers for known media content, and the media content can be matched or identified as a function of the comparison.

Abstract: A method and system generates and compares fingerprints for videos in a video library. The video fingerprints provide a compact representation of the temporal locations of discontinuities in the video that can be used to quickly and efficiently identify video content. Discontinuities can be, for example, shot boundaries in the video frame sequence or silent points in the audio stream. Because the fingerprints are based on structural discontinuity characteristics rather than exact bit sequences, visual content of videos can be effectively compared even when there are small differences between the videos in compression factors, source resolutions, start and stop times, frame rates, and so on. Comparison of video fingerprints can be used, for example, to search for and remove copyright protected videos from a video library. Furthermore, duplicate videos can be detected and discarded in order to preserve storage space.

Abstract: A volume identification system identifies a set of unlabeled spatio-temporal volumes within each of a set of videos, each volume representing a distinct object or action. The volume identification system further determines, for each of the videos, a set of volume-level features characterizing the volume as a whole. In one embodiment, the features are based on a codebook and describe the temporal and spatial relationships of different codebook entries of the volume. The volume identification system uses the volume-level features, in conjunction with existing labels assigned to the videos as a whole, to label with high confidence some subset of the identified volumes, e.g., by employing consistency learning or training and application of weak volume classifiers. The labeled volumes may be used for a number of applications, such as training strong volume classifiers, improving video search (including locating individual volumes), and creating composite videos based on identified volumes.

Abstract: A method includes identifying a named entity, retrieving images associated with the named entity, and using a face detection algorithm to perform face detection on the retrieved images to detect faces in the retrieved images. At least one representative face image from the retrieved images is identified, and the representative face image is used to identify one or more additional images representing the at least one named entity.

Abstract: The disclosed embodiments describe a method, an apparatus, an application specific integrated circuit, and a server that provides a fast and efficient look up for data analysis. The apparatus and server may be configured to obtain data segments from a plurality of input devices. The data segments may be individual unique subsets of the entire data set obtained by a plurality input devices. A hash function may be applied to an aggregated set of the data segments. A result of the hash function may be stored in a data structure. A codebook may be generated from the hash function results.

Abstract: Methods and systems for processing an image to facilitate automated object recognition are disclosed. More particularly, an image is processed based on a perceptual grouping for the image (e.g., derived via segmentation, derived via contour detection, etc.) and a geometric-configuration model for the image (e.g., a bounding box model, a constellation, a k-fan, etc.).

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for clustering images. In one aspect a system includes one or more computers configured to, for each of a plurality of digital images, associate extrinsic image-related information with each individual image, the extrinsic image-related information including text information and co-click data for the individual image, assign images from the plurality of images to one or more of the clusters of images based on the extrinsic information associated with each of the plurality of images, receive in the search system a user query from a user device, identify by operation of the search system one or more clusters of images that match the query, and provide one or more cluster results, where each cluster result provides information about an identified cluster.

Abstract: A demographics analysis trains classifier models for predicting demographic attribute values of videos and users not already having known demographics. In one embodiment, the demographics analysis system trains classifier models for predicting demographics of videos using video features such as demographics of video uploaders, textual metadata, and/or audiovisual content of videos. In one embodiment, the demographics analysis system trains classifier models for predicting demographics of users (e.g., anonymous users) using user features based on prior video viewing periods of users. For example, viewing-period based user features can include individual viewing period statistics such as total videos viewed. Further, the viewing-period based features can include distributions of values over the viewing period, such as distributions in demographic attribute values of video uploaders, and/or distributions of viewings over hours of the day, days of the week, and the like.

Abstract: A volume identification system identifies a set of unlabeled spatio-temporal volumes within each of a set of videos, each volume representing a distinct object or action. The volume identification system further determines, for each of the videos, a set of volume-level features characterizing the volume as a whole. In one embodiment, the features are based on a codebook and describe the temporal and spatial relationships of different codebook entries of the volume. The volume identification system uses the volume-level features, in conjunction with existing labels assigned to the videos as a whole, to label with high confidence some subset of the identified volumes, e.g., by employing consistency learning or training and application of weak volume classifiers. The labeled volumes may be used for a number of applications, such as training strong volume classifiers, improving video search (including locating individual volumes), and creating composite videos based on identified volumes.

Abstract: This disclosure relates to filter based object detection using hash functions. A hashing component can compute respective hash values for a set of object windows that are associated with an image to be scanned. The hashing component can employ various hash functions in connection with computing the hash values, such as a winner takes all (WTA) hash function. A filter selection component can compare the respective hash values of the object windows against a hash table of object filters, and can select one or more object filters for recognizing or localizing at least one of an object within the image as a function of the comparison.

Abstract: Described herein are methods and system for analyzing music audio. An example method includes obtaining a music audio track, calculating acoustic features of the music audio track, calculating geometric features of the music audio track in view of the acoustic features, and determining a mood of the music audio track in view of the geometric features.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training scoring models. One method includes storing data identifying a plurality of positive and a plurality of negative training images for a query. The method further includes selecting a first image from either the positive group of images or the negative group of images, and applying a scoring model to the first image. The method further includes selecting a plurality of candidate images from the other group of images, applying the scoring model to each of the candidate images, and then selecting a second image from the candidate images according to scores for the images. The method further includes determining that the scores for the first image and the second image fail to satisfy a criterion, updating the scoring model, and storing the updated scoring model.