Abstract: Computing a scale factor to insert a first set of shapes into a second set of shapes to form a combined image includes receiving the two sets of shapes, using a processor to convert the first set of shapes into a set of rectangles and the second set of shapes into a set of intervals and computing the scale factor for either the set of intervals or the set of rectangles to generate the combined image by iteratively inserting the set of rectangles into the set of intervals and updating the scale factor in response to a residual area or an overflow area until all the rectangles in the set of rectangles have been inserted into the set of intervals and the residual area in the set of intervals is below a threshold, and storing the combined image in memory.

Abstract: Generating a tag layout from a set of tags and an ordering of the set of tags, wherein each tag includes a text label and a size for the text label, is disclosed. The method further includes receiving at least one closed shape corresponding to a space for the tag layout. A processor computes a scale factor for at least one of the closed shape or the size of the text labels in the set of tags to generate the tag layout of the set of tags within the closed shape such that all the tags in the set of tags fit within the closed shape and the tags are placed in the space based at least upon the ordering of the tags in the set of tags.

Abstract: A method for determining a semantic concept associated with an audio signal captured using an audio sensor. A data processor is used to automatically analyze the audio signal using a plurality of semantic concept detectors to determine corresponding preliminary semantic concept detection values, each semantic concept detector being adapted to detect a particular semantic concept. The preliminary semantic concept detection values are analyzed using a joint likelihood model based on predetermined pair-wise likelihoods that particular pairs of semantic concepts co-occur to determine updated semantic concept detection values. One or more semantic concepts are determined based on the updated semantic concept detection values. The semantic concept detectors and the joint likelihood model are trained together with a joint training process using training audio signals, at least some of which are known to be associated with a plurality of semantic concepts.

Abstract: A method for controlling a device responsive to an audio signal captured using an audio sensor. A data processor is used to automatically analyze the audio signal using a plurality of semantic concept detectors to determine corresponding preliminary semantic concept detection values, each semantic concept detector being adapted to detect a particular semantic concept. The preliminary semantic concept detection values are analyzed using a joint likelihood model based on predetermined pair-wise likelihoods that particular pairs of semantic concepts co-occur to determine updated semantic concept detection values. One or more semantic concepts are determined based on the updated semantic concept detection values, and the device is controlled responsive to identified semantic concepts. The semantic concept detectors and the joint likelihood model are trained together with a joint training process using training audio signals, at least some of which are known to be associated with a plurality of semantic concepts.

Abstract: A method of automatically classifying images in a consumer digital image collection, includes generating an event representation of the image collection; computing global time-based features for each event within the hierarchical event representation; computing content-based features for each image in an event within the hierarchical event representation; combining content-based features for each image in an event to generate event-level content-based features; and using time-based features and content-based features for each event to classify an event into one of a pre-determined set of semantic categories.

Abstract: A method for identifying a set of key video frames from a video sequence comprising extracting feature vectors for each video frame and applying a group sparsity algorithm to represent the feature vector for a particular video frame as a group sparse combination of the feature vectors for the other video frames. Weighting coefficients associated with the group sparse combination are analyzed to determine video frame clusters of temporally-contiguous, similar video frames. A summary is formed based on the determined video frame clusters.

Abstract: A method for identifying a set of key video frames from a video sequence comprising extracting feature vectors for each video frame and applying a group sparsity algorithm to represent the feature vector for a particular video frame as a group sparse combination of the feature vectors for the other video frames. Weighting coefficients associated with the group sparse combination are analyzed to determine video frame clusters of temporally-contiguous, similar video frames. A set of key video frames are selected based on the determined video frame clusters.

Abstract: A method for identifying a set of key video frames from a video sequence comprising extracting feature vectors for each video frame and applying a group sparsity algorithm to represent the feature vector for a particular video frame as a group sparse combination of the feature vectors for the other video frames. Weighting coefficients associated with the group sparse combination are analyzed to determine video frame clusters of temporally-contiguous, similar video frames. The video sequence is segmented into scenes by identifying scene boundaries based on the determined video frame clusters.

Abstract: A method of detecting recurring events in a digital image collection taken over a pre-determined period of time is disclosed. The method uses a processor for analyzing the digital image collection to produce a two-dimensional representation of the distribution of image capture activity over time and detecting recurring events by identifying spatial clusters in the two-dimensional representation.

Abstract: A method of identifying groups of related digital images in a digital image collection, comprising: analyzing each of the digital images to generate associated feature descriptors related to image content or image capture conditions; storing the feature descriptors associated with the digital images in a metadata database; automatically analyzing the metadata database to identify a plurality of frequent itemsets, wherein each of the frequent itemsets is a co-occurring feature descriptor group that occurs in at least a predefined fraction of the digital images; determining a probability of occurrence for each the identified frequent itemsets; determining a quality score for each of the identified frequent itemsets responsive to the determined probability of occurrence; ranking the frequent itemsets based at least on the determined quality scores; and identifying one or more groups of related digital images corresponding to one or more of the top ranked frequent itemsets.

Abstract: A method of automatically classifying images in a consumer digital image collection, includes generating an event representation of the image collection; computing global time-based features for each event within the hierarchical event representation; computing content-based features for each image in an event within the hierarchical event representation; combining content-based features for each image in an event to generate event-level content-based features; and using time-based features and content-based features for each event to classify an event into one of a pre-determined set of semantic categories.

Abstract: A method of identifying one or more particular images from an image collection, includes indexing the image collection to provide image descriptors for each image in the image collection such that each image is described by one or more of the image descriptors; receiving a query from a user specifying at least one keyword for an image search; and using the keyword(s) to search a second collection of tagged images to identify co-occurrence keywords. The method further includes using the identified co-occurrence keywords to provide an expanded list of keywords; using the expanded list of keywords to search the image descriptors to identify a set of candidate images satisfying the keywords; grouping the set of candidate images according to at least one of the image descriptors, and selecting one or more representative images from each grouping; and displaying the representative images to the user.

Abstract: A method for identifying digital images having matching backgrounds from a collection of digital images, comprising using a processor to perform the steps of: determining a set of one or more feature values for each digital image in the collection of digital images, wherein the set of feature values includes an edge compactness feature value that is an indication of the number of objects in the digital image that are useful for scene matching; determining a subset of the collection of digital images that are good candidates for scene matching by applying a classifier responsive to the determined feature values; applying a scene matching algorithm to the subset of the collection of digital images to identify groups of digital images having matching backgrounds; and storing an indication of the identified groups of digital images having matching backgrounds in a processor-accessible memory.

Abstract: A method for determining a semantic concept classification for a digital video clip based on a grouplet dictionary that includes a plurality of temporally-correlated grouplets. The temporally-correlated grouplets include textual codewords and either visual codewords or audio codewords, wherein the codewords in a particular temporally-correlated grouplet were determined to be correlated with each other based on analysis of a set of training videos. Reference video codeword similarity scores are determined for a set of reference video clips, and codeword similarity scores are determined for the digital video clip. A reference video similarity score is determined for each reference video clip representing a similarity between the digital video clip and the reference video clip based on the reference video codeword similarity scores, the codeword similarity scores, and the temporally-correlated grouplets.

Abstract: A method for representing a video sequence including a time sequence of input video frames, the input video frames including some common scene content that is common to all of the input video frames and some dynamic scene content that changes between at least some of the input video frames. Affine transform are determined to align the common scene content in the input video frames. A common video frame including the common scene content is determined by forming a sparse combination of a first basis functions. A dynamic video frame is determined for each input video frame by forming a sparse combination of a second basis functions, wherein the dynamic video frames can be combined with the respective affine transforms and the common video frame to provide reconstructed video frames.

Abstract: A method for determining a scene boundary location dividing a first scene and a second scene in an input video sequence. The scene boundary location is determined responsive to a merit function value, which is a function of the candidate scene boundary location. The merit function value for a particular candidate scene boundary location is determined by representing the dynamic scene content for the input video frames before and after candidate scene boundary using sparse combinations of a set of basis functions, wherein the sparse combinations of the basis functions are determined by finding a sparse vector of weighting coefficients for each of the basis functions. The weighting coefficients determined for each of the input video frames are combined to determine the merit function value. The candidate scene boundary providing the smallest merit function value is designated to be the scene boundary location.

Abstract: A method for determining a semantic concept classification for a digital video clip, comprising: receiving an audio-visual dictionary including a plurality of audio-visual grouplets, the audio-visual grouplets including visual background and foreground codewords, audio background and foreground codewords, wherein the codewords in a particular audio-visual grouplet were determined to be correlated with each other; analyzing the digital video clip to determine a set of visual features and a set of audio features; determining similarity scores between the digital video clip and each of the audio-visual grouplets by comparing the set of visual features to any visual background and foreground codewords associated with a particular audio-visual grouplet, and comparing the set of audio features to any audio background and foreground codewords associated with the particular audio-visual grouplet; and determining one or more semantic concept classifications using trained semantic classifiers.

Abstract: A method for determining a semantic concept classification for a digital video clip, comprising: receiving an audio-visual dictionary including a plurality of audio-visual grouplets, the audio-visual grouplets including visual background and foreground codewords, audio background and foreground codewords, wherein the codewords in a particular audio-visual grouplet were determined to be correlated with each other; determining reference video codeword similarity scores for a set of reference video clips; determining codeword similarity scores for the digital video clip; determining a reference video similarity score for each reference video clip representing a similarity between the digital video clip and the reference video clip responsive to the audio-visual grouplets, the codeword similarity scores and the reference video codeword similarity scores; and determining one or more semantic concept classifications using trained semantic classifiers responsive to the determined reference video similarity scores.

Abstract: A method for identifying high saliency regions in a digital image, comprising: segmenting the digital image into a plurality of segmented regions; determining a saliency value for each segmented region, merging neighboring segmented regions that share a common boundary in response to determining that one or more specified merging criteria are satisfied; and designating one or more of the segmented regions to be high saliency regions. The determination of the saliency value for a segmented region includes: determining a surround region including a set of image pixels surrounding the segmented region; analyzing the image pixels in the segmented region to determine one or more segmented region attributes; analyzing the image pixels in the surround region to determine one or more corresponding surround region attributes; determining a region saliency value responsive to differences between the one or more segmented region attributes and the corresponding surround region attributes.

Abstract: A method of identifying groups of related digital images in a digital image collection, comprising: analyzing each of the digital images to generate associated feature descriptors related to image content or image capture conditions; storing the feature descriptors associated with the digital images in a metadata database; automatically analyzing the metadata database to identify a plurality of frequent itemsets, wherein each of the frequent itemsets is a co-occurring feature descriptor group that occurs in at least a predefined fraction of the digital images; determining a probability of occurrence for each the identified frequent itemsets; determining a quality score for each of the identified frequent itemsets responsive to the determined probability of occurrence; ranking the frequent itemsets based at least on the determined quality scores; and identifying one or more groups of related digital images corresponding to one or more of the top ranked frequent itemsets.