Abstract: A system and methods use music features extracted from music to detect a music mood within a hierarchical mood detection framework. A two-dimensional mood model divides music into four moods which include contentment, depression, exuberance, and anxious/frantic. A mood detection algorithm uses a hierarchical mood detection framework to determine which of the four moods is associated with a music clip based on the extracted features. In a first tier of the hierarchical detection process, the algorithm determines one of two mood groups to which the music clip belongs. In a second tier of the hierarchical detection process, the algorithm then determines which mood from within the selected mood group is the appropriate, exact mood for the music clip. Benefits of the mood detection system include automatic detection of music mood which can be used as music metadata to manage music through music representation and classification.

Abstract: Improvements are provided to effectively assess a user's face and head pose such that a computer or like device can track the user's attention towards a display device(s). Then the region of the display or graphical user interface that the user is turned towards can be automatically selected without requiring the user to provide further inputs. A frontal face detector is applied to detect the user's frontal face and then key facial points such as left/right eye center, left/right mouth corner, nose tip, etc., are detected by component detectors. The system then tracks the user's head by an image tracker and determines yaw, tilt and roll angle and other pose information of the user's head through a coarse to fine process according to key facial points and/or confidence outputs by pose estimator.

Abstract: A process for comparing two digital images is described. The process includes comparing texture moment data for the two images to provide a similarity index, combining the similarity index with other data to provide a similarity value and determining that the two images match when the similarity value exceeds a first threshold value.

Abstract: A multimedia object retrieval and annotation system integrates an annotation process with object retrieval and relevance feedback processes. The annotation process annotates multimedia objects, such as digital images, with semantically relevant keywords. The annotation process is performed in background, hidden from the user, as the user conducts normal searches. The annotation process is “semi-automatic” in that it utilizes both keyword-based information retrieval and content-based image retrieval techniques to automatically search for multimedia objects, and then encourages users to provide feedback on the retrieved objects. The user identifies objects as either relevant or irrelevant to the query keywords and based on this feedback, the system automatically annotates the objects with semantically relevant keywords and/or updates associations between the keywords and objects. As the retrieval-feedback-annotation cycle is repeated, the annotation coverage and accuracy of future searches continues to improve.

Abstract: A “music video parser” automatically detects and segments music videos in a combined audio-video media stream. Automatic detection and segmentation is achieved by integrating shot boundary detection, video text detection and audio analysis to automatically detect temporal boundaries of each music video in the media stream. In one embodiment, song identification information, such as, for example, a song name, artist name, album name, etc., is automatically extracted from the media stream using video optical character recognition (OCR). This information is then used in alternate embodiments for cataloging, indexing and selecting particular music videos, and in maintaining statistics such as the times particular music videos were played, and the number of times each music video was played.

Abstract: Methods and apparatuses are provided for detecting blur within digital images using Cepstrum analysis blur detection techniques that are able to detect motion blur and/or out-of-focus blur.

Abstract: A robust camera motion analysis method is described. In an implementation, a method includes analyzing video having sequential frames to determine one or more camera motions that occurred when sequential frames of the video were captured. The one or more camera motions for each frame are described by a set of displacement curves, a mean absolute difference (MAD) curve, and a major motion (MAJ) curve. The set of displacement curves describe the one or more camera motions in respective horizontal (H), vertical (V), and radial (R) directions. The MAD curve relates a minimum MAD value from the set of displacement curves. The MAJ curve is generated from the minimum MAD value and provides one or more qualitative descriptions that describe the one or more camera motions as at least one of still, vertical, horizontal and radial.

Abstract: Systems and methods for representing sequential motion patterns are described. In one aspect, video frames are converted into a sequence of energy redistribution measurements. One or more motion filters are then applied to the ER measurements to generate one or more temporal sequences of motion patterns, the number of temporal sequences being a function of the number of motion filters.

Abstract: Systems and methods for adapting images for substantially optimal presentation by heterogeneous client display sizes are described. In one aspect, an image is modeled with respect to multiple visual attentions to generate respective attention objects for each of the visual attentions. For each of one or more image adaptation schemes, an objective measure of information fidelity (IF) is determined for a region R of the image. The objective measures are determined as a function of a resource constraint of the display device and as a function of a weighted sum of IF of each attention object in the region R. A substantially optimal adaptation scheme is then selected as a function of the calculated objective measures. The image is then adapted via the selected substantially optimal adaptation scheme to generate an adapted image as a function of at least the target area of the client display.

Abstract: A system and process for video characterization that facilitates video classification and retrieval, as well as motion detection, applications. This involves characterizing a video sequence with a gray scale image having pixel levels that reflect the intensity of motion associated with a corresponding region in the sequence of video frames. The intensity of motion is defined using any of three characterizing processes. Namely, a perceived motion energy spectrum (PMES) characterizing process that represents object-based motion intensity over the sequence of frames, a spatio-temporal entropy (STE) characterizing process that represents the intensity of motion based on color variation at each pixel location, a motion vector angle entropy (MVAE) characterizing process which represents the intensity of motion based on the variation of motion vector angles.

Abstract: Systems and methods for shape registration are described. In one aspect, training shape vectors are generated from images in an image database. The training shape vectors identify landmark points associated with one or more object types. A distribution of shape in the training shape vectors is represented as a prior of tangent shape in tangent shape space. The prior of tangent shape is then incorporated into a unified Bayesian framework for shape registration.

Abstract: An exemplary system includes a browser to browse a web page based on a web page definition having a slicing tree defining an arrangement of rectangular regions in the web page. The web page definition can include parametric data describing adaptability parameters associated with a rectangular region. A rendering module renders an adapted web page based on the web page definition, and a proxy module generates an intermediary adapted web page definition. A method includes rendering the web page according to a slicing tree and block property data in an associated web page definition. The method may include determining a set of unsummarized blocks that maximize information fidelity.

Abstract: The described arrangements and procedures provide an intelligent media agent to autonomously collect semantic multimedia data text descriptions on behalf of a user whenever and wherever the user accesses media content. The media agent analyzes these semantic multimedia data text descriptions in view of user behavior patterns and actions to assist the user in identifying multimedia content and related information that is appropriate to the context within which the user is operating or working. For instance, the media agent detects insertion of text and analyzes the inserted text. Based on the analysis, the agent predicts whether a user intends to access media content. If so, the agent retrieves information corresponding to media content from a media content source and presents the information to a user as a suggestion.

Abstract: Systems and methods for annotating a face in a digital image are described. In one aspect, a probability model is trained by mapping one or more sets of sample facial features to corresponding names of individuals. A face from an input data set of at least one the digital image is then detected. Facial features are then automatically extracted from the detected face. A similarity measure is them modeled as a posterior probability that the facial features match a particular set of features identified in the probability model. The similarity measure is statistically learned. A name is then inferred as a function of the similarity measure. The face is then annotated with the name.

Abstract: Systems and methods to generate an attention model for computational analysis of video data are described. In one aspect, feature components from a video data sequence are extracted. Attention data is generated by applying multiple attention models to the extracted feature components. The generated attention data is integrated into a comprehensive user attention model for the computational analysis of the video data sequence.

Abstract: A video skim is assembled by identifying one or more key frames from a video shot. Certain lengths of frames to the left and right of the key frame are measured for visual content variety. Depending upon the measured visual content variety to the left and right of the key frame, the video skim is assembled that has L frames to the left of the key frame and R frames to the right of the key frame. Measuring the visual content variety to the left and right of the key frame, provides a video skim that incorporates the more salient features of a shot.

Abstract: Systems and methods for adapting images for substantially optimal presentation by heterogeneous client display sizes are described. In one aspect, an image is modeled with respect to multiple visual attentions to generate respective attention objects for each of the visual attentions. For each of one or more image adaptation schemes, an objective measure of information fidelity (IF) is determined for a region R of the image. The objective measures are determined as a function of a resource constraint of the display device and as a function of a weighted sum of IF of each attention object in the region R. A substantially optimal adaptation scheme is then selected as a function of the calculated objective measures. The image is then adapted via the selected substantially optimal adaptation scheme to generate an adapted image as a function of at least the target area of the client display.

Abstract: Methods and apparatuses are provided for detecting blur within digital images using wavelet transform and/or Cepstrum analysis blur detection techniques that are able to detect motion blur and/or out-of-focus blur.

Abstract: A portion of an audio signal is separated into multiple frames from which one or more different features are extracted. These different features are used, in combination with a set of rules, to classify the portion of the audio signal into one of multiple different classifications (for example, speech, non-speech, music, environment sound, silence, etc.). In one embodiment, these different features include one or more of line spectrum pairs (LSPs), a noise frame ratio, periodicity of particular bands, spectrum flux features, and energy distribution in one or more of the bands. The line spectrum pairs are also optionally used to segment the audio signal, identifying audio classification changes as well as speaker changes when the audio signal is speech.

Abstract: Systems and methods for indexing and retrieving images are described herein. The systems and methods analyze an image to determine its texture moments. The pixels of the image are converted to gray scale. Textural attributes of the pixels are determined. The textural attributes are associated with the local texture of the pixels and are derived from coefficients of Discrete Fourier Transform associated with the pixels. Statistical values associated with the textural attributes of the pixels are calculated. The texture moments of the image are determined from the statistical value.