Abstract: A system that analyzes music to detect musical beats and to rectify beats that are out of sync with the actual beat phase of the music. The music analysis includes onset detection, tempo/meter estimation, and beat analysis, which includes the rectification of out-of-sync beats.

Abstract: Systems and methods for smart media content thumbnail extraction are described. In one aspect program metadata is generated from recorded video content. The program metadata includes one or more key-frames from one or more corresponding shots. An objectively representative key-frame is identified from among the key-frames as a function of shot duration and frequency of appearance of key-frame content across multiple shots. The objectively representative key-frame is an image frame representative of the recorded video content. A thumbnail is created from the objectively representative key-frame.

Abstract: A system and method that includes a virtual human face generation technique which synthesizes images of a human face at a variety of poses. This is preferably accomplished using just a frontal and profile image of a specific subject. An automatic deformation technique is used to align the features of a generic 3-D graphic face model with the corresponding features of these pre-provided images of the subject. Specifically, a generic frontal face model is aligned with the frontal image and a generic profile face model is aligned with the profile image. The deformation procedure results in a single 3-D face model of the specific human face. It precisely reflects the geometric features of the specific subject. After that, subdivision spline surface construction and multi-direction texture mapping techniques are used to smooth the model and endow photometric detail to the specific 3-D geometric face model.

Abstract: A multimedia object retrieval and annotation system integrates an annotation process with object retrieval and relevance feedback processes. The annotation process annotates multimedia objects, such as digital images, with semantically relevant keywords. The annotation process is performed in background, hidden from the user, as the user conducts normal searches. The annotation process is “semi-automatic” in that it utilizes both keyword-based information retrieval and content-based image retrieval techniques to automatically search for multimedia objects, and then encourages users to provide feedback on the retrieved objects. The user identifies objects as either relevant or irrelevant to the query keywords and based on this feedback, the system automatically annotates the objects with semantically relevant keywords and/or updates associations between the keywords and objects. As the retrieval-feedback-annotation cycle is repeated, the annotation coverage and accuracy of future searches continues to improve.

Abstract: A system and process for video characterization that facilitates video classification and retrieval, as well as motion detection, applications. This involves characterizing a video sequence with a gray scale image having pixel levels that reflect the intensity of motion associated with a corresponding region in the sequence of video frames. The intensity of motion is defined using any of three characterizing processes. Namely, a perceived motion energy spectrum (PMES) characterizing process that represents object-based motion intensity over the sequence of frames, a spatio-temporal entropy (STE) characterizing process that represents the intensity of motion based on color variation at each pixel location, a motion vector angle entropy (MVAE) characterizing process which represents the intensity of motion based on the variation of motion vector angles.

Abstract: A system and methods analyze music to detect musical beats and to rectify beats that are out of sync with the actual beat phase of the music. The music analysis includes onset detection, tempo/meter estimation, and beat analysis, which includes the rectification of out-of-sync beats.

Abstract: A system and methods use music features extracted from music to detect a music mood within a hierarchical mood detection framework. A two-dimensional mood model divides music into four moods which include contentment, depression, exuberance, and anxious/frantic. A mood detection algorithm uses a hierarchical mood detection framework to determine which of the four moods is associated with a music clip based on the extracted features. In a first tier of the hierarchical detection process, the algorithm determines one of two mood groups to which the music clip belongs. In a second tier of the hierarchical detection process, the algorithm then determines which mood from within the selected mood group is the appropriate, exact mood for the music clip. Benefits of the mood detection system include automatic detection of music mood which can be used as music metadata to manage music through music representation and classification.

Abstract: A face recognition system and process for identifying a person depicted in an input image and their face pose. This system and process entails locating and extracting face regions belonging to known people from a set of model images, and determining the face pose for each of the face regions extracted. All the extracted face regions are preprocessed by normalizing, cropping, categorizing and finally abstracting them. More specifically, the images are normalized and cropped to show only a person's face, categorized according to the face pose of the depicted person's face by assigning them to one of a series of face pose ranges, and abstracted preferably via an eigenface approach.

Abstract: The disclosed subject matter improves iterative results of content-based image retrieval (CBIR) using a bigram model to correlate relevance feedback. Specifically, multiple images are received responsive to multiple image search sessions. Relevance feedback is used to determine whether the received images are semantically relevant. A respective semantic correlation between each of at least one pair of the images is then estimated using respective bigram frequencies. The bigram frequencies are based on multiple search sessions in which each image of a pair of images is semantically relevant.

Abstract: Methods and systems for generic adaptive multimedia content delivery are described. In one embodiment, a novel framework features an abstract content model and an abstract adaptive delivery decision engine. The abstract content model recognizes important aspects of contents while hiding their physical details from other parts of the framework. The decision engine then makes content adaptation plans based on the abstracted model of the contents and needs little knowledge of any physical details of the actual contents. Thus, under the same framework, adaptive delivery of generic contents is possible.

Abstract: Methods and systems for generic adaptive multimedia content delivery are described. In one embodiment, a novel framework features an abstract content model and an abstract adaptive delivery decision engine. The abstract content model recognizes important aspects of contents while hiding their physical details from other parts of the framework. The decision engine then makes content adaptation plans based on the abstracted model of the contents and needs little knowledge of any physical details of the actual contents. Thus, under the same framework, adaptive delivery of generic contents is possible.

Abstract: A face recognition system and process for identifying a person depicted in an input image and their face pose. This system and process entails locating and extracting face regions belonging to known people from a set of model images, and determining the face pose for each of the face regions extracted. All the extracted face regions are preprocessed by normalizing, cropping, categorizing and finally abstracting them. More specifically, the images are normalized and cropped to show only a persons face, categorized according to the face pose of the depicted person's face by assigning them to one of a series of face pose ranges, and abstracted preferably via an eigenface approach.

Abstract: A system and process for video characterization that facilitates video classification and retrieval, as well as motion detection, applications. This involves characterizing a video sequence with a gray scale image having pixel levels that reflect the intensity of motion associated with a corresponding region in the sequence of video frames. The intensity of motion is defined using any of three characterizing processes. Namely, a perceived motion energy spectrum (PMES) characterizing process that represents object-based motion intensity over the sequence of frames, a spatio-temporal entropy (STE) characterizing process that represents the intensity of motion based on color variation at each pixel location, a motion vector angle entropy (MVAE) characterizing process which represents the intensity of motion based on the variation of motion vector angles.

Abstract: A face recognition system and process for identifying a person depicted in an input image and their face pose. This system and process entails locating and extracting face regions belonging to known people from a set of model images, and determining the face pose for each of the face regions extracted. All the extracted face regions are preprocessed by normalizing, cropping, categorizing and finally abstracting them. More specifically, the images are normalized and cropped to show only a persons face, categorized according to the face pose of the depicted person's face by assigning them to one of a series of face pose ranges, and abstracted preferably via an eigenface approach.

Abstract: A system and method that includes a virtual human face generation technique which synthesizes images of a human face at a variety of poses. This is preferably accomplished using just a frontal and profile image of a specific subject. An automatic deformation technique is used to align the features of a generic 3-D graphic face model with the corresponding features of these pre-provided images of the subject. Specifically, a generic frontal face model is aligned with the frontal image and a generic profile face model is aligned with the profile image. The deformation procedure results in a single 3-D face model of the specific human face. It precisely reflects the geometric features of the specific subject. After that, subdivision spline surface construction and multi-direction texture mapping techniques are used to smooth the model and endow photometric detail to the specific 3-D geometric face model.

Abstract: The described arrangements and procedures identify an image's orientation by extracting features from peripheral portions of the image. The procedure evaluates the extracted features based on training image feature orientation classification models to identify the image's orientation.

Abstract: An implementation of a technology, described herein, for relevance-feedback, content-based facilitating accurate and efficient image retrieval minimizes the number of iterations for user feedback regarding the semantic relevance of exemplary images while maximizing the resulting relevance of each iteration. One technique for accomplishing this is to use a Bayesian classifier to treat positive and negative feedback examples with different strategies. In addition, query refinement techniques are applied to pinpoint the users' intended queries with respect to their feedbacks. These techniques further enhance the accuracy and usability of relevance feedback. This abstract itself is not intended to limit the scope of this patent. The scope of the present invention is pointed out in the appending claims.

Abstract: Systems and methods are described that implement personalized karaoke, wherein a user's personal home video and photographs are used to form a background for the lyrics during a karaoke performance. An exemplary karaoke apparatus is configured to segment visual content to produce a plurality of sub-shots and to segment music to produce a plurality of music sub-clips. Having produced the visual content sub-shots and music sub-clips, the exemplary karaoke apparatus shortens some of the plurality of sub-shots to a length of a corresponding music sub-clip from within the plurality of music sub-clips. The plurality of sub-shots is then displayed as a background to lyrics associated with the music, thereby adding interest to a karaoke performance.

Abstract: The disclosed subject matter improves iterative results of content-based image retrieval (CBIR) using a bigram model to correlate relevance feedback. Specifically, multiple images are received responsive to multiple image search sessions. Relevance feedback is used to determine whether the received images are semantically relevant. A respective semantic correlation between each of at least one pair of the images is then estimated using respective bigram frequencies. The bigram frequencies are based on multiple search sessions in which each image of a pair of images is semantically relevant.

Abstract: A multimedia object retrieval and annotation system integrates an annotation process with object retrieval and relevance feedback processes. The annotation process annotates multimedia objects, such as digital images, with semantically relevant keywords. The annotation process is performed in background, hidden from the user, as the user conducts normal searches. The annotation process is “semi-automatic” in that it utilizes both keyword-based information retrieval and content-based image retrieval techniques to automatically search for multimedia objects, and then encourages users to provide feedback on the retrieved objects. The user identifies objects as either relevant or irrelevant to the query keywords and based on this feedback, the system automatically annotates the objects with semantically relevant keywords and/or updates associations between the keywords and objects. As the retrieval-feedback-annotation cycle is repeated, the annotation coverage and accuracy of future searches continues to improve.