Abstract: The present invention may be embodied in a method, and in a related apparatus, for classifying a feature in an image frame. In the method, an original image frame having an array of pixels is transformed using Gabor-wavelet transformations to generate a transformed image frame. Each pixel of the transformed image is associated with a respective pixel of the original image frame and is represented by a predetermined number of wavelet component values. A pixel of the transformed image frame associated with the feature is selected for analysis. A neural network is provided that has an output and a predetermined number of inputs. Each input of the neural network is associated with a respective wavelet component value of the selected pixel. The neural network classifies the local feature based on the wavelet component values, and indicates a class of the feature at an output of the neural network.

Abstract: The present invention provides a technique for translating facial animation values to head mesh positions for rendering facial features of an animated avatar. In the method, an animation vector of dimension Na is provided. Na is the number of facial animation values in the animation vector. A mapping algorithm F is applied to the animation vector to generate a target mix vector of dimension M. M is the number of targets associated with the head mesh positions. The head mesh positions are deformed based on the target mix vector.

Abstract: The present invention provides a technique for translating an animation vector to a target mix vector.

Abstract: The present invention relates to a technique for optimizing off-line facial feature tracking. Facial features in a sequence of image frames are automatically tracked while a visual indication is presented of the plurality of tracking node locations on the respective image frames. The sequence of image frames may be manually paused at a particular image frame in the sequence of image frames if the visual indication of the tracking node locations indicates that at least one location of a tracking node for a respective facial feature is not adequately tracking the respective facial feature. The location of the tracking node may be reinitialized by manually placing the tracking node location at a position on the particular image frame in the monitor window that corresponds to the respective facial feature. Automatic tracking of the facial feature may be continued based on the reinitialized tracking node location.

Abstract: The present invention is embodied in a method and system for customizing a visual sensor for facial feature tracking using a neutral face image of an actor. The method may include generating a corrector graph to improve the sensor's performance in tracking an actor's facial features.

Abstract: The present invention is embodied in an apparatus, and related method, for sensing a person's facial movements, features and characteristics and the like to generate and animate an avatar image based on facial sensing. The avatar apparatus uses an image processing technique based on model graphs and bunch graphs that efficiently represent image features as jets. The jets are composed of wavelet transforms processed at node or landmark locations on an image corresponding to readily identifiable features. The nodes are acquired and tracked to animate an avatar image in accordance with the person's facial movements. Also, the facial sensing may use jet similarity to determine the person's facial features and characteristic thus allows tracking of a person's natural characteristics without any unnatural elements that may interfere or inhibit the person's natural characteristics.

Abstract: The present invention provides a technique for translating facial animation values to head mesh positions for rendering facial features of an animated avatar. In the method, an animation vector of dimension Na is provided. Na is the number of facial animation values in the animation vector. A mapping algorithm F is applied to the animation vector to generate a target mix vector of dimension M. M is the number of targets associated with the head mesh positions. The head mesh positions are deformed based on the target mix vector.

Abstract: The present invention provides a technique for translating an animation vector to a target mix vector.

Abstract: The present invention relates to a technique for optimizing off-line facial feature tracking. Facial features in a sequence of image frames are automatically tracked while a visual indication is presented of the plurality of tracking node locations on the respective image frames. The sequence of image frames may be manually paused at a particular image frame in the sequence of image frames if the visual indication of the tracking node locations indicates that at least one location of a tracking node for a respective facial feature is not adequately tracking the respective facial feature. The location of the tracking node may be reinitialized by manually placing the tracking node location at a position on the particular image frame in the monitor window that corresponds to the respective facial feature. Automatic tracking of the facial feature may be continued based on the reinitialized tracking node location.

Abstract: The invention relates to a technique for generating an animated three-dimensional video head based on sensed locations of facial features and texture mapping of corresponding two dimensional video image frames onto a shaped head mesh generated using the sensed locations.

Abstract: The present invention is embodied in a method and apparatus for relief texture map flipping. The relief texture map flipping technique provides realistic avatar animation in a computationally efficient manner.

Abstract: Facial animation values are generated using a sequence of facial image frames and synchronously captured audio data of a speaking actor. In the technique, a plurality of visual-facial-animation values are provided based on tracking of facial features in the sequence of facial image frames of the speaking actor, and a plurality of audio-facial-animation values are provided based on visemes detected using the synchronously captured audio voice data of the speaking actor. The plurality of visual facial animation values and the plurality of audio facial animation values are combined to generate output facial animation values for use in facial animation.

Abstract: The present invention is embodied in a method and system for generating an animation transform using a neutral face image. An avatar editor uses a frontal head image and a side head image of a neutral face model for generating an avatar. The avatar is generated by automatically finding head feature locations on the front and side head images using elastic bunch graph matching. Significant time savings may be accomplished by a generating an animation transform using the neutral face features. The animation transform for the neutral face features may be applied to the other facial expression avatar meshes to improve the quality of the resulting avatar. The neutral-face-based animation transform provides significant improvement to the facial expression head models without the significant editing time incurred by generating a particular animation transform for each particular facial expression (and/or pose) features.

Abstract: The present invention is directed to a method and related system for determining a feature location in multiple dimensions including depth. The method includes providing left and right camera images of the feature and locating the feature in the left camera image and in the right camera image using bunch graph matching. The feature location is determined in multiple dimensions including depth based on the feature locations in the left camera image and the right camera image.

Abstract: The present invention is embodied in a method and system for customizing a visual sensor for facial feature tracking using a neutral face image of an actor. The method may include generating a corrector graph to improve the sensor's performance in tracking an actor's facial features.

Abstract: The present invention is embodied in an apparatus, and related method, for sensing a person's facial movements, features and characteristics and the like to generate and animate an avatar image based on facial sensing. The avatar apparatus uses an image processing technique based on model graphs and bunch graphs that efficiently represent image features as jets. The jets are composed of wavelet transforms processed at node or landmark locations on an image corresponding to readily identifiable features. The nodes are acquired and tracked to animate an avatar image in accordance with the person's facial movements. Also, the facial sensing may use jet similarity to determine the person's facial features and characteristic thus allows tracking of a person's natural characteristics without any unnatural elements that may interfere or inhibit the person's natural characteristics.

Abstract: The present invention is embodied in an apparatus, and related method, for detecting and recognizing an object in an image frame. The object may be, for example, a head having particular facial characteristics. The object detection process uses robust and computationally efficient techniques. The object identification and recognition process uses an image processing technique based on model graphs and bunch graphs that efficiently represent image features as jets. The jets are composed of wavelet transforms and are processed at nodes or landmark locations on an image corresponding to readily identifiable features. The system of the invention is particularly advantageous for recognizing a person over a wide variety of pose angles.

Abstract: The present invention is embodied in an apparatus, and related method, for sensing a person's facial movements, features and characteristics and the like to generate and animate an avatar image based on facial sensing. The avatar apparatus uses an image processing technique based on model graphs and bunch graphs that efficiently represent image features as jets. The jets are composed of wavelet transforms processed at node or landmark locations on an image corresponding to readily identifiable features. The nodes are acquired and tracked to animate an avatar image in accordance with the person's facial movements. Also, the facial sensing may use jet similarity to determine the person's facial features and characteristic thus allows tracking of a person's natural characteristics without any unnatural elements that may interfere or inhibit the person's natural characteristics.