Abstract: A super-resolution image is generated from a pixel image by first performing an initial image interpolation, creating an interpolated low resolution image. The interpolated low resolution image is then partitioned into overlapping low resolution patches. The low resolution patches are then processed in a raster scan order. For each low-resolution patch, a scaled mid band patch is generated. A search vector is constructed from pixels in the scaled mid band input patch, and pixels in an overlap region of adjacent previously predicted high band patches. A nearest index vector to the search vector is located in a training database, the nearest index vector has an associated high band output patch. The high band output patch is then combined with the interpolated low resolution patch to predict pixel values for the corresponding high resolution patch of the super-resolution image.

Abstract: A method determines the probabilities of states of a system represented by a model including of nodes connected by links. Each node represents possible states of a corresponding part of the system, and each link represents statistical dependencies between possible states of related nodes. The nodes are grouped into arbitrary sized clusters such that every node is included in at least one cluster. A minimal number of marginalization constraints to be satisfied between the clusters are determined. A super-node network is constructed so that each cluster of nodes is represented by exactly one super-node. Super-nodes that share one of the marginalization constraints are connected by super-links. The super-node network is searched to locate closed loops of super-nodes containing at least one common node. A normalization operator for each closed loop is determined, and messages between the super-nodes are defined.

Abstract: The present invention provides a system for analyzing multi-factor data according to a separable mixture model to classify, extrapolate, and translate data. A separable mixture model represents each data point as a function of underlying factors and corresponding parameters. With two factors, the factors can be referred to as style and content. In particular, a particular data point can be represented according to a separable mixture model, as a combination of the product of parameter vectors for each of the factors. The system accommodates symmetric and asymmetric models of interactions of parameter vectors, which are selected based upon the known data and the desired analysis. Using a selected model and known data, the system generates relevant parameters describing the known data. The parameter vector information can then be used to analyze and synthesize test data. In classification, unknown data in a new style is classified as to content, or vice versa.

Abstract: A method infers a scene from a test image. During a training phase, a plurality of images and corresponding scenes are acquired. Each of the images and corresponding scenes are partitioned respectively into a plurality of image patches and scene patches. Each image patch is represented as an image vector, and each scene patch is represented as a scene vector. The image vectors and scene vectors are modeled as a network. During an inference phase, the test image is acquired. The test image is partitioned into a plurality of test image patches. Each test image patch is represented as a test image vector. Candidate scene vectors corresponding to the test image vectors are located in the network. Compatibility matrices for the candidate scene vectors are determined, and probabilities of the compatibility matrices are propagated in the network until convergence to infer the scene from the test image.

Abstract: A super-resolution image is generated from a pixel image by first performing an initial image interpolation, creating an interpolated low resolution image. The interpolated low resolution image is then partitioned into overlapping low resolution patches. The low resolution patches are then processed in a raster scan order. For each low-resolution patch, a scaled mid band patch is generated. A search vector is constructed from pixels in the scaled mid band input patch, and pixels in an overlap region of adjacent previously predicted high band patches. A nearest index vector to the search vector is located in a training database, the nearest index vector has an associated high band output patch. The high band output patch is then combined with the interpolated low resolution patch to predict pixel values for the corresponding high resolution patch of the super-resolution image.

Abstract: The invention provides an image-based method for generating novel visual appearance in a new image. Synthetic texture is stitching together from small patches in existing images. First, we use a least cost path determination to determine the local boundaries between the patches. Second, we perform texture transfer by rendering an arbitrary object with a synthetic texture taken from a different object. More generally, we provide methods for rendering entire images in styles of different images. The method works directly on pixel images, and does not require 3D information.

Abstract: A method for estimating scenes from images generates a plurality of scenes and renders an image for each scene. The scenes and corresponding images are partitioned into patches. Each patch is quantitized as a vector and probability density function is fitted to each vector. The patches and probabilities are organized as a Markov network where local probability information is propagated to neighboring nodes. After propagation, the probability densities at each node is used to estimate the scene.

Abstract: To identify an image characteristic, a tree structured code having coding representing each of a plurality of training images is stored in a memory. Coding representing a sample image, not included in the plurality of training images, is compared to the tree structured code to identify the training image coding in the structured tree code closest to the sample image coding. A characteristic of the sample image is identified based upon the training image represented by the closest training image coding.

Abstract: A compact video image recording device for recording video images before and after a triggering event and which utilizes no moving parts is disclosed. The recording device includes at least one camera wherein each camera comprises a lens and a video image sensor. Each video image sensor generates an electronic signal representative of a video image impinging the respective sensor. The output of each image sensor is processed, compressed and generally employed to produce frame data which are successively stored in a successive frame locations of a semiconductor memory organized as a circular buffer memory. Upon the occurrence of a triggering event, a additional frames are stored in the buffer memory and further storage of frames then terminates. A video record is thus created of video images received both before and after the triggering event via a device which contains no moving parts and which can withstand substantial shock and vibration.

Abstract: A method for estimating scenes from images generates a plurality of scenes and renders an image for each scene. The scenes and corresponding images are partitioned into patches. Each patch is quantitized as a vector and probability density function is fitted to each vector. The patches and probability densities are organized as a Markov network where local probability information is propagated to neighboring nodes. After propagation, the probability density at each node is used to estimate the scene.

Abstract: A method and apparatus are disclosed for determining a direction in which a subject's head is facing. The present invention has particular relevance to monitoring driver conditions in automobiles. By monitoring head positions and motions, the system can estimate driver awareness. Signals or alarms can be activated if the driver does not appear to be aware of driving conditions. The present invention provides a rapid, low cost device for determining pose position, which would be useful in automobiles. A large number of images of various subjects in different poses are stored a memory. The poses are then organized according to tree-structured vector quantization to allow comparison with a sample image of an actual driver. The tree structure is traversed to find a leaf node constituting one of the stored images which is closest to the sample image. The pose of the closest image is outputted as the pose of the sample image.

Abstract: A system is provided for reconstructing the 3-dimensional motions of a hu figure from a monocularly viewed image sequence in which a statistical approach is used coupled with the use of a set of motion capture examples to build a gaussian probability model for short human motion sequences. In a simplified rendering domain, this yields an optimal 3-d estimate of human motion used to create realistic animations given a 2-d temporal sequence. The subject system is also useful to identify which motion modes are difficult to estimate. In one embodiment, a stick figure is overlaid on the input video image to allow manual correction of incorrectly tracked body parts. The utilization of stick figure correction permits manual correction for a more accurate 3-d reconstruction of the motion depicted in the input video sequence. An interactive tracking system processes real video sequences, and achieves good 3-d reconstructions of the human figure motion.

Abstract: In a graphical object manipulation system, a system for modifying shapes ludes representing a reference shape in terms of particles making up the shape, providing a Coulomb warp between the particles of the reference shape and those of a target shape, modifying the Coulomb warp and using the modified warp to generate a modified target shape from the reference shape by displacing particles in the reference shape to modified positions in the target shape based on modified displacements. The system permits shape averaging or providing a shape with a distinctive style.

Abstract: A system is provided for rapidly recognizing hand gestures for the control of computer graphics, in which image moment calculations are utilized to determine an overall equivalent rectangle corresponding to hand position, orientation and size, with size in one embodiment correlating to the width of the hand. In a further embodiment, a hole generated through the utilization of the touching of the forefinger with the thumb provides a special trigger gesture recognized through the corresponding hole in the binary representation of the hand. In a further embodiment, image moments of images of other objects are detected for controlling or directing onscreen images.

Abstract: A system for creating lighting for a photograph generates a large number of mages based upon structures determined from the photograph and randomly selected light positions, types and directions. One or more photographs are analyzed to determine the three-dimensional structures in the photographs. These structures are represented as gray surfaces, to which lighting is applied. The images are culled to provide a set of images which best spans the lighting space represented by the large number of images. The culling process is iterative; at each iteration, the image which is most dissimilar from a nearest neighbor in the selected set is added to the selected set. The images are organized in a hierarchical structure. A user interface allows the user to review and select images in the hierarchical structure. The images selected by the user are combined to create a final image with composite lighting. The lighting specifications can be saved for future reference by the lighting system or other applications.

Abstract: A system for the control from a distance of machines having displays incls hand gesture detection in which the hand gesture causes movement of an on-screen hand icon over an on-screen machine control icon, with the hand icon moving the machine control icon in accordance with sensed hand movements to effectuate machine control. In one embodiment, TV control led by hand signals includes detecting a single hand gesture and providing a hand icon on the screen along with the provision of icons representing TV controls such as volume, channel, color, density, etc., in which a television camera detects the hand in a noisy background through correlation techniques based on values of local image orientation. In order to trigger the system into operation, a trigger gesture such as the "how" sign is distinguished from the background through the utilization of orientation angle differences.

Abstract: A technique is provided to remove noise from images and to enhance their visual appearance through the utilization of a technique which converts an image into a set of coefficients in a multi-scale image decomposition process, followed by modification of each coefficient based on its value and the value of coefficients of related orientation, position, or scale, which is in turn followed by a reconstruction or synthesis process to generate the enhanced image. Also contributing to the improved enhancement is a set of orientation tuned filters of a specialized design to permit steering, with the analysis and synthesis filters also having a self-inverting characteristic. Additionally, steerable pyramid architecture is used for image enhancement for the first time, with the steering being provided by the above orientation tuned filters.

Abstract: A low-level model-free dynamic and static hand gesture recognition system utilizes either a 1-D histogram of frequency of occurrence vs. spatial orientation angle for static gestures or a 2-D histogram of frequency of occurrence vs. space-time orientation for dynamic gestures. In each case the histogram constitutes the signature of the gesture which is used for gesture recognition. For moving gesture detection, a 3-D space-time orientation map is merged or converted into the 2-D space-time orientation histogram which graphs frequency of occurrence vs. both orientation and movement. It is against this representation or template that an incoming moving gesture is matched.

Abstract: Apparatus and method for encoding and recording electronic image information signals in a manner whereby luminance signals for odd field lines are recorded in association respectively with chrominance signals alternately selected from both the even field lines and the odd field lines and luminance signals for even field lines are recorded in association respectively with the remaining chrominance signals alternately selected from both the even field lines and the odd field lines. Chrominance samples recorded in this manner are spaced only two field lines apart thereby allowing the chrominance signals to be interpolated by a playback device specially configured to provide for increased vertical chromianance resolution.

Abstract: An improved method and apparatus is provided for sensing scene light and providing sampled image data in three colors in response thereto. The sampled image data is subsequently interpolated for the non-sampled colors and thereafter subtracted to provide two color difference signals. The two color difference signals, in turn, are each median filtered and subsequently reconstructed in conjunction with the originally sampled image data to provide an image of the subject with substantially reduced color fringing. The median filter preferably filters color difference signals corresponding to a select number of image sensing elements separated with respect to each other by non-select image sensing elements.