Abstract: Method and a system for correlating pixels between two digital images. In general, the present invention uses a progressive iterative technique that finds generally unambiguous pixel matches by beginning with a few reliable pixel matches and finding progressively more unambiguous pixel matches. Unambiguous pixel matches in the current iteration then are found using the correlation technique and based on a correlation score associated with a pixel match. The search range is capable of being rotated, and is part of a novel correlation technique of the present invention that provides a more robust estimate of pixel match reliability. Potential pixel matches found in the search ranges are tested for ambiguity and any unambiguous matches are selected and added to the set of reliable pixel matches. The ambiguity testing includes determining a correlation score for the pixel match and classifying the match based on the correlation score.

Abstract: System and method for progressive stereo matching of digital images representing a scene. In general, the progressive stereo matching system includes an initialization module, that obtains a set of previous unambiguous matches of pixels within the digital images, and a search range module, that determines a search range in the first and second digital images to look for a candidate pixel match. The progressive stereo matching system also includes a correlation module, that computes a correlation score between the candidate pixel match, and a classification module, that classifies the pixel match based on its correlation score.

Abstract: An incremental motion estimation system and process for estimating the camera pose parameters associated with each image of a long image sequence. Unlike previous approaches, which rely on point matches across three or more views, the present system and process also includes those points shared only by two views. The problem is formulated as a series of localized bundle adjustments in such a way that the estimated camera motions in the whole sequence are consistent with each other. The result of the inclusion of two-view matching points and the localized bundle adjustment approach is more accurate estimates of the camera pose parameters for each image in the sequence than previous incremental techniques, and providing an accuracy approaching that of global bundle adjustment techniques except with processing times about 100 to 700 times faster than the global approaches.

Abstract: A system and method for mapping facial expressions. Facial expressions exhibit not only facial feature motions, but also subtle changes in illumination and appearance (e.g., facial creases and wrinkles). These details are important visual cues, but they are difficult to synthesize. Traditional expression mapping techniques consider feature motions while the details in illumination changes are ignored. A novel technique for facial expression mapping is presented. The illumination change of one person's expression is captured in what is called expression ratio image (ERI). Together with geometric warping, an ERI is mapped to any other person's face image to generate more expressive facial expressions.

Abstract: A system and method for turning a regular computer monitor screen into a touch screen using an ordinary camera. It includes an image-screen mapping procedure to correct for the non-flatness of the computer screen. It also includes a segmentation method to distinguish the foreground, for example an indicator such as a finger, from the background of a computer screen. Additionally, this system and method includes a robust technique of finding the tip point location of the indicator (such as the finger tip). The screen coordinates of the tip point are then used to control the position of the system indicator.

Abstract: A system and method for turning a regular computer monitor screen into a touch screen using an ordinary camera. It includes an image-screen mapping procedure to correct for the non-flatness of the computer screen. It also includes a segmentation method to distinguish the foreground, for example an indicator such as a finger, from the background of a computer screen. Additionally, this system and method includes a robust technique of finding the tip point location of the indicator (such as the finger tip). The screen coordinates of the tip point are then used to control the position of the system indicator.

Abstract: The present invention is embodied in a system and method for curve matching multiple images of a scene. The curve matching produces a geometrical representation of the scene from the images, which can be used for any suitable application, such as computer and stereo vision applications. In general, first, multiple images depicting a scene are digitally received by the system. The images are graphical images digitally received and processed as two dimensional image data, such as bitmap or raster image data. Curve matching of the images is then performed to correlate the two images of the scene for creating three dimensional (3D) curve information, such as 3D vector information, of the scene. This 3D vector information can then be used in any suitable manner, for example, to digitally reconstruct the scene for stereo vision applications.

Abstract: A system and method for mapping facial expressions. Facial expressions exhibit not only facial feature motions, but also subtle changes in illumination and appearance (e.g., facial creases and wrinkles). These details are important visual cues, but they are difficult to synthesize. Traditional expression mapping techniques consider feature motions while the details in illumination changes are ignored. A novel technique for facial expression mapping is presented. The illumination change of one person's expression is captured in what is called expression ratio image (ERI).

Abstract: A method and a system for progressive stereo matching of digital images representing a scene. In general, the present invention uses a progressive iterative technique that includes a disparity gradient limit principle and a least commitment strategy. The progressive iterative technique finds generally unambiguous pixel matches by beginning with a few reliable pixel matches and finding progressively more unambiguous pixel matches. Unambiguous pixel matches found in the previous iteration are used to define the search ranges for each pixel to guide matching in the current iteration. Unambiguous pixel matches in the current iteration then are found using a novel correlation technique and based on a correlation score associated with a pixel match. The search range is capable of being rotated, and is part of a novel correlation technique of the present invention that provides a more robust estimate of pixel match reliability.

Abstract: The present invention is embodied in a curve matching system and method that is guided by a set of matched corners. The corner guided curve matching produces a geometrical representation of the scene from the images, which can be used for any suitable application, such as computer and stereo vision applications. In general, first, multiple images depicting a scene are digitally received by the system. The images are graphical images digitally received and processed. For example, the images can be two dimensional image data, such as bitmap or raster image data. Curves of the images are then matched to correlate the two images of the scene for creating three dimensional (3D) curve information, such as 3D vector or mathematical information, of the scene. This 3D vector information can then be used in any suitable manner, for example, to digitally reconstruct the scene for stereo vision applications.

Abstract: A method and apparatus for performing georegistration using both a telemetry based rendering technique and an interative rendering technique. The method begins with a telemetry based rendering that produces reference imagery that substantially matches a view being imaged by the camera. The reference imagery is rendered using the telemetry of the present camera orientation. Upon obtaining a certain level of accuracy, the method proceeds to perform iterative rendering. During iterative rendering, the method uses image motion information from the video to enhance rendering of the reference imagery. A further embodiment uses sequential statistical framework to provide a unified approach to georegistration.

Abstract: A system and method, called Image-Based Surface Detail Transfer, to transfer geometric details from one surface of an object in an image to another with simple 2D image operations. The basic observation is that, without knowing its 3D geometry, geometric details (local deformations) can be extracted from a single image of an object in a way independent of its surface reflectance, and furthermore, these geometric details can be transferred to modify the appearance of other objects directly in images. Examples are shown including surface detail transfer between real objects, as well as between real and synthesized objects.

Abstract: A method and a system for progressive stereo matching of digital images representing a scene. In general, the present invention uses a progressive iterative technique that includes a disparity gradient limit principle and a least commitment strategy. The progressive iterative technique finds generally unambiguous pixel matches by beginning with a few reliable pixel matches and finding progressively more unambiguous pixel matches. Unambiguous pixel matches found in the previous iteration are used to define the search ranges for each pixel to guide matching in the current iteration. Unambiguous pixel matches in the current iteration then are found using a novel correlation technique and based on a correlation score associated with a pixel match. The search range is capable of being rotated, and is part of a novel correlation technique of the present invention that provides a more robust estimate of pixel match reliability.

Abstract: A vision-based gesture interface system, which employs an arbitrary quadrangle-shaped panel and a pointer tip like a fingertip as an intuitive input device. Taking advantage of the panel, the system can fulfill many tasks such as controlling a remote and large display, and simulating a physical keyboard. Users can naturally use their fingers and other pointer tips to issue commands and type texts. The system is facilitated by accurately and reliably tracking the panel and the pointer tip and detecting clicking and dragging actions.

Abstract: An incremental motion estimation system and process for estimating the camera pose parameters associated with each image of a long image sequence. Unlike previous approaches, which rely on point matches across three or more views, the present system and process also includes those points shared only by two views. The problem is formulated as a series of localized bundle adjustments in such a way that the estimated camera motions in the whole sequence are consistent with each other. The result of the inclusion of two-view matching points and the localized bundle adjustment approach is more accurate estimates of the camera pose parameters for each image in the sequence than previous incremental techniques, and providing an accuracy approaching that of global bundle adjustment techniques except with processing times about 100 to 700 times faster than the global approaches.

Abstract: A vision-based gesture interface system, which employs an arbitrary quadrangle-shaped panel and a pointer tip like a fingertip as an intuitive input device. Taking advantage of the panel, the system can fulfill many tasks such as controlling a remote and large display, and simulating a physical keyboard. Users can naturally use their fingers and other pointer tips to issue commands and type texts. The system is facilitated by accurately and reliably tracking the panel and the pointer tip and detecting clicking and dragging actions.

Abstract: A system and process for computer modeling of an object involving first capturing images of the object that collectively depict all the object's surfaces which are to be modeled. A series of 3D reconstructions are then derived from the images. Each of the reconstructions represent a portion of the object's surfaces. Noise elimination techniques are employed to reduce the number of extraneous reconstruction points. The individual 3D reconstructions are then registered to a common coordinate system to produce an overall 3D reconstruction of the object's surfaces. A surface representation of the object is extracted from the overall 3D reconstruction using robust surface extraction techniques, and if desired, a texture map for the surface representation of the object can be computed using the previously captured images to produce a photorealistic model of the object.