Abstract: A detection method is based on a statistical analysis of the appearance of model patches from all possible viewpoints in the scene, and incorporates 3D geometry during both matching and pose estimation processes. By analyzing the computed probability distribution of the visibility of each patch from different viewpoints, a reliability measure for each patch is estimated. That reliability measure is useful for developing industrial augmented reality applications. Using the method, the pose of complex objects can be estimated efficiently given a single test image.

Abstract: A method of tracking a pose of a moving camera includes receiving a first image from a camera, receiving a sequence of digitized images from said camera, recording, for each of said sequence of digitized images, the pose and 2D correspondences of landmarks, reconstructing a location and appearance of a 2-dimensional texture patch from 2D correspondences of the landmarks by triangulation and optimization, computing back-projection errors by comparing said reconstructed texture patch with said first received image; and reconstructing said location and appearance of said 2-dimensional texture patch from the 2D correspondences of the landmarks of said sequence of digitized images by triangulation and optimization after eliminating those landmarks with large back-projection errors.

Abstract: A method for modeling a vehicle, includes: receiving an image that includes a vehicle; and constructing a three-dimensional (3D) model of the vehicle, wherein the 3D model is constructed by: (a) taking a predetermined set of base shapes that are extracted from a subset of vehicles; (b) multiplying each of the base shapes by a parameter; (c) adding the resultant of each multiplication to form a vector that represents the vehicle's shape; (d) fitting the vector to the vehicle in the image; and (e) repeating steps (a)-(d) by modifying the parameters until a difference between a fit vector and the vehicle in the image is minimized.

Abstract: A method wherein images of different types of objects within a class are partitioned into region stacks. For each one of the stacks, the method: (a) applies a template to extract fragments having a predetermined size and one of a plurality of different spatial orientations, to generate extracted templates; (b) determines, from the extracted templates, a most frequent one thereof having only a first number of fragments with a common spatial orientations; (c) records the number of images having the determined most frequent extracted template; (d) repeats (b) and (c) with successively increasing number of fragments until the number of recoded images falls below a threshold; and (e) selects as a master extracted template the one of the most frequent templates having the largest recorded number of fragments. The master extracted templates for the stacks are combined into a map that is then compared with background images to remove extracted templates matching segment in the background.

Abstract: A method for reconstructing three-dimensional, plural views of images from two dimensional image data. The method includes: obtaining two-dimensional, stereo digital data from images of an object; processing the digital data to generate an initial three-dimensional candidate of the object, such process using projective geometric constraints imposed on edge points of the object; refining the initial candidate comprising examining spatial coherency of neighboring edge points along a surface of the candidate.

Abstract: A method of using a Graphic User Interface (GUI) for interactive virtual inspection of modeled objects. The method includes acquiring a three-dimensional model of a modeled object and displaying a first view of the modeled object for a user to identify locations of interest on a surface of the modeled object visible within the first view. The user enters information to create a markup tag that annotates the location of interest, and the markup tag is automatically associated with the location of interest on the modeled object. A second view of the modeled object is displayed including the user identified location of interest and the markup tag.

Abstract: A method of coalescing information about inspected objects. The method includes acquiring an image set of an object to be inspected, the image set having a three-dimensional model of the object and a plurality of two-dimensional images of the object. A location of interest is identified on a surface of the modeled object and global coordinate points of the three-dimensional model are designated that characterize the location of interest of the modeled object. A markup tag annotating the location of interest is associate with the designated global coordinate points of the three-dimensional model, and the markup tag is conveyed when viewing any one of the plurality of two-dimensional images of the image set that have at least one image point that correlates to a corresponding designated global coordinate point of the three-dimensional model that characterize the location of interest.

Abstract: A computer-implemented method for object localization includes providing a plurality of tags having fixed positions within an environment and known identifications, and providing a reader, for detecting the tags and reading the identifications, wherein the reader detects a plurality of tags within a scanning volume having a known shape. The method further includes determining positions for the plurality of tags within the scanning volume, and determining a position of the reader according to the positions for the plurality of tags within the scanning volume and the known shape of the scanning volume.

Abstract: A method for calibrating real and virtual views includes tracking a calibration screen, wherein a real reference point, generated by a real reference point generator, is projected on the calibration screen, aligning a view of a virtual reference point to a view of the real reference point in a display, wherein the real reference point generator and the display have a fixed relative position, determining a point correspondence between the virtual reference point and the real reference point, and determining one or more parameters for rendering a virtual object in the real scene.

Abstract: A method of generating a resulting image includes generating a superimposed image by aligning and superimposing one or more transposed images with a reference image by using offsets of the one or more transposed images from the reference image, generating an intermediate image from the superimposed image, generating a new superimposed image by aligning and superimposing the intermediate image, the one or more transposed images and the reference image by using offsets of the one or more transposed images and the reference image from the intermediate image, and generating a resulting image from the new superimposed image.

Abstract: An augmented reality system includes a video source, and a database. The video source resides at a location and produces an image. At least one encoded marker resides within the image. A marker detector is coupled to the video source and is adapted to derive encoded data from the marker residing within the image. A localization processor is adapted to receive data from the marker detector and to generate data regarding location and orientation of the marker. The localization processor retrieves information from the database that is related to the location and orientation of the marker. The localization processor makes the information retrieved from the database available to a user.

Abstract: A new interaction/input device for an augmented reality system is provided. In an augmented reality system using infrared video-based tracking, the interaction/input device is employed by placing markers, e.g., small disks, at a predetermined location in a scene viewed by a user which are augmented to simulate physical buttons. These augmented markers, as viewed through the augmented reality system, can then be physically manipulated by the user. The user will put their fingers on one of these markers, and in turn, the infrared video-based tracker will recognize this action and process it accordingly. The augmented reality system can also augment simulated menus in the user's view giving the user the necessary feedback for interaction. By exploiting conventional tracking technology, the interaction/input device can be implemented with minimal additional hardware and minimal additional processing required by the augmented reality system.

Abstract: A method for determining registration accuracy of an augmented reality system is provided. The method includes the steps of providing an input device for inputting a position of a user's touch; registering the input device with a workspace coordinate system of the augmented reality system; displaying a virtual object in a known position; recording a user's touch position in response to the user attempting to touch the virtual object; and calculating a registration error of the augmented reality system by comparing the user's touch position to the known position of the virtual object. The input device may be a passive device used in conjunction with a tracking system of the augmented reality system or an active device, such as a touch-sensitive device, coupled to the augmented reality system which outputs the user's touch position.

Abstract: A detection method is based on a statistical analysis of the appearance of model patches from all possible viewpoints in the scene, and incorporates 3D geometry during both matching and pose estimation processes. By analyzing the computed probability distribution of the visibility of each patch from different viewpoints, a reliability measure for each patch is estimated. That reliability measure is useful for developing industrial augmented reality applications. Using the method, the pose of complex objects can be estimated efficiently given a single test image. In a fast method to detect objects in a given image or video sequence, a series of hierarchical feature descriptors permit balancing between the complexity of feature extraction and cost of combinatorial matching. The feature descriptors are derived from the 3D model of the object of interest along with the available calibrated real-life images or videos. The variability of the real-life images defines the granularity of the feature descriptors.

Abstract: A method of tracking a pose of a moving camera includes receiving a first image from a camera, receiving a sequence of digitized images from said camera, recording, for each of said sequence of digitized images, the pose and 2D correspondences of landmarks, reconstructing a location and appearance of a 2-dimensional texture patch from 2D correspondences of the landmarks by triangulation and optimization, computing back-projection errors by comparing said reconstructed texture patch with said first received image; and reconstructing said location and appearance of said 2-dimensional texture patch from the 2D correspondences of the landmarks of said sequence of digitized images by triangulation and optimization after eliminating those landmarks with large back-projection errors.

Abstract: A method for calibrating real and virtual views includes tracking a calibration screen, wherein a real reference point, generated by a real reference point generator, is projected on the calibration screen, aligning a view of a virtual reference point to a view of the real reference point in a display, wherein the real reference point generator and the display have a fixed relative position, determining a point correspondence between the virtual reference point and the real reference point, and determining one or more parameters for rendering a virtual object in the real scene.

Abstract: An interaction/input device for use in systems utilizing infrared video-based trackers is provided. The interaction/input device is employed by placing markers, e.g., small retroreflective disks, in a predetermined configuration on a planar surface to simulate physical buttons. Once in view of the infrared tracker, the system will recognize the predetermined configuration as the input device which can then be physically manipulated by the user. The user will put their fingers on one of these markers or disks, and in turn, the infrared video-based tracker will recognize this action and process it accordingly. Optionally, an augmented reality system employing the input device of the present invention can also augment simulated menus in the user's view, via a head-mounted display, giving the user the necessary feedback for interaction.

Abstract: A computer-implemented method for object localization includes providing a plurality of tags having fixed positions within an environment and known identifications, and providing a reader, for detecting the tags and reading the identifications, wherein the reader detects a plurality of tags within a scanning volume having a known shape. The method further includes determining positions for the plurality of tags within the scanning volume, and determining a position of the reader according to the positions for the plurality of tags within the scanning volume and the known shape of the scanning volume.

Abstract: A system and method for tracking parcels on a planar surface is provided. The method comprising: acquiring an image of the parcel located on the planar surface; determining edges of the parcel; projecting the edges onto the planar surface; determining which edges belong to each side of the parcel; calculating a cost function associated with the edges belonging to each side of the parcel; searching the edges belonging to each side of the parcel to find edges having a lowest cost; and constructing a matching configuration of the parcel using the edges having the lowest cost.

Abstract: A system and method for calibrating a stereo optical see-through HMD (head-mounted display). A preferred method integrates measurement for an optical see-through HMD and a six degrees of freedom tracker that is fixedly attached to the HMD to perform calibration. Calibration is based on the alignment of a stereoscopically fused marker image, which is perceived in depth, with a single 3D reference point in a world coordinate system from various viewpoints. The user interaction to perform the calibration is extremely easy compared to conventional methods and does not require keeping the head static during the calibration process.