Abstract: A system and method for calibrating a stereo optical see-through HMD (head-mounted display). A preferred method integrates measurement for an optical see-through HMD and a six degrees of freedom tracker that is fixedly attached to the HMD to perform calibration. Calibration is based on the alignment of a stereoscopically fused marker image, which is perceived in depth, with a single 3D reference point in a world coordinate system from various viewpoints. The user interaction to perform the calibration is extremely easy compared to conventional methods and does not require keeping the head static during the calibration process.

Abstract: A new interaction/input device for an augmented reality system is provided. In an augmented reality system using infrared video-based tracking, the interaction/input device is employed by placing markers, e.g., small disks, at a predetermined location in a scene viewed by a user which are augmented to simulate physical buttons. These augmented markers, as viewed through the augmented reality system, can then be physically manipulated by the user. The user will put their fingers on one of these markers, and in turn, the infrared video-based tracker will recognize this action and process it accordingly. The augmented reality system can also augment simulated menus in the user's view giving the user the necessary feedback for interaction. By exploiting conventional tracking technology, the interaction/input device can be implemented with minimal additional hardware and minimal additional processing required by the augmented reality system.

Abstract: A method for determining registration accuracy of an augmented reality system is provided. The method includes the steps of providing an input device for inputting a position of a user's touch; registering the input device with a workspace coordinate system of the augmented reality system; displaying a virtual object in a known position; recording a user's touch position in response to the user attempting to touch the virtual object; and calculating a registration error of the augmented reality system by comparing the user's touch position to the known position of the virtual object. The input device may be a passive device used in conjunction with a tracking system of the augmented reality system or an active device, such as a touch-sensitive device, coupled to the augmented reality system which outputs the user's touch position.

Abstract: A single-computer real-time augmented reality system includes a computer having a processor coupled in signal communication with a PCI bus, a head-mounted display coupled with the computer, a first frame grabber disposed relative to the computer and coupled with the processor, the first frame grabber having a direct digital video output, a left video camera disposed relative to the head-mounted display and coupled with the first frame grabber, a left video display disposed relative to the head-mounted display and coupled with the direct video output of the first frame grabber, a second frame grabber disposed relative to the computer and coupled with the processor, the second frame grabber having a direct digital video output, a right video camera disposed relative to the head-mounted display and coupled with the second frame grabber, and a right video display disposed relative to the head-mounted display and coupled with the direct video output of the second frame grabber.

Abstract: A method for removing a portion of a foreground of an image comprises determining a portion of a foreground to remove from a reference image, determining a plurality of source views of a background obscured in the reference image, determining a correlated portion in each source view corresponding to the portion of the foreground to remove, and displaying the correlated portion in the reference image.

Abstract: An augmented reality system includes a video source, and a database. The video source resides at a location and produces an image. At least one encoded marker resides within the image. A marker detector is coupled to the video source and is adapted to derive encoded data from the marker residing within the image. A localization processor is adapted to receive data from the marker detector and to generate data regarding location and orientation of the marker. The localization processor retrieves information from the database that is related to the location and orientation of the marker. The localization processor makes the information retrieved from the database available to a user.

Abstract: A method and system for tracking a position and orientation (pose) of a camera using real scene features is provided. The method includes the steps of capturing a video sequence by the camera; extracting features from the video sequence; estimating a first pose of the camera by an external tracking system; constructing a model of the features from the first pose; and estimating a second pose by tracking the model of the features, wherein after the second pose is estimated, the external tracking system is eliminated. The system includes an external tracker for estimating a reference pose; a camera for capturing a video sequence; a feature extractor for extracting features from the video sequence; a model builder for constructing a model of the features from the reference pose; and a pose estimator for estimating a pose of the camera by tracking the model of the features.

Abstract: An interaction/input device for use in systems utilizing infrared video-based trackers is provided. The interaction/input device is employed by placing markers, e.g., small retroreflective disks, in a predetermined configuration on a planar surface to simulate physical buttons. Once in view of the infrared tracker, the system will recognize the predetermined configuration as the input device which can then be physically manipulated by the user. The user will put their fingers on one of these markers or disks, and in turn, the infrared video-based tracker will recognize this action and process it accordingly. Optionally, an augmented reality system employing the input device of the present invention can also augment simulated menus in the user's view, via a head-mounted display, giving the user the necessary feedback for interaction.

Abstract: A system and method for calibrating a stereo optical see-through HMD (head-mounted display). A preferred method integrates measurement for an optical see-through HMD and a six degrees of freedom tracker that is fixedly attached to the HMD to perform calibration. Calibration is based on the alignment of a stereoscopically fused marker image, which is perceived in depth, with a single 3D reference point in a world coordinate system from various viewpoints. The user interaction to perform the calibration is extremely easy compared to conventional methods and does not require keeping the head static during the calibration process.

Abstract: A method for computer assisted localization and navigation in a defined environment, comprises the steps of: obtaining a floor map of the defined environment; placing a set of identifiable markers on the floor in given locations; determining the position of a user with respect to ones of the markers; determining the global position of the user by utilizing the floor map and the given locations; and indexing a database for data associated with the global position.