Abstract: This invention relates to positional tracking, locating, and guiding individuals and robotic devices within buildings. An embodiment of the invention includes a method for tracking, locating, and guiding an entity within a building, that has the steps of emergency personnel arriving at the location, determining if digital floor plans are available, downloading digital floor plan data of a building, entering the digital floor plan data of the building into a server, displaying placeholder locations within the building, activating the tracking devices by affixing the tracking devices to the placeholders, determining if the server placeholder locations were used, inputting placeholder locations that were used, and displaying a combination of said location data and said building floor plans to a user.

Abstract: A method for determining registration accuracy of an augmented reality system is provided. The method includes the steps of providing an input device for inputting a position of a user's touch; registering the input device with a workspace coordinate system of the augmented reality system; displaying a virtual object in a known position; recording a user's touch position in response to the user attempting to touch the virtual object; and calculating a registration error of the augmented reality system by comparing the user's touch position to the known position of the virtual object. The input device may be a passive device used in conjunction with a tracking system of the augmented reality system or an active device, such as a touch-sensitive device, coupled to the augmented reality system which outputs the user's touch position.

Abstract: A single-computer real-time augmented reality system includes a computer having a processor coupled in signal communication with a PCI bus, a head-mounted display coupled with the computer, a first frame grabber disposed relative to the computer and coupled with the processor, the first frame grabber having a direct digital video output, a left video camera disposed relative to the head-mounted display and coupled with the first frame grabber, a left video display disposed relative to the head-mounted display and coupled with the direct video output of the first frame grabber, a second frame grabber disposed relative to the computer and coupled with the processor, the second frame grabber having a direct digital video output, a right video camera disposed relative to the head-mounted display and coupled with the second frame grabber, and a right video display disposed relative to the head-mounted display and coupled with the direct video output of the second frame grabber.

Abstract: A method for removing a portion of a foreground of an image comprises determining a portion of a foreground to remove from a reference image, determining a plurality of source views of a background obscured in the reference image, determining a correlated portion in each source view corresponding to the portion of the foreground to remove, and displaying the correlated portion in the reference image.

Abstract: A method for three-dimensional (3D) image reconstruction includes the steps of registering an orthographic image of a scene, combining a photogrammetric image and a technical drawing of the scene to form a co-registered orthographic and perspective (COP) image, and reconstructing a 3D image from the COP image.

Abstract: This invention relates to positional tracking, locating, and guiding individuals and robotic devices within buildings. An embodiment of the invention includes a method for tracking, locating, and guiding an entity within a building, that has the steps of emergency personnel arriving at the location, determining if digital floor plans are available, downloading digital floor plan data of a building, entering the digital floor plan data of the building into a server, displaying placeholder locations within the building, activating the tracking devices by affixing the tracking devices to the placeholders, determining if the server placeholder locations were used, inputting placeholder locations that were used, and displaying a combination of said location data and said building floor plans to a user.

Abstract: A method and system for computer assisted site navigation includes capturing an image of a scene for providing real-time image data, calibrating a camera and localizing the image by utilizing the image data and retrieved marker registration and floor-map data for providing position and orientation data, transmitting the data to a web server, utilizing the image data and information on the scene retrieved from a database for deriving an augmented view, and displaying the augmented view.

Abstract: An augmented reality system includes a video source, and a database. The video source resides at a location and produces an image. At least one encoded marker resides within the image. A marker detector is coupled to the video source and is adapted to derive encoded data from the marker residing within the image. A localization processor is adapted to receive data from the marker detector and to generate data regarding location and orientation of the marker. The localization processor retrieves information from the database that is related to the location and orientation of the marker. The localization processor makes the information retrieved from the database available to a user.

Abstract: A method for determining a pose of a user is provided including the steps of capturing a video image sequence of an environment including at least one coded marker; detecting if the coded marker is present in the video images; if the marker is present, extracting feature correspondences of the coded marker; determining a code of the coded marker using the feature correspondences; and comparing the determined code with a database of predetermined codes to determine the pose of the user. According to an embodiment, the coded marker includes four color blocks arranged in a square formation and the determining a code of the at least one marker further includes determining a color of each of the four blocks. According to another embodiment, the marker includes a coding matrix and a code of the marker being determined by numbered squares of the coding matrix being covered by a circle.

Abstract: There is provided a method for augmented reality guided instrument positioning. A viewpoint is established, from which a line of sight to a point on a target defines a path for an instrument to follow during a positioning of the instrument to the point on the target. The instrument is aligned along the line of sight to the point on the target.

Abstract: A method and system for tracking a position and orientation (pose) of a camera using real scene features is provided. The method includes the steps of capturing a video sequence by the camera; extracting features from the video sequence; estimating a first pose of the camera by an external tracking system; constructing a model of the features from the first pose; and estimating a second pose by tracking the model of the features, wherein after the second pose is estimated, the external tracking system is eliminated. The system includes an external tracker for estimating a reference pose; a camera for capturing a video sequence; a feature extractor for extracting features from the video sequence; a model builder for constructing a model of the features from the reference pose; and a pose estimator for estimating a pose of the camera by tracking the model of the features.

Abstract: An interaction/input device for use in systems utilizing infrared video-based trackers is provided. The interaction/input device is employed by placing markers, e.g., small retroreflective disks, in a predetermined configuration on a planar surface to simulate physical buttons. Once in view of the infrared tracker, the system will recognize the predetermined configuration as the input device which can then be physically manipulated by the user. The user will put their fingers on one of these markers or disks, and in turn, the infrared video-based tracker will recognize this action and process it accordingly. Optionally, an augmented reality system employing the input device of the present invention can also augment simulated menus in the user's view, via a head-mounted display, giving the user the necessary feedback for interaction.

Abstract: Superimposed X-ray and video images can be obtained by acquiring the respective images from the optically equivalent points in space. One or more mirrors may be used to acquire the images and direct them towards the camera. The images can then be combined by warping one onto the other.

Abstract: Superimposed X-ray and video images can be obtained by acquiring the respective images from the optically equivalent points in space. One or more mirrors may be used to acquire the images. Alignment of one camera with respect to the X-ray source may be achieved using images of reference points in space. The procedure of alignment also provides correlated X-ray and optical images.

Abstract: A system and method for calibrating a stereo optical see-through HMD (head-mounted display). A preferred method integrates measurement for an optical see-through HMD and a six degrees of freedom tracker that is fixedly attached to the HMD to perform calibration. Calibration is based on the alignment of a stereoscopically fused marker image, which is perceived in depth, with a single 3D reference point in a world coordinate system from various viewpoints. The user interaction to perform the calibration is extremely easy compared to conventional methods and does not require keeping the head static during the calibration process.

Abstract: A system and method for calibrating a monocular optical see-through HMD (head-mounted display). A preferred method integrates measurement for an optical see-through HMD and a six degrees of freedom tracker that is fixedly attached to the HMD to perform calibration. Calibration is based on the alignment of image points with a single 3D reference point in a world coordinate system from various viewpoints. The user interaction to perform the calibration is extremely easy compared to conventional methods and does not require keeping the head static during the calibration process. In one aspect, a calibration method comprises the steps of: displaying a marker image on one display of an HMD for view by a user; aligning the displayed marker image with a preselected reference point; collecting calibration data associated with the alignment; and computing a model of the HMD using the collected calibration data.

Abstract: A video editing system or tool for E-commerce utilizing augmented reality (AR) technology combines real and virtual worlds together to provide an interface for a user to sense and interact with virtual objects in the real world. The AR video editing system is usable in conjunction with an ordinary desktop computer and a low-cost parallel port camera. A known camera calibration algorithm is utilized together with a set of specially designed markers for camera calibration and pose estimation of the markers. OpenGL and VRML (Virtual Reality Modeling Language) for 3D virtual model rendering and superimposition. are utilized. Marker-based calibration is utilized to calibrate the camera and estimate the pose of the markers in the AR video editing system. The system comprises video input/output, image feature extraction and marker recognition, camera calibration/pose estimation, and virtual reality (VR) model rendering/augmentation.

Abstract: A system and method for illuminating a target point in a real scene comprises an image capture device for capturing image data of a scene, an illumination device for projecting a beam of light at a target point in the scene; and a data processing device comprising computer readable program code embodied therein for processing image data associated with the target point and generating control signals to control the illumination system to direct a beam of light at the target point in the scene. Preferably, the imaging system and illumination system comprise an integrated system wherein the optical path for image formation is identical to that of light projection, so as to eliminate occlusion and eliminate the need for calibration.

Abstract: A method and system for computer assisted site navigation comprises the steps of capturing an image of a scene for providing real-time image data; calibrating the camera and localizing the image by utilizing the image data and retrieved marker registration and floor-map data for providing position and orientation data; transmitting the data to a web server; utilizing the image data and information on the scene retrieved from a database for deriving an augmented view; and displaying the augmented view.

Abstract: A method for determining the best entry point for a percutaneous procedure, such as with a biopsy needle, comprises selecting first and second arbitrary entry points on a patient; determining the three dimensional (3-D) orientation of the needle at the first arbitrary entry point for pointing the needle at the primary target; determining the 3-D orientation of the needle at the first arbitrary entry point for pointing the needle at the secondary target; determining the 3-D dimensional orientation of the needle at the second arbitrary entry point for pointing the needle at the primary target; determining the 3-D orientation of the needle at the second arbitrary entry point for pointing the needle at the secondary target; determining a 3-D line representing the intersection of a first plane containing the first arbitrary entry point, the primary target point, and the secondary target point, and a second plane containing the second arbitrary entry point, the primary target, and the secondary target point, whereb