Abstract: A surgical robot system includes a robot. The robot includes a robot base and a robot arm coupled to the robot base. The robot also includes an end-effector coupled to the robot arm. The robot is configured to control movement of the end-effector to perform a surgical procedure. The robot also includes an inertial measurement unit coupled to the robot arm. The surgical robot system also includes camera that is configured to capture one or more pictures or videos used to determine a location of the end-effector. The inertial measurement unit is configured to capture one or more measurements used to determine the location of the end-effector when a view of the camera is occluded.

Abstract: A system and method for tracking positions of objects in an actual scene in a wide range of lighting conditions. A vision-inertial navigation system is provided that may be configured to track relative positions of objects of interest in lighting conditions ranging from, e.g., typical indoor lighting conditions to direct sunlight to caves with virtually with no light. The navigation system may include two or more helmet-mounted cameras each configured to capture images of the actual scene in a different lighting condition. A remote processing element may combine data from the two or more helmet-mounted cameras with data from a helmet-mounted inertial sensor to estimate the position (e.g., 3D location) of the objects.

Abstract: An apparatus, system, and method for dual mode imaging under different lighting conditions. A sensor is configured to image a target. A dual-mode illumination source is configured to illuminate the target while the sensor images the target. The dual-mode illumination source is configured to illuminate the target using a first wavelength of light under a first lighting condition and to illuminate the target using a second wavelength of light under a second lighting condition. The system may be used in an optical tracking system to track the motion of an object.

Abstract: A method, a system, and a computer-readable medium for tracking position and orientation of a pedestrian. The system tracks a head position or head orientation of the pedestrian, and a foot position or foot orientation of the pedestrian. The system determines a first heading or position uncertainty associated with the head position or the head orientation of the pedestrian, and determines a second heading or position uncertainty associated with the foot position or the foot orientation of the pedestrian. Moreover, the system determines which of the first heading or position uncertainty or the second heading or position uncertainty is smaller. The system transfers the first heading or position uncertainty to a device or system portion for foot position, or transfers the second heading or position uncertainty to a device or system portion for head position.

Abstract: An augmented reality surgical system includes a head mounted display (HMD) with a see-through display screen, a motion sensor, a camera, and computer equipment. The motion sensor outputs a head motion signal indicating measured movement of the HMD. The computer equipment computes the relative location and orientation of reference markers connected to the HMD and to the patient based on processing a video signal from the camera. The computer equipment generates a three dimensional anatomical model using patient data created by medical imaging equipment, and rotates and scales at least a portion of the three dimensional anatomical model based on the relative location and orientation of the reference markers, and further rotate at least a portion of the three dimensional anatomical model based on the head motion signal to track measured movement of the HMD. The rotated and scaled three dimensional anatomical model is displayed on the display screen.

Abstract: An augmented reality surgical system includes a head mounted display (HMD) with a see-through display screen, a motion sensor, a camera, and computer equipment. The motion sensor outputs a head motion signal indicating measured movement of the HMD. The computer equipment computes the relative location and orientation of reference markers connected to the HMD and to the patient based on processing a video signal from the camera. The computer equipment generates a three dimensional anatomical model using patient data created by medical imaging equipment, and rotates and scales at least a portion of the three dimensional anatomical model based on the relative location and orientation of the reference markers, and further rotate at least a portion of the three dimensional anatomical model based on the head motion signal to track measured movement of the HMD. The rotated and scaled three dimensional anatomical model is displayed on the display screen.

Abstract: An augmented reality surgical system includes a head mounted display (HMD) with a see-through display screen, a motion sensor, a camera, and computer equipment. The motion sensor outputs a head motion signal indicating measured movement of the HMD. The computer equipment computes the relative location and orientation of reference markers connected to the HMD and to the patient based on processing a video signal from the camera. The computer equipment generates a three dimensional anatomical model using patient data created by medical imaging equipment, and rotates and scales at least a portion of the three dimensional anatomical model based on the relative location and orientation of the reference markers, and further rotate at least a portion of the three dimensional anatomical model based on the head motion signal to track measured movement of the HMD. The rotated and scaled three dimensional anatomical model is displayed on the display screen.

Abstract: An augmented reality surgical system includes a head mounted display (HMD) with a see-through display screen, a motion sensor, a camera, and computer equipment. The motion sensor outputs a head motion signal indicating measured movement of the HMD. The computer equipment computes the relative location and orientation of reference markers connected to the HMD and to the patient based on processing a video signal from the camera. The computer equipment generates a three dimensional anatomical model using patient data created by medical imaging equipment, and rotates and scales at least a portion of the three dimensional anatomical model based on the relative location and orientation of the reference markers, and further rotate at least a portion of the three dimensional anatomical model based on the head motion signal to track measured movement of the HMD. The rotated and scaled three dimensional anatomical model is displayed on the display screen.

Abstract: A system and method for tracking positions of objects in an actual scene in a wide range of lighting conditions. A vision-inertial navigation system is provided that may be configured to track relative positions of objects of interest in lighting conditions ranging from, e.g., typical indoor lighting conditions to direct sunlight to caves with virtually with no light. The navigation system may include two or more helmet-mounted cameras each configured to capture images of the actual scene in a different lighting condition. A remote processing element may combine data from the two or more helmet-mounted cameras with data from a helmet-mounted inertial sensor to estimate the position (e.g., 3D location) of the objects.

Abstract: An apparatus, system, and method for dual mode imaging under different lighting conditions. A sensor is configured to image a target. A dual-mode illumination source is configured to illuminate the target while the sensor images the target. The dual-mode illumination source is configured to illuminate the target using a first wavelength of light under a first lighting condition and to illuminate the target using a second wavelength of light under a second lighting condition. The system may be used in an optical tracking system to track the motion of an object.

Abstract: An augmented reality surgical system includes a head mounted display (HMD) with a see-through display screen, a motion sensor, a camera, and computer equipment. The motion sensor outputs a head motion signal indicating measured movement of the HMD. The computer equipment computes the relative location and orientation of reference markers connected to the HMD and to the patient based on processing a video signal from the camera. The computer equipment generates a three dimensional anatomical model using patient data created by medical imaging equipment, and rotates and scales at least a portion of the three dimensional anatomical model based on the relative location and orientation of the reference markers, and further rotate at least a portion of the three dimensional anatomical model based on the head motion signal to track measured movement of the HMD. The rotated and scaled three dimensional anatomical model is displayed on the display screen.