Abstract: A system for autonomous camera control includes a first robot having a surgical tool mounted as an end effector and a second robot having a camera mounted as an end effector. A controller may be provided for manipulating the second robot, where the controller stores a first kinematic model for the first robot and a second kinematic model for the second robot. The controller may be configured to automatically manipulate the second robot to position the camera based on the second kinematic model and an expected position of the surgical tool according the first kinematic model of the first robot. The controller is further configured to identify a threshold angle from a viewing axis of the camera, calculate a tool angle from the viewing axis of the camera, and move the camera toward or further away from the tool depending on if the tool angle is greater than or less than the threshold angle.

Abstract: A system for autonomous camera control is provided. The system may include a first robot having a surgical tool mounted as an end effector and a second robot having a camera mounted as an end effector. A controller may be provided for manipulating the second robot, where the controller stores a first kinematic model for the first robot and a second kinematic model for the second robot. The controller may be configured to automatically manipulate the second robot to position the camera based on the second kinematic model and an expected position of the surgical tool according the first kinematic model of the first robot.

Abstract: A system for generating an augmented reality image comprises a video camera for obtaining video data and a sensor for obtaining sensed data. An augmented reality processor is coupled to the video camera and to the sensor. The augmented reality processor is configured to receive the video data from the video camera and to receive the sensed data from the sensor and to receive other types of computed data, such as any imaging modality. A display device is coupled to the augmented reality processor. The augmented reality processor is further configured to generate for display on the display device a video image from the video data received from the video camera and to generate a corresponding data image from the sensed data received from the sensor or other computed data. The augmented reality processor is further configured to merge the video image and the corresponding data image generated from computed data or sensed data so as to generate the augmented reality image.