Abstract: In one embodiment of the invention, a digital zoom and panning system for digital video is disclosed including an image acquisition device to capture digital video images; an image buffer to store one or more frames of digital video images as source pixels; a display device having first pixels to display images; a user interface to accept user input including a source rectangle to select source pixels within frames of the digital video images, a destination rectangle to select target pixels within the display device to display images, and a region of interest within the digital video images to display in the destination rectangle; and a digital mapping and filtering device to selectively map and filter source pixels in the region of interest from the image buffer into target pixels of the display device in response to the user input.

Abstract: In one embodiment of the invention, a method is disclosed to locate a robotic instrument in the field of view of a camera. The method includes capturing sequential images in a field of view of a camera. The sequential images are correlated between successive views. The method further includes receiving a kinematic datum to provide an approximate location of the robotic instrument and then analyzing the sequential images in response to the approximate location of the robotic instrument. An additional method for robotic systems is disclosed. Further disclosed is a method for indicating tool entrance into the field of view of a camera.

Abstract: In one embodiment of the invention, a method for a robotic system is disclosed to track one or more robotic instruments. The method includes generating kinematics information for the robotic instrument within a field of view of a camera; capturing image information in the field of view of the camera; and adaptively fusing the kinematics information and the image information together to determine pose information of the robotic instrument. Additionally disclosed is a robotic medical system with a tool tracking sub-system. The tool tracking sub-system receives raw kinematics information and video image information of the robotic instrument to generate corrected kinematics information for the robotic instrument by adaptively fusing the raw kinematics information and the video image information together.

Abstract: In one embodiment of the invention, a method is disclosed to locate a robotic instrument in the field of view of a camera. The method includes capturing sequential images in a field of view of a camera. The sequential images are correlated between successive views. The method further includes receiving a kinematic datum to provide an approximate location of the robotic instrument and then analyzing the sequential images in response to the approximate location of the robotic instrument. An additional method for robotic systems is disclosed. Further disclosed is a method for indicating tool entrance into the field of view of a camera.

Abstract: A minimally invasive surgical includes a scene anti-bloom process that allows switching between imaging modes on a stereoscopic display without causing a surgeon to look-away or being momentarily distracted by sudden changes in overall scene luminance. The process receives a switch from a first imaging mode to a second imaging mode. An overall scene luminance of a scene in the first imaging mode is less than an overall scene luminance of a scene in the second imaging mode. The process delays the switch to the second imaging mode until after an illumination output level of a visible illumination source has changed to a higher output level, and then switches to the second imaging mode.

Abstract: In one embodiment of the invention, a tool tracking system is disclosed including a computer usable medium having computer readable program code to receive images of video frames from at least one camera and to perform image matching of a robotic instrument to determine video pose information of the robotic instrument within the images. The tool tracking system further includes computer readable program code to provide a state-space model of a sequence of states of corrected kinematics information for accurate pose information of the robotic instrument. The state-space model receives raw kinematics information of mechanical pose information and adaptively fuses the mechanical pose information and the video pose information together to generate the sequence of states of the corrected kinematics information for the robotic instrument. Additionally disclosed are methods for image guided surgery.

Abstract: Systems, methods, and devices are used to match images. Points of interest from a first image are identified for matching to a second image. In response to the identified points of interest, regions and features can be identified and used to match the points of interest to a corresponding second image or second series of images. Regions can be used to match the points of interest when regions of the first image are matched to the second image with high confidence scores, for example above a threshold. Features of the first image can be matched to the second image, and these matched features may be used to match the points of interest to the second image, for example when the confidence scores for the regions are below the threshold value. Constraint can be used to evaluate the matched points of interest, for example by excluding bad points.

Abstract: The present disclosure relates to systems, methods, and tools for tool tracking using image-derived data from one or more tool-located reference features. A method includes: capturing a first image of a tool that includes multiple features that define a first marker, where at least one of the features of the first marker includes an identification feature; determining a position for the first marker by processing the first image; determining an identification for the first marker by using the at least one identification feature by processing the first image; and determining a tool state for the tool by using the position and the identification of the first marker.

Abstract: An apparatus is configured to show telestration in 3-D to a surgeon in real time. A proctor is shown one side of a stereo image pair, such that the proctor can draw a telestration line on the one side with an input device. Points of interest are identified for matching to the other side of the stereo image pair. In response to the identified points of interest, regions and features are identified and used to match the points of interest to the other side. Regions can be used to match the points of interest. Features of the first image can be matched to the second image and used to match the points of interest to the second image, for example when the confidence scores for the regions are below a threshold value. Constraints can be used to evaluate the matched points of interest, for example by excluding bad points.

Abstract: An endoscope with a stereoscopic optical channel is held and positioned by a robotic surgical system. A capture unit captures (1) a visible first image and (2) a visible second image combined with a fluorescence second image from the light. An intelligent image processing system receives (1) the visible first image and (2) the visible second image combined with the fluorescence second image and generates at least one fluorescence image of a stereoscopic pair of fluorescence images and a visible second image. An augmented stereoscopic display system outputs a real-time stereoscopic image including a three-dimensional presentation including in one eye, a blend of the at least one fluorescence image of a stereoscopic pair of fluorescence images and one of the visible first and second images; and in the other eye, the other of the visible first and second images.

Abstract: An illumination channel, a stereoscopic optical channel and another optical channel are held and positioned by a robotic surgical system. A first capture unit captures a stereoscopic visible image from the first light from the stereoscopic optical channel while a second capture unit captures a fluorescence image from the second light from the other optical channel. An intelligent image processing system receives the captured stereoscopic visible image and the captured fluorescence image and generates a stereoscopic pair of fluorescence images. An augmented stereoscopic display system outputs a real-time stereoscopic image comprising a three-dimensional presentation of a blend of the stereoscopic visible image and the stereoscopic pair of fluorescence images.

Abstract: An endoscope with a stereoscopic optical channel is again held and positioned by a robotic surgical system. A capture unit captures (1) at a first time, a first image from light from the channel; and (2) at a second time different from the first time, a second image from the light. Only one of the first image and the second image includes a combination of a fluorescence image and a visible image. The other of the first image and the second image is a visible image. An intelligent image processing system generates an artificial fluorescence image using the captured fluorescence image. An augmented stereoscopic display system outputs an augmented stereoscopic image of at least a portion of the tissue comprising the artificial fluorescence image.

Abstract: A robotic surgical system positions and holds an endoscope. A visible imaging system is coupled to the endoscope. The visible imaging system captures a visible image of tissue. An alternate imaging system is also coupled to the endoscope. The alternate imaging system captures a fluorescence image of at least a portion of the tissue. A stereoscopic video display system is coupled to the visible imaging system and to the alternate imaging system. The stereoscopic video display system outputs a real-time stereoscopic image comprising a three-dimensional presentation of a blend of a fluorescence image associated with the captured fluorescence image, and the visible image.

Abstract: An endoscope with a stereoscopic optical channel is held and positioned by a robotic surgical system. A first capture unit captures: a visible first color component of a visible left image combined with a fluorescence left image from first light from one channel in the endoscope; a visible second color component of the visible left image from the first light; and a visible third color component of the visible left image from the first light. A second capture unit captures: a visible first color component of a visible right image combined with a fluorescence right image from second light from the other channel in the endoscope; a visible second color component of the visible right image from the second light; and a visible third color component of the visible right image from the second light. An augmented stereoscopic outputs a real-time stereoscopic image including a three-dimensional presentation including the visible left and right images and the fluorescence left and right images.

Abstract: In one embodiment of the invention, a method for controlling a robotic surgical tool is disclosed. The method for controlling a robotic surgical tool includes moving a monitor displaying an image of a robotic surgical tool; sensing motion of the monitor; and translating the sensed motion of the monitor into motion of the robotic surgical tool.

Abstract: A synthetic representation of a tool for display on a user interface of a robotic system. The synthetic representation may be used to show force on the tool, an actual position of the tool, or to show the location of the tool when out of a field of view. A three-dimensional pointer is also provided for a viewer in the surgeon console of a telesurgical system.

Abstract: A synthetic representation of a robot tool for display on a user interface of a robotic system. The synthetic representation may be used to show the position of a view volume of an image capture device with respect to the robot. The synthetic representation may also be used to find a tool that is outside of the field of view, to display range of motion limits for a tool, to remotely communicate information about the robot, and to detect collisions.

Abstract: In one embodiment of the invention, a tool tracking system is disclosed including a computer usable medium having computer readable program code to receive images of video frames from at least one camera and to perform image matching of a robotic instrument to determine video pose information of the robotic instrument within the images. The tool tracking system further includes computer readable program code to provide a state-space model of a sequence of states of corrected kinematics information for accurate pose information of the robotic instrument. The state-space model receives raw kinematics information of mechanical pose information and adaptively fuses the mechanical pose information and the video pose information together to generate the sequence of states of the corrected kinematics information for the robotic instrument. Additionally disclosed are methods for image guided surgery.

Abstract: In one embodiment of the invention, a method for a robotic system is disclosed to track one or more robotic instruments. The method includes generating kinematics information for the robotic instrument within a field of view of a camera; capturing image information in the field of view of the camera; and adaptively fusing the kinematics information and the image information together to determine pose information of the robotic instrument. Additionally disclosed is a robotic medical system with a tool tracking sub-system. The tool tracking sub-system receives raw kinematics information and video image information of the robotic instrument to generate corrected kinematics information for the robotic instrument by adaptively fusing the raw kinematics information and the video image information together.

Abstract: In one embodiment of the invention, a method is disclosed to locate a robotic instrument in the field of view of a camera. The method includes capturing sequential images in a field of view of a camera. The sequential images are correlated between successive views. The method further includes receiving a kinematic datum to provide an approximate location of the robotic instrument and then analyzing the sequential images in response to the approximate location of the robotic instrument. An additional method for robotic systems is disclosed. Further disclosed is a method for indicating tool entrance into the field of view of a camera.