Abstract: In one embodiment of the invention, a digital zoom and panning system for digital video is disclosed including an image acquisition device to capture digital video images; an image buffer to store one or more frames of digital video images as source pixels; a display device having first pixels to display images; a user interface to accept user input including a source rectangle to select source pixels within frames of the digital video images, a destination rectangle to select target pixels within the display device to display images, and a region of interest within the digital video images to display in the destination rectangle; and a digital mapping and filtering device to selectively map and filter source pixels in the region of interest from the image buffer into target pixels of the display device in response to the user input.

Abstract: In one embodiment of the invention, a method is disclosed to locate a robotic instrument in the field of view of a camera. The method includes capturing sequential images in a field of view of a camera. The sequential images are correlated between successive views. The method further includes receiving a kinematic datum to provide an approximate location of the robotic instrument and then analyzing the sequential images in response to the approximate location of the robotic instrument. An additional method for robotic systems is disclosed. Further disclosed is a method for indicating tool entrance into the field of view of a camera.

Abstract: In one embodiment of the invention, a method is disclosed to locate a robotic instrument in the field of view of a camera. The method includes capturing sequential images in a field of view of a camera. The sequential images are correlated between successive views. The method further includes receiving a kinematic datum to provide an approximate location of the robotic instrument and then analyzing the sequential images in response to the approximate location of the robotic instrument. An additional method for robotic systems is disclosed. Further disclosed is a method for indicating tool entrance into the field of view of a camera.

Abstract: In a minimally invasive surgical system, an illuminator includes a visible color component illumination source and a hardware non-visible fluorescence emission illumination source. Thus, the illuminator outputs target image illumination light in a first spectrum where the first spectrum includes at least a portion of the visible spectrum. The illuminator also outputs target image illumination light in a second spectrum, where the second spectrum includes non-visible light with a wavelength the same as a wavelength in an emission from a fluorophore.

Abstract: A surgical site is simultaneously illuminated by less than all the visible color components that make up visible white light, and a fluorescence excitation illumination component by an illuminator in a minimally invasive surgical system. An image capture system acquires an image for each of the visible color components illuminating the surgical site and a fluorescence image, which is excited by the fluorescence excitation component from the illuminator. The minimally invasive surgical system uses the acquired images to generate a background black and white image of the surgical site. The acquired fluorescence image is superimposed on the background black and white image, and is highlighted in a selected color, e.g., green. The background black and white image with the superimposed highlighted fluorescence image is displayed for a user of the system. The highlighted fluorescence image identifies tissue of clinical interest.

Abstract: In one embodiment of the invention, a tool tracking system is disclosed including a computer usable medium having computer readable program code to receive images of video frames from at least one camera and to perform image matching of a robotic instrument to determine video pose information of the robotic instrument within the images. The tool tracking system further includes computer readable program code to provide a state-space model of a sequence of states of corrected kinematics information for accurate pose information of the robotic instrument. The state-space model receives raw kinematics information of mechanical pose information and adaptively fuses the mechanical pose information and the video pose information together to generate the sequence of states of the corrected kinematics information for the robotic instrument. Additionally disclosed are methods for image guided surgery.

Abstract: Robotic and/or measurement devices, systems, and methods for telesurgical and other applications employ input devices operatively coupled to tools so as to allow a system user to manipulate tissues and other structures being measured. The system may make use of three dimensional position information from stereoscopic images. Two or more discrete points can be designated in three dimensions so as to provide a cumulative length along a straight or curving structure, an area measurement, a volume measurement, or the like. The discrete points may be identified by a single surgical tool or by distances separating two or more surgical tools, with the user optionally measuring a structure longer than a field of view of the stereoscopic image capture device by walking a pair of tools “hand-over-hand” along the structure.

Abstract: Robotic and/or measurement devices, systems, and methods for telesurgical and other applications employ input devices operatively coupled to tools so as to allow a system user to manipulate tissues and other structures being measured. The system may make use of three dimensional position information from stereoscopic images. Two or more discrete points can be designated in three dimensions so as to provide a cumulative length along a straight or curving structure, an area measurement, a volume measurement, or the like. The discrete points may be identified by a single surgical tool or by distances separating two or more surgical tools, with the user optionally measuring a structure longer than a field of view of the stereoscopic image capture device by walking a pair of tools “hand-over-hand” along the structure.

Abstract: In one embodiment, a surgical instrument includes a housing linkable with a manipulator arm of a robotic surgical system, a shaft operably coupled to the housing, a force transducer on a distal end of the shaft, and a plurality of fiber optic strain gauges on the force transducer. In one example, the plurality of strain gauges are operably coupled to a fiber optic splitter or an arrayed waveguide grating (AWG) multiplexer. A fiber optic connector is operably coupled to the fiber optic splitter or the AWG multiplexer. A wrist joint is operably coupled to a distal end of the force transducer, and an end effector is operably coupled to the wrist joint. In another embodiment, a robotic surgical manipulator includes a base link operably coupled to a distal end of a manipulator positioning system, and a distal link movably coupled to the base link, wherein the distal link includes an instrument interface and a fiber optic connector optically linkable to a surgical instrument.

Abstract: An apparatus is configured to show telestration in 3-D to a surgeon in real time. A proctor is shown one side of a stereo image pair, such that the proctor can draw a telestration line on the one side with an input device. Points of interest are identified for matching to the other side of the stereo image pair. In response to the identified points of interest, regions and features are identified and used to match the points of interest to the other side. Regions can be used to match the points of interest. Features of the first image can be matched to the second image and used to match the points of interest to the second image, for example when the confidence scores for the regions are below a threshold value. Constraints can be used to evaluate the matched points of interest, for example by excluding bad points.

Abstract: A robotic surgical system positions and holds an endoscope. A visible imaging system is coupled to the endoscope. The visible imaging system captures a visible image of tissue. An alternate imaging system is also coupled to the endoscope. The alternate imaging system captures a fluorescence image of at least a portion of the tissue. A stereoscopic video display system is coupled to the visible imaging system and to the alternate imaging system. The stereoscopic video display system outputs a real-time stereoscopic image comprising a three-dimensional presentation of a blend of a fluorescence image associated with the captured fluorescence image, and the visible image.

Abstract: An endoscope with a stereoscopic optical channel is again held and positioned by a robotic surgical system. A capture unit captures (1) at a first time, a first image from light from the channel; and (2) at a second time different from the first time, a second image from the light. Only one of the first image and the second image includes a combination of a fluorescence image and a visible image. The other of the first image and the second image is a visible image. An intelligent image processing system generates an artificial fluorescence image using the captured fluorescence image. An augmented stereoscopic display system outputs an augmented stereoscopic image of at least a portion of the tissue comprising the artificial fluorescence image.

Abstract: An illumination channel, a stereoscopic optical channel and another optical channel are held and positioned by a robotic surgical system. A first capture unit captures a stereoscopic visible image from the first light from the stereoscopic optical channel while a second capture unit captures a fluorescence image from the second light from the other optical channel. An intelligent image processing system receives the captured stereoscopic visible image and the captured fluorescence image and generates a stereoscopic pair of fluorescence images. An augmented stereoscopic display system outputs a real-time stereoscopic image comprising a three-dimensional presentation of a blend of the stereoscopic visible image and the stereoscopic pair of fluorescence images.

Abstract: An apparatus, system, and method for improving force and torque sensing and feedback to the surgeon performing a telerobotic surgery are provided. In one embodiment, a robotic surgical manipulator system, a robotic surgical system, and a method for improved sensing of forces on a robotic surgical instrument and/or manipulator arm are disclosed.

Abstract: In one embodiment of the invention, a digital zoom and panning system for digital video is disclosed including an image acquisition device to capture digital video images; an image buffer to store one or more frames of digital video images as source pixels; a display device having first pixels to display images; a user interface to accept user input including a source rectangle to select source pixels within frames of the digital video images, a destination rectangle to select target pixels within the display device to display images, and a region of interest within the digital video images to display in the destination rectangle; and a digital mapping and filtering device to selectively map and filter source pixels in the region of interest from the image buffer into target pixels of the display device in response to the user input.

Abstract: A synthetic representation of a robot tool for display on a user interface of a robotic system. The synthetic representation may be used to show the position of a view volume of an image capture device with respect to the robot. The synthetic representation may also be used to find a tool that is outside of the field of view, to display range of motion limits for a tool, to remotely communicate information about the robot, and to detect collisions.

Abstract: A haptic feedback control device, such as a handheld remote control or handheld game controller, for controlling a graphical object within a graphical display and for outputting forces to a user. A housing includes a button, wherein the user engages the button with a finger. The button is depressible along a degree of freedom by the user. An actuator applies forces to the user through the button along the degree of freedom. A sensor detects displacement of the button along the degree of freedom when the button is depressed by the user. A process, local to the device, controls the actuator to generate the forces upon the button in the degree of freedom to provide a tactile sensation to the user contacting the button.

Abstract: In one embodiment, a force sensor apparatus is provided including a tube portion having a plurality of radial ribs and a strain gauge positioned over each of the plurality of radial ribs, a proximal end of the tube portion that operably couples to a shaft of a surgical instrument that operably couples to a manipulator arm of a robotic surgical system, and a distal end of the tube portion that proximally couples to a wrist joint coupled to an end effector.

Abstract: In one embodiment of the invention, a tool tracking system is disclosed including a computer usable medium having computer readable program code to receive images of video frames from at least one camera and to perform image matching of a robotic instrument to determine video pose information of the robotic instrument within the images. The tool tracking system further includes computer readable program code to provide a state-space model of a sequence of states of corrected kinematics information for accurate pose information of the robotic instrument. The state-space model receives raw kinematics information of mechanical pose information and adaptively fuses the mechanical pose information and the video pose information together to generate the sequence of states of the corrected kinematics information for the robotic instrument. Additionally disclosed are methods for image guided surgery.

Abstract: In one embodiment of the invention, a method for a robotic system is disclosed to track one or more robotic instruments. The method includes generating kinematics information for the robotic instrument within a field of view of a camera; capturing image information in the field of view of the camera; and adaptively fusing the kinematics information and the image information together to determine pose information of the robotic instrument. Additionally disclosed is a robotic medical system with a tool tracking sub-system. The tool tracking sub-system receives raw kinematics information and video image information of the robotic instrument to generate corrected kinematics information for the robotic instrument by adaptively fusing the raw kinematics information and the video image information together.