Abstract: A stereo imaging system comprises a stereoscopic camera having left and right image capturing elements for capturing stereo images; a stereo viewer; and a processor configured to modify the stereo images prior to being displayed on the stereo viewer so that a disparity between corresponding points of the stereo images is adjusted as a function of a depth value within a region of interest in the stereo images after the depth value reaches a target depth value.

Abstract: Methods of and a system for providing force information for a robotic surgical system. The method includes storing first kinematic position information and first actual position information for a first position of an end effector; moving the end effector via the robotic surgical system from the first position to a second position; storing second kinematic position information and second actual position information for the second position; and providing force information regarding force applied to the end effector at the second position utilizing the first actual position information, the second actual position information, the first kinematic position information, and the second kinematic position information. Visual force feedback is also provided via superimposing an estimated position of an end effector without force over an image of the actual position of the end effector. Similarly, tissue elasticity visual displays may be shown.

Abstract: In one embodiment of the invention, an apparatus is disclosed including an image sensor, a color filter array, and an image processor. The image sensor has an active area with a matrix of camera pixels. The color filter array is in optical alignment over the matrix of the camera pixels. The color filter array assigns alternating single colors to each camera pixel. The image processor receives the camera pixels and includes a correlation detector to detect spatial correlation of color information between pairs of colors in the pixel data captured by the camera pixels. The correlation detector further controls demosaicing of the camera pixels into full color pixels with improved resolution. The apparatus may further include demosaicing logic to demosaic the camera pixels into the full color pixels with improved resolution in response to the spatial correlation of the color information between pairs of colors.

Abstract: An operator telerobotically controls tools to perform a procedure on an object at a work site while viewing real-time images of the work site on a display. Tool information is provided in the operator's current gaze area on the display by rendering the tool information over the tool so as not to obscure objects being worked on at the time by the tool nor to require eyes of the user to refocus when looking at the tool information and the image of the tool on a stereo viewer.

Abstract: Methods of and a system for providing a visual representation of force information in a robotic surgical system. A real position of a surgical end effector is determined. A projected position of the surgical end effector if no force were applied against the end effector is also determined. Images representing the real and projected positions are output superimposed on a display. The offset between the two images provides a visual indication of a force applied to the end effector or to the kinematic chain that supports the end effector. In addition, tissue deformation information is determined and displayed.

Abstract: A bleeding detection unit in a surgical system processes information in an acquired scene before that scene is presented on a display unit in the operating room. For example, the bleeding detection unit analyzes the pixel data in the acquired scene and determines whether there are one or more initial sites of blood in the scene. Upon detection of an initial site of blood, the region is identified by an initial site icon in the scene displayed on the display unit. In one aspect, the processing is done in real-time which means that there is no substantial delay in presenting the acquired scene to the surgeon.

Abstract: Systems, methods, and devices are used to match images. Points of interest from a first image are identified for matching to a second image. In response to the identified points of interest, regions and features can be identified and used to match the points of interest to a corresponding second image or second series of images. Regions can be used to match the points of interest when regions of the first image are matched to the second image with high confidence scores, for example above a threshold. Features of the first image can be matched to the second image, and these matched features may be used to match the points of interest to the second image, for example when the confidence scores for the regions are below the threshold value. Constraint can be used to evaluate the matched points of interest, for example by excluding bad points.

Abstract: Methods of and a system for providing a visual representation of force information in a robotic surgical system. A real position of a surgical end effector is determined. A projected position of the surgical end effector if no force were applied against the end effector is also determined. Images representing the real and projected positions are output superimposed on a display. The offset between the two images provides a visual indication of a force applied to the end effector or to the kinematic chain that supports the end effector. In addition, tissue deformation information is determined and displayed.

Abstract: Mixed mode imaging is implemented using a single-chip image capture sensor with a color filter array. The single-chip image capture sensor captures a frame including a first set of pixel data and a second set of pixel data. The first set of pixel data includes a first combined scene, and the second set of pixel data includes a second combined scene. The first combined scene is a first weighted combination of a fluorescence scene component and a visible scene component due to the leakage of a color filter array. The second combined scene includes a second weighted combination of the fluorescence scene component and the visible scene component. Two display scene components are extracted from the captured pixel data in the frame and presented on a display unit.

Abstract: Mixed mode imaging is implemented using a single-chip image capture sensor with a color filter array. The single-chip image capture sensor captures a frame including a first set of pixel data and a second set of pixel data. The first set of pixel data includes a first combined scene, and the second set of pixel data includes a second combined scene. The first combined scene is a first weighted combination of a fluorescence scene component and a visible scene component due to the leakage of a color filter array. The second combined scene includes a second weighted combination of the fluorescence scene component and the visible scene component. Two display scene components are extracted from the captured pixel data in the frame and presented on a display unit.

Abstract: Stereo gaze tracking estimates a 3-D gaze point by projecting determined right and left eye gaze points on left and right stereo images. The determined right and left eye gaze points are based on one or more tracked eye gaze points, estimates for non-tracked eye gaze points based upon the tracked gaze points and image matching in the left and right stereo images, and confidence scores indicative of the reliability of the tracked gaze points and/or the image matching.

Abstract: Methods of and a system for providing force information for a robotic surgical system. The method includes storing first kinematic position information and first actual position information for a first position of an end effector; moving the end effector via the robotic surgical system from the first position to a second position; storing second kinematic position information and second actual position information for the second position; and providing force information regarding force applied to the end effector at the second position utilizing the first actual position information, the second actual position information, the first kinematic position information, and the second kinematic position information. Visual force feedback is also provided via superimposing an estimated position of an end effector without force over an image of the actual position of the end effector. Similarly, tissue elasticity visual displays may be shown.

Abstract: Systems, methods, and devices are used to match images. Points of interest from a first image are identified for matching to a second image. In response to the identified points of interest, regions and features can be identified and used to match the points of interest to a corresponding second image or second series of images. Regions can be used to match the points of interest when regions of the first image are matched to the second image with high confidence scores, for example above a threshold. Features of the first image can be matched to the second image, and these matched features may be used to match the points of interest to the second image, for example when the confidence scores for the regions are below the threshold value. Constraint can be used to evaluate the matched points of interest, for example by excluding bad points.

Abstract: In one embodiment of the invention, an apparatus is disclosed including an image sensor, a color filter array, and an image processor. The image sensor has an active area with a matrix of camera pixels. The color filter array is in optical alignment over the matrix of the camera pixels. The color filter array assigns alternating single colors to each camera pixel. The image processor receives the camera pixels and includes a correlation detector to detect spatial correlation of color information between pairs of colors in the pixel data captured by the camera pixels. The correlation detector further controls demosaicing of the camera pixels into full color pixels with improved resolution. The apparatus may further include demosaicing logic to demosaic the camera pixels into the full color pixels with improved resolution in response to the spatial correlation of the color information between pairs of colors.

Abstract: A local contrast enhancement method transforms a first plurality of color components of a first visual color image into a modified brightness component by using a first transformation. The first plurality of color components are in a first color space. The modified brightness component is a brightness component of a second color space. The second color space also includes a plurality of chromatic components. The method transforms all the color components of the first color space into the chromatic components of the second color space. The method then transforms the modified brightness component and the chromatic components of the second color space into a plurality of new color components, in the first color space, of a second visual color image. The method transmits the plurality of new color components to a device such as a display device. The second visual color image has enhanced contrast in comparison to the first visual color image.

Abstract: An illumination channel, a stereoscopic optical channel and another optical channel are held and positioned by a robotic surgical system. A first capture unit captures a stereoscopic visible image from the first light from the stereoscopic optical channel while a second capture unit captures a fluorescence image from the second light from the other optical channel. An intelligent image processing system receives the captured stereoscopic visible image and the captured fluorescence image and generates a stereoscopic pair of fluorescence images. An augmented stereoscopic display system outputs a real-time stereoscopic image comprising a three-dimensional presentation of a blend of the stereoscopic visible image and the stereoscopic pair of fluorescence images.

Abstract: The present disclosure relates to calibration assemblies and methods for use with an imaging system, such as an endoscopic imaging system. A calibration assembly includes: an interface for constraining engagement with an endoscopic imaging system; a target coupled with the interface so as to be within the field of view of the imaging system, the target including multiple of markers having calibration features that include identification features; and a processor configured to identify from first and second images obtained at first and second relative spatial arrangements between the imaging system and the target, respectively, at least some of the markers from the identification features, and using the identified markers and calibration feature positions within the images to generate calibration data.

Abstract: Systems, methods, and devices are used to match images. Points of interest from a first image are identified for matching to a second image. In response to the identified points of interest, regions and features can be identified and used to match the points of interest to a corresponding second image or second series of images. Regions can be used to match the points of interest when regions of the first image are matched to the second image with high confidence scores, for example above a threshold. Features of the first image can be matched to the second image, and these matched features may be used to match the points of interest to the second image, for example when the confidence scores for the regions are below the threshold value. Constraint can be used to evaluate the matched points of interest, for example by excluding bad points.

Abstract: Provided are a system and method for image sharpening is provided that involves capturing an image, and then decomposing the image into a plurality of image-representation components, such as RGB components for example. Each image-representation component is transformed to obtain an unsharpened multi-resolution representation for each image-representation component. A multi-resolution representation includes a plurality of transformation level representations. Sharpness information is transported from an unsharpened transformation level representation of a first one of the image-representation components to a transformation level representation of an unsharpened multi-resolution representation of a second one of the image-representation components to create a sharpened multi-resolution representation of the second one of the image-representation components. The sharpened multi-resolution representation of the second one of the image-representation components is then transformed to obtain a sharpened image.

Abstract: An endoscope with a stereoscopic optical channel is again held and positioned by a robotic surgical system. A capture unit captures (1) at a first time, a first image from light from the channel; and (2) at a second time different from the first time, a second image from the light. Only one of the first image and the second image includes a combination of a fluorescence image and a visible image. The other of the first image and the second image is a visible image. An intelligent image processing system generates an artificial fluorescence image using the captured fluorescence image. An augmented stereoscopic display system outputs an augmented stereoscopic image of at least a portion of the tissue comprising the artificial fluorescence image.