Patents by Inventor Wenyi Zhao

Wenyi Zhao has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 8706301
    Abstract: Methods of and a system for providing force information for a robotic surgical system. The method includes storing first kinematic position information and first actual position information for a first position of an end effector; moving the end effector via the robotic surgical system from the first position to a second position; storing second kinematic position information and second actual position information for the second position; and providing force information regarding force applied to the end effector at the second position utilizing the first actual position information, the second actual position information, the first kinematic position information, and the second kinematic position information. Visual force feedback is also provided via superimposing an estimated position of an end effector without force over an image of the actual position of the end effector. Similarly, tissue elasticity visual displays may be shown.
    Type: Grant
    Filed: January 8, 2013
    Date of Patent: April 22, 2014
    Assignee: Intuitive Surgical Operations, Inc.
    Inventors: Wenyi Zhao, Tao Zhao, David Q. Larkin
  • Patent number: 8698885
    Abstract: In one embodiment of the invention, an apparatus is disclosed including an image sensor, a color filter array, and an image processor. The image sensor has an active area with a matrix of camera pixels. The color filter array is in optical alignment over the matrix of the camera pixels. The color filter array assigns alternating single colors to each camera pixel. The image processor receives the camera pixels and includes a correlation detector to detect spatial correlation of color information between pairs of colors in the pixel data captured by the camera pixels. The correlation detector further controls demosaicing of the camera pixels into full color pixels with improved resolution. The apparatus may further include demosaicing logic to demosaic the camera pixels into the full color pixels with improved resolution in response to the spatial correlation of the color information between pairs of colors.
    Type: Grant
    Filed: February 14, 2011
    Date of Patent: April 15, 2014
    Assignee: Intuitive Surgical Operations, Inc.
    Inventors: Jeffrey DiCarlo, David D. Scott, Wenyi Zhao
  • Publication number: 20140058564
    Abstract: Methods of and a system for providing a visual representation of force information in a robotic surgical system. A real position of a surgical end effector is determined. A projected position of the surgical end effector if no force were applied against the end effector is also determined. Images representing the real and projected positions are output superimposed on a display. The offset between the two images provides a visual indication of a force applied to the end effector or to the kinematic chain that supports the end effector. In addition, tissue deformation information is determined and displayed.
    Type: Application
    Filed: October 30, 2013
    Publication date: February 27, 2014
    Applicant: Intuitive Surgical Operations, Inc.
    Inventors: Wenyi Zhao, Tao Zhao, David Q. Larkin
  • Publication number: 20140055489
    Abstract: An operator telerobotically controls tools to perform a procedure on an object at a work site while viewing real-time images of the work site on a display. Tool information is provided in the operator's current gaze area on the display by rendering the tool information over the tool so as not to obscure objects being worked on at the time by the tool nor to require eyes of the user to refocus when looking at the tool information and the image of the tool on a stereo viewer.
    Type: Application
    Filed: February 15, 2013
    Publication date: February 27, 2014
    Applicant: Intuitive Surgical Operations, Inc.
    Inventors: Brandon D. Itkowitz, Daniel J. Halabe, Tao Zhao, Simon P. DiMaio, Christopher J. Hasser, Catherine J. Mohr, Paul W. Mohr, David Q. Larkin, Brian David Hoffman, Wenyi Zhao
  • Publication number: 20140031659
    Abstract: A bleeding detection unit in a surgical system processes information in an acquired scene before that scene is presented on a display unit in the operating room. For example, the bleeding detection unit analyzes the pixel data in the acquired scene and determines whether there are one or more initial sites of blood in the scene. Upon detection of an initial site of blood, the region is identified by an initial site icon in the scene displayed on the display unit. In one aspect, the processing is done in real-time which means that there is no substantial delay in presenting the acquired scene to the surgeon.
    Type: Application
    Filed: July 22, 2013
    Publication date: January 30, 2014
    Applicant: Intuitive Surgical Operations, Inc.
    Inventors: Wenyi ZHAO, Catherine J. Mohr, Jonthan M. Sorger
  • Patent number: 8639000
    Abstract: Systems, methods, and devices are used to match images. Points of interest from a first image are identified for matching to a second image. In response to the identified points of interest, regions and features can be identified and used to match the points of interest to a corresponding second image or second series of images. Regions can be used to match the points of interest when regions of the first image are matched to the second image with high confidence scores, for example above a threshold. Features of the first image can be matched to the second image, and these matched features may be used to match the points of interest to the second image, for example when the confidence scores for the regions are below the threshold value. Constraint can be used to evaluate the matched points of interest, for example by excluding bad points.
    Type: Grant
    Filed: April 24, 2012
    Date of Patent: January 28, 2014
    Assignee: Intuitive Surgical Operations, Inc.
    Inventors: Wenyi Zhao, Chenyu Wu, David Hirvonen, Tao Zhao, Brian D. Hoffman, Simon Di Maio
  • Patent number: 8594841
    Abstract: Methods of and a system for providing a visual representation of force information in a robotic surgical system. A real position of a surgical end effector is determined. A projected position of the surgical end effector if no force were applied against the end effector is also determined. Images representing the real and projected positions are output superimposed on a display. The offset between the two images provides a visual indication of a force applied to the end effector or to the kinematic chain that supports the end effector. In addition, tissue deformation information is determined and displayed.
    Type: Grant
    Filed: April 22, 2009
    Date of Patent: November 26, 2013
    Assignee: Intuitive Surgical Operations, Inc.
    Inventors: Wenyi Zhao, Tao Zhao, David Q. Larkin
  • Publication number: 20130300837
    Abstract: Mixed mode imaging is implemented using a single-chip image capture sensor with a color filter array. The single-chip image capture sensor captures a frame including a first set of pixel data and a second set of pixel data. The first set of pixel data includes a first combined scene, and the second set of pixel data includes a second combined scene. The first combined scene is a first weighted combination of a fluorescence scene component and a visible scene component due to the leakage of a color filter array. The second combined scene includes a second weighted combination of the fluorescence scene component and the visible scene component. Two display scene components are extracted from the captured pixel data in the frame and presented on a display unit.
    Type: Application
    Filed: May 14, 2013
    Publication date: November 14, 2013
    Inventors: Jeffrey DiCarlo, Ian E. McDowall, Wenyi Zhao
  • Publication number: 20130300836
    Abstract: Mixed mode imaging is implemented using a single-chip image capture sensor with a color filter array. The single-chip image capture sensor captures a frame including a first set of pixel data and a second set of pixel data. The first set of pixel data includes a first combined scene, and the second set of pixel data includes a second combined scene. The first combined scene is a first weighted combination of a fluorescence scene component and a visible scene component due to the leakage of a color filter array. The second combined scene includes a second weighted combination of the fluorescence scene component and the visible scene component. Two display scene components are extracted from the captured pixel data in the frame and presented on a display unit.
    Type: Application
    Filed: May 14, 2013
    Publication date: November 14, 2013
    Applicant: Intuitive Surgical Operations, Inc.
    Inventors: Wenyi Zhao, Jeffrey DiCarlo, Ian E. McDowall
  • Publication number: 20130107207
    Abstract: Stereo gaze tracking estimates a 3-D gaze point by projecting determined right and left eye gaze points on left and right stereo images. The determined right and left eye gaze points are based on one or more tracked eye gaze points, estimates for non-tracked eye gaze points based upon the tracked gaze points and image matching in the left and right stereo images, and confidence scores indicative of the reliability of the tracked gaze points and/or the image matching.
    Type: Application
    Filed: October 29, 2012
    Publication date: May 2, 2013
    Applicant: Intuitive Surgical Operations, Inc.
    Inventors: Wenyi Zhao, Brandon D. Itkowitz
  • Patent number: 8374723
    Abstract: Methods of and a system for providing force information for a robotic surgical system. The method includes storing first kinematic position information and first actual position information for a first position of an end effector; moving the end effector via the robotic surgical system from the first position to a second position; storing second kinematic position information and second actual position information for the second position; and providing force information regarding force applied to the end effector at the second position utilizing the first actual position information, the second actual position information, the first kinematic position information, and the second kinematic position information. Visual force feedback is also provided via superimposing an estimated position of an end effector without force over an image of the actual position of the end effector. Similarly, tissue elasticity visual displays may be shown.
    Type: Grant
    Filed: April 22, 2009
    Date of Patent: February 12, 2013
    Assignee: Intuitive Surgical Operations, Inc.
    Inventors: Wenyi Zhao, Tao Zhao, David Q. Larkin
  • Publication number: 20120237095
    Abstract: Systems, methods, and devices are used to match images. Points of interest from a first image are identified for matching to a second image. In response to the identified points of interest, regions and features can be identified and used to match the points of interest to a corresponding second image or second series of images. Regions can be used to match the points of interest when regions of the first image are matched to the second image with high confidence scores, for example above a threshold. Features of the first image can be matched to the second image, and these matched features may be used to match the points of interest to the second image, for example when the confidence scores for the regions are below the threshold value. Constraint can be used to evaluate the matched points of interest, for example by excluding bad points.
    Type: Application
    Filed: April 24, 2012
    Publication date: September 20, 2012
    Applicant: Intuitive Surgical Operations, Inc.
    Inventors: WENYI ZHAO, CHENYU WU, DAVID HIRVONEN, TAO ZHAO, BRIAN D. HOFFMAN, SIMON DIMAIO
  • Publication number: 20120209287
    Abstract: A local contrast enhancement method transforms a first plurality of color components of a first visual color image into a modified brightness component by using a first transformation. The first plurality of color components are in a first color space. The modified brightness component is a brightness component of a second color space. The second color space also includes a plurality of chromatic components. The method transforms all the color components of the first color space into the chromatic components of the second color space. The method then transforms the modified brightness component and the chromatic components of the second color space into a plurality of new color components, in the first color space, of a second visual color image. The method transmits the plurality of new color components to a device such as a display device. The second visual color image has enhanced contrast in comparison to the first visual color image.
    Type: Application
    Filed: February 14, 2011
    Publication date: August 16, 2012
    Applicant: Intuitive Surgical Operations, Inc.
    Inventors: Wenyi Zhao, Simon P. DiMaio, Catherine J. Mohr, David D. Scott, Tao Zhao
  • Publication number: 20120206582
    Abstract: In one embodiment of the invention, an apparatus is disclosed including an image sensor, a color filter array, and an image processor. The image sensor has an active area with a matrix of camera pixels. The color filter array is in optical alignment over the matrix of the camera pixels. The color filter array assigns alternating single colors to each camera pixel. The image processor receives the camera pixels and includes a correlation detector to detect spatial correlation of color information between pairs of colors in the pixel data captured by the camera pixels. The correlation detector further controls demosaicing of the camera pixels into full color pixels with improved resolution. The apparatus may further include demosaicing logic to demosaic the camera pixels into the full color pixels with improved resolution in response to the spatial correlation of the color information between pairs of colors.
    Type: Application
    Filed: February 14, 2011
    Publication date: August 16, 2012
    Applicant: Intuitive Surgical Operations, Inc.
    Inventors: Jeffrey DiCarlo, David D. Scott, Wenyi Zhao
  • Patent number: 8228368
    Abstract: An illumination channel, a stereoscopic optical channel and another optical channel are held and positioned by a robotic surgical system. A first capture unit captures a stereoscopic visible image from the first light from the stereoscopic optical channel while a second capture unit captures a fluorescence image from the second light from the other optical channel. An intelligent image processing system receives the captured stereoscopic visible image and the captured fluorescence image and generates a stereoscopic pair of fluorescence images. An augmented stereoscopic display system outputs a real-time stereoscopic image comprising a three-dimensional presentation of a blend of the stereoscopic visible image and the stereoscopic pair of fluorescence images.
    Type: Grant
    Filed: June 30, 2008
    Date of Patent: July 24, 2012
    Assignee: Intuitive Surgical Operations, Inc.
    Inventors: Wenyi Zhao, Catherine J. Mohr, Christopher J. Hasser, Brian D. Hoffman, John D. Stern, Tao Zhao
  • Patent number: 8223193
    Abstract: The present disclosure relates to calibration assemblies and methods for use with an imaging system, such as an endoscopic imaging system. A calibration assembly includes: an interface for constraining engagement with an endoscopic imaging system; a target coupled with the interface so as to be within the field of view of the imaging system, the target including multiple of markers having calibration features that include identification features; and a processor configured to identify from first and second images obtained at first and second relative spatial arrangements between the imaging system and the target, respectively, at least some of the markers from the identification features, and using the identified markers and calibration feature positions within the images to generate calibration data.
    Type: Grant
    Filed: March 31, 2009
    Date of Patent: July 17, 2012
    Assignee: Intuitive Surgical Operations, Inc.
    Inventors: Tao Zhao, Wenyi Zhao, David D. Scott
  • Patent number: 8184880
    Abstract: Systems, methods, and devices are used to match images. Points of interest from a first image are identified for matching to a second image. In response to the identified points of interest, regions and features can be identified and used to match the points of interest to a corresponding second image or second series of images. Regions can be used to match the points of interest when regions of the first image are matched to the second image with high confidence scores, for example above a threshold. Features of the first image can be matched to the second image, and these matched features may be used to match the points of interest to the second image, for example when the confidence scores for the regions are below the threshold value. Constraint can be used to evaluate the matched points of interest, for example by excluding bad points.
    Type: Grant
    Filed: May 13, 2009
    Date of Patent: May 22, 2012
    Assignee: Intuitive Surgical Operations, Inc.
    Inventors: Wenyi Zhao, Chenyu Wu, David Hirvonen, Tao Zhao, Brian D. Hoffman, Simon Di Maio
  • Publication number: 20120120245
    Abstract: Provided are a system and method for image sharpening is provided that involves capturing an image, and then decomposing the image into a plurality of image-representation components, such as RGB components for example. Each image-representation component is transformed to obtain an unsharpened multi-resolution representation for each image-representation component. A multi-resolution representation includes a plurality of transformation level representations. Sharpness information is transported from an unsharpened transformation level representation of a first one of the image-representation components to a transformation level representation of an unsharpened multi-resolution representation of a second one of the image-representation components to create a sharpened multi-resolution representation of the second one of the image-representation components. The sharpened multi-resolution representation of the second one of the image-representation components is then transformed to obtain a sharpened image.
    Type: Application
    Filed: November 15, 2010
    Publication date: May 17, 2012
    Applicant: Intuitive Surgical Operations, Inc.
    Inventors: Wenyi Zhao, Simon P. DiMaio, David D. Scott
  • Patent number: 8169468
    Abstract: A robotic surgical system positions and holds an endoscope. A visible imaging system is coupled to the endoscope. The visible imaging system captures a visible image of tissue. An alternate imaging system is also coupled to the endoscope. The alternate imaging system captures a fluorescence image of at least a portion of the tissue. A stereoscopic video display system is coupled to the visible imaging system and to the alternate imaging system. The stereoscopic video display system outputs a real-time stereoscopic image comprising a three-dimensional presentation of a blend of a fluorescence image associated with the captured fluorescence image, and the visible image.
    Type: Grant
    Filed: June 30, 2008
    Date of Patent: May 1, 2012
    Assignee: Intuitive Surgical Operations, Inc.
    Inventors: David Scott, Wenyi Zhao, Christopher J. Hasser, Brian D. Hoffman, Paul Lilagan, Ian McDowall, Catherine J. Mohr, John D. Stern, Tao Zhao
  • Patent number: 8167793
    Abstract: An endoscope with a stereoscopic optical channel is again held and positioned by a robotic surgical system. A capture unit captures (1) at a first time, a first image from light from the channel; and (2) at a second time different from the first time, a second image from the light. Only one of the first image and the second image includes a combination of a fluorescence image and a visible image. The other of the first image and the second image is a visible image. An intelligent image processing system generates an artificial fluorescence image using the captured fluorescence image. An augmented stereoscopic display system outputs an augmented stereoscopic image of at least a portion of the tissue comprising the artificial fluorescence image.
    Type: Grant
    Filed: June 30, 2008
    Date of Patent: May 1, 2012
    Assignee: Intuitive Surgical Operations, Inc.
    Inventors: David Scott, Wenyi Zhao, Christopher J. Hasser, Brian D. Hoffman, Ian McDowall, Catherine J. Mohr, John D. Stern