Abstract: A medical imaging system comprises a teleoperational assembly and a processing unit configured for receiving a roll position indicator for an imaging instrument coupled to the teleoperational assembly. The imaging instrument has a view angle other than 0° relative to an optical axis of the imaging instrument, and the imaging instrument is a stereoscopic imaging instrument including first and second image sources. The processing unit is further configured for: obtaining first image data from the imaging instrument at a first roll position; obtaining subsequent image data from the imaging instrument at a second roll position; and, responsive to a roll movement of the imaging instrument between the first and second roll positions, transitioning between presentation of the first image data on a user display and presentation of the subsequent image data on the user display. The transition includes changing a position of presentation of the first and second image sources.

Abstract: In a minimally invasive surgical system, a hand tracking system tracks a location of a sensor element mounted on part of a human hand. A system control parameter is generated based on the location of the part of the human hand. Operation of the minimally invasive surgical system is controlled using the system control parameter. Thus, the minimally invasive surgical system includes a hand tracking system. The hand tracking system tracks a location of part of a human hand. A controller coupled to the hand tracking system converts the location to a system control parameter, and injects into the minimally invasive surgical system a command based on the system control parameter.

Abstract: A teleoperational system comprises a teleoperational control system and a teleoperational manipulator configured for operating an instrument in an environment. The teleoperational system also comprises an operator controller in communication with the teleoperational control system. The teleoperational control system includes a processing unit including one or more processors. The processing unit is configured to determine whether an operator of the operator controller has a head portion directed toward a display region of a display device and based on a determination that the operator's head portion is directed toward the display region, initiate an operator following mode in which movement of the operator controller provides a corresponding movement to the teleoperational manipulator. The teleoperational system may be a teleoperational medical system.

Abstract: A teleoperational assembly is disclosed which includes an operator control system and a plurality of manipulators configured to control the movement of medical instruments in a surgical environment. The manipulators are teleoperationally controlled by the operator control system. The system further includes a processing unit configured to display an image of a field of view of the surgical environment, project the position of distal end portions of the medical instruments into the image's coordinate space, determine initial positions for badges associated with the distal end portions of each medical instrument, evaluate a display factor for each badge based on its initial position, and determine a final display position for each badge in the image's coordinate space based on the display factor.

Abstract: A system comprises a medical tool including a shaft having proximal and distal ends and an articulatable distal portion coupled to the distal end of the shaft. The system also comprises a processing unit including one or more processors. The processing unit is configured to determine a target in a medical environment. The articulatable distal portion is directed toward the target. The processing unit is also configured to determine a motion of at least a portion of the shaft, and in response to the determined motion, control a pose of the articulatable distal portion so that the articulatable distal portion remains directed toward the target.

Abstract: An operator telerobotically controls tools to perform a procedure on an object at a work site while viewing real-time images of the object, tools and work site on a display. Tool information is provided by filtering a part of the real-time images for enhancement or degradation to indicate a state of a tool and displaying the filtered images on the display.

Abstract: A teleoperational medical system for performing a medical procedure in a surgical field includes a dynamic guided setup system having step-by-step setup instructions for setting up a teleoperational assembly having at least one motorized surgical arm configured to assist in a surgical procedure. It also includes a user interface configured to communicate the step-by-step setup instructions to a user. The dynamic guided setup system is configured to automatically recognize completion of a first setup step based on detected physical arrangement of at least one surgical arm on a teleoperational assembly and automatically display a prompt for a subsequent setup step after the recognizing completion of the first setup step.

Abstract: In a minimally invasive surgical system, a plurality of video images is acquired. Each image includes a hand pose image. Depth data for the hand pose image is also acquired or synthesized. The hand pose image is segmented from the image using the depth data. The segmented image is combined with an acquired surgical site image using the depth data. The combined image is displayed to a person at a surgeon's console of the minimally invasive surgical system. Processing each of the video images in the plurality video images in this way reproduces the hand gesture overlaid on the video of the surgical site in the display.

Abstract: A method and apparatus for gaze-based communications. A gaze of a user is detected being directed at an element in a graphical user interface. The graphical user interface is displayed on a display system located in an environment. A state of the element is changed in response to detecting that the gaze is directed at the element. Audio information is received through an audio input device. An output operation is controlled based on the state of the element such that the audio information received through the audio input device is directed to an information conveyor corresponding to the element.

Abstract: Systems and methods of recentering an input control include a control unit configured to suspend teleoperated control of an end effector by the input control in response to a recentering request, determine a recentering move for the input control to provide positional and orientational harmony between the input control and the end effector, execute the recentering move, and reinstate teleoperated control of the end effector by the input control. In some embodiments, to determine the recentering move the control unit is configured to determine one or more first positions associated with the end effector, map the first positions to a view coordinate system, map the first positions from the view coordinate system to a console workspace coordinate system, and determine one or more second positions for one or more control points on the input control, the control points corresponding to the mapped first positions in the console workspace coordinate system.

Abstract: A teleoperational assembly is disclosed which includes an operator control system and a plurality of manipulators configured to control the movement of medical instruments in a surgical environment. The manipulators are teleoperationally controlled by the operator control system. The system further includes a processing unit configured to display an image of a field of view of the surgical environment, project the position of distal end portions of the medical instruments into the image's coordinate space, determine initial positions for badges associated with the distal end portions of each medical instrument, evaluate a display factor for each badge based on its initial position, and determine a final display position for each badge in the image's coordinate space based on the display factor.

Abstract: In a minimally invasive surgical system, a plurality of video images is acquired. Each image includes a hand pose image. Depth data for the hand pose image is also acquired or synthesized. The hand pose image is segmented from the image using the depth data. The segmented image is combined with an acquired surgical site image using the depth data. The combined image is displayed to a person at a surgeon's console of the minimally invasive surgical system. Processing each of the video images in the plurality video images in this way reproduces the hand gesture overlaid on the video of the surgical site in the display.

Abstract: A system and method of recentering imaging devices and input controls includes a medical device having one or more end effectors, an imaging device, one or more input controls for teleoperating the one or more end effectors, and a control unit including one or more processors coupled to the end effectors, the imaging device, and the input controls. The control unit suspends teleoperated control of the end effectors by the input controls in response to a recentering request, determines a view recentering move for the imaging device so that the end effectors are contained within a view space of the imaging device, determines one or more input control recentering moves to provide positional and orientational harmony between each of the input controls and a corresponding one of the end effectors, executes the view and input control recentering moves, and reinstates teleoperated control of the end effectors by the input controls.

Abstract: A system and method of assisting operator engagement with input devices includes an input device, a hand detection system, and a control unit. The control unit is configured to detect the hand of the operator using the hand detection system; in response to a trigger condition, command the input device to move from a first to a second position; and in response to a signal indicating an input provided by the operator to the input device, command a motion of an end effector associated with the input device. The second position is closer to a grasping position for the hand than the first position. In some embodiments, the control unit is further configured to, in response to the trigger condition, command the input device to move from a first to a second orientation. The second orientation is closer to a grasping orientation for the hand than the first orientation.

Abstract: A teleoperated system includes a master grip and a ratcheting system coupled to the master grip. The ratcheting system is configured to be coupled to a slave instrument. The ratcheting system is further configured to align the master grip with the slave instrument by determining a commanded angular velocity for the slave instrument based on a command from the master grip, determining a current alignment error between an alignment of the master grip and an alignment of the slave instrument, and altering the commanded angular velocity for the slave instrument based on the current alignment error. In some embodiments, the ratcheting system is further configured to introduce a control system error to alter the commanded angular velocity. In some embodiments, the current alignment error is a rotation angle error between the alignment of the master grip and the alignment of the slave instrument.

Abstract: An operator telerobotically controls tools to perform a procedure on an object at a work site while viewing real-time images of the work site on a display. Tool information is provided in the operator's current gaze area on the display by rendering the tool information over the tool so as not to obscure objects being worked on at the time by the tool nor to require eyes of the user to refocus when looking at the tool information and the image of the tool on a stereo viewer.

Abstract: A system and method for integrated table motion includes a device. The device includes an articulated arm having joints and a distal portion distal to the joints and a control unit. To support integrated motion with a separate table, the control unit is configured to receive a table movement request from the table, determine whether allowing the table movement request would result in a first joint of the joints being at a range of motion limit, allow the table to perform the table movement request based on determining that allowing the table to perform the table movement request would not result in the first joint being at the range of motion limit, track movement of the table due to performing the table movement request, and maintain, using the joints and based on the tracked movement, a position and/or an orientation of the distal portion relative to the table.

Abstract: An operator telerobotically controls tools to perform a procedure on an object at a work site while viewing real-time images of the work site on a display. Tool information is provided in the operator's current gaze area on the display by rendering the tool information over the tool so as not to obscure objects being worked on at the time by the tool nor to require eyes of the user to refocus when looking at the tool information and the image of the tool on a stereo viewer.

Abstract: A minimally-invasive surgical system includes a slave surgical instrument having a slave surgical instrument tip and a master grip. The slave surgical instrument tip has an alignment in a common frame of reference and the master grip, which is coupled to the slave surgical instrument, has an alignment in the common frame of reference. An alignment error, in the common frame of reference, is a difference in alignment between the alignment of the slave surgical instrument tip and the alignment of the master grip. A ratcheting system (i) coupled to the master grip to receive the alignment of the master grip and (ii) coupled to the slave surgical instrument, to control motion of the slave by continuously reducing the alignment error, as the master grip moves, without autonomous motion of the slave surgical instrument tip and without autonomous motion of the master grip.

Abstract: A teleoperational medical system for performing a medical procedure in a surgical field includes a teleoperational assembly having a plurality of motorized surgical arms configured to assist in a surgical procedure. The motorized surgical arms have a motion limit defining a boundary beyond which the surgical arm cannot pass when the surgical arm is attached to a patient. The teleoperational medical system also includes a control system having a surgical threshold limit stored therein. The surgical threshold limit is an edge of a boundary to be potentially travelled by the surgical arm to suitably perform a surgical procedure. The control system is configured to compare the motion limit to the surgical threshold limit and notify an operator via an output device when the threshold limit is outside a range of motion bounded by the motion limit.