Abstract: A method and apparatus for gaze-based communications. A gaze of a user is detected being directed at an element in a graphical user interface. The graphical user interface is displayed on a display system located in an environment. A state of the element is changed in response to detecting that the gaze is directed at the element. Audio information is received through an audio input device. An output operation is controlled based on the state of the element such that the audio information received through the audio input device is directed to an information conveyor corresponding to the element.

Abstract: Systems and methods of recentering an input control include a control unit configured to suspend teleoperated control of an end effector by the input control in response to a recentering request, determine a recentering move for the input control to provide positional and orientational harmony between the input control and the end effector, execute the recentering move, and reinstate teleoperated control of the end effector by the input control. In some embodiments, to determine the recentering move the control unit is configured to determine one or more first positions associated with the end effector, map the first positions to a view coordinate system, map the first positions from the view coordinate system to a console workspace coordinate system, and determine one or more second positions for one or more control points on the input control, the control points corresponding to the mapped first positions in the console workspace coordinate system.

Abstract: A teleoperational assembly is disclosed which includes an operator control system and a plurality of manipulators configured to control the movement of medical instruments in a surgical environment. The manipulators are teleoperationally controlled by the operator control system. The system further includes a processing unit configured to display an image of a field of view of the surgical environment, project the position of distal end portions of the medical instruments into the image's coordinate space, determine initial positions for badges associated with the distal end portions of each medical instrument, evaluate a display factor for each badge based on its initial position, and determine a final display position for each badge in the image's coordinate space based on the display factor.

Abstract: In a minimally invasive surgical system, a plurality of video images is acquired. Each image includes a hand pose image. Depth data for the hand pose image is also acquired or synthesized. The hand pose image is segmented from the image using the depth data. The segmented image is combined with an acquired surgical site image using the depth data. The combined image is displayed to a person at a surgeon's console of the minimally invasive surgical system. Processing each of the video images in the plurality video images in this way reproduces the hand gesture overlaid on the video of the surgical site in the display.

Abstract: A system and method of recentering imaging devices and input controls includes a medical device having one or more end effectors, an imaging device, one or more input controls for teleoperating the one or more end effectors, and a control unit including one or more processors coupled to the end effectors, the imaging device, and the input controls. The control unit suspends teleoperated control of the end effectors by the input controls in response to a recentering request, determines a view recentering move for the imaging device so that the end effectors are contained within a view space of the imaging device, determines one or more input control recentering moves to provide positional and orientational harmony between each of the input controls and a corresponding one of the end effectors, executes the view and input control recentering moves, and reinstates teleoperated control of the end effectors by the input controls.

Abstract: A system and method of assisting operator engagement with input devices includes an input device, a hand detection system, and a control unit. The control unit is configured to detect the hand of the operator using the hand detection system; in response to a trigger condition, command the input device to move from a first to a second position; and in response to a signal indicating an input provided by the operator to the input device, command a motion of an end effector associated with the input device. The second position is closer to a grasping position for the hand than the first position. In some embodiments, the control unit is further configured to, in response to the trigger condition, command the input device to move from a first to a second orientation. The second orientation is closer to a grasping orientation for the hand than the first orientation.

Abstract: A teleoperated system includes a master grip and a ratcheting system coupled to the master grip. The ratcheting system is configured to be coupled to a slave instrument. The ratcheting system is further configured to align the master grip with the slave instrument by determining a commanded angular velocity for the slave instrument based on a command from the master grip, determining a current alignment error between an alignment of the master grip and an alignment of the slave instrument, and altering the commanded angular velocity for the slave instrument based on the current alignment error. In some embodiments, the ratcheting system is further configured to introduce a control system error to alter the commanded angular velocity. In some embodiments, the current alignment error is a rotation angle error between the alignment of the master grip and the alignment of the slave instrument.

Abstract: An operator telerobotically controls tools to perform a procedure on an object at a work site while viewing real-time images of the work site on a display. Tool information is provided in the operator's current gaze area on the display by rendering the tool information over the tool so as not to obscure objects being worked on at the time by the tool nor to require eyes of the user to refocus when looking at the tool information and the image of the tool on a stereo viewer.

Abstract: A system and method for integrated table motion includes a device. The device includes an articulated arm having joints and a distal portion distal to the joints and a control unit. To support integrated motion with a separate table, the control unit is configured to receive a table movement request from the table, determine whether allowing the table movement request would result in a first joint of the joints being at a range of motion limit, allow the table to perform the table movement request based on determining that allowing the table to perform the table movement request would not result in the first joint being at the range of motion limit, track movement of the table due to performing the table movement request, and maintain, using the joints and based on the tracked movement, a position and/or an orientation of the distal portion relative to the table.

Abstract: An operator telerobotically controls tools to perform a procedure on an object at a work site while viewing real-time images of the work site on a display. Tool information is provided in the operator's current gaze area on the display by rendering the tool information over the tool so as not to obscure objects being worked on at the time by the tool nor to require eyes of the user to refocus when looking at the tool information and the image of the tool on a stereo viewer.

Abstract: A minimally-invasive surgical system includes a slave surgical instrument having a slave surgical instrument tip and a master grip. The slave surgical instrument tip has an alignment in a common frame of reference and the master grip, which is coupled to the slave surgical instrument, has an alignment in the common frame of reference. An alignment error, in the common frame of reference, is a difference in alignment between the alignment of the slave surgical instrument tip and the alignment of the master grip. A ratcheting system (i) coupled to the master grip to receive the alignment of the master grip and (ii) coupled to the slave surgical instrument, to control motion of the slave by continuously reducing the alignment error, as the master grip moves, without autonomous motion of the slave surgical instrument tip and without autonomous motion of the master grip.

Abstract: A teleoperational medical system for performing a medical procedure in a surgical field includes a teleoperational assembly having a plurality of motorized surgical arms configured to assist in a surgical procedure. The motorized surgical arms have a motion limit defining a boundary beyond which the surgical arm cannot pass when the surgical arm is attached to a patient. The teleoperational medical system also includes a control system having a surgical threshold limit stored therein. The surgical threshold limit is an edge of a boundary to be potentially travelled by the surgical arm to suitably perform a surgical procedure. The control system is configured to compare the motion limit to the surgical threshold limit and notify an operator via an output device when the threshold limit is outside a range of motion bounded by the motion limit.

Abstract: A system and method for an integrated surgical table includes a medical device including an articulated arm having one or more first and second joints and a control unit. The articulated arm has at least a cannula, an endoscope, or an instrument mounted distal to the first and second joints, which is inserted into a patient at a body opening. The control unit unlocks the first joints, receives a surgical table movement request, determines whether the surgical table movement request should be granted, allows the surgical table to perform the requested movement based on the determining, uses the first joints to allow the articulated arm to track movement of the body opening based on forces applied by a body wall at the body opening, and compensates for changes in a pose of the cannula, endoscope, or instrument due to the tracked movement by performing compensating motions in the second joints.

Abstract: A synthetic representation of a robot tool for display on a user interface of a robotic system. The synthetic representation may be used to show the position of a view volume of an image capture device with respect to the robot. The synthetic representation may also be used to find a tool that is outside of the field of view, to display range of motion limits for a tool, to remotely communicate information about the robot, and to detect collisions.

Abstract: A teleoperational medical system for performing a medical procedure in a surgical field includes a teleoperational assembly having a plurality of motorized surgical arms configured to assist in a surgical procedure. It also includes an input device configured to receive an input to move all the arms of the plurality of motorized surgical arms to a pre-established position. A processing system is configured to receive the input form the input device and output control signals to each arm of the plurality of motorized surgical arms to move each arm to the pre-established position.

Abstract: A teleoperational medical system for performing a medical procedure in a surgical field includes a teleoperational assembly having a plurality of motorized surgical arms configured to assist in a surgical procedure. The motorized surgical arms have a motion limit defining a boundary beyond which the surgical arm cannot pass when the surgical arm is attached to a patient. The teleoperational medical system also includes a control system having a surgical threshold limit stored therein. The surgical threshold limit is an edge of a boundary to be potentially travelled by the surgical arm to suitably perform a surgical procedure. The control system is configured to compare the motion limit to the surgical threshold limit and notify an operator via an output device when the threshold limit is outside a range of motion bounded by the motion limit.

Abstract: A system and method of maintaining a tool position and orientation for a computer-assisted medical device include a device with an articulated structure including a plurality of joints and a control unit. The articulated structure is configured to support a tool distal to the joints. The control unit is configured to determine an initial position and orientation of the tool prior to movement of a first joint, determine, while the first joint is moving, a current position and orientation of the tool, determine a difference between the current and the initial position and orientation, and drive at least a second joint based on the difference to maintain a position and orientation of the tool at the initial position and orientation. In some embodiments, the position and orientation of the tool are maintained relative to a reference coordinate frame based on a cause of the moving of the first joint.

Abstract: A system and method of aligning with a reference target includes a computer-assisted device. The computer-assisted device includes a link, one or more first joints coupled proximally to the link, an articulated arm comprising one or more second joints coupled distally to the link, the articulated arm configured to couple to an instrument, and a control unit. The control unit is configured to position or orient the link, using the one or more first joints, based on at least one reference of the instrument selected from the group consisting of a reference point associated with the instrument and a reference orientation of the instrument. The control unit is further configured to maintain, while positioning or orienting the link and by using the one or more second joints, a position or an orientation of the instrument relative to a workspace in accordance with the at least one reference of the instrument.

Abstract: A synthetic representation of a robot tool for display on a user interface of a robotic system. The synthetic representation may be used to show the position of a view volume of an image capture device with respect to the robot. The synthetic representation may also be used to find a tool that is outside of the field of view, to display range of motion limits for a tool, to remotely communicate information about the robot, and to detect collisions.

Abstract: A teleoperational medical system for performing a medical procedure in a surgical field includes a teleoperational assembly having a plurlity of motorized surgical arms configured to assist in a surgical procedure. It also includes an input device configured to receive an input to move all the arms of the plurality of motorized surgical arms to a pre-established position. A processing system is configured to receive the input form the input device and output control signals to each arm of the plurality of motorized surgical arms to move each arm to the pre-established position.