Abstract: A robotic system provides user selectable actions associated with gaze tracking according to user interface types. User initiated correction and/or recalibration of the gaze tracking may be performed during the processing of individual of the user selectable actions.

Abstract: A system comprises a first robotic arm adapted to support and move a tool and a second robotic arm adapted to support and move a camera configured to capture an image of a camera field of view. The system further comprises an input device, a display, and a processor. The processor is configured to display a first synthetic image including a first synthetic image of the tool. The first synthetic image of the tool includes a portion of the tool outside of the camera field of view. The processor is also configured to receive a user input at the input device and responsive to the user input, change the display of the first synthetic image to a display of a second synthetic image including a second synthetic image of the tool that is different from the first synthetic image of the tool.

Abstract: A system includes a robotic manipulator including a serial chain comprising a first joint, a first link, and a second link. The second link is between the first joint and the first link in the serial chain. The system further includes a processing unit including one or more processors. The processing unit is configured to receive first link data from a first sensor system located at the first link, generate a first joint state estimate of the first joint based on the first link data and a kinematic model of the robotic manipulator, and control the first joint based on the first joint state estimate.

Abstract: A device comprises one or more movable elements and one or more processors. The device is configured to receive a first movement assignment of a first type, receive tracking data from a motion tracking system configured to monitor a position of personnel in a vicinity of the device, plan a first motion for a first movable element of the one or more movable elements based on the first type of the first movement assignment and the tracking data, and execute the first motion.

Abstract: A method and system for hand tracking in a robotic system includes a hand tracking system and a controller coupled to the hand tracking system. The controller is configured to receive, from the hand tracking system, a plurality of locations of a hand; determine if the hand is in a first hand pose based on the plurality of locations; in response to determining that the hand is in the first hand pose, and switch the robotic system to a hand trajectory detection mode. While in the hand trajectory detection mode, the control unit is configured to detect, based on hand tracking information from the hand tracking system, that the hand has performed a first hand trajectory of a plurality of known hand trajectories; and in response to detecting the first hand trajectory, change a mode of operation of the robotic system.

Abstract: A method comprises obtaining an endoscopic image dataset of a patient anatomy from an endoscopic imaging system and retrieving an anatomic model dataset of the patient anatomy obtained by an anatomic imaging system. The method also comprises mapping the endoscopic image dataset to the anatomic model dataset and displaying a first vantage point image using the mapped endoscopic image dataset. The first vantage point image is presented from a first vantage point at a distal end of the endoscopic imaging system. The method also comprises displaying a second vantage point image using at least a portion of the mapped endoscopic image dataset. The second vantage point image is presented from a second vantage point, different from the first vantage point.

Abstract: A system includes a robotic manipulator including a serial chain comprising a first joint, a second joint, and a first link. The system further includes a processing unit including one or more processors. The processing unit is configured to receive first link data from a first sensor system located at the first link, generate a first joint state estimate of the first joint based on the first link data, and generate a second joint state estimate of the second joint. The processing unit is further configured to apply a first weight to the first joint state estimate to generate a first weighted joint state estimate, apply a second weight to the second joint state estimate to generate a second weighted joint state estimate, and control the first and second joints based on the first weighted joint state estimate and second weighted joint state estimate.

Abstract: A region-based augmentation system is configured to determine that a viewpoint of a user of a display device is directed within an anchor region of a physical world containing the user. In response, the system directs the display device to present augmented content in an evident manner The evident manner comprises anchoring the augmented content relative to the viewpoint, such that the augmented content follows the viewpoint as the user moves the viewpoint within the anchor region. The system further determines that the viewpoint is directed outside of the anchor region by the user. In response, the system directs the display device to present the augmented content in a less evident manner. The presenting of the augmented content in the less evident manner comprises presenting the augmented content less visibly than the evident manner and/or unanchoring the augmented content relative to the viewpoint. Corresponding methods and systems are also disclosed.

Abstract: A teleoperated system includes a master grip and a ratcheting system coupled to the master grip. The ratcheting system is configured to align the master grip with a slave instrument commanded by the master grip by determining grip rotation values describing an orientation of the master grip, determining instrument rotation values describing an orientation of the instrument, determining an orientation error between an orientation of the master grip and the orientation of the instrument based on the grip rotation values and the instrument rotation values, and reducing the orientation error by low pass filtering the grip rotation values or the instrument rotation values.

Abstract: A surgical system includes a flexible steerable needle and a shape sensor for measuring the shape of the needle. The surgical system can be manual (e.g., laparoscopic), robotic, or any combination of the two. By directly measuring the shape of the needle, complex and potentially inaccurate modeling of the needle to determine trajectory and insertion depth can be avoided in favor of much more robust direct measurement and modeling of needle shape and/or pose.

Abstract: A stereo imaging system comprises a depth sensing system, a stereo viewer, and a processor configured to receive captured stereo worksite images, compare a depth value to a predetermined threshold value, and generate stereo images for display by adjusting or not adjusting native disparities based on the comparison of the depth value to the threshold value.

Abstract: Systems and methods for registration feature integrity checking include a repositionable arm configured to support a repositionable device and a control unit. The control unit is configured to receive a feature set including one or more features extracted from one or more images of a repositionable structure obtained from an imaging device, determine an expected feature corresponding to an extracted feature in the feature set based on one or more models of the repositionable structure, determine an error between the extracted feature and the expected feature, determine whether to remove the extracted feature from the feature set based on the determined error, remove the extracted feature from the feature set in response to determining that the extracted feature should be removed from the feature set, and provide the feature set to a registration module. The repositionable structure includes the repositionable arm and/or the repositionable device.

Abstract: A system comprises a first robotic arm adapted to support and move a tool and a second robotic arm adapted to support and move a camera. The system also comprises an input device, a display, and a processor. The processor is configured to, in a first mode, command the first robotic arm to move the camera in response to a first input received from the input device to capture an image of the tool and present the image as a displayed image on the display. The processor is configured to, in a second mode, display a synthetic image of the first robotic arm in a boundary area around the captured image on the display, and in response to a second input, change a size of the boundary area relative a size of the displayed image.

Abstract: A robotic system includes a processor that is programmed to determine and cause work site measurements for user specified points in the work site to be graphically displayed in order to provide geometrically appropriate tool selection assistance to the user. The processor is also programmed to determine an optimal one of a plurality of tools of varying geometries for use at the work site and to cause graphical representations of at least the optimal tool to be displayed along with the work site measurements.

Abstract: A teleoperated system comprises a display, a master input device, and a control system. The control system is configured to determine an orientation of an end effector reference frame relative to a field of view reference frame, determine an orientation of a master input device reference frame relative to a display reference frame, establish an alignment relationship between the master input device reference frame and the display reference frame, and command, based on the alignment relationship, a change in a pose of the end effector in response to a change in a pose of the master input device. The alignment relationship is independent of a position relationship between the master input device reference frame and the display reference frame. In one aspect, the teleoperated system is a telemedical system such as a telesurgical system.

Abstract: A teleoperational system comprises a teleoperational control system and a teleoperational manipulator configured for operating an instrument in an environment. The teleoperational system also comprises an operator controller in communication with the teleoperational control system. The teleoperational control system includes a processing unit including one or more processors. The processing unit is configured to determine whether an operator of the operator controller has a head portion directed toward a display region of a display device and based on a determination that the operator’s head portion is directed toward the display region, initiate an operator following mode in which movement of the operator controller provides a corresponding movement to the teleoperational manipulator. The teleoperational system may be a teleoperational medical system.

Abstract: The systems and methods of the present disclosure are used for guiding a medical instrument towards a target, the method positioning a medical instrument at a first location within a patient anatomy, wherein the medical instrument comprises at least one sensor, determining a first biomarker measurement using the at least one sensor, determining a second biomarker measurement using the at least one sensor, comparing the first biomarker measurement with the second biomarker measurement to determine a proximity to the target to provide a first comparison, and providing guidance for moving the medical instrument based on results of the first comparison.

Abstract: A method comprises obtaining an endoscopic image dataset of a patient anatomy from an endoscopic imaging system and retrieving an anatomic model dataset of the patient anatomy obtained by an anatomic imaging system. The method also comprises mapping the endoscopic image dataset to the anatomic model dataset and displaying a first vantage point image using the mapped endoscopic image dataset. The first vantage point image is presented from a first vantage point at a distal end of the endoscopic imaging system. The method also comprises displaying a second vantage point image using at least a portion of the mapped endoscopic image dataset. The second vantage point image is presented from a second vantage point, different from the first vantage point.

Abstract: A minimally invasive system comprises an elongate medical instrument including a flexible body. The flexible body includes a wall including a channel, and the channel includes a groove. The flexible body further includes a lumen defined by an interior surface of the wall and a curved distal tip portion. The elongate medical instrument further includes a shape sensor coupled to the flexible body. The shape sensor is at least partially positioned within the groove, and the shape sensor is configured to detect shape characteristics of at least a portion of the flexible body. The system further includes an actuator for manipulating the elongate medical instrument.

Abstract: Systems and methods for inter-arm registration include a computer-assisted system having a control unit coupled to a repositionable arm of a computer-assisted device. The control unit is configured to: receive, from an imaging device, successive images of an instrument mounted to the repositionable arm; determine an observed velocity of a feature of the instrument; determine an expected velocity of the feature of the instrument based on kinematics of the repositionable arm; transform the observed velocity and/or the expected velocity to a common coordinate system using a registration transform; determine an error between directions of the observed and expected velocities in the common coordinate system; and update the registration transform based on the determined error. In some embodiments, the instrument is a medical instrument and the imaging device is an endoscope. In some embodiments, the control unit is further configured to control the instrument using the registration transform.