Abstract: Surgical robotic systems, and methods of verifying functionality of a user interface device of such systems, are described. During a surgical procedure, the user interface device controls motion of a surgical tool. Proximity sensors of the user interface device generate proximity measures throughout the surgical procedure. The proximity measures are used to detect whether the user interface device is dropped and to responsively halt motion of the surgical tool. To verify an accuracy of the proximity sensors that provide the drop detection, a test is performed when the user interface device is placed in a dock. The test compares the generated proximity measures to expected proximity data. When the proximity measures match the expected proximity data, the system determines that the proximity sensors are functioning accurately and verifies that the user interface device is functioning safely. Other embodiments are described and claimed.

Abstract: Surgical robotic systems, and methods of controlling such systems based on a user's grip on a user interface device, are described. During a surgical procedure, a surgical robotic system generates control commands to actuate a component of the surgical robotic system. The component can be a surgical tool that is actuated to move based on tracking data from the user interface device. The component can be the user interface device, which is actuated to render haptic feedback to the user's hand based on load data from a surgical robotic arm holding the surgical tool. In either case, the actuation is based on a center of rotation of the user interface device that corresponds to the user's grip on the user interface device. Other embodiments are described and claimed.

Abstract: A method performed by a surgical robotic system that includes a seat that is arranged for a user to sit and a display column that includes at least one display for displaying a three-dimensional (3D) surgical presentation. The method includes receiving an indication that the user has manually adjusted the seat and in response, determining, while the user is sitting on the seat, a position of the user's eyes, determining a configuration for the display column based on the determined position of the user's eyes, and adjusting the display column by actuating one or more actuators of the display column according to the determined configuration.

Abstract: A method for disengaging a surgical instrument of a surgical robotic system comprising receiving a gaze input from an eye tracker; determining, by one or more processors, whether the gaze input indicates the gaze of the user is outside or inside of the display; in response to determining the gaze input indicates the gaze of the user is outside of the display, determining an amount of time the gaze of the user is outside of the display; in response to determining the gaze of the user is outside of the display for less than a maximum amount of time, pause the surgical robotic system from a teleoperation mode; and in response to determining the gaze of the user is outside of the display for more than the maximum amount of time, disengage the surgical robotic system from the teleoperation mode.

Abstract: Embodiments described herein provide various examples of a machine-learning-based visual-haptic system for constructing visual-haptic models for various interactions between surgical tools and tissues. In one aspect, a process for constructing a visual-haptic model is disclosed. This process can begin by receiving a set of training videos. The process then processes each training video in the set of training videos to extract one or more video segments that depict a target tool-tissue interaction from the training video, wherein the target tool-tissue interaction involves exerting a force by one or more surgical tools on a tissue. Next, for each video segment in the set of video segments, the process annotates each video image in the video segment with a set of force levels predefined for the target tool-tissue interaction. The process subsequently trains a machine-learning model using the annotated video images to obtain a trained machine-learning model for the target tool-tissue interaction.

Abstract: Disclosed herein are methods to detect a free-falling or other non-surgical motions of the user interface device (UID) of a surgical robotic system so that the surgical robotic system may pause the robotic arm controlled by the UID to prevent the robotic arm from mimicking the unintentional movement of the UID. Contact sensors embedded in the UID may be used to detect conditions indicating that a user does not possess full control of the UID. After determining that the user does not have full control of the UID, the UID may detect if the UID is experiencing non-surgical motions using motion sensors such as inertial sensors. By conditioning analysis of the data from the motion sensors by the initial determination that the UID is not being held based on the contact sensors, the method increases the robustness of the detection of non-surgical motions and reduces the probability of false positives.

Abstract: Communication apparatus and devices for surgical robotic systems are described. The communication apparatus can include a user console in communication with a communication device having a surgical tool. The communication device can include a microphone to convert a sound input into an acoustic input signal. The communication device can transmit the acoustic input signal to the user console for reproduction as a sound output for a remote operator. The surgical tool can include an endoscope having several microphones mounted on a housing. The surgical tool can be a sterile barrier having a microphone and a drape. The microphone(s) of the surgical tools can face a surrounding environment such that a tableside staff is a source of the sound input that causes the sound output, and a surgeon and the tableside staff can communicate in a noisy environment. Other embodiments are also described and claimed.

Abstract: Embodiments described herein provide various examples of a visual-haptic feedback system for generating a haptic feedback signal based on captured endoscopy images. In one aspect, the process for generating the haptic feedback signal includes the steps of: receiving an endoscopic video captured for a surgical procedure performed on a robotic surgical system; detecting a surgical task in the endoscopic video involving a given type of surgical tool-tissue interaction; selecting, a machine learning model constructed for analyzing the given type of surgical tool-tissue interaction; for a video image associated with the detected surgical task depicting the given type of surgical tool-tissue interaction, applying the selected machine learning model to the video image to predict a strength level of the depicted surgical tool-tissue interaction; and then providing the predicted strength level to a surgeon performing the surgical task as a haptic feedback signal for the given type of surgical tool-tissue interaction.

Abstract: Communication apparatus and devices for surgical robotic systems are described. The communication apparatus can include a user console in communication with a communication device having a surgical tool. The communication device can include a microphone to convert a sound input into an acoustic input signal. The communication device can transmit the acoustic input signal to the user console for reproduction as a sound output for a remote operator. The surgical tool can include an endoscope having several microphones mounted on a housing. The surgical tool can be a sterile barrier having a microphone and a drape. The microphone(s) of the surgical tools can face a surrounding environment such that a tableside staff is a source of the sound input that causes the sound output, and a surgeon and the table side staff can communicate in a noisy environment. Other embodiments are also described and claimed.

Abstract: Surgical systems including a user console for controlling a surgical robotic tool are described. A witness sensor and a reference sensor can be mounted on the user console to measure an electromagnetic field distortion near a location, and to measure deformation of the location, respectively. Distortion in the electromagnetic field can be detected based on the measurements from the witness sensor and the reference sensor. An alert can be generated, or teleoperation of the surgical tool can be adjusted or paused, when a user interface device used to control the surgical tool is within a range of the distortion. The distortion can be from a known source, such as from actuation of a haptic motor of the user interface device, and the user console can adjust the actuation to reduce the likelihood that the distortion will disrupt surgical tool control. Other embodiments are described and claimed.

Abstract: A method for disengaging a surgical instrument of a surgical robotic system comprising receiving a gaze input from an eye tracker; determining, by one or more processors, whether the gaze input indicates the gaze of the user is outside or inside of the display; in response to determining the gaze input indicates the gaze of the user is outside of the display, determining an amount of time the gaze of the user is outside of the display; in response to determining the gaze of the user is outside of the display for less than a maximum amount of time, pause the surgical robotic system from a teleoperation mode; and in response to determining the gaze of the user is outside of the display for more than the maximum amount of time, disengage the surgical robotic system from the teleoperation mode.

Abstract: A method for engaging and disengaging a surgical instrument of a surgical robotic system comprising: receiving a plurality of interlock inputs from one or more interlock detection components of the surgical robotic system; determining, by one or more processors communicatively coupled to the interlock detection components, whether the plurality of interlock inputs indicate each of the following interlock requirements are satisfied: (1) a user is looking toward a display, (2) at least one or more user interface devices of the surgical robotic system are configured in a usable manner, and (3) a surgical workspace of the surgical robotic system is configured in a usable manner; in response to determining each of the interlock requirements are satisfied, transition the surgical robotic system into a teleoperation mode; and in response to determining less than all of the interlock requirements are satisfied, transition the surgical robotic system out of a teleoperation mode.

Abstract: Wearable user interface devices are described. A wearable user interface device can include a wearable base connected to a trackable device component by a linkage. The linkage can connect to a pivoted support that the trackable device is mounted on, and which maintains poses when the user interface device is not manipulated by a user's hand The pivoted support has several orthogonal axes intersecting at a center of rotation located inside a device body of the trackable device. Other embodiments are also described and claimed.

Abstract: Communication apparatus and devices for surgical robotic systems are described. The communication apparatus can include a user console in communication with a communication device having a surgical tool. The communication device can include a microphone to convert a sound input into an acoustic input signal. The communication device can transmit the acoustic input signal to the user console for reproduction as a sound output for a remote operator. The surgical tool can include an endoscope having several microphones mounted on a housing. The surgical tool can be a sterile barrier having a microphone and a drape. The microphone(s) of the surgical tools can face a surrounding environment such that a tableside staff is a source of the sound input that causes the sound output, and a surgeon and the table side staff can communicate in a noisy environment. Other embodiments are also described and claimed.

Abstract: Embodiments described herein provide various examples of a visual-haptic feedback system for generating a haptic feedback signal based on captured endoscopy images. In one aspect, the process for generating the haptic feedback signal includes the steps of: receiving an endoscopic video captured for a surgical procedure performed on a robotic surgical system; detecting a surgical task in the endoscopic video involving a given type of surgical tool-tissue interaction; selecting, a machine learning model constructed for analyzing the given type of surgical tool-tissue interaction; for a video image associated with the detected surgical task depicting the given type of surgical tool-tissue interaction, applying the selected machine learning model to the video image to predict a strength level of the depicted surgical tool-tissue interaction; and then providing the predicted strength level to a surgeon performing the surgical task as a haptic feedback signal for the given type of surgical tool-tissue interaction.

Abstract: Communication apparatus and devices for surgical robotic systems are described. The communication apparatus can include a user console in communication with a communication device having a surgical tool. The communication device can include a microphone to convert a sound input into an acoustic input signal. The communication device can transmit the acoustic input signal to the user console for reproduction as a sound output for a remote operator. The surgical tool can include an endoscope having several microphones mounted on a housing. The surgical tool can be a sterile barrier having a microphone and a drape. The microphone(s) of the surgical tools can face a surrounding environment such that a tableside staff is a source of the sound input that causes the sound output, and a surgeon and the tableside staff can communicate in a noisy environment. Other embodiments are also described and claimed.

Abstract: Communication apparatus and devices for surgical robotic systems are described. The communication apparatus can include a user console in communication with a communication device having a surgical tool. The communication device can include a microphone to convert a sound input into an acoustic input signal. The communication device can transmit the acoustic input signal to the user console for reproduction as a sound output for a remote operator. The surgical tool can include an endoscope having several microphones mounted on a housing. The surgical tool can be a sterile barrier having a microphone and a drape. The microphone(s) of the surgical tools can face a surrounding environment such that a tableside staff is a source of the sound input that causes the sound output, and a surgeon and the tableside staff can communicate in a noisy environment. Other embodiments are also described and claimed.

Abstract: Embodiments described herein provide various examples of a visual-haptic feedback system for generating a haptic feedback signal based on captured endoscopy images. In one aspect, the process for generating the haptic feedback signal includes the steps of: receiving an endoscopic video captured for a surgical procedure performed on a robotic surgical system; detecting a surgical task in the endoscopic video involving a given type of surgical tool-tissue interaction; selecting a machine learning model constructed for analyzing the given type of surgical tool-tissue interaction; for a video image associated with the detected surgical task depicting the given type of surgical tool-tissue interaction, applying the selected machine learning model to the video image to predict a strength level of the depicted surgical tool-tissue interaction; and then providing the predicted strength level to a surgeon performing the surgical task as a haptic feedback signal for the given type of surgical tool-tissue interaction.

Abstract: A valve body assembly for a hybrid drive unit includes a pressure regulator valve, a first on/off shift valve for controlling a stationary clutch, a second on/off shift valve for controlling a rotating clutch, and a third on/off shift valve for controlling a damper bypass clutch. The control system controls the stationary clutch and the rotating clutch to operate the hybrid drive unit in four different modes. The first shift valve, the second shift valve and the third shift valve do not include a regulated pressure feedback port, nor a pressure switch for indicating a clutch fill level.

Abstract: A valve body assembly for a hybrid drive unit includes a pressure regulator valve, a first on/off shift valve for controlling a stationary clutch, a second on/off shift valve for controlling a rotating clutch, and a third on/off shift valve for controlling a damper bypass clutch. The control system controls the stationary clutch and the rotating clutch to operate the hybrid drive unit in four different modes. The first shift valve, the second shift valve and the third shift valve do not include a regulated pressure feedback port, nor a pressure switch for indicating a clutch fill level.