Abstract: For control about a remote center of motion (RCM) of a surgical robotic system, possible configurations of a robotic manipulator are searched to find the configuration providing a greatest overlap of the workspace of the surgical instrument with the target anatomy. The force at the RCM may be measured, such as with one or more sensors on the cannula or in an adaptor connecting the robotic manipulator to the cannula. The measured force is used to determine a change in the RCM to minimize the force exerted on the patient at the RCM. Given this change, the configuration of the robotic manipulator may be dynamically updated. Various aspects of this RCM control may be used alone or in combination, such as to optimize the alignment of workspace to the target anatomy, to minimize force at the RCM, and/or to dynamically control the robotic manipulator configuration based on workspace alignment and force measurement.

Abstract: Mobile virtual reality system for simulation, training or demonstration of a surgical robotic system can include a virtual reality processor. The processor can generate a virtual surgical robot and render the virtual surgical robot on a display. The virtual surgical robot can include a virtual surgical tool. A handheld user input device (UID) can sense a hand input from a hand. A foot input device can sense a foot input from a foot. The virtual reality processor can be configured to control a movement or action of the virtual surgical robot based on the hand input, and change which of the virtual surgical instruments is controlled by the one or more handheld UIDs based on the foot input. Other embodiments and aspects are disclosed and claimed.

Abstract: This patent disclosure provides various embodiments of combining multiple modalities of non-text surgical data of different formats, in particular in forms of videos, images, and audios in a meaningful manner so that the combined data from the multiple modalities are compatible with text data. In some embodiments, prior to combining the multiple modalities of surgical data, multiple segmentation engines are used to segment and convert a corresponding modality of surgical data into a corresponding set of metrics and parameters. The multiple sets of metrics and parameters corresponding to the multiple modalities are then combined to generate a combined feature set. The combined feature set can be provided to a data analytics tool for performing comprehensive data analyses on the combined feature set to generate one or more predictions for the surgical procedure.

Abstract: A surgical robotic system senses position or orientation of an object, which may be a trocar that has a magnetic field. Magnetic field sensors are coupled to a surgical robotic arm. A machine learning model coupled to the magnetic field sensors is trained to output three-dimensional position and/or three-dimensional orientation of the trocar or other object. Other aspects are also described.

Abstract: Several solutions are described for making it easier to move a surgical robotics patient table, either from rest or to turn it. The table has a base, a support, and a table top on which the patient is supported. A number of casters are mounted to a body of the base. In one case, controllable jacks lift the body to release the casters thereby allowing them to swivel to a desired orientation. In another case, a drive assembly is rotatably coupled to the body of the base and includes an electric drive motor and a drive wheel to drive the table across the floor. Other aspects are also described and claimed.

Abstract: A surgical robotic system has a tool drive coupled to a distal end of a robotic arm that has a plurality of actuators. The tool drive has a docking interface to receive a trocar. One or more sensors in the docking interface sense a magnetic field generated by the trocar. One or more processors are configured to determine a position and orientation of the trocar based on the sensed magnetic field, and then drive the actuators to orient the docking interface to the determined orientation of the trocar, or otherwise guide the robotic arm toward the determined position of the trocar. Other aspects are also described and claimed.

Abstract: This patent disclosure provides various embodiments for anonymizing raw surgical procedure videos recorded by a recording device, such as an endoscope camera, during a surgical procedure performed on a patient inside an operating room (OR). In one aspect, a process for anonymizing raw surgical procedure videos recorded by a recording device within an OR is disclosed. This process can begin by receiving a set of raw surgical videos corresponding to a surgical procedure performed within the OR. The process next merges the set of raw surgical videos to generate a surgical procedure video corresponding to the surgical procedure. Next, the process detects image-based personally-identifiable information embedded in the set of raw video images of the surgical procedure video. When image-based personally-identifiable information is detected, the process automatically de-identifies the detected image-based personally-identifiable information in the surgical procedure video.

Abstract: Embodiments described herein provide various examples of a system for extracting an actual procedure duration composed of actual surgical tool-tissue interactions from an overall procedure duration of a surgical procedure on a patient. In one aspect, the system is configured to obtain the actual procedure duration by: obtaining an overall procedure duration of the surgical procedure; receiving a set of operating room (OR) data from a set of OR data sources collected during the surgical procedure, wherein the set of OR data includes an endoscope video captured during the surgical procedure; analyzing the set of OR data to detect a set of non-surgical events during the surgical procedure that do not involve surgical tool-tissue interactions; extracting a set of durations corresponding to the set of non-surgical events; and determining the actual procedure duration by subtracting the set of extracted durations from the overall procedure duration.

Abstract: A surgical robotic system has a tool drive coupled to a distal end of a robotic arm that has a plurality of actuators. The tool drive has a docking interface to receive a trocar. The system also includes one or more sensors that are operable to visually sense a surface feature of the trocar. One or more processors determine a position and orientation of the trocar, based on the visually sensed surface feature. In response, the processor controls the actuators to orient the docking interface to the determined orientation of the trocar and to guide the robotic arm toward the determined position of the trocar. Other aspects are also described and claimed.

Abstract: A virtual operating room (OR) is generated that includes virtual surgical equipment based on existing OR models and existing equipment models. Sensor feedback is received that defines a physical OR having physical equipment, the physical equipment including a surgical robotic system within the physical OR. The virtual OR and the virtual surgical equipment is updated based on the sensor feedback. A layout of the virtual surgical equipment is optimized in the virtual OR. The virtual OR and the virtual surgical equipment are rendered on a display.

Abstract: Planning surgical robotic workflow with a surgical robotic system can include generating a virtual surgical environment, the virtual surgical environment including a virtual surgical robotic arm and a virtual patient. A workspace can be determined in the virtual patient. A position of a virtual tool, attached to the virtual surgical robotic arm, can be determined. A position of the virtual surgical robotic arm can be determined to maintain a reach of the virtual tool in the workspace.

Abstract: An autostereoscopic three-dimensional display system for surgical robotics has an autostereoscopic three-dimensional display configured to receive and display video from a surgical robotics camera, and a first sensor assembly and a second sensor assembly. A processor is configured to detect and track an eye position or a head position of a user relative to the display based on processing output data of the first sensor assembly, and to detect and track a gaze of the user based on processing output data of the second sensor assembly. The processor further is configured to modify or control an operation of the display system based on the detected gaze of the user. A spatial relationship of the display also can be automatically adjusted in relation to the user based on the detected eye or head position of the user to optimize the user's visualization of three-dimensional images on the display.

Abstract: Robotic arms and surgical robotic systems incorporating such arms are described. A robotic arm includes a roll joint connected to a prismatic link by a pitch joint and a tool drive connected to the prismatic link by another pitch joint. The prismatic link includes several prismatic sublinks that are connected by a prismatic joint. A surgical tool supported by the tool drive can insert into a patient along an insertion axis through a remote center of motion of the robotic arm. Movement of the robotic arm can be controlled to telescopically move the prismatic sublinks relative to each other by the prismatic joint while maintaining the remote center of motion fixed. Other embodiments are also described and claimed.

Abstract: Embodiments described herein provide various examples of a surgical procedure analysis system for extracting an actual procedure duration that involves actual surgical tool-tissue interactions from a total procedure duration of a surgical procedure. In one aspect, the process for generating the haptic feedback signal includes the steps of: obtaining the total procedure duration of the surgical procedure; receiving a set of operating room (OR) data from a set of OR data sources collected during the surgical procedure; analyzing the set of OR data to detect a set of non-surgical events during the surgical procedure that do not involve surgical tool-tissue interactions; extracting a set of durations corresponding to the set of events; and determining the actual procedure duration by subtracting the combined set of durations corresponding to the set of events from the total procedure duration.

Abstract: An autostereoscopic three-dimensional display system for surgical robotics has an autostereoscopic three-dimensional display configured to receive and display video from a surgical robotics camera, and a first sensor assembly and a second sensor assembly. A processor is configured to detect and track an eye position or a head position of a user relative to the display based on processing output data of the first sensor assembly, and to detect and track a gaze of the user based on processing output data of the second sensor assembly. The processor further is configured to modify or control an operation of the display system based on the detected gaze of the user. A spatial relationship of the display also can be automatically adjusted in relation to the user based on the detected eye or head position of the user to optimize the user's visualization of three-dimensional images on the display.

Abstract: Embodiments described herein provide various examples of synchronizing the playback of a recorded video of a surgical procedure with a live video feed of a user performing the surgical procedure. In one aspect, a system can simultaneously receive a recorded video of a surgical procedure and a live video feed of a user performing the surgical procedure in a training session. More specifically, the recorded video is shown to the user as a training reference, and the surgical procedure includes a set of surgical tasks. The system next simultaneously monitors the playback of a current surgical task in the set of surgical tasks in the recorded video and the live video feed depicting the user performing the current surgical task. Next, the system detects that the end of the current surgical task has been reached during the playback of the recorded video.

Abstract: This patent disclosure provides various embodiments of combining multiple modalities of non-text surgical data of different formats, in particular in forms of videos, images, and audios in a meaningful manner so that the combined data from the multiple modalities are compatible with text data. In some embodiments, prior to combining the multiple modalities of surgical data, multiple segmentation engines are used to segment and convert a corresponding modality of surgical data into a corresponding set of metrics and parameters. The multiple sets of metrics and parameters corresponding to the multiple modalities are then combined to generate a combined feature set. The combined feature set can be provided to a data analytics tool for performing comprehensive data analyses on the combined feature set to generate one or more predictions for the surgical procedure.

Abstract: This patent disclosure provides various embodiments for anonymizing raw surgical procedure videos recorded by a recording device, such as an endoscope camera, during a surgical procedure performed on a patient inside an operating room (OR). In one aspect, a process for anonymizing raw surgical procedure videos recorded by a recording device within an OR is disclosed. This process can begin by receiving a set of raw surgical videos corresponding to a surgical procedure performed within the OR. The process next merges the set of raw surgical videos to generate a surgical procedure video corresponding to the surgical procedure. Next, the process detects image-based personally-identifiable information embedded in the set of raw video images of the surgical procedure video. When image-based personally-identifiable information is detected, the process automatically de-identifies the detected image-based personally-identifiable information in the surgical procedure video.

Abstract: Embodiments described herein provide various examples of preparing two procedure videos, in particular two surgical procedure videos for comparative learning. In some embodiments, to allow comparative learning of two recorded surgical videos, each of the two recorded surgical videos is segmented into a sequence of predefined phases/steps. Next, corresponding phases/steps of the two segmented videos are individually time-synchronized in pair-wise manner so that a given phase/step of one recorded video and a corresponding phase/step of the other segmented video can have the same or substantially the same starting time and ending timing during comparative playbacks of the two recorded videos.

Abstract: A surgical robotic system has a tool drive coupled to a distal end of a robotic arm that has a plurality of actuators. The tool drive has a docking interface to receive a trocar. One or more sensors in the docking interface sense a magnetic field generated by the trocar. One or more processors are configured to determine a position and orientation of the trocar based on the sensed magnetic field, and then drive the actuators to orient the docking interface to the determined orientation of the trocar, or otherwise guide the robotic arm toward the determined position of the trocar. Other aspects are also described and claimed.