Abstract: A virtual surgical robot being built from kinematic data is rendered to a display. A user input is received to effect a movement or a configuration of the virtual surgical robot. The kinematic data is modified based on evaluation of the movement or the configuration of the virtual surgical robot.

Abstract: This patent disclosure provides various embodiments of combining multiple modalities of non-text surgical data in forms of videos, images, and audios in a meaningful manner so that the combined data can be used to perform comprehensive data analytics for a surgical procedure. In some embodiments, the disclosed system can begin by receiving two or more modalities of surgical data during the surgical procedure. The system then time-synchronizes the two or more modalities of surgical data to generate two or more modalities of time-synchronized surgical data. Next, the system converts each modality of the time-synchronized surgical data into a corresponding array of values of a common format. The system then combines the two or more arrays of values to generate a combined set of values. The system subsequently performs comprehensive data analytics on the combined set of values to generate a surgical decision for the surgical procedure.

Abstract: Several solutions are described for making it easier to move a surgical robotics patient table, either from rest or to turn it. The table has a base, a support, and a table top on which the patient is supported. A number of casters are mounted to a body of the base. In one case, controllable jacks lift the body to release the casters thereby allowing them to swivel to a desired orientation. In another case, a drive assembly is rotatably coupled to the body of the base and includes an electric drive motor and a drive wheel to drive the table across the floor. Other aspects are also described and claimed.

Abstract: Errors in a blended stream that would result in non-display or obscuring of a live video stream from a medical device may be automatically detected, and a failover stream corresponding to the first live video stream may be displayed to medical personnel. For example, one or more second input streams that are being blended may contain no data or invalid data which may result in the blended stream not displaying (or obscuring) the live video stream (if the blended were displayed). Switching from blending to a failover buffer may occur within the time to process a single video image frame. Upon detection (prior to display) that the blended stream would not display the live video stream, display of the live video stream from the failover buffer may be initiated. Other aspects are also described and claimed.

Abstract: A surgical robotic system has a tool drive coupled to a distal end of a robotic arm that has a plurality of actuators. The tool drive has a docking interface to receive a trocar. The system also includes one or more sensors that are operable to visually sense a surface feature of the trocar. One or more processors determine a position and orientation of the trocar, based on the visually sensed surface feature. In response, the processor controls the actuators to orient the docking interface to the determined orientation of the trocar and to guide the robotic arm toward the determined position of the trocar. Other aspects are also described and claimed.

Abstract: A virtual operating room (OR) is generated that includes virtual surgical equipment based on existing OR models and existing equipment models. Sensor feedback is received that defines a physical OR having physical equipment, the physical equipment including a surgical robotic system within the physical OR. The virtual OR and the virtual surgical equipment is updated based on the sensor feedback. A layout of the virtual surgical equipment is optimized in the virtual OR. The virtual OR and the virtual surgical equipment are rendered on a display.

Abstract: An autostereoscopic three-dimensional display system for surgical robotics has an autostereoscopic three-dimensional display configured to receive and display video from a surgical robotics camera, and a first sensor assembly and a second sensor assembly. A processor is configured to detect and track an eye position or a head position of a user relative to the display based on processing output data of the first sensor assembly, and to detect and track a gaze of the user based on processing output data of the second sensor assembly. The processor further is configured to modify or control an operation of the display system based on the detected gaze of the user. A spatial relationship of the display also can be automatically adjusted in relation to the user based on the detected eye or head position of the user to optimize the user's visualization of three-dimensional images on the display.

Abstract: Embodiments described herein provide various examples of a system for extracting an actual procedure duration composed of actual surgical tool-tissue interactions from an overall procedure duration of a surgical procedure on a patient. In one aspect, the system is configured to obtain the actual procedure duration by: obtaining an overall procedure duration of the surgical procedure; receiving a set of operating room (OR) data from a set of OR data sources collected during the surgical procedure, wherein the set of OR data includes an endoscope video captured during the surgical procedure; analyzing the set of OR data to detect a set of non-surgical events during the surgical procedure that do not involve surgical tool-tissue interactions; extracting a set of durations corresponding to the set of non-surgical events; and determining the actual procedure duration by subtracting the set of extracted durations from the overall procedure duration.

Abstract: A surgical robotic system senses position or orientation of an object, which may be a trocar that has a magnetic field. Magnetic field sensors are coupled to a surgical robotic arm. A machine learning model coupled to the magnetic field sensors is trained to output three-dimensional position and/or three-dimensional orientation of the trocar or other object. Other aspects are also described.

Abstract: This patent disclosure provides various verification techniques to ensure that anonymized surgical procedure videos are indeed free of any personally-identifiable information (PII). In a particular aspect, a process for verifying that an anonymized surgical procedure video is free of PII is disclosed. This process can begin by receiving a surgical video corresponding to a surgery. The process next removes personally-identifiable information (PII) from the surgical video to generate an anonymized surgical video. Next, the process selects a set of verification video segments from the anonymized surgical procedure video. The process subsequently determines whether each segment in the set of verification video segments is free of PII. If so, the process replaces the surgical video with the anonymized surgical video for storage. If not, the process performs additional PII removal steps on the anonymized surgical video to generate an updated anonymized surgical procedure video.

Abstract: An autostereoscopic three-dimensional display system for surgical robotics has an autostereoscopic three-dimensional display configured to receive and display video from a surgical robotics camera, and a first sensor assembly and a second sensor assembly. A processor is configured to detect and track an eye position or a head position of a user relative to the display based on processing output data of the first sensor assembly, and to detect and track a gaze of the user based on processing output data of the second sensor assembly. The processor further is configured to modify or control an operation of the display system based on the detected gaze of the user. A spatial relationship of the display also can be automatically adjusted in relation to the user based on the detected eye or head position of the user to optimize the user's visualization of three-dimensional images on the display.

Abstract: Mobile virtual reality system for simulation, training or demonstration of a surgical robotic system can include a virtual reality processor. The processor can generate a virtual surgical robot and render the virtual surgical robot on a display. The virtual surgical robot can include a virtual surgical tool. A handheld user input device (UID) can sense a hand input from a hand. A foot input device can sense a foot input from a foot. The virtual reality processor can be configured to control a movement or action of the virtual surgical robot based on the hand input, and change which of the virtual surgical instruments is controlled by the one or more handheld UIDs based on the foot input. Other embodiments and aspects are disclosed and claimed.

Abstract: This disclosure provides techniques of synchronizing the playback of two recorded videos of the same surgical procedure. In one aspect, a process for generating a composite video from two recorded videos of a surgical procedure is disclosed. This process begins by receiving a first and second surgical videos of the same surgical procedure. The process then performs phase segmentation on each of the first and second surgical videos to segment the first and second surgical videos into a first set of video segments and a second set of video segments, respectively, corresponding to a sequence of predefined phases. Next, the process time-aligns each video segment of a given predefined phase in the first video with a corresponding video segment of the given predefined phase in the second video. The process next displays the time-aligned first and second surgical videos for comparative viewing.

Abstract: Embodiments described herein provide various examples of a system for extracting an actual procedure duration composed of actual surgical tool-tissue interactions from an overall procedure duration of a surgical procedure on a patient. In one aspect, the system is configured to obtain the actual procedure duration by: obtaining an overall procedure duration of the surgical procedure; receiving a set of operating room (OR) data from a set of OR data sources collected during the surgical procedure, wherein the set of OR data includes an endoscope video captured during the surgical procedure; analyzing the set of OR data to detect a set of non-surgical events during the surgical procedure that do not involve surgical tool-tissue interactions; extracting a set of durations corresponding to the set of non-surgical events; and determining the actual procedure duration by subtracting the set of extracted durations from the overall procedure duration.

Abstract: A surgical robotic system has a tool drive coupled to a distal end of a robotic arm that has a plurality of actuators. The tool drive has a docking interface to receive a trocar. The system also includes one or more sensors that are operable to visually sense a surface feature of the trocar. One or more processors determine a position and orientation of the trocar, based on the visually sensed surface feature. In response, the processor controls the actuators to orient the docking interface to the determined orientation of the trocar and to guide the robotic arm toward the determined position of the trocar. Other aspects are also described and claimed.

Abstract: Mobile virtual reality system for simulation, training or demonstration of a surgical robotic system can include a virtual reality processor. The processor can generate a virtual surgical robot and render the virtual surgical robot on a display. The virtual surgical robot can include a virtual surgical tool. A handheld user input device (UID) can sense a hand input from a hand. A foot input device can sense a foot input from a foot. The virtual reality processor can be configured to control a movement or action of the virtual surgical robot based on the hand input, and change which of the virtual surgical instruments is controlled by the one or more handheld UIDs based on the foot input. Other embodiments and aspects are disclosed and claimed.

Abstract: This patent disclosure provides various verification techniques to ensure that anonymized surgical procedure videos are indeed free of any personally-identifiable information (PII). In a particular aspect, a process for verifying that an anonymized surgical procedure video is free of PII is disclosed. This process can begin by receiving a surgical video corresponding to a surgery. The process next removes personally-identifiable information (PII) from the surgical video to generate an anonymized surgical video. Next, the process selects a set of verification video segments from the anonymized surgical procedure video. The process subsequently determines whether each segment in the set of verification video segments is free of PII. If so, the process replaces the surgical video with the anonymized surgical video for storage. If not, the process performs additional PII removal steps on the anonymized surgical video to generate an updated anonymized surgical procedure video.

Abstract: This disclosure provides techniques of synchronizing the playback of two recorded videos of the same surgical procedure. In one aspect, a process for generating a composite video from two recorded videos of a surgical procedure is disclosed. This process begins by receiving a first and second surgical videos of the same surgical procedure. The process then performs phase segmentation on each of the first and second surgical videos to segment the first and second surgical videos into a first set of video segments and a second set of video segments, respectively, corresponding to a sequence of predefined phases. Next, the process time-aligns each video segment of a given predefined phase in the first video with a corresponding video segment of the given predefined phase in the second video. The process next displays the time-aligned first and second surgical videos for comparative viewing.

Abstract: Errors in a blended stream that would result in non-display or obscuring of a live video stream from a medical device may be automatically detected, and a failover stream corresponding to the first live video stream may be displayed to medical personnel. For example, one or more second input streams that are being blended may contain no data or invalid data which may result in the blended stream not displaying (or obscuring) the live video stream (if the blended were displayed). Switching from blending to a failover buffer may occur within the time to process a single video image frame. Upon detection (prior to display) that the blended stream would not display the live video stream, display of the live video stream from the failover buffer may be initiated. Other aspects are also described and claimed.

Abstract: A surgical stapler for a surgical robotic system, the surgical stapler including a jaw coupled to a base, the jaw having a first anvil that moves relative to a second anvil between an open position and a closed position; and a force sensor operable to detect a force applied to the jaw.