Abstract: One example method for identifying useful segments in surgical videos includes accessing a video of a surgical procedure and user activities of a plurality of users who have watched the video of the surgical procedure. The user activities include operations performed during playback of the video. The method further includes dividing the video into multiple segments and determining a popularity score for each of the multiple segments based on the operations. Useful segments are identified from the segments based on the popularity scores. The method further includes generating metadata for the video of the surgical procedure to include an indication of the identified useful segments and associating the metadata with the video of the surgical procedure.

Abstract: One example method for detecting phases of a surgical procedure via video processing includes accessing a video of the surgical procedure and dividing the video into one or more blocks, each of the blocks containing one or more video frames. For each of the blocks, the method includes applying a prediction model on the video frames of the respective block to obtain a phase prediction for each of the video frames. The prediction model is configured to predict, for an input video frame, one of the plurality of phases of the surgical procedure. The method further includes generating an aggregated phase prediction for the respective block by aggregating the phase predictions of the video frames, and modifying the video of the surgical procedure to include an indication of a predicted phase of the respective block based on the aggregated phase prediction.

Abstract: One method for fleet management of robotic surgical systems includes receiving, by a management server from a robotic surgery system, a provisioning request; in response to receiving the provisioning request: generating an encryption key pair for the robotic surgery system, the encryption key pair comprising a private key and a public key, communicating the private key to the robotic surgery system, and communicating a set of secure certificates to the robotic surgery system, at least one of the secure certificates enabling secure communications between the robotic surgery system and the management server; receiving from the robotic surgery system, and using the at least one secure certificate enabling secure communications, a message indicating one or more software packages, each software package indicating a version of an installed software package on the robotic surgery system; communicating one or more software updates to the robotic surgery system based on the message; and registering, at the management

Abstract: Systems and methods for surgical training are provided. The system may include an operating display console for a first surgeon to view a video feed of a surgery while the first surgeon performs the surgery. The system may also include a training display console for a second surgeon to view the video feed of the surgery while the surgery is performed. The system may further include a controller that interfaces with the operating display console and the training display console. The training display console may include a training interface for pausing the video feed displayed on the training display console, and for switching from a display mode to a simulation mode for simulating one or more surgical steps of the surgery.

Abstract: One example method for detecting phases of a surgical procedure via video processing includes accessing a video of the surgical procedure and dividing the video into one or more blocks, each of the blocks containing one or more video frames. For each of the blocks, the method includes applying a prediction model on the video frames of the respective block to obtain a phase prediction for each of the video frames. The prediction model is configured to predict, for an input video frame, one of the plurality of phases of the surgical procedure. The method further includes generating an aggregated phase prediction for the respective block by aggregating the phase predictions of the video frames, and modifying the video of the surgical procedure to include an indication of a predicted phase of the respective block based on the aggregated phase prediction.

Abstract: One example method for improving robotic surgical safety via video processing includes identifying, during a robotic surgical procedure, one or more surgical tools using one or more images of the surgical procedure captured by a camera, the robotic surgical procedure employing a robotic surgical device controlling the one or more identified surgical tools; predicting, for at least one of the one or more images, one or more loaded surgical tools that are controlled by the robotic surgical device and should be in a field of view of the camera; comparing the one or more identified surgical tools with the one or more loaded surgical tools; determining that at least one of the one or more loaded surgical tools does not match any of the one or more identified surgical tools; and causing the at least one loaded tool of the robotic surgical device to be disabled.

Abstract: A system embodied in an article of clothing for synchronizing one or more breast-pumping sessions of an individual (such as a mother) and milk consumption by a second individual (such as a baby or infant) is described. In particular, based on measurements of a volume of the collected milk as a function of time and received information specifying milk consumption by the second individual as a function of time, a control circuit may determine a need for milk. Then, the control circuit may provide feedback based on the determined need for milk that synchronizes the one or more breast-pumping sessions and the milk consumption. For example, the feedback may alert the individual to initiate a breast-pumping session and/or may provide a signal to a breast pump that initiates a breast-pumping session.

Abstract: An apparatus, system and process for guiding a surgeon during a medical procedure to prevent surgical mistakes are described. The system may include a machine learning medical procedure server that generates one or more machine learning medical procedure models using, at least, medical procedure data captured during medical procedures performed at a plurality of different medical procedure systems. The system may also include a medical procedure system communicably coupled with the machine learning medical procedure server that receives a selected machine learning medical procedure model from the machine learning medical procedure server, and utilizes the selected machine learning medical procedure model during a corresponding medical procedure to control one or more operations of the medical procedure system.

Abstract: Systems and methods for surgical training are provided. The system may include an operating display console for a first surgeon to view a video feed of a surgery while the first surgeon performs the surgery. The system may also include a training display console for a second surgeon to view the video feed of the surgery while the surgery is performed. The system may further include a controller that interfaces with the operating display console and the training display console. The training display console may include a training interface for pausing the video feed displayed on the training display console, and for switching from a display mode to a simulation mode for simulating one or more surgical steps of the surgery.

Abstract: One example method for improving robotic surgical safety via video processing includes identifying, during a robotic surgical procedure, one or more surgical tools using one or more images of the surgical procedure captured by a camera, the robotic surgical procedure employing a robotic surgical device controlling the one or more identified surgical tools; predicting, for at least one of the one or more images, one or more loaded surgical tools that are controlled by the robotic surgical device and should be in a field of view of the camera; comparing the one or more identified surgical tools with the one or more loaded surgical tools; determining that at least one of the one or more loaded surgical tools does not match any of the one or more identified surgical tools; and causing the at least one loaded tool of the robotic surgical device to be disabled.

Abstract: A system for synchronizing one or more breast-pumping sessions of an individual (such as a mother) and milk consumption by a second individual (such as a baby or infant) is described. In particular, based on measurements of a volume of the collected milk as a function of time and received information specifying milk consumption by the second individual as a function of time, a control circuit may determine a need for milk. Then, the control circuit may provide feedback based on the determined need for milk that synchronizes the one or more breast-pumping sessions and the milk consumption. For example, the feedback may alert the individual to initiate a breast-pumping session and/or may provide a signal to a breast pump that initiates a breast-pumping session.

Abstract: One example method for identifying useful segments in surgical videos includes accessing a video of a surgical procedure and user activities of a plurality of users who have watched the video of the surgical procedure. The user activities include operations performed during playback of the video. The method further includes dividing the video into multiple segments and determining a popularity score for each of the multiple segments based on the operations. Useful segments are identified from the segments based on the popularity scores. The method further includes generating metadata for the video of the surgical procedure to include an indication of the identified useful segments and associating the metadata with the video of the surgical procedure.

Abstract: In some embodiments, a surgery assistance system is provided. The surgery assistance system comprises an image sensor, a video capture computing device, a notification computing device, and a machine learning (ML) processing computing device. The ML processing computing device is configured to receive video data, generate copies of the video data downsampled as appropriate for each of a plurality of machine learning models, process the copies of the video data using the machine learning models, and cause the notification computing device to provide at least one notification based on the machine learning models detecting at least one instance of an item.

Abstract: An apparatus, system and process for guiding a surgeon during a medical procedure to prevent surgical mistakes are described. The system may include a machine learning medical procedure server that generates one or more machine learning medical procedure models using, at least, medical procedure data captured during medical procedures performed at a plurality of different medical procedure systems. The system may also include a medical procedure system communicably coupled with the machine learning medical procedure server that receives a selected machine learning medical procedure model from the machine learning medical procedure server, and utilizes the selected machine learning medical procedure model during a corresponding medical procedure to control one or more operations of the medical procedure system.

Abstract: One example method for identifying useful segments in surgical videos includes accessing a video of a surgical procedure and user activities of a plurality of users who have watched the video of the surgical procedure. The user activities include operations performed during playback of the video. The method further includes dividing the video into multiple segments and determining a popularity score for each of the multiple segments based on the operations. Useful segments are identified from the segments based on the popularity scores. The method further includes generating metadata for the video of the surgical procedure to include an indication of the identified useful segments and associating the metadata with the video of the surgical procedure.

Abstract: One example method for detecting events during a surgery includes receiving surgical video of a surgery comprising a plurality of video frames; identifying, by a first trained machine-learning (“ML”) model, an event during a surgical procedure based on the surgical video; determining, by the first trained ML model, a subset of the plurality of video frames, the subset of the plurality of video frames corresponding to the event; determining, by a second trained ML model, a characteristic of the event based on the subset of the plurality of video frames; and generating metadata corresponding to event based on the characteristic of the event.

Abstract: Systems and methods for surgical training are provided. The system may include an operating display console for a first surgeon to view a video feed of a surgery while the first surgeon performs the surgery. The system may also include a training display console for a second surgeon to view the video feed of the surgery while the surgery is performed. The system may further include a controller that interfaces with the operating display console and the training display console. The training display console may include a training interface for pausing the video feed displayed on the training display console, and for switching from a display mode to a simulation mode for simulating one or more surgical steps of the surgery.

Abstract: One example method for comprehensively recording and reporting messages generated within a robotic surgical system includes capturing and storing network data on an internal network of the robotic surgical system. The method further includes subscribing to and receiving published messages from processes of interest executing on one or more task devices of the robotic surgical system. The received published messages each is associated with a priority. The method further involves sending at least a portion of the published messages according to the respective message priorities to a server device for analysis. Based on the analysis result, the method involves causing the surgical robotic system to be modified to improve its performance.

Abstract: Systems and methods for depth detection and virtual modeling using surgical robots during a surgical procedure are described. The robotic surgical systems include robotic arms with interchangeable surgical tools. An endoscope at the end of one of the robotic arms includes a depth sensor for detecting a distance from the camera to patient anatomy. The depth data and image data are used to generate a feature model of the patient anatomy for reference by a surgeon. The feature model is displayed to the surgeon in an augmented reality view with preoperative and intraoperative images mapped thereon for reference and guidance of a surgical procedure.

Abstract: Systems and methods for setting up and evaluating positioning of robotic arms of a surgical device using procedure data including previous surgeries is described herein. Robotic surgical systems described herein include color-configurable lights connected to each robotic arm of the system. A first color and a second color are selected and emitted from each light and the color displayed by the light of each robotic arm is also displayed at the surgical console of the system. Further, systems and methods for providing a position quality score for each robotic arm and surgical port as well as the overall system are described. The position quality score is determined based on comparing the position of each robotic arm to a database of positions for the robotic arms during a procedure and lowering the score for deviations from the database of positions.