Abstract: Various of the disclosed embodiments relate to systems and methods for recognizing types of surgical operations from data gathered in a surgical theater, such as recognizing a surgery procedure and corresponding specialty from endoscopic video data. Some embodiments select discrete frame sets from the data for individual consideration by a corpus of machine learning models, Some embodiments may include an uncertainty indication with each classification to guide downstream decision-making based upon the classification. For example, where the system is used as part of a data annotation pipeline, uncertain classifications may be flagged for downstream confirmation and review by a human reviewer.

Abstract: Various of the disclosed embodiments are directed to computer-implemented systems and methods for recognizing surgical tasks from surgical data, In some embodiments an ensemble model configured to receive video data, kinematics data, and system event data from the surgical theater may be implemented. The ensemble model may implement modular streams for processing the data, facilitating predictions even when less than all the data types are available. In some embodiments, smoothing operations may help facilitate more accurate prediction results, Various of the embodiments may be employed in real-time during surgery, providing predictions at per-second intervals.

Abstract: Various of the disclosed embodiments relate to systems and methods for processing surgical data to facilitate further downstream operations. For example, some embodiments may include machine learning systems trained to recognize whether video from surgical visualization tools, such as endoscopes, depicts a field of view inside or outside the patient body. The system may excise or whiteout frames of video appearing outside the patient so as to remove potentially compromising personal information, such as the identities of members of the surgical team, the patients identity, configurations of the surgical theater, etc. Appropriate removal of such non-surgical data may facilitate downstream processing, e.g., by complying with regulatory requirements as well as by removing extraneous data potentially inimical to further downstream processing, such as training a downstream classifier.

Abstract: Various of the disclosed embodiments relate to computer systems and computer-implemented methods for measuring and monitoring surgical performance, For example, the system may receive raw data acquired from the surgical theater, generate and select features from the data amenable to analysis, and then train a machine learning classifier using the selected features to facilitate subsequent assessment of other surgeons' performances. Generation and selection of the features may itself involve application of a machine learning classifier in some embodiments. While some embodiments contemplate raw data acquired from surgical robotic systems, some embodiments facilitate assessments upon data acquired from non-robotic surgical theaters.