Abstract: A location of an anatomical structure in image(s) from a surgical procedure is predicted using machine learning. An image/video capture device such as an endoscope, a wearable camera, a stationary camera, etc., can be used to capture the image(s). A confidence score of the prediction of the machine learning is determined. A surgeon can be provided an augmented visualization of the surgical procedure by displaying one or more graphical overlays based on the findings of the machine learning to enhance the surgeon-s information.

Abstract: An aspect includes a computer-implemented method that accesses input data including spatial data and/or sensor data temporally associated with a video stream of a surgical procedure. One or more machine-learning models predict a state of the surgical procedure based on the input data. The one or more machine-learning models detect one or more surgical instruments at least partially depicted in the video stream based on the input data. A state indicator and one or more surgical instrument indicators temporally correlated with the video stream are output.