Abstract: A machine learning model has two stages. In a first stage, features from one or more frames of a surgical video are extracted, wherein the features include presence of a surgical instrument and type of the surgical instrument. A second stage analyzes the surgical video based on the extracted features to recognize a video segment, wherein the recognized video segment includes a detected presence of the surgical instrument, and where the video segment is recognized by a multi-stage temporal convolution network (MS-TCN) or a vision transformer. Other aspects are also described and claimed.