Abstract: An annotation system facilitates collection of labels for images, video, or other content items relevant to training machine learning models associated with surgical applications or other medical applications. The annotation system enables an administrator to configure annotation jobs associated with training a machine learning model. The job configuration controls presentation of content items to various participating annotators via an annotation application and collection of the labels via a user interface of the annotation application. The annotation application enables the participating annotators to provide inputs in a simple and efficient manner, such as by providing gesture-based inputs or selecting graphical elements associated with different possible labels.