Abstract: For intraoperative guidance to an object, a machine-learned model is used to predict the intra-operative location of an object identified in pre-operative planning. The predicted location is used by the surgeon or controller during the operation, such as during a bronchoscopy.

Abstract: Systems, methods, and instrumentalities are disclosed for computer vision-based surgical workflow recognition using natural language processing (NLP) techniques. Surgical video of surgical procedures may be processed and analyzed, for example, to achieve workflow recognition. Surgical phases may be determined based on the surgical video and segmented to generate an annotated video representation. The annotated video representation of the surgical video may provide information associated with the surgical procedure. For example, the annotated video representation may provide information on surgical phases, surgical events, surgical tool usage, and/or the like.

Abstract: For anatomy measurement in a surgical system, a machine-learned model is used to determine the location or length. For example, in RYGB, the machine-learned model indicates how much of the bowel has been pulled or run. As another RYGB example, the machine-learned model indicates a length along the bowel to a current location. In yet another RYGB example, the machine-learned model indicates a location for incision based on a desired length. Using an image or images, such as a video stream from an endoscope, the artificial intelligence provides the measurement, avoiding running with estimated steps size or use an inserted ruler.