Abstract: A method for processing content being collected during a gastrointestinal (GI) procedure. The method includes: displaying a live video feed from a scope within a GI tract of a patient; capturing and displaying an image from the live video feed in response to a natural language (NL) command recognized during the GI procedure with an artificial intelligence (AI) language processor; extracting a label and a GI location associated with the image from uttered NL using the AI language processor during the GI procedure; annotating the image with the label and GI location while the image is displayed during the GI procedure; and displaying a GI tract replica during the procedure, wherein the replica includes a visual indicator that maps the GI location of the image in the GI tract.

Abstract: An AI based platform for processing information collected during a medical procedure. A method includes capturing images and speech during a medical procedure; processing the images using a trained classifier to identify image-based quality-of-care indicators (QIs); converting the speech into text; parsing the text into sentences; performing a search and replace on predefined text patterns in the sentences; identifying text-based QIs in the sentences; classifying sentences into sentence types using a trained model; updating sentences by integrating the image-based QIs with text-based QIs; and outputting structured data that includes sentences organized by sentence type.