Abstract: A system and method of identifying an adverse event using a video of a surgery and a two-stage surgical phase recognition module. The method includes receiving, by the module, a video of the surgery, where the video comprises a sequence of video frames. The module comprises a first stage that includes a neural network and a second stage that includes a multi-stage temporal convolution network. The method includes extracting, using the first stage, visual information content of a single frame based on the single frame; identifying, using the second stage, surgical phases captured in the frames of the video based on the visual information content from the first stage; and identifying, using the identified surgical phases, an adverse event during the surgery. An adverse event includes the omission of a surgical phase and an injury to the patient. The identification can occur in real-time or near-real-time.

Abstract: A computer system may perform an image-related function using a segmentation (e.g., an image mask) that has been generated by a custom segmentation machine learning model. To begin, the system may receive image data corresponding to a surgical scene including a background that includes an anatomical feature and at least one surgical tool. The system may also generate segmentation data using the custom segmentation machine learning model based on inputting the first image data to the custom segmentation machine learning model. The system may also include generating a segmentation of the at least one surgical tool using the segmentation data. Once the segmentation has been generated, the system may perform an image-related function using the segmentation.

Abstract: The present disclosure is directed towards systems and methods that leverage machine-learned models to decrease the rate at which abnormal sites are missed during a gastroenterological procedure. In particular, the system and methods of the present disclosure can use machine-learning techniques to determine the coverage rate achieved during a gastroenterological procedure. Measuring the coverage rate of the gastroenterological procedure can allow medical professionals to be alerted when the coverage output is deficient and thus allow an additional coverage to be achieved and as a result increase in the detection rate for abnormal sites (e.g., adenoma, polyp, lesion, tumor, etc.) during the gastroenterological procedure.

Abstract: A system for generating a set of digital image features, comprising at least one hardware processor adapted for: producing a plurality of input groups of features, each produced by extracting a plurality of features from one of a plurality of digital images; computing an output group of features by inputting the plurality of input groups of features into at least one prediction model trained to produce a model group of features in response to at least two groups of features, such that a model set of labels indicative of the model group of features is similar, according to at least one similarity test, to a target set of labels computed by applying at least one set operator to a plurality of input sets of labels each indicative of one of the at least two groups of features; and providing the output group of features to at least one other processor.

Abstract: The present disclosure is directed towards systems and methods that leverage machine-learned models to decrease the rate at which abnormal sites are missed during a gastroenterological procedure. In particular, the system and methods of the present disclosure can use machine-learning techniques to determine the coverage rate achieved during a gastroenterological procedure. Measuring the coverage rate of the gastroenterological procedure can allow medical professionals to be alerted when the coverage output is deficient and thus allow an additional coverage to be achieved and as a result increase in the detection rate for abnormal sites (e.g., adenoma, polyp, lesion, tumor, etc.) during the gastroenterological procedure.

Abstract: A system for generating a set of digital image features, comprising at least one hardware processor adapted for: producing a plurality of input groups of features, each produced by extracting a plurality of features from one of a plurality of digital images; computing an output group of features by inputting the plurality of input groups of features into at least one prediction model trained to produce a model group of features in response to at least two groups of features, such that a model set of labels indicative of the model group of features is similar, according to at least one similarity test, to a target set of labels computed by applying at least one set operator to a plurality of input sets of labels each indicative of one of the at least two groups of features; and providing the output group of features to at least one other processor

Abstract: Embodiments of the present systems and methods may provide techniques for handling acoustic-lips timing mismatch in audiovisual processing. In embodiments, the context-dependent time shift between the audio and visual streams may be explicitly modeled using an attention mechanism.

Abstract: Embodiments of the present systems and methods may provide techniques for handling acoustic-lips timing mismatch in audiovisual processing. In embodiments, the context-dependent time shift between the audio and visual streams may be explicitly modeled using an attention mechanism.

Abstract: Methods, computing systems and computer program products implement embodiments of the present invention that include receiving, at a first time, first video and first audio signals generated in response to a user uttering a passphrase, and receiving, at a second time subsequent to the first time, second video and second audio signals generated in response the user uttering the passphrase. Upon computing an audio temporal alignment between the first and the second audio signals and computing a video temporal alignment between the first and the second video signal, the user can be authenticated by comparing the audio temporal alignment to the video temporal alignment.

Abstract: A system for wide-scale light field imaging comprising: a network of multiple terrestrial cameras positioned at multiple known locations, where each of the cameras comprises a wide-angle lens directed towards the sky and configured to capture one or more images of the sky, and a processor configured to operate a wired and/or wireless communications infrastructure for transmitting the captured images; and a server configured to receive the wirelessly transmitted images and jointly process the received images, thereby producing a wide-scale light field image of the sky.

Abstract: Methods, computing systems and computer program products implement embodiments of the present invention that include receiving, at a first time, first video and first audio signals generated in response to a user uttering a passphrase, and receiving, at a second time subsequent to the first time, second video and second audio signals generated in response the user uttering the passphrase. Upon computing an audio temporal alignment between the first and the second audio signals and computing a video temporal alignment between the first and the second video signal, the user can be authenticated by comparing the audio temporal alignment to the video temporal alignment.

Abstract: A system for wide-scale light field imaging comprising: a network of multiple terrestrial cameras positioned (at multiple known locations, where each of the cameras comprises a wide-angle lens directed towards the sky and configured to capture one or more images of the sky, and a processor configured to operate a wired and/or wireless communications infrastructure for transmitting the captured images; and a server configured to receive the wirelessly transmitted images and jointly process the received images, thereby producing a wide-scale light field image of the sky.