Abstract: A system is described for generating diagnostic information from a video sequence of ultrasound images acquired in “blind sweeps”, i.e., without operator seeing ultrasound images as they are acquired. We disclose two different types of machine learning systems for predicting diagnostic information: a “Temporal Accumulation” system and a “3-D Modeling Component” system. These machine learning systems could be implemented in several possible ways: using just one or the other of them in any given implementation, or using both of them in combination. We also disclose a computing system which implements (a) an image selection system including at least one machine learning model trained to identify clinically suitable images from the sequence of ultrasound images and (b) an image diagnosis/measurement system including of one or more machine learning models, configured to obtain the clinically suitable images identified by the image selection system and further process such images to predict health states.

Abstract: A method is disclosed of processing a set of images. Each image in the set has an associated counterpart image. One or more regions of interest (ROIs) are identified in one or more of the images in the set of images. For ROI identified, a reference region is identified in the associated counterpart image. ROIs and associated reference regions are cropped out, thereby forming cropped pairs of images 1 . . . n1, that are fed to a deep learning model trained to make a prediction of probability of a state of the ROI, e.g., disease state, which generates a prediction Pi-, (i=1 . . . n) for each cropped pair. The model generates an overall prediction P from each of the predictions Pi. A visualization of the set of medical images and the associated counterpart images including the cropped pair of images is generated.

Abstract: A system is described for conducting an ultrasound scan on a human subject. The system includes an ultrasound probe generating ultrasound image data and provisioned with one or more position sensors generating real time position and orientation data as to the position of the ultrasound probe and orientation in three-dimensional space during use of the probe; one or more machine learning models trained to correlate ultrasound images with probe position and orientation, wherein the one or more machine learning models receive images generated from the ultrasound probe; a feedback generator generating feedback data based on the current probe position determined by the position sensors; and a feedback display receiving the feedback data providing real-time suggestions to the user of the ultrasound probe for adjusting the probe position, orientation, pressure and/or other parameters of the ultrasound probe to improve the quality of the images generated from the ultrasound probe.

Abstract: A computer-implemented system is described for identifying and retrieving similar radiology images to a query image. The system includes one or more fetchers receiving the query image and retrieving a set of candidate similar radiology images from a data store. One or more scorers receive the query image and the set of candidate similar radiology images and generate a similarity score between the query image and each candidate image. A pooler receives the similarity scores from the one or more scorers, ranks the candidate images, and returns a list of the candidate images reflecting the ranking. The scorers implement a modelling technique to generate the similarity score capturing a plurality of similarity attributes of the query image and the set of candidate similar radiology images and annotations associated therewith.

Abstract: A method and system to generate a probabilistic prediction of the presence/absence of cancer in longitudinal and current image datasets, and/or multimodal image datasets, and the location of the cancer, is described. The method and system uses an ensemble of deep learning models. The ensemble includes a global model in the form of a 3D convolutional neural network (CNN) extracting features in the datasets indicative of the presence of cancer on a global basis. The ensemble also includes a two-stage prediction model which includes a first stage or detection model which identifies cancer detection candidates (different cropped volumes of 3D data in the a dataset containing candidates which may be cancer) and a second stage or probability model which incorporates the longitudinal datasets (or multimodal images in a multimodal dataset) and the extracted features from the global model and assigns a cancer probability p to each of the cancer detection candidates.

Abstract: A method is provided for processing medical text and associated medical images. A natural language processor configured as a deep conventional neural network is trained on a first corpus of curated free-text, medical reports each of which having one or more structured labels assigned by an medical expert. The network is trained to learn to read additional free-text medical reports and produce predicted structured labels. The natural language processor is applied to a second corpus of free-text medical reports that are associated with medical images. The natural language processor generates structured labels for the associated medical images. A computer vision model is trained using the medical images and the structured labels generated. The computer vision model can thereafter assign a structured label to a further input medical image. In one example, the medical images are chest X-rays.

Abstract: A computer-implemented system is described for identifying and retrieving similar radiology images to a query image. The system includes one or more fetchers receiving the query image and retrieving a set of candidate similar radiology images from a data store. One or more scorers receive the query image and the set of candidate similar radiology images and generate a similarity score between the query image and each candidate image. A pooler receives the similarity scores from the one or more scorers, ranks the candidate images, and returns a list of the candidate images reflecting the ranking. The scorers implement a modelling technique to generate the similarity score capturing a plurality of similarity attributes of the query image and the set of candidate similar radiology images and annotations associated therewith.