Abstract: Described herein are systems, methods, and instrumentalities associated with processing mammogram images using machine learning based techniques. An apparatus as described herein may obtain a mammographic image of a person, extract features from the mammographic image using a feature encoder, and predict a health condition of the person based on the extracted features. The feature encoder may be trained using a self-supervised technique and based on multi-view mammogram images that may belong to a same person or to different people.

Abstract: A method for coronary road mapping may include receiving a current image associated with a medical scan, determining an image that matches the current image from a sequence of images for which respective blood vessel maps have been determined, and overlaying the current image with the blood vessel map associated with the matching image. The sequence of images may be captured when a contrast agent was present, and the matching image may be identified in response to determining that the physiological phase (e.g., cardiac cycle) and/or view depicted by the image match those depicted by the current image. Motion compensation may be performed as part of the blood vessel overlay, which may enable better visualization of blood vessels and medical device(s) (e.g., a catheter) placed in the blood vessels even when the contrast agent is worn out.

Abstract: An object or person in a medical environment may be identified based on images of the medical environment. The identification may include determining an identifier associated with the object or the person, a position of the object or the person in the medical environment, and a three-dimensional (3D) shape/pose of the object or the person. Representation information that indicates at least the determined identifier, position in the medical environment, and 3D shape/pose of the object or the person may be generated and then used (e.g., by a visualization device) together with one or more predetermined 3D models to determine a 3D model for the object or the person identified in the medical environment and generate a visual depiction of at least the object or the person in the medical environment based on the determined 3D model and the position of the object or the person in the medical environment.

Abstract: Detecting motions associated with a body part of a patient may include using an image sensor installed inside a medical scanner to capture first and second images of the patient inside the medical scanner, wherein the first image may depict the patient in a first state and the second image may depict the patient in a second state. A first area, in the first image, that corresponds to the body part of the patient may be identified and a second area, in the second image, that corresponds to the body part may also be identified so that a first plurality of features may be extracted from the first area of the first image and a second plurality of features may be extracted from the second area of the second image. A motion associated with the body part of the patient may be determined based on the first and second pluralities of features.

Abstract: The motion estimation of an anatomical structure may be performed using a machine-learned (ML) model trained based on medical training images of the anatomical structure and corresponding segmentation masks for the anatomical structure. During the training of the ML model, the model may be used to predict a motion field that may indicate a change between a first training image and a second training image, and to transform the first training image and a corresponding first segmentation mask based on the motion field. The parameters of the ML model may then be adjusted to maintain a correspondence between the transformed first training image and the second training image and between the transformed first segmentation mask or a second segmentation mask associated with the second training image. The correspondence may be assessed based on at least a boundary region shared by the anatomical structure and one or more other anatomical structures.

Abstract: Disclosed herein are systems, methods, and instrumentalities associated with MRI image reconstruction. According to embodiments of the disclosure, an apparatus configured to perform the MRI image reconstruction task may be configured to obtain an under-sampled MRI image and generate a reconstructed MRI image based on the under-sampled MRI image and a machine-learned (ML) model. The ML model may be trained via contrastive learning, during which randomly selected locations of the reconstructed MRI data generated by the ML model may be replaced with values that are different than the under-sampled MRI data, and the MRI data thus derived may be used as a negative example for the training.

Abstract: A system for physiological motion measurement is provided. The system may acquire a reference image corresponding to a reference motion phase of an ROI and a target image of the ROI corresponding to a target motion phase, wherein the reference motion phase may be different from the target motion phase. The system may identify one or more feature points relating to the ROI from the reference image, and determine a motion field of the feature points from the reference motion phase to the target motion phase using a motion prediction model. An input of the motion prediction model may include at least the reference image and the target image. The system may further determine a physiological condition of the ROI based on the motion field.

Abstract: A power management apparatus for a workflow to enable low power MR patient positioning on edge devices is disclosed. The power management apparatus changes an operational mode of an edge device from a first power mode to a second power mode after a defined time-interval. The power management apparatus further controls the edge device to capture a first image of a first scene. The power management apparatus further determines a trigger point based on a detection of a plurality of objects in the captured first image. The power management apparatus further changes the operational mode of the edge device from the second power mode to a third power mode to control a consumption of electric power while a set of operations is executed at the edge device. The operational mode of the edge device may be changed at the determined trigger point.

Abstract: A person's privacy is protected by the law in many settings and disclosed herein are systems, methods, and instrumentalities associated with anonymizing an image of a person while still preserving the visual saliency and/or utility of the image for one or more downstream tasks. These objectives may be accomplished using various machine-learning (ML) techniques such as ML models trained for extracting identifying and residual features from the input image as well as ML models trained for transforming the identifying features into identity-concealing features and for preserving the utility features of the image. An output image may be generated based on the various ML models, wherein the identity of the person may be substantially disguised in the output image while the background and utility attributes of the original image may be substantially maintained in the output image.

Abstract: An apparatus may obtain a sequence of medical images of a target structure and determine, using a first ANN, a first segmentation and a second segmentation of the target structure based on a first medical image and a second medical image, respectively. The first segmentation may indicate a first plurality of pixels that may belong to the target structure. The second segmentation may indicate a second plurality of pixels that may belong to the target structure. The apparatus may identify, using a second ANN, a first subset of true positive pixels among the first plurality of pixels that may belong to the target structure, and a second subset of true positive pixels among the second plurality of pixels that may belong to the target structure. The apparatus may determine a first refined segmentation and a second refined segmentation of the target structure based on the true positive pixels.

Abstract: Disclosed herein are systems, methods, and instrumentalities associated with magnetic resonance (MR) image reconstruction. An under-sampled MR image may be reconstructed through an iterative process (e.g., over multiple iterations) based on a machine-learning (ML) model. The ML model may be obtained through a reinforcement learning process during which the ML model may be used to predict a correction to an input MR image of at least one of the multiple iterations, apply the correction to the input MR image to obtain a reconstructed MR image, determine a reward for the ML model based on the reconstructed MR image, and adjust the parameters of the ML model based on the reward. The reward may be determined using a pre-trained reward neural network and the ML model may also be pre-trained in a supervised manner before being refined through the reinforcement learning process.

Abstract: Digital breast tomosynthesis (DBT) may provide richer information than full-field digital mammography (FFDM). DBT data such as DBT slices may be processed based on deep learning techniques such as using a neural network, and the DBT slices may be divided into groups and a pre-determined number of representative images may be derived based on the grouping. The neural network may be configured to process the representative images to predict the presence or non-presence of a breast disease such as breast cancer.

Abstract: Described herein are systems, methods, and instrumentalities associated with automatically annotating a 3D image dataset. The 3D automatic annotation may be accomplished based on a 2D annotation provided by an annotator and by propagating the 2D annotation through multiple images of a sequence of 2D images associated with the 3D image dataset. The automatically annotated 3D image dataset may then be used to annotate other 3D image datasets based on similarities between the first 3D image dataset and the other 3D image datasets. The automatic annotation of the first 3D image dataset and/or the other 3D image datasets may be conducted based on one or more machine-learning models trained for performing those tasks.

Abstract: Deep learning-based systems, methods, and instrumentalities are described herein for MRI reconstruction and/or refinement. An MRI image may be reconstructed based on under-sampled MRI information and a generative model may be trained to refine the reconstructed image, for example, by increasing the sharpness of the MRI image without introducing artifacts into the image. The generative model may be implemented using various types of artificial neural networks including a generative adversarial network. The model may be trained based on an adversarial loss and a pixel-wise image loss, and once trained, the model may be used to improve the quality of a wide range of 2D or 3D MRI images including those of a knee, brain, or heart.

Abstract: The decision process of a first machine learning (ML) model may be explained based on a second ML model implemented on an apparatus. The apparatus may obtain a prediction about an image made based on the first ML model. The apparatus may further determine visual concepts associated with the image that may have been used by the first ML model to make the prediction, and determine respective contributions of the visual concepts to the prediction made by the first ML model. The apparatus may then generate, based on the second ML model, a textual description that explains the respective contributions of the visual concepts to the prediction made by the first ML model. The second ML model may determine respective image features associated with the visual concepts, map the determined image features to corresponding text features, and generate the textual description based at least on the text features.

Abstract: Described herein are systems, methods, and instrumentalities associated with using a multi-layer perceptron (MLP) neural network to process medical images of an anatomical structure. The processing may include padding an input image in accordance with the training of the MLP neural network, splitting the input image (e.g., the padded input image) into patches of a same size, and processing the patches through the MLP neural network over one or more iterations. During an iteration of the processing, the patches may be processed separately and re-combined into an intermediate image before the intermediate image is shifted to concatenate portions of the image that are derived from different patches. This way, global features of the anatomical structure may be learned and used to improve the quality of the image generated by the MLP neural network, without incurring significant computation or memory costs.

Abstract: A video of medical scan images associated with an anatomical structure may be arranged into multiple image pairs. The multiple image pairs may be provided to a machine learning (ML) model successively and the ML model may determine respective first sets of image features associated with the multiple image pairs and, for each of the multiple image pairs, refine the first set of image features associated with the image pair based on the respective first sets of image features associated with one or more other image pairs. A motion field associated with the image pair may be determined based at least on the refined first set of image features associated with the image pair and a task may be performed based on the respective motion fields.

Abstract: The decision process of a first machine learning (ML) model may be explained based on a second ML model implemented on an apparatus. The apparatus may obtain a prediction about an image made based on the first ML model. The apparatus may further determine visual concepts associated with the image that may have been used by the first ML model to make the prediction, and determine respective contributions of the visual concepts to the prediction made by the first ML model. The apparatus may then generate, based on the second ML model, a textual description that explains the respective contributions of the visual concepts to the prediction made by the first ML model. The second ML model may determine respective image features associated with the visual concepts, map the determined image features to corresponding text features, and generate the textual description based at least on the text features.

Abstract: Disclosed herein are systems, methods, and instrumentalities associated with medical image denoising. An apparatus configured to perform the medical image denoising task may be configured to obtain a medical image of an object and separate the medical image into a background layer and a foreground layer. The apparatus may then denoise the background layer using a first neural network pre-trained to suit the characteristics of the background layer, denoise the foreground layer using a second neural network pre-trained to suit the characteristics of the foreground layer, and merge the denoised background layer and the denoised foreground layer back into a clean medical image that depicts the object with improved image quality.

Abstract: Deblurring and denoising a medical image such as X-ray fluoroscopy images may be challenging, and deep-learning based techniques may be employed to meet the challenge. An artificial neural network (ANN) may be trained using training images with synthetic noise and as well as training images with real noise. The parameters of the ANN may be adjusted during the training based on at least a first loss designed to maintain continuity between consecutive medical images generated by the ANN and a second loss designed to maintain similarity of patches inside a medical image generated by the ANN. The parameters of the ANN may be further adjusted based on a third loss that may be calculated from ground truth associated with the synthetic training images. Transfer learning between the synthetic images and the real images may be accomplished using these techniques.