Abstract: Cardiac images such as cardiac magnetic resonance (CMR) images and tissue characterization maps (e.g., T1/T2 maps) may be analyzed automatically using machine learning (ML) techniques, and reports may be generated to summarize the analysis. The ML techniques may include training one or more of an image classification model, a heart segmentation model, or a cardiac pathology detection model to automate the image analysis and/or reporting process. The image classification model may be capable of grouping the cardiac images into different categories, the heart segmentation model may be capable of delineating different anatomical regions of the heart, and the pathology detection model may be capable of detecting a medical abnormality in one or more of the anatomical regions based on tissue patterns or parameters automatically recognized by the detection model. Image registration that compensates for the impact of motions or movements may also be conducted automatically using ML techniques.

Abstract: Described herein are systems, methods, and instrumentalities associated with automatically annotating a 3D image dataset. The 3D automatic annotation may be accomplished based on a 2D annotation provided by an annotator and by propagating the 2D annotation through multiple images of a sequence of 2D images associated with the 3D image dataset. The automatically annotated 3D image dataset may then be used to annotate other 3D image datasets based on similarities between the first 3D image dataset and the other 3D image datasets. The automatic annotation of the first 3D image dataset and/or the other 3D image datasets may be conducted based on one or more machine-learning models trained for performing those tasks.

Abstract: Described herein are systems, methods, and instrumentalities associated with automatically annotating a 3D image dataset. The 3D automatic annotation may be accomplished based on a 2D manual annotation provided by an annotator and by propagating, using a set of machine-learning (ML) based techniques, the 2D manual annotation through sequences of 2D images associated with the 3D image dataset. The automatically annotated 3D image dataset may then be used to annotate other 3D image datasets upon passing a readiness assessment conducted using another set of ML based techniques. The automatic annotation of the images may be performed progressively, e.g., by processing a subset or batch of images at a time, and the ML based techniques may be trained to ensure consistency between a forward propagation and a backward propagation.

Abstract: Described herein are systems, methods, and instrumentalities associated with landmark detection. The detection may be accomplished by determining a graph representation of a plurality of hypothetical landmarks detected in one or more medical images. The graph representation may include nodes that represent the hypothetical landmarks and edges that represent the relationships between paired hypothetical landmarks. The graph representation may be processed using a graph neural network such a message passing graph neural network, by which the landmark detection problem may be converted and solved as a graph node labeling problem.

Abstract: The pose and shape of a human body may be recovered based on joint location information associated with the human body. The joint location information may be derived based on an image of the human body or from an output of a human motion capture system. The recovery of the pose and shape of the human body may be performed by a computer-implemented artificial neural network (ANN) trained to perform the recovery task using training datasets that include paired joint location information and human model parameters. The training of the ANN may be conducted in accordance with multiple constraints designed to improve the accuracy of the recovery and by artificially manipulating the training data so that the ANN can learn to recover the pose and shape of the human body even with partially observed joint locations.

Abstract: A neural network system implements a model for generating an output image based on a received input image. The model is learned through a training process during which parameters associated with the model are adjusted so as to maximize a difference between a first image predicted using first parameter values of the model and a second image predicted using second parameter values of the model, and to minimize a difference between the second image and a ground truth image. During a first iteration of the training process the first image is predicted and during a second iteration the second image is predicted. The first parameter values are obtained during the first iteration by minimizing a difference between the first image and the ground truth image, and the second parameter values are obtained during the second iteration by maximizing the difference between the first image and the second image.

Abstract: Image-based key points detection using a convolutional neural network (CNN) may be impacted if the key points are occluded in the image. Images obtained from additional imaging modalities such as depth and/or thermal images may be used in conjunction with RGB images to reduce or minimize the impact of the occlusion. The additional images may be used to determine adjustment values that are then applied to the weights of the CNN so that the convolution operations may be performed in a modality aware manner to increase the robustness, accuracy, and efficiency of key point detection.

Abstract: In Multiplex MRI image reconstruction, a hardware processor acquires sub-sampled Multiplex MRI data and reconstructs parametric images from the sub-sampled Multiplex MRI data. A machine learning model or deep learning model uses the subsampled Multiplex MRI data as the input and parametric maps calculated from the fully sampled data, or reconstructed fully sample data, as the ground truth. The model learns to reconstruct the parametric maps directly from the subsampled Multiplex MRI data.

Abstract: The present disclosure generally provides systems and methods for situation awareness. When executing a set of instructions stored in at least one non-transitory storage medium, at least one processor may be configured to cause the system to perform operations including obtaining, from at least one of one or more sensors, environmental data associated with an environment corresponding to a first time point, generating a first static global representation of an environment corresponding to the first time point based at least in part on the environmental data, generating a first dynamic global representation of the environment corresponding to the first time point based at least in part on the environmental data, and estimating, based on the first static global representation and the first dynamic global representation, a target state of the environment at a target time point using a target estimation model.

Abstract: Disclosed herein are systems, methods, and instrumentalities associated with reconstructing magnetic resonance (MR) images based on under-sampled MR data. The MR data include 2D or 3D information, and may encompass multiple contrasts and multiple coils. The MR images are reconstructed using deep learning (DL) methods, which may accelerate the scan and/or image generation process. Challenges imposed by the large quantity of the MR data and hardware limitations are overcome by separately reconstructing MR images based on respective subsets of contrasts, coils, and/or readout segments, and then combining the reconstructed MR images to obtain desired multi-contrast results.

Abstract: Disclosed is a method and a system for reviewing annotated medical images. The method includes receiving a dataset of medical images comprising one or more pre-existing annotations therein. The method also includes displaying, via a first graphical user interface, at a given instance, one of the medical images, and detecting a first input comprising a modification of at least one pre-existing annotation in the one of the medical images being displayed to define at least one modified annotation therefor and a reference for the at least one modified annotation to be associated therewith. The method also includes displaying, via a second graphical user interface, the one of the medical images having the at least one modified annotation and the associated reference for the at least one modified annotation, and detecting a second input comprising one of verification, correction, or rejection of the at least one modified annotation.

Abstract: Click based contour editing includes detecting a selection input with respect to an image presented on a graphical user interface; designating an area of the image corresponding to the selection input as a region of interest; detecting at least one other selection input on the graphical user interface with respect to the image; determining if the at least one other selection input is within the region of interest or outside of the region of interest; and if the at least one other selection input is within the region of interest, excluding the portion of the image corresponding to the other input; or if the other selection input is outside of the region of interest, including the portion of the image corresponding to an area of the image associated with the other selection input.

Abstract: Disclosed is a method and a system for automatic positioning of a medical equipment with respect to a patient. The method includes obtaining sensor data related to the patient, from a plurality of sensors fixed relative to the medical equipment. The method further includes processing the sensor data to determine at least one pose characteristic of the patient and at least one shape characteristic of the patient. The method further includes determining at least one adjustment parameter for the medical equipment based on the at least one pose characteristic of the patient and the at least one shape characteristic of the patient. The method further includes adjusting the medical equipment based on the at least one adjustment parameter.

Abstract: The present disclosure is related to systems and methods for image processing. The method includes obtaining a first image of a first modality. The method includes generating a second image of a second modality by processing, based on a trained machine learning model, the first image. The second modality may be different from the first modality.

Abstract: Motion contaminated magnetic resonance (MR) images for training an artificial neural network to remove motion artifacts from the MR images are difficult to obtain. Described herein are systems, methods, and instrumentalities for injecting motion artifacts into clean MR images and using the artificially contaminated images for machine learning and neural network training. The motion contaminated MR images may be created based on clean source MR images that are associated with multiple physiological cycles of a scanned object, and by deriving MR data segments for the multiple physiological cycles based on the source MR images. The MR data segments thus derived may be combined to obtain a simulated MR data set, from which one or more target MR images may be generated to exhibit a motion artifact. The motion artifact may be created by manipulating the source MR images and/or controlling the manner in which the MR data set or the target MR images are generated.

Abstract: Systems, methods, and instrumentalities are described herein for constructing a multi-view patient model (e.g., a 3D human mesh model) based on multiple single-view models of the patient. Each of the single-view models may be generated based on images captured by a sensing device and, dependent on the field of the view of the sensing device, may depict some keypoints of the patient's body with a higher accuracy and other keypoints of the patient's body with a lower accuracy. The multi-view patient model may be constructed using respective portions of the single-view models that correspond to accurately depicted keypoints. This way, a comprehensive and accurate depiction of the patient's body shape and pose may be obtained via the multi-view model even if some keypoints of the patient's body are blocked from a specific sensing device.

Abstract: An anatomy-aware contouring editing method includes receiving an image, wherein the image represents an anatomically recognizable structure; identifying a first image segment representing part of the anatomically recognizable structure; annotating the first image segment to generate a label of the part; drawing a contour along a boundary of the part; receiving a first input from a user device indicative of a region of contour failure, wherein the region of contour failure includes a portion of a contour that requires editing; editing the contour for generating an edited contour based on the first input and anatomical information; and updating another contour of another part of the anatomically recognizable structure based on the edited contour, wherein the another part is anatomically related to the part.

Abstract: Systems, methods and instrumentalities are described herein for automatically devising and executing a surgical plan associated with a patient in a medical environment, e.g., under the supervision of a medical professional. The surgical plan may be devised based on images of the medical environment captured by one or more sensing devices. A processing device may determine, based on all or a first subset of the images, a patient model that may indicate a location and a shape of an anatomical structure of the patient and determine, based on all or a second subset of the images, an environment model that may indicate a three-dimensional (3D) spatial layout of the medical environment. The surgical plan may be devised based on the patient model and the environment model, and may indicate at least a movement path of a medical device towards the anatomical structure of the patient.

Abstract: A three-dimensional (3D) model of a person may be obtained using a pre-trained neural network based on one or more images of the person. Such a model may be subject to estimation bias and/or other types of defects or errors. Described herein are systems, methods, and instrumentalities for refining the 3D model and/or the neural network used to generate the 3D model. The proposed techniques may extract information such as key body locations and/or a body shape from the images and refine the 3D model and/or the neural network using the extracted information. In examples, the 3D model and/or the neural network may be refined by minimizing a difference between the key body locations and/or body shape extracted from the images and corresponding key body locations and/or body shape determined from the 3D model. The refinement may be performed in an iterative and alternating manner.

Abstract: Disclosed herein are systems, methods, and instrumentalities associated with reconstructing magnetic resonance (MR) images based on multi-slice, under-sampled MRI data (e.g., k-space data). The multi-slice MRI data may be acquired using a simultaneous multi-slice (SMS) technique and MRI information associated with multiple MRI slices may be entangled in the multi-slice MRI data. A neural network may be trained and used to disentangle the MRI information and reconstruct MRI images for the different slices. A data consistency component may be used to estimate k-space data based on estimates made by the neural network, from which respective MRI images associated with multiple MRI slices may be obtained by applying a Fourier transform to the k-space data.