Abstract: Cardiac images such as cardiac magnetic resonance (CMR) images and tissue characterization maps (e.g., T1/T2 maps) may be analyzed automatically using machine learning (ML) techniques, and reports may be generated to summarize the analysis. The ML techniques may include training one or more of an image classification model, a heart segmentation model, or a cardiac pathology detection model to automate the image analysis and/or reporting process. The image classification model may be capable of grouping the cardiac images into different categories, the heart segmentation model may be capable of delineating different anatomical regions of the heart, and the pathology detection model may be capable of detecting a medical abnormality in one or more of the anatomical regions based on tissue patterns or parameters automatically recognized by the detection model. Image registration that compensates for the impact of motions or movements may also be conducted automatically using ML techniques.

Abstract: Described herein are systems, methods, and instrumentalities associated with landmark detection. The detection may be accomplished by determining a graph representation of a plurality of hypothetical landmarks detected in one or more medical images. The graph representation may include nodes that represent the hypothetical landmarks and edges that represent the relationships between paired hypothetical landmarks. The graph representation may be processed using a graph neural network such a message passing graph neural network, by which the landmark detection problem may be converted and solved as a graph node labeling problem.

Abstract: Disclosed herein are systems, methods, and instrumentalities associated with reconstructing magnetic resonance (MR) images based on under-sampled MR data. The MR data include 2D or 3D information, and may encompass multiple contrasts and multiple coils. The MR images are reconstructed using deep learning (DL) methods, which may accelerate the scan and/or image generation process. Challenges imposed by the large quantity of the MR data and hardware limitations are overcome by separately reconstructing MR images based on respective subsets of contrasts, coils, and/or readout segments, and then combining the reconstructed MR images to obtain desired multi-contrast results.

Abstract: A neural network system implements a model for generating an output image based on a received input image. The model is learned through a training process during which parameters associated with the model are adjusted so as to maximize a difference between a first image predicted using first parameter values of the model and a second image predicted using second parameter values of the model, and to minimize a difference between the second image and a ground truth image. During a first iteration of the training process the first image is predicted and during a second iteration the second image is predicted. The first parameter values are obtained during the first iteration by minimizing a difference between the first image and the ground truth image, and the second parameter values are obtained during the second iteration by maximizing the difference between the first image and the second image.

Abstract: In Multiplex MRI image reconstruction, a hardware processor acquires sub-sampled Multiplex MRI data and reconstructs parametric images from the sub-sampled Multiplex MRI data. A machine learning model or deep learning model uses the subsampled Multiplex MRI data as the input and parametric maps calculated from the fully sampled data, or reconstructed fully sample data, as the ground truth. The model learns to reconstruct the parametric maps directly from the subsampled Multiplex MRI data.

Abstract: Motion contaminated magnetic resonance (MR) images for training an artificial neural network to remove motion artifacts from the MR images are difficult to obtain. Described herein are systems, methods, and instrumentalities for injecting motion artifacts into clean MR images and using the artificially contaminated images for machine learning and neural network training. The motion contaminated MR images may be created based on clean source MR images that are associated with multiple physiological cycles of a scanned object, and by deriving MR data segments for the multiple physiological cycles based on the source MR images. The MR data segments thus derived may be combined to obtain a simulated MR data set, from which one or more target MR images may be generated to exhibit a motion artifact. The motion artifact may be created by manipulating the source MR images and/or controlling the manner in which the MR data set or the target MR images are generated.

Abstract: An anatomy-aware contouring editing method includes receiving an image, wherein the image represents an anatomically recognizable structure; identifying a first image segment representing part of the anatomically recognizable structure; annotating the first image segment to generate a label of the part; drawing a contour along a boundary of the part; receiving a first input from a user device indicative of a region of contour failure, wherein the region of contour failure includes a portion of a contour that requires editing; editing the contour for generating an edited contour based on the first input and anatomical information; and updating another contour of another part of the anatomically recognizable structure based on the edited contour, wherein the another part is anatomically related to the part.

Abstract: Disclosed herein are systems, methods, and instrumentalities associated with reconstructing magnetic resonance (MR) images based on multi-slice, under-sampled MRI data (e.g., k-space data). The multi-slice MRI data may be acquired using a simultaneous multi-slice (SMS) technique and MRI information associated with multiple MRI slices may be entangled in the multi-slice MRI data. A neural network may be trained and used to disentangle the MRI information and reconstruct MRI images for the different slices. A data consistency component may be used to estimate k-space data based on estimates made by the neural network, from which respective MRI images associated with multiple MRI slices may be obtained by applying a Fourier transform to the k-space data.

Abstract: Digital breast tomosynthesis (DBT) may provide richer information than full-field digital mammography (FFDM). DBT data such as DBT slices may be processed based on deep learning techniques such as using a neural network, and the DBT slices may be divided into groups and a pre-determined number of representative images may be derived based on the grouping. The neural network may be configured to process the representative images to predict the presence or non-presence of a breast disease such as breast cancer.

Abstract: A 3D anatomical model of one or more blood vessels of a patient may be obtained using CT angiography, while a 2D image of the blood vessels may be obtained based on fluoroscopy. The 3D model may be registered with the 2D image based on a contrast injection site identified on the 3D model and/or in the 2D image. A fused image may then be created to depict the overlaid 3D model and 2D image, for example, on a monitor or through a virtual reality headset. The injection site may be determined automatically or based on a user input that may include a bounding box drawn around the injection site on the 3D model, a selection of an automatically segmented area in the 3D model, etc.

Abstract: Mammography data such as DBT and/or FFDM images may be processed using deep learning based techniques, but labeled training data that may facilitate the learning may be difficult to obtained. Described herein are systems, methods, and instrumentalities associated with automatically generating and/or augmenting labeled mammography training data, and training a deep learn model based on the auto-generated/augmented data for detecting a breast disease (e.g., breast cancer) in a mammography image.

Abstract: A magnification system for magnifying an image based on trained neural networks is disclosed. The magnification system receives a first user input associated with a selection of a region of interest (ROI) within an input image of a site and a second user input associated with a first magnification factor of the selected ROI. The first magnification factor is associated with a magnification of the ROI in the input image. The ROI is modified based on an application of a first neural network model on the ROI. The modification of the ROI corresponds to a magnified image that is predicted in accordance with the first magnification factor. A display device is controlled to display the modified ROI.

Abstract: A method for surgery planning is provided. The method may include: generating a surgery video displaying a process of a virtual surgery performed by a virtual surgical robot on a virtual patient based on a surgery plan relating to a robotic surgery to be performed on a patient by a surgical robot; transmitting the surgery video to a display component of an XR assembly used to render the surgery video for display to a user; recording one or more actions that the user performed on the virtual surgery based on user input received via an input component of the XR assembly; modifying the surgery plan based on the one or more recorded actions; and causing the surgical robot to implement the robotic surgery on the patient according to the modified surgery plan.

Abstract: The present disclosure provides systems and methods for medical assistant. The systems may obtain position information of a target inside a subject during an operation. The systems may also determine depth information of the target with respect to an operational region of the subject based on the position information. The systems may further direct an optical projection device to project an optical signal representing the depth information on a surface of the subject.

Abstract: The present disclosure provides a method for surgical automation. The method may include: obtaining video data generated in a surgical process using a camera; identifying a surgical stage in the surgical process based on the video data; and triggering an activation of a surgical equipment used for a surgical operation and/or providing a guidance of the surgical operation based on the surgical stage.

Abstract: Described herein are systems, methods, and instrumentalities associated with using an invertible neural network to complete various medical imaging tasks. Unlike traditional neural networks that may learn to map input data (e.g., a blurry reconstructed MRI image) to ground truth (e.g., a fully-sampled MRI image), the invertible neural network may be trained to learn a mapping from the ground truth to the input data, and may subsequently apply an inverse of the mapping (e.g., at an inference time) to complete a medical imaging task. The medical imaging task may include, for example, MRI image reconstruction (e.g., to increase the sharpness of a reconstructed MRI image), image denoising, image super-resolution, and/or the like.

Abstract: A method and a system for managing healthcare records of a user are provided. The method includes storing an electronic medical record related to the user in form of a non-fungible token (NFT) written to a blockchain, associating a smart contract to the NFT in the blockchain, authorizing a request to access the electronic medical record related to the user based on the defined ownership of the electronic medical record stored in the blockchain, identifying one or more NFTs from the blockchain comprising one or more electronic medical records related to the user based on processing of the identifier information in associated one or more smart contracts therewith, in response to the request, and sending the one or more electronic medical records corresponding to the identified one or more NFTs to a requestor associated with the request.

Abstract: Deep learning-based systems, methods, and instrumentalities are described herein for MRI reconstruction and/or refinement. An MRI image may be reconstructed based on under-sampled MRI information and a generative model may be trained to refine the reconstructed image, for example, by increasing the sharpness of the MRI image without introducing artifacts into the image. The generative model may be implemented using various types of artificial neural networks including a generative adversarial network. The model may be trained based on an adversarial loss and a pixel-wise image loss, and once trained, the model may be used to improve the quality of a wide range of 2D or 3D MRI images including those of a knee, brain, or heart.

Abstract: The present disclosure provides a system and method for fetus monitoring. The method may include obtaining ultrasound data relating to a fetus collected by an ultrasound imaging device; generating a 4D image of the fetus based on the ultrasound data; directing a display component of a virtual reality (VR) device to display the 4D image to an operator; detecting motion of the fetus based on the ultrasound data; and directing a haptic component of the VR device to provide haptic feedback with respect to the motion to the operator.

Abstract: Described herein are systems, methods, and instrumentalities associated with tracking groups of small objects in medical images. The tracking may be accomplished by, for each one of a sequence of medical images, determining a plurality of candidate objects captured in the medical image, grouping the plurality of candidate objects into a plurality of groups of candidate objects and dividing the medical image into a plurality of regions that each surrounds a corresponding group of candidate objects. Each of the plurality of regions may be examined to extract respective features associated with each corresponding group of candidate objects. A match between a first group of candidate objects in a first medical image and a second group of candidate objects in a second medical image may be determined based on first features associated with the first group and second features associated with the second group.