Abstract: Systems or techniques for facilitating learnable visual prompt engineering are provided. In various embodiments, a system can access a medical image and a pre-trained machine learning model that is configured to perform a diagnostic or prognostic inferencing task. In various aspects, the system can apply a pre-processing transformation to one or more pixels or voxels of the medical image, thereby yielding a transformed version of the medical image, wherein the pre-processing transformation can convert an input pixel or voxel intensity value to an output pixel or voxel intensity value via one or more parameters that are iteratively learned. In various instances, the system can perform the diagnostic or prognostic inferencing task, by executing the pre-trained machine learning model on the transformed version of the medical image.

Abstract: Systems/techniques that facilitate deep learning multi-planar reformatting of medical images are provided. In various embodiments, a system can access a three-dimensional medical image. In various aspects, the system can localize, via execution of a machine learning model, a set of landmarks depicted in the three-dimensional medical image, a set of principal anatomical planes depicted in the three-dimensional medical image, and a set of organs depicted in the three-dimensional medical image. In various instances, the system can determine an anatomical orientation exhibited by the three-dimensional medical image, based on the set of landmarks, the set of principal anatomical planes, or the set of organs. In various cases, the system can rotate the three-dimensional medical image, such that the anatomical orientation now matches a predetermined anatomical orientation.

Abstract: The present disclosure relates to use of a workflow for automatic prescription of different radiological imaging scan planes across different anatomies and modalities. The automated prescription of such imaging scan planes helps ensure contiguous visualization of the different landmark structures. Unlike prior approaches, the disclosed technique determines the necessary planes using the localizer images itself and does not explicitly segment or delineate the landmark structures to perform plane prescription.

Abstract: A method for performing one-shot anatomy localization includes obtaining a medical image of a subject. The method includes receiving a selection of both a template image and a region of interest within the template image, wherein the template image includes one or more anatomical landmarks assigned a respective anatomical label. The method includes inputting both the medical image and the template image into a trained vision transformer model. The method includes outputting from the trained vision transformer model both patch level features and image level features for both the medical image and the template image. The method still further includes interpolating pixel level features from the patch level features for both the medical image and the template image. The method includes utilizing the pixel level features within the region of interest of the template image to locate and label corresponding pixel level features in the medical image.

Abstract: The current disclosure provides methods for an automated clinical recommendation system that generates clinical recommendations for patients of a health care system based on natural language queries submitted by care providers of the health care system. The natural language queries may be typed into a user interface (UI) of the clinical recommendation system, or submitted by voice via a microphone. The clinical recommendation system provides a clinically explainable disease state of a patient based on patient data included in a query, and recommends a next course of action (e.g., a treatment) based on clinical guidelines and population statistics, in a manner that reduces a current burden of clinicians in consulting digital clinical manuals via a series of time-consuming and cumbersome interactions with a graphical user interface (GUI) of the digital clinical manuals.

Abstract: Systems/techniques that facilitate interpretable task-specific dimensionality-reduction are provided. In various embodiments, a system can access a three-dimensional medical image. In various aspects, the system can generate, via execution of a first deep learning neural network, a voxel-wise weight map corresponding to the three-dimensional medical image and a set of projection vectors corresponding to the three-dimensional medical image. In various instances, the system can generate a set of two-dimensional projection images of the three-dimensional medical image, based on the voxel-wise weight map and the set of projection vectors. In various cases, the first deep learning neural network can be trained in a serial pipeline with a second deep learning neural network that is configured to perform an inferencing task on two-dimensional inputs. This can cause the set of two-dimensional projection images to be tailored to the inferencing task.

Abstract: Techniques are described that facilitate dynamic multimodal segmentation selection and fusion in medical imaging. In one example embodiment, a computer processing system receives a segmentation dataset comprising a combination of different image segmentations of an anatomical object of interest respectively segmented via different segmentation models from different medical images captured of the (same) anatomical object, wherein the different medical images and the different image segmentations vary with respect to at least one of, capture modality, acquisition protocol, or acquisition parameters. The system employs a dynamic ranking protocol as opposed to a static ranking protocol to determine ranking scores for the different image segmentations that control relative contributions of the different image segmentations in association with combining the different image segmentations into a fused segmentation for the anatomical object.

Abstract: The current disclosure provides systems and methods for automatic image alignment of three-dimensional (3D) medical image volumes. The method includes pre-processing the 3D medical image volume by selecting a sub-volume of interest, detecting anatomical landmarks in the sub-volume using a deep neural network, estimating transformation parameters based on the anatomical landmarks to adjust rotation angles and translation of the sub-volume, adjusting the rotation angles and translation to produce a first aligned sub-volume, determining confidence in the transformation parameters based on the first aligned sub-volume, and iteratively refining the transformation parameters if the confidence is below a predetermined threshold. The disclosed approach for automated image alignment reduces the need for manual alignment and, increases a probability of the 3D image volume converging to a desired orientation compared to conventional approaches.

Abstract: Systems/techniques that facilitate explainable confidence estimation for landmark localization are provided. In various embodiments, a system can access a three-dimensional voxel array captured by a medical imaging scanner and can localize, via execution of a first deep learning neural network, a set of anatomical landmarks depicted in the three-dimensional voxel array. In various aspects, the system can generate a multi-tiered confidence score collection based on the set of anatomical landmarks and based on a training dataset on which the first deep learning neural network was trained. In various instances, the system can, in response to one or more confidence scores from the multi-tiered confidence score collection failing to satisfy a threshold, generate, via execution of a second deep learning neural network, a classification label that indicates an explanatory factor for why the one or more confidence scores failed to satisfy the threshold.

Abstract: Various methods and systems are provided for reducing parametric heterogeneity in quantitative magnetic resonance (qMR) images, to increase robustness of in-field machine learning model inferences. In one example, a method for reducing qMR image heterogeneity includes, receiving a first qMR image, acquired using a first value of an acquisition parameter, determining a target value of the acquisition parameter based on a training dataset of a machine learning model, generating a synthetic qMR image, wherein the synthetic qMR image simulates a qMR image acquired using the target value of the acquisition parameter, by mapping the first qMR image to the synthetic qMR image using an analytical model, and feeding the synthetic qMR image to the machine learning model.

Abstract: Systems/techniques that facilitate interpretable task-specific dimensionality-reduction are provided. In various embodiments, a system can access a three-dimensional medical image. In various aspects, the system can generate, via execution of a first deep learning neural network, a voxel-wise weight map corresponding to the three-dimensional medical image and a set of projection vectors corresponding to the three-dimensional medical image. In various instances, the system can generate a set of two-dimensional projection images of the three-dimensional medical image, based on the voxel-wise weight map and the set of projection vectors. In various cases, the first deep learning neural network can be trained in a serial pipeline with a second deep learning neural network that is configured to perform an inferencing task on two-dimensional inputs. This can cause the set of two-dimensional projection images to be tailored to the inferencing task.

Abstract: Various methods and systems are provided for computer-aided diagnosis. In one embodiment, a method comprises acquiring, with an imaging system, a medical image of a subject, generating, with a radiologist model associated with a radiologist of an institution, a computer-aided diagnosis for the medical image, the radiologist model comprising a deep neural network trained on a plurality of diagnoses provided by the radiologist, displaying, to the radiologist via a display device, the medical image and the computer-aided diagnosis, and selectively updating, based on the medical image, one or more of the radiologist model, an institution model associated with the institution, and a geographic model associated with a geographic area containing the institution. In this way, a radiologist may be assisted by a deep neural network model configured as a digital twin of the radiologist.

Abstract: Techniques are described for generating reformatted views of a three-dimensional (3D) anatomy scan using deep-learning estimated scan prescription masks. According to an embodiment, a system is provided that comprises a memory that stores computer executable components, and a processor that executes the computer executable components stored in the memory. The computer executable components comprise a mask generation component that employs a pre-trained neural network model to generate masks for different anatomical landmarks depicted in one or more calibration images captured of an anatomical region of a patient. The computer executable components further comprise a reformatting component that reformats 3D image data captured of the anatomical region of the patient using the masks to generate different representations of the 3D image data that correspond to the different anatomical landmarks.

Abstract: Methods and systems are provided for inferring thickness and volume of one or more object classes of interest in two-dimensional (2D) medical images, using deep neural networks. In an exemplary embodiment, a thickness of an object class of interest may be inferred by acquiring a 2D medical image, extracting features from the 2D medical image, mapping the features to a segmentation mask for an object class of interest using a first convolutional neural network (CNN), mapping the features to a thickness mask for the object class of interest using a second CNN, wherein the thickness mask indicates a thickness of the object class of interest at each pixel of a plurality of pixels of the 2D medical image; and determining a volume of the object class of interest based on the thickness mask and the segmentation mask.

Abstract: Methods and systems are provided for inferring thickness and volume of one or more object classes of interest in two-dimensional (2D) medical images, using deep neural networks. In an exemplary embodiment, a thickness of an object class of interest may be inferred by acquiring a 2D medical image, extracting features from the 2D medical image, mapping the features to a segmentation mask for an object class of interest using a first convolutional neural network (CNN), mapping the features to a thickness mask for the object class of interest using a second CNN, wherein the thickness mask indicates a thickness of the object class of interest at each pixel of a plurality of pixels of the 2D medical image; and determining a volume of the object class of interest based on the thickness mask and the segmentation mask.

Abstract: Techniques are described that facilitate dynamic multimodal segmentation selection and fusion in medical imaging. In one example embodiment, a computer processing system receives a segmentation dataset comprising a combination of different image segmentations of an anatomical object of interest respectively segmented via different segmentation models from different medical images captured of the (same) anatomical object, wherein the different medical images and the different image segmentations vary with respect to at least one of, capture modality, acquisition protocol, or acquisition parameters. The system employs a dynamic ranking protocol as opposed to a static ranking protocol to determine ranking scores for the different image segmentations that control relative contributions of the different image segmentations in association with combining the different image segmentations into a fused segmentation for the anatomical object.

Abstract: Techniques are described for generating mono-modality training image data from multi-modality image data and using the mono-modality training image data to train and develop mono-modality image inferencing models. A method embodiment comprises generating, by a system comprising a processor, a synthetic 2D image from a 3D image of a first capture modality, wherein the synthetic 2D image corresponds to a 2D version of the 3D image in a second capture modality, and wherein the 3D image and the synthetic 2D image depict a same anatomical region of a same patient. The method further comprises transferring, by the system, ground truth data for the 3D image to the synthetic 2D image. In some embodiments, the method further comprises employing the synthetic 2D image to facilitate transfer of the ground truth data to a native 2D image captured of the same anatomical region of the same patient using the second capture modality.

Abstract: Techniques are described for generating high quality training data collections for training artificial intelligence (AI) models in the medical imaging domain. A method embodiment comprises receiving, by a system comprising processor, input indicating a clinical context associated with usage of a medical image dataset, and selecting, by the system, one or more data scrutiny metrics for filtering the medical image dataset based on the clinical context. The method further comprises applying, by the system, one or more image processing functions to the medical image dataset to generate metric values of the one or more data scrutiny metrics for respective medical images included in the medical image dataset, filtering, by the system, the medical image dataset into one or more subsets based on one or more acceptability criteria for the metric values.

Abstract: Techniques are described for generating reformatted views of a three-dimensional (3D) anatomy scan using deep-learning estimated scan prescription masks. According to an embodiment, a system is provided that comprises a memory that stores computer executable components, and a processor that executes the computer executable components stored in the memory. The computer executable components comprise a mask generation component that employs a pre-trained neural network model to generate masks for different anatomical landmarks depicted in one or more calibration images captured of an anatomical region of a patient. The computer executable components further comprise a reformatting component that reformats 3D image data captured of the anatomical region of the patient using the masks to generate different representations of the 3D image data that correspond to the different anatomical landmarks.

Abstract: Techniques are described for generating reformatted views of a three-dimensional (3D) anatomy scan using deep-learning estimated scan prescription masks. According to an embodiment, a system is provided that comprises a memory that stores computer executable components, and a processor that executes the computer executable components stored in the memory. The computer executable components comprise a mask generation component that employs a pre-trained neural network model to generate masks for different anatomical landmarks depicted in one or more calibration images captured of an anatomical region of a patient. The computer executable components further comprise a reformatting component that reformats 3D image data captured of the anatomical region of the patient using the masks to generate different representations of the 3D image data that correspond to the different anatomical landmarks.