Abstract: An apparatus may obtain a sequence of medical images of a target structure and determine, using a first ANN, a first segmentation and a second segmentation of the target structure based on a first medical image and a second medical image, respectively. The first segmentation may indicate a first plurality of pixels that may belong to the target structure. The second segmentation may indicate a second plurality of pixels that may belong to the target structure. The apparatus may identify, using a second ANN, a first subset of true positive pixels among the first plurality of pixels that may belong to the target structure, and a second subset of true positive pixels among the second plurality of pixels that may belong to the target structure. The apparatus may determine a first refined segmentation and a second refined segmentation of the target structure based on the true positive pixels.

Abstract: Disclosed herein are systems, methods and instrumentalities associated with multi-view 3D human model estimation using machine learning (ML) based techniques. These techniques may use synthetically generated data to train an ML model that may be used to progressively regress a 3D human body model based on multi-view 2D images. The training data may be synthetically generated based on statistical distributions of human poses and human body shapes, as well as a statistical distribution of camera viewpoints. The progressive regression may be performed based on consensus features shared by the multi-view images and diversity features derived from at least one of the multi-view images. Consistency between the multi-view images may also be maintained during the regression process.

Abstract: The physical characteristics of one or more anatomical structures of a person may change in accordance with conditions surrounding the determination of such physical characteristics. Machine learning based techniques may be used to determine a template representation of the one or more anatomical structures that may indicate the physical characteristics of the one or more anatomical structures free of the impact imposed by changing conditions. The template representation may then be used to predict the physical characteristics of the one or more anatomical structures under a new set of conditions, without subjecting the person to additional medical scans.

Abstract: Detecting motions associated with a body part of a patient may include using an image sensor installed inside a medical scanner to capture first and second images of the patient inside the medical scanner, wherein the first image may depict the patient in a first state and the second image may depict the patient in a second state. A first area, in the first image, that corresponds to the body part of the patient may be identified and a second area, in the second image, that corresponds to the body part may also be identified so that a first plurality of features may be extracted from the first area of the first image and a second plurality of features may be extracted from the second area of the second image. A motion associated with the body part of the patient may be determined based on the first and second pluralities of features.

Abstract: Described herein are systems, methods, and instrumentalities associated with processing complex-valued MRI data using a machine learning (ML) model. The ML model may be learned based on synthetically generated MRI training data and by applying one or more meta-learning techniques. The MRI training data may be generated by adding phase information to real-valued MRI data and/or by converting single-coil MRI data into multi-coil MRI data based on coil sensitivity maps. The meta-learning process may include using portions of the training data to conduct a first round of learning to determine updated model parameters and using remaining portions of the training data to test the updated model parameters. Losses associated with the testing may then be determined and used to refine the model parameters. The ML model learned using these techniques may be adopted for a variety of tasks including, for example, MRI image reconstruction and/or de-noising.

Abstract: A medical system may utilize a modular and extensible sensing device to derive a two-dimensional (2D) or three-dimensional (3D) human model for a patient in real-time based on images of the patient captured by a sensor such as a digital camera. The 2D or 3D human model may be visually presented on one or more devices of the medical system and used to facilitate a healthcare service provided to the patient. In examples, the 2D or 3D human model may be used to improve the speed, accuracy and consistency of patient positioning for a medical procedure. In examples, the 2D or 3D human model may be used to enable unified analysis of the patient's medical conditions by linking different scan images of the patient through the 2D or 3D human model. In examples, the 2D or 3D human model may be used to facilitate surgical navigation, patient monitoring, process automation, and/or the like.

Abstract: Deep learning based systems, methods, and instrumentalities are described herein for registering images from a same imaging modality and different imaging modalities. Transformation parameters associated with the image registration task are determined using a neural ordinary differential equation (ODE) network that comprises multiple layers, each configured to determine a respective gradient update for the transformation parameters based on a current state of the transformation parameters received by the layer. The gradient updates determined by the multiple ODE layers are then integrated and applied to initial values of the transformation parameters to obtain final parameters for completing the image registration task. The operations of the ODE network may be facilitated by a feature extraction network pre-trained to determine content features shared by the input images.

Abstract: Click based contour editing includes detecting a selection input with respect to an image presented on a graphical user interface; designating an area of the image corresponding to the selection input as a region of interest; detecting at least one other selection input on the graphical user interface with respect to the image; determining if the at least one other selection input is within the region of interest or outside of the region of interest; and if the at least one other selection input is within the region of interest, excluding the portion of the image corresponding to the other input; or if the other selection input is outside of the region of interest, including the portion of the image corresponding to an area of the image associated with the other selection input.

Abstract: Human model recovery may be realized utilizing pre-trained artificially neural networks. A first neural network may be trained to determine body keypoints of a person based on image(s) of a person. A second neural network may be trained to predict pose parameters associated with the person based on the body keypoints. A third neural network may be trained to predict shape parameters associated with the person based on depth image(s) of the person. A 3D human model may then be generated based on the pose and shape parameters respectively predicted by the second and third neural networks. The training of the second neural network may be conducted using synthetically generated body keypoints and the training of the third neural network may be conducted using normal maps. The pose and shape parameters predicted by the second and third neural networks may be further optimized through an iterative optimization process.

Abstract: A system and a method for bypass vessel reconstruction may be provided. A target image including at least a cardiac region of a subject may be obtained. A first segmentation result and a second segmentation result may be determined based on the target image. The first segmentation result and the second segmentation result may respectively indicate the heart and vessels of the subject segmented from the target image. A target segment result may be determined using a vessel segment model based on the first segmentation result and the second segmentation result. The target segment result may include a plurality of segment labels of a plurality of points on the vessels of the subject, and the plurality of segment labels may include a segment label corresponding to bypass vessels. Further, data relating to one or more bypass vessels of the subject may be determined based on the target segment result.

Abstract: Disclosed herein are systems, methods and instrumentalities associated with multi-person joint location and pose estimation based on an image that depicts multiple people in a scene, where at least some of the joint locations of a person may be blocked or obstructed by other people or objects in the scene. The estimation may be performed by detecting and grouping joint locations in the image using a bottom-up approach, and refining each group of detected joint locations by recovering obstructed joint location(s) that may be missing from the group. The detection, grouping, and/or refinement may be accomplished based on one or more machine learning (ML) models that may be implemented using artificial neural networks such as convolutional neural networks.

Abstract: An object or person in a medical environment may be identified based on images of the medical environment. The identification may include determining an identifier associated with the object or the person, a position of the object or the person in the medical environment, and a three-dimensional (3D) shape/pose of the object or the person. Representation information that indicates at least the determined identifier, position in the medical environment, and 3D shape/pose of the object or the person may be generated and then used (e.g., by a visualization device) together with one or more predetermined 3D models to determine a 3D model for the object or the person identified in the medical environment and generate a visual depiction of at least the object or the person in the medical environment based on the determined 3D model and the position of the object or the person in the medical environment.

Abstract: Disclosed herein are systems, methods, and instrumentalities associated with magnetic resonance (MR) image reconstruction. An under-sampled MR image may be reconstructed through an iterative process (e.g., over multiple iterations) based on a machine-learning (ML) model. The ML model may be obtained through a reinforcement learning process during which the ML model may be used to predict a correction to an input MR image of at least one of the multiple iterations, apply the correction to the input MR image to obtain a reconstructed MR image, determine a reward for the ML model based on the reconstructed MR image, and adjust the parameters of the ML model based on the reward. The reward may be determined using a pre-trained reward neural network and the ML model may also be pre-trained in a supervised manner before being refined through the reinforcement learning process.

Abstract: Automatic hand gesture determination may be a challenging task considering the complex anatomy and high dimensionality of the human hand. Disclosed herein are systems, methods, and instrumentalities associated with recognizing a hand gesture in spite of the challenges. An apparatus in accordance with embodiments of the present disclosure may use machine learning based techniques to identify the area of an image that may contain a hand and to determine an orientation of the hand relative to a pre-defined direction. The apparatus may then adjust the area of the image containing the hand to align the orientation of the hand with the pre-defined direction and/or to scale the image area to a pre-defined size. Based on the adjusted image area, the apparatus may detect a plurality of hand landmarks and predict a gesture indicated by the hand based on the plurality of detected landmarks.

Abstract: A magnification system for magnifying an image based on trained neural networks is disclosed. The magnification system receives a first user input associated with a selection of a region of interest (ROI) within an input image of a site and a second user input associated with a first magnification factor of the selected ROI. The first magnification factor is associated with a magnification of the ROI in the input image. The ROI is modified based on an application of a first neural network model on the ROI. The modification of the ROI corresponds to a magnified image that is predicted in accordance with the first magnification factor. A display device is controlled to display the modified ROI.

Abstract: The motion estimation of an anatomical structure may be performed using a machine-learned (ML) model trained based on medical training images of the anatomical structure and corresponding segmentation masks for the anatomical structure. During the training of the ML model, the model may be used to predict a motion field that may indicate a change between a first training image and a second training image, and to transform the first training image and a corresponding first segmentation mask based on the motion field. The parameters of the ML model may then be adjusted to maintain a correspondence between the transformed first training image and the second training image and between the transformed first segmentation mask or a second segmentation mask associated with the second training image. The correspondence may be assessed based on at least a boundary region shared by the anatomical structure and one or more other anatomical structures.

Abstract: Disclosed herein are systems, methods, and instrumentalities associated with cardiac motion tracking and/or analysis. In accordance with embodiments of the disclosure, the motion of a heart such as an anatomical component of the heart may be tracked through multiple medical images and a contour of the anatomical component may be outlined in the medical images and presented to a user. The user may adjust the contour in one or more of the medical images and the adjustment may trigger modifications of motion field(s) associated with the one or more medical images, re-tracking of the contour in the one or more medical images, and/or re-determination of a physiological characteristic (e.g., a myocardial strain) of the heart. The adjustment may be made selectively, for example, to a specific medical image or one or more additional medical images selected by the user, without triggering a modification of all of the medical images.

Abstract: Described herein are systems, methods, and instrumentalities associated with denoising medical images such as fluoroscopic images using deep learning techniques. A first artificial neural network (ANN) is trained to denoise an input medical image in accordance with a provided target noise level. The training of the first ANN is conducted by pairing a noisy input image with target denoised images that include different levels of noise. These target denoised images are generated using a second ANN as intermediate outputs of the second ANN during different training iterations. As such, the first ANN may learn to perform the denoising task in an unsupervised manner without requiring noise-free training images as the ground truth.

Abstract: A system for fault diagnosis is provided. The system may acquire a vibration signal of a target device, and determine one or more feature values of the vibration signal. The system may further determine a fault condition of the target device by applying a fault diagnosis model to the feature values. The fault diagnosis model may include a trained first component including a plurality of stacked trained RBMs, and a trained second component connected to the trained first component. The trained second component may include a trained fully connected layer and a trained output layer connected to the trained fully connected layer.

Abstract: An apparatus for stent visualization includes a hardware processor that is configured to input one or more stent images from a sequence of X-ray images and corresponding balloon marker location data to a cascaded spatial transform network. The background is separated from the one or more stent images using the cascaded spatial transform network and a transformed stent image with a clear background and a non-stent background image is generated. The stent layer and non-stent layer are generated using a neural network without online optimization. A mapping function f maps the inputs, the sequence images and marker coordinates, into the two single image outputs.