Abstract: A computer-implemented method for analyzing digital holographic microscopy (DHM) data for hematology applications includes receiving a DHM image acquired using a digital holographic microscopy system. The DHM image comprises depictions of one or more cell objects and background. A reference image is generated based on the DHM image. This reference image may then be used to reconstruct a fringe pattern in the DHM image into an optical depth map.

Abstract: Independent subspace analysis (ISA) is used to learn (42) filter kernels for CLE images in brain tumor classification. Convolution (46) and stacking are used for unsupervised learning (44, 48) with ISA to derive the filter kernels. A classifier is trained (56) to classify CLE brain images based on features extracted using the filter kernels. The resulting filter kernels and trained classifier are used (60, 64) to assist in diagnosis of occurrence of brain tumors during or as part of neurosurgical resection. The classification may assist a physician in detecting whether CLE examined brain tissue is healthy or not and/or a type of tumor.

Abstract: Systems and methods for model augmentation include receiving intra-operative imaging data of an anatomical object of interest at a deformed state. The intra-operative imaging data is stitched into an intra-operative model of the anatomical object of interest at the deformed state. The intra-operative model of the anatomical object of interest at the deformed state is registered with a pre-operative model of the anatomical object of interest at an initial state by deforming the pre-operative model of the anatomical object of interest at the initial state based on a biomechanical model. Texture information from the intra-operative model of the anatomical object of interest at the deformed state is mapped to the deformed pre-operative model to generate a deformed, texture-mapped pre-operative model of the anatomical object of interest.

Abstract: A method and system for scene parsing and model fusion in laparoscopic and endoscopic 2D/2.5D image data is disclosed. A current frame of an intra-operative image stream including a 2D image channel and a 2.5D depth channel is received. A 3D pre-operative model of a target organ segmented in pre-operative 3D medical image data is fused to the current frame of the intra-operative image stream. Semantic label information is propagated from the pre-operative 3D medical image data to each of a plurality of pixels in the current frame of the intra-operative image stream based on the fused pre-operative 3D model of the target organ, resulting in a rendered label map for the current frame of the intra-operative image stream. A semantic classifier is trained based on the rendered label map for the current frame of the intra-operative image stream.

Abstract: A method and system for registration of 2D/2.5D laparoscopic or endoscopic image data to 3D volumetric image data is disclosed. A plurality of 2D/2.5D intra-operative images of a target organ are received, together with corresponding relative orientation measurements for the intraoperative images. A 3D medical image volume of the target organ is registered to the plurality of 2D/2.5D intra-operative images by calculating pose parameters to match simulated projection images of the 3D medical image volume to the plurality of 2D/2.5D intra-operative images, and the registration is constrained by the relative orientation measurements for the intra-operative images.

Abstract: A method for analyzing digital holographic microscopy (DHM) data for hematology applications includes receiving a DHM image acquired using a digital holographic microscopy system and identifying one or more erythrocytes in the DHM image. For each respective erythrocyte included in the one or more erythrocytes, a cell thickness value for the respective erythrocyte using a parametric model is estimated, and a cell volume value is calculated for the respective erythrocyte using the cell thickness value.

Abstract: A method and system for determining fractional flow reserve (FFR) for a coronary artery stenosis of a patient is disclosed. In one embodiment, medical image data of the patient including the stenosis is received, a set of features for the stenosis is extracted from the medical image data of the patient, and an FFR value for the stenosis is determined based on the extracted set of features using a trained machine-learning based mapping. In another embodiment, a medical image of the patient including the stenosis of interest is received, image patches corresponding to the stenosis of interest and a coronary tree of the patient are detected, an FFR value for the stenosis of interest is determined using a trained deep neural network regressor applied directly to the detected image patches.

Abstract: Systems and methods for image classification include receiving imaging data of in-vivo or excised tissue of a patient during a surgical procedure. Local image features are extracted from the imaging data. A vocabulary histogram for the imaging data is computed based on the extracted local image features. A classification of the in-vivo or excised tissue of the patient in the imaging data is determined based on the vocabulary histogram using a trained classifier, which is trained based on a set of sample images with confirmed tissue types.

Abstract: A method and system for semantic segmentation laparoscopic and endoscopic 2D/2.5D image data is disclosed. Statistical image features that integrate a 2D image channel and a 2.5D depth channel of a 2D/2.5 laparoscopic or endoscopic image are extracted for each pixel in the image. Semantic segmentation of the laparoscopic or endoscopic image is then performed using a trained classifier to classify each pixel in the image with respect to a semantic object class of a target organ based on the extracted statistical image features. Segmented image masks resulting from the semantic segmentation of multiple frames of a laparoscopic or endoscopic image sequence can be used to guide organ specific 3D stitching of the frames to generate a 3D model of the target organ.

Abstract: A method and system for classifying tissue endomicroscopy images are disclosed. Local feature descriptors are extracted from an endomicroscopy image. Each of the local feature descriptors is encoded using a learnt discriminative dictionary. The learnt discriminative dictionary includes class-specific sub-dictionaries and penalizes correlation between bases of sub-dictionaries associated with different classes. Tissue in the endomicroscopy image is classified using a trained machine learning based classifier based on the coded local feature descriptors encoded using a learnt discriminative dictionary.

Abstract: A method for performing cellular classification includes using a convolution sparse coding process to generate a plurality of feature maps based on a set of input images and a plurality of biologically-specific filters. A feature pooling operation is applied on each of the plurality of feature maps to yield a plurality of image representations. Each image representation is classified as one of a plurality of cell types.

Abstract: A method for performing cellular classification includes extracting a plurality of local feature descriptors from a set of input images and applying a coding process to covert each of the plurality of local feature descriptors into a multi-dimensional code. A feature pooling operation is applied on each of the plurality of local feature descriptors to yield a plurality of image representations and each image representation is classified as one of a plurality of cell types.

Abstract: A projector in an endoscope is used to project visible light onto tissue. The projected intensity, color, and/or wavelength vary by spatial location in the field of view to provide an overlay. Rather than relying on a rendered overlay alpha-blended on a captured image, the illumination with spatial variation physically highlights one or more regions of interest or physically overlays on the tissue.

Abstract: A method and system for tumor ablation planning and guidance based on a patient-specific model of liver tumor ablation is disclosed. A patient-specific anatomical model of the liver and circulatory system of the liver is estimated from 3D medical image data of a patient. Blood flow in the liver and the circulatory system of the liver is simulated based on the patient-specific anatomical model. Heat diffusion due to ablation is simulated based on a virtual ablation probe position and the simulated blood flow in the liver and the venous system of the liver. Cellular necrosis in the liver is simulated based on the simulated heat diffusion. A visualization of a simulated necrosis region is generated and displayed to the user for decision making and optimal therapy planning and guidance.

Abstract: Machine training and application of machine-trained classifier are used for image-based tumor phenotyping in a medical system. To create a training database with known phenotype information, synthetic medical images are created. A computational tumor model creates various examples of tumors in tissue. Using the computational tumor model allows one to create examples not available from actual patients, increasing the number and variance of examples used for machine-learning to predict tumor phenotype. A model of an imaging system generates synthetic images from the examples. The machine-trained classifier is applied to images from actual patients to predict tumor phenotype for that patient based on the knowledge learned from the synthetic images.

Abstract: Methods and systems for image registration using an intelligent artificial agent are disclosed. In an intelligent artificial agent based registration method, a current state observation of an artificial agent is determined based on the medical images to be registered and current transformation parameters. Action-values are calculated for a plurality of actions available to the artificial agent based on the current state observation using a machine learning based model, such as a trained deep neural network (DNN). The actions correspond to predetermined adjustments of the transformation parameters. An action having a highest action-value is selected from the plurality of actions and the transformation parameters are adjusted by the predetermined adjustment corresponding to the selected action. The determining, calculating, and selecting steps are repeated for a plurality of iterations, and the medical images are registered using final transformation parameters resulting from the plurality of iterations.

Abstract: A method and system for automatic non-invasive estimation of shear modulus and viscosity of biological tissue from shear-wave imaging is disclosed. Shear-wave images are acquired to evaluate the mechanical properties of an organ of a patient. Shear-wave propagation in the tissue in the shear-wave images is simulated based on shear modulus and viscosity values for the tissue using a computational model of shear-wave propagation. The simulated shear-wave propagation is compared to observed shear-wave propagation in the shear-wave images of the tissue using a cost function. Patient-specific shear modulus and viscosity values for the tissue are estimated to optimize the cost function comparing the simulated shear-wave propagation to the observed shear-wave propagation.

Abstract: A method and system for determining fractional flow reserve (FFR) for a coronary artery stenosis of a patient is disclosed. In one embodiment, medical image data of the patient including the stenosis is received, a set of features for the stenosis is extracted from the medical image data of the patient, and an FFR value for the stenosis is determined based on the extracted set of features using a trained machine-learning based mapping. In another embodiment, a medical image of the patient including the stenosis of interest is received, image patches corresponding to the stenosis of interest and a coronary tree of the patient are detected, an FFR value for the stenosis of interest is determined using a trained deep neural network regressor applied directly to the detected image patches.

Abstract: A method and system for determining fractional flow reserve (FFR) for a coronary artery stenosis of a patient is disclosed. In one embodiment, medical image data of the patient including the stenosis is received, a set of features for the stenosis is extracted from the medical image data of the patient, and an FFR value for the stenosis is determined based on the extracted set of features using a trained machine-learning based mapping. In another embodiment, a medical image of the patient including the stenosis of interest is received, image patches corresponding to the stenosis of interest and a coronary tree of the patient are detected, an FFR value for the stenosis of interest is determined using a trained deep neural network regressor applied directly to the detected image patches.

Abstract: Methods and systems for estimating patient-specific cardiac electrical properties from medical image data and non-invasive electrocardiography measurements of a patient are disclosed. A patient-specific anatomical heart model is generated from medical image data of a patient. Patient-specific cardiac electrical properties are estimated by simulating cardiac electrophysiology over time in the patient-specific anatomical heart model using a computational cardiac electrophysiology model and adjusting cardiac electrical parameters based on the simulation results and the non-invasive electrocardiography measurements. A patient-specific cardiac electrophysiology model with the patient-specific cardiac electrical parameters can then be used to perform virtual cardiac electrophysiology interventions for planning and guidance of cardiac electrophysiology interventions.