Abstract: A method and a system are for providing a medical data structure for a patient. The system includes a plurality of data sources, each data source to provide medical data of the patient; a computing device to implement an artificial neural network structure a plurality of encoding modules, each being realized as an artificial neural network configured and trained to generate, from the medical data from the corresponding data source, a corresponding encoded output matrix; a weighting gate module for each of the encoding modules; a concatenation module configured to concatenate weighted output matrices of the weighting gates to a concatenated output matrix; and an aggregation module realized as an artificial neural network configured and trained to receive the concatenated output matrix and to generate therefrom the medical data structure for the patient, the artificial neural network structure being trained as a whole using a cost function.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for planning robotic movements to perform a given task while satisfying object pose estimation accuracy requirements. One of the methods includes generating a plurality of candidate measurement configurations for measuring an object to be manipulated by a robot; determining respective measurement accuracies for the plurality of candidate measurement configurations; determining a measurement accuracy landscape for the object including defining a high measurement accuracy region based on the respective measurement accuracies for the plurality of candidate measurement configurations; and generating a motion plan for manipulating the object in the robotic process that moves the robot, a sensor, or both, through the high measurement accuracy region when performing pose estimation for the object.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for planning robotic movements to perform a given task while satisfying object pose estimation accuracy requirements. One of the methods includes generating a plurality of candidate measurement configurations for measuring an object to be manipulated by a robot; determining respective measurement accuracies for the plurality of candidate measurement configurations; determining a measurement accuracy landscape for the object including defining a high measurement accuracy region based on the respective measurement accuracies for the plurality of candidate measurement configurations; and generating a motion plan for manipulating the object in the robotic process that moves the robot, a sensor, or both, through the high measurement accuracy region when performing pose estimation for the object.

Abstract: In a method and a computer for determining a training function in order to generate annotated training images, a training image and training-image information are provided to a computer, together with an isolated item of image information that is independent of the training image. A first calculation is made in the computer by applying an image-information-processing first function to the isolated item of image information, and a second calculation is made by applying an image-information-processing second function to the training image. Adjustments to the first and second functions are made based on these calculation results, from which a determination of a training function is then made in the computer.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing image data. One of the method includes receiving an input image from a source domain, the input image showing an object to be manipulated by a robot in a robotic process; processing the input image to generate an intermediate representation of the input image, comprising: generating a gradient orientation representation and a gradient magnitude representation of the input image; and generating the intermediate representation of the input image from the gradient orientation representation and the gradient magnitude representation; processing the intermediate representation of the input image using a neural network trained to make predictions about objects in images to generate a network output that represents a prediction about physical characteristics of the object in the input image.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a feature extraction neural network to generate domain-invariant feature representations from domain-varying input images. In one aspect, the method includes obtaining a training dataset comprising a first set of target domain images and a second set of real domain images that each have pixel-wise level alignment with a corresponding target domain image, and training the feature extraction neural network on the training dataset based on optimizing an objective function that includes a term that depends on a similarity between respective feature representations generated by the network for a pair of target and source domain images.

Abstract: In order to compare high-dimensional, multi-modal data for a patient to data for other patients, deep learning is used to encode original, multi-modal data for a patient into a compact signature. The compact signature is compared to predetermined compact signatures generated for other patients, and similar predetermined compact signatures are identified based on the comparison. A clinical outcome may be predicted based on the similar predetermined compact signatures that are identified.

Abstract: Systems and methods are disclosed for medical image processing using neural networks. A first and a second controller network share a memory to which both the first and second controller network can write data and from which both the first and the second controller network can read data. Reading and writing is performed by respective read and write heads which are advantageously neural networks trained how to write and read in an optimal way. The memory thus provides each controller network with context data generated by the respective other controller network.

Abstract: A framework for defective pixel correction using adversarial networks. In accordance with one aspect, the framework receives first and second training image datasets. The framework performs adversarial training of a corrector and a classifier with the first and second training image datasets respectively. The corrector may be trained to correct a first input image and the classifier may be trained to recognize whether a second input image is real or generated by the corrector. The framework applies the trained corrector to a current image to correct any defective pixels and generate a corrected image. The corrected image may then be presented.

Abstract: A method, apparatus and non-transitory computer readable medium are for segmenting different types of structures, including cancerous lesions and regular structures like vessels and skin, in a digital breast tomosynthesis (DBT) volume. In an embodiment, the method includes: pre-classification of the DBT volume in dense and fatty tissue and based on the result; localizing a set of structures in the DBT volume by using a multi-stream deep convolutional neural network; and segmenting the localized structures by calculating a probability for belonging to a specific type of structure for each voxel in the DBT volume by using a deep convolutional neural network for providing a three-dimensional probabilistic map.

Abstract: A method is for determining a two-dimensional mammography dataset. The method includes the receipt of a three-dimensional mammography dataset of an examination region via an interface. The method furthermore includes the first determination of a two-dimensional mammography dataset of the examination region by application of a trained generator function to the three-dimensional mammography dataset via a processing unit, wherein the trained generator function is based on a trained GA network. Through this method, it is possible efficiently to create two-dimensional mammography datasets, which are visually similar to real two-dimensional mammography datasets and can therefore be appraised with standardized methods.

Abstract: A framework for defective pixel correction using adversarial networks. In accordance with one aspect, the framework receives first and second training image datasets. The framework performs adversarial training of a corrector and a classifier with the first and second training image datasets respectively. The corrector may be trained to correct a first input image and the classifier may be trained to recognize whether a second input image is real or generated by the corrector. The framework applies the trained corrector to a current image to correct any defective pixels and generate a corrected image. The corrected image may then be presented.

Abstract: A computer-implemented method for identifying features in 3D image volumes includes dividing a 3D volume into a plurality of 2D slices and applying a pre-trained 2D multi-channel global convolutional network (MC-GCN) to the plurality of 2D slices until convergence. Following convergence of the 2D MC-GCN, a plurality of parameters are extracted from a first feature encoder network in the 2D MC-GCN. The plurality of parameters are transferred to a second feature encoder network in a 3D Anisotropic Hybrid Network (AH-Net). The 3D AH-Net is applied to the 3D volume to yield a probability map;. Then, using the probability map, one or more of (a) coordinates of the objects with non-maximum suppression or (b) a label map of objects of interest in the 3D volume are generated.

Abstract: A method and a system are for providing a medical data structure for a patient. The system includes a plurality of data sources, each data source to provide medical data of the patient; a computing device to implement an artificial neural network structure a plurality of encoding modules, each being realized as an artificial neural network configured and trained to generate, from the medical data from the corresponding data source, a corresponding encoded output matrix; a weighting gate module for each of the encoding modules; a concatenation module configured to concatenate weighted output matrices of the weighting gates to a concatenated output matrix; and an aggregation module realized as an artificial neural network configured and trained to receive the concatenated output matrix and to generate therefrom the medical data structure for the patient, the artificial neural network structure being trained as a whole using a cost function.

Abstract: Systems and methods are disclosed for medical image processing using neural networks. A first and a second controller network share a memory to which both the first and second controller network can write data and from which both the first and the second controller network can read data. Reading and writing is performed by respective read and write heads which are advantageously neural networks trained how to write and read in an optimal way. The memory thus provides each controller network with context data generated by the respective other controller network.

Abstract: Both object-oriented analysis and the faster pixel-oriented analysis are used to recognize patterns in an image of stained tissue. Object-oriented image analysis is used to segment a small portion of the image into object classes. Then the object class to which each pixel in the remainder of the image most probably belongs is determined using decision trees with pixelwise descriptors. The pixels in the remaining image are assigned object classes without segmenting the remainder of the image into objects. After the small portion is segmented into object classes, characteristics of object classes are determined. The pixelwise descriptors describe which pixels are associated with particular object classes by matching the characteristics of object classes to the comparison between pixels at predetermined offsets. A pixel heat map is generated by giving each pixel the color assigned to the object class that the pixelwise descriptors indicate is most probably associated with that pixel.

Abstract: A computer-implemented method for identifying features in 3D image volumes includes dividing a 3D volume into a plurality of 2D slices and applying a pre-trained 2D multi-channel global convolutional network (MC-GCN) to the plurality of 2D slices until convergence. Following convergence of the 2D MC-GCN, a plurality of parameters are extracted from a first feature encoder network in the 2D MC-GCN. The plurality of parameters are transferred to a second feature encoder network in a 3D Anisotropic Hybrid Network (AH-Net). The 3D AH-Net is applied to the 3D volume to yield a probability map;. Then, using the probability map, one or more of (a) coordinates of the objects with non-maximum suppression or (b) a label map of objects of interest in the 3D volume are generated.

Abstract: A method is for determining a two-dimensional mammography dataset. The method includes the receipt of a three-dimensional mammography dataset of an examination region via an interface. The method furthermore includes the first determination of a two-dimensional mammography dataset of the examination region by application of a trained generator function to the three-dimensional mammography dataset via a processing unit, wherein the trained generator function is based on a trained GA network. Through this method, it is possible efficiently to create two-dimensional mammography datasets, which are visually similar to real two-dimensional mammography datasets and can therefore be appraised with standardized methods.

Abstract: A method, apparatus and non-transitory computer readable medium are for segmenting different types of structures, including cancerous lesions and regular structures like vessels and skin, in a digital breast tomosynthesis (DBT) volume. In an embodiment, the method includes: pre-classification of the DBT volume in dense and fatty tissue and based on the result; localizing a set of structures in the DBT volume by using a multi-stream deep convolutional neural network; and segmenting the localized structures by calculating a probability for belonging to a specific type of structure for each voxel in the DBT volume by using a deep convolutional neural network for providing a three-dimensional probabilistic map.

Abstract: A first interface for reading a medical patient image record is provided. Furthermore, provision is made of an encoding module for machine-based learning of data encodings of image patterns by an unsupervised deep learning and for establishing a deep-learning-reduced data encoding of a patient image pattern contained in the patient image record. Furthermore, provision is made of a comparison module for comparing the established data encoding with reference encodings of reference image patterns stored in a database and for selecting a reference image pattern with a reference encoding which is similar to the established data encoding. An assignment module serves to establish a key term assigned to the selected reference image pattern and to assign the established key term to the patient image pattern. A second interface is provided for outputting the established key term with assignment to the patient image pattern.