Abstract: Mechanisms are provided to implement a visual analytics pipeline. The mechanisms generate, from an input database of records, a chronology-aware graph data structure of a plurality of records based features specified in an ontology data structure. The chronology-aware graph data structure has vertices representing one or more of events or records based features corresponding to events, and edges representing chronological relationships between events. The mechanisms execute a chronology-aware graph query on the chronology-aware graph data structure to generate a filtered set of vertices and corresponding features corresponding to criteria of the chronology-aware graph query.

Abstract: A computer implemented method includes transforming a set of received samples from a set of data into a multiplexed graph, by creating a plurality of planes, each plane having the set of nodes and the set of edges. Each set of edges is associated with a given relation type from the set of relation types. Message passing walks are alternated within and across the plurality of planes of the multiplexed graph using a graph neural network (GNN) layer. The GNN layer has a plurality of units where each unit outputs an aggregation of two parallel sub-units. Sub-units include a typed GNN layer that allows different permutations of connectivity patterns between intra-planar and inter-planar nodes. A task-specific supervision is used to train a set of weights of the GNN for the machine learning task.

Abstract: Mechanisms are provided to implement a visual analytics pipeline. The mechanisms generate, from an input database of records, a chronology-aware graph data structure of a plurality of records based features specified in an ontology data structure. The chronology-aware graph data structure has vertices representing one or more of events or records based features corresponding to events, and edges representing chronological relationships between events. The mechanisms execute a chronology-aware graph query on the chronology-aware graph data structure to generate a filtered set of vertices and corresponding features corresponding to criteria of the chronology-aware graph query.

Abstract: Mechanisms are provided to improve an output of a trained machine learning (ML) computer model based on label co-occurrence statistics. For a corpus, label vector representations of the knowledge data structures are generated. Co-occurrence scores for each pairing of labels, across the label vector representations, are generated. A vector output of the ML computer model is received and a knowledge driven reasoning (KDR) computer model is configured with threshold(s) and delta value(s) specifying condition(s) of a co-occurrence of a first label in the output with a second label in the plurality of labels which, if present, causes the delta value(s) to be applied to modify a probability value associated with the second label in the output of the ML computer model. The KDR computer model is executed on the output of the ML computer model to modify probability value(s) in the output.

Abstract: Techniques for spatially preserving flattening in deep learning neural networks are provided. In one aspect, a spatially preserving flattening module includes: a predictor for generating image feature maps from at least one convolutional layer of a feature extraction phase of a deep learning neural network applied to input image data; an auto-encoder for producing encodings of the image feature maps that preserve location and shape information associated with objects in the input image data; and a flattener for concatenating the encodings of the image feature maps to form a spatially preserving flattened encoding vector. A deep learning neural network that includes the present spatially preserving flattening module is also provided, as is a method for spatially preserving flattening.

Abstract: Computer-implemented methods are provided for generating machine learning model for multimodal data inference tasks. Such a method includes, for each sample in a training dataset of multimodal data samples, encoding the sample to produce a compressed vector representation of the sample in a k-dimensional latent space, and perturbing features of the sample to identify, for each dimension of the latent space, a set of active features perturbation of each of which produces more than a threshold change in the vector representation in that dimension. The method further comprises generating a sample graph having nodes interconnected by edges, wherein the nodes comprise nodes representing respective said features of the sample and edges interconnecting nodes indicate the active features for each dimension. The sample graph is then used to train a graph neural network model to perform the multimodal data inference task. Multimodal data inference systems employing such models are also provided.

Abstract: Mechanisms are provided to implement a fine-grained finding descriptor generation computing tool that automatically generates fine-grained labels for downstream computer system operations. The mechanisms process medical report content based on a core finding lexicon, to extract core finding instances from the medical report content. The mechanisms execute, for each core finding instance, automated computer NLP operations that generate a parse tree for the portion of the medical report content corresponding to the core finding instance, perform phrasal grouping on the parse tree to thereby associate one or more modifiers of core findings specified in the portion of the medical report content with the core finding instance, and generate a fine-grained finding descriptor data structure for the core finding instance based on the association of one or more modifiers of the core finding with the core finding instance.

Abstract: Mechanisms for implementing a text encoder and text encoder operations are provided. A contrastive machine learning training operation trains an encoder of a machine learning computer model, to learn a sense and similarity preserving embedding that operates to encode input natural language text data to generate encoded natural language text data based on a sense attribute of one or more terms in the input natural language text data. The contrastive machine learning training operation learns to separate positive samples in training data from negative samples in the training data. The trained encoder processes a term specified in an input natural language text to generate an encoded natural language text based on the learned embedding and inputs, to a downstream computing system, the encoded natural language text to cause the downstream computing system to perform a computer natural language processing operation based on the embedding.

Abstract: A joint multimodal fusion computer model architecture is provided that receives prediction output data from a machine learning (ML) computer model set comprising a plurality of different subsets of ML computer models operating on input data of different modalities and generating different prediction outputs. Prediction outputs are fused by executing an uncertainty and correlation weighted (UCW) joint multimodal fusion operation on the prediction outputs to generate a fused output providing multimodal prediction output data. The UCW joint multimodal fusion operation applies different weights to different ones of prediction outputs from the different subsets of ML computer models operating on input data of different modalities. The different weights are determined based on an estimation of uncertainty in each of the different subsets of ML computer models and an estimate of a correlation between different modalities.

Abstract: A method for automatically training and applying automatic segmentation in digital image processing is provided. The method may include, in response to receiving a plurality of digital images wherein each digital image associated with the plurality of digital images comprises only one annotated structure out of a plurality of structures included in each digital image, applying a predictive algorithm to each digital image that determines a predicted probability of each annotation in each digital image, determines a predicted background for each digital image, and merges the predicted probability of each annotation with the predicted background in each digital image. The method may further include, in response to applying the predictive algorithm, using the received plurality of digital images to train and apply an application for automatically segmenting unlabeled digital images.

Abstract: Mechanisms are provided to improve an output of a trained machine learning (ML) computer model based on label co-occurrence statistics. For a corpus, label vector representations of the knowledge data structures are generated. Co-occurrence scores for each pairing of labels, across the label vector representations, are generated. A vector output of the ML computer model is received and a knowledge driven reasoning (KDR) computer model is configured with threshold(s) and delta value(s) specifying condition(s) of a co-occurrence of a first label in the output with a second label in the plurality of labels which, if present, causes the delta value(s) to be applied to modify a probability value associated with the second label in the output of the ML computer model. The KDR computer model is executed on the output of the ML computer model to modify probability value(s) in the output.

Abstract: Mechanisms are provided to implement a visual analytics pipeline. The mechanisms generate, from an input database of records, a chronology-aware graph data structure of a plurality of records based features specified in an ontology data structure. The chronology-aware graph data structure has vertices representing one or more of events or records based features corresponding to events, and edges representing chronological relationships between events. The mechanisms execute a chronology-aware graph query on the chronology-aware graph data structure to generate a filtered set of vertices and corresponding features corresponding to criteria of the chronology-aware graph query.

Abstract: A joint multimodal fusion computer model architecture is provided that receives prediction output data from a machine learning (ML) computer model set comprising a plurality of different subsets of ML computer models operating on input data of different modalities and generating different prediction outputs. Prediction outputs are fused by executing an uncertainty and correlation weighted (UCW) joint multimodal fusion operation on the prediction outputs to generate a fused output providing multimodal prediction output data. The UCW joint multimodal fusion operation applies different weights to different ones of prediction outputs from the different subsets of ML computer models operating on input data of different modalities. The different weights are determined based on an estimation of uncertainty in each of the different subsets of ML computer models and an estimate of a correlation between different modalities.

Abstract: Mechanisms are provided to implement a fine-grained finding descriptor generation computing tool that automatically generates fine-grained labels for downstream computer system operations. The mechanisms process medical report content based on a core finding lexicon, to extract core finding instances from the medical report content. The mechanisms execute, for each core finding instance, automated computer NLP operations that generate a parse tree for the portion of the medical report content corresponding to the core finding instance, perform phrasal grouping on the parse tree to thereby associate one or more modifiers of core findings specified in the portion of the medical report content with the core finding instance, and generate a fine-grained finding descriptor data structure for the core finding instance based on the association of one or more modifiers of the core finding with the core finding instance.

Abstract: A generalized distributed framework is provided for parallel search and retrieval of unstructured and structured patient data across zones with hierarchical ranking. In various embodiments, patient data is ingested from a plurality of data sources. A plurality of data models is populated based on the ingested patient data, each data model comprising an abstract data type. The plurality of data models is stored in an index. A search request is processed against the index, the search request comprising one or more attribute of the abstract data type.

Abstract: A method for automatically training and applying automatic segmentation in digital image processing is provided. The method may include, in response to receiving a plurality of digital images wherein each digital image associated with the plurality of digital images comprises only one annotated structure out of a plurality of structures included in each digital image, applying a predictive algorithm to each digital image that determines a predicted probability of each annotation in each digital image, determines a predicted background for each digital image, and merges the predicted probability of each annotation with the predicted background in each digital image. The method may further include, in response to applying the predictive algorithm, using the received plurality of digital images to train and apply an application for automatically segmenting unlabeled digital images.

Abstract: A system for automated detection and type classification of central venous catheters. The system includes an electronic processor that is configured to, based on an image, generate a segmentation of a potential central venous catheter using a segmentation method and extract, from the segmentation, one or more image features associated with the potential central venous catheter. The electronic processor is also configured to, based on the one or more image features, determine, using a first classifier, whether the image includes a central venous catheters and determine, using a second classifier, a type of central venous catheter included in the image.

Abstract: Mechanisms are provided to implement an automated medical imaging report generator which receives an input medical image and inputs the input medical image into a machine learning (ML) computer model trained to predict finding labels based on patterns of image features extracted from the medical image. The ML computer model generates a prediction of a finding label applicable to the input medical image in terms of a finding label prediction output vector. Based on the finding label prediction output vector, a lookup operation is performed, in a medical report database of previously processed medical imaging report data structures, to find a matching medical imaging report data structure corresponding to the finding label. An output medical imaging report is generated for the input medical image based on natural language content of the matching medical imaging report data structure.

Abstract: Workflows for automatic measurement of Doppler is provided. In various embodiments, a plurality of frames of a medical video are read. A mode label indicative of a mode of each of the plurality of frames is determined. At least one of the plurality of frames is provided to a trained feature generator. The at least one of the plurality of frames have the same mode label. At least one feature vector is obtained from the trained feature generator corresponding to the at least one of the plurality of frames. At least one feature vector is provided to a trained classifier. A valve label indicative of a valve is obtained from the trained classifier corresponding to the at least one of the plurality of frames. One or more measurement is extracted indicative of a disease condition from those of the at least one of the plurality of frames matching a predetermined valve label.

Abstract: Annotation of large image datasets is provided. In various embodiments, a plurality of medical images is received. At least one collection is formed containing a subset of the plurality of medical images. One or more image from the at least one collection is provided to each of a plurality of remote users. An annotation template is provided to each of the plurality of remote users. Annotations for the one or more image are received from each of the plurality of remote users. The annotations and the plurality of medical images are stored together.