Abstract: A computer implemented method includes transforming a set of received samples from a set of data into a multiplexed graph, by creating a plurality of planes, each plane having the set of nodes and the set of edges. Each set of edges is associated with a given relation type from the set of relation types. Message passing walks are alternated within and across the plurality of planes of the multiplexed graph using a graph neural network (GNN) layer. The GNN layer has a plurality of units where each unit outputs an aggregation of two parallel sub-units. Sub-units include a typed GNN layer that allows different permutations of connectivity patterns between intra-planar and inter-planar nodes. A task-specific supervision is used to train a set of weights of the GNN for the machine learning task.

Abstract: Computer-implemented methods are provided for generating machine learning model for multimodal data inference tasks. Such a method includes, for each sample in a training dataset of multimodal data samples, encoding the sample to produce a compressed vector representation of the sample in a k-dimensional latent space, and perturbing features of the sample to identify, for each dimension of the latent space, a set of active features perturbation of each of which produces more than a threshold change in the vector representation in that dimension. The method further comprises generating a sample graph having nodes interconnected by edges, wherein the nodes comprise nodes representing respective said features of the sample and edges interconnecting nodes indicate the active features for each dimension. The sample graph is then used to train a graph neural network model to perform the multimodal data inference task. Multimodal data inference systems employing such models are also provided.

Abstract: Methods and systems are provided for processing different-modality digital images of tissue. The method includes, for each image, detecting biological entities in the image and generating an entity graph comprising entity nodes, representing respective biological entities, interconnected by edges representing interactions between entities represented by the entity nodes. The method also includes selecting, from each image, anchor elements comprising elements corresponding to anchor elements of at least one other image, and generating an anchor graph in which anchor nodes, representing respective anchor elements, are interconnected with entity nodes of the entity graph for the image by edges indicating relations between entity nodes and anchor nodes.

Abstract: Computer-implemented methods and systems are provided for interpreting a whole-slide image of a tissue specimen. Such a method includes detecting biological entities in a region of interest independently at each magnification level in a whole-slide image. An entity graph is generated for each magnification level, the entity graph comprising nodes, representing respective biological entities detected at that magnification level, interconnected by edges representing interactions between entities. The method also includes, for each region of interest, generating a hierarchical graph, defining a hierarchy of the entity graphs for that region, in which nodes of different entity graphs are interconnected by hierarchical edges representing hierarchical relations between nodes of the entity graphs.

Abstract: Methods and systems are provided for processing different-modality digital images of tissue. The method includes, for each image, detecting biological entities in the image and generating an entity graph comprising entity nodes, representing respective biological entities, interconnected by edges representing interactions between entities represented by the entity nodes. The method also includes selecting, from each image, anchor elements comprising elements corresponding to anchor elements of at least one other image, and generating an anchor graph in which anchor nodes, representing respective anchor elements, are interconnected with entity nodes of the entity graph for the image by edges indicating relations between entity nodes and anchor nodes.

Abstract: A system and process for extracting information from filled form images is described. In one example, the claimed invention first extracts textual information and the hierarchy in a blank form. This information is then used to extract and understand the content of filled forms. In this way, the system does not have to analyze from the beginning each filled form. The system is designed so that it remains as generic as possible. The number of hard-coded rules in the whole pipeline was minimized to offer an adaptive solution able to address the largest number of forms, with various structures and typography. The system is also created to be integrated as a built-in function in a larger pipeline. The form understanding pipeline could be the starting point of any advanced Natural Language Processing application.

Abstract: Computer-implemented methods and systems are provided for interpreting a whole-slide image of a tissue specimen. Such a method includes detecting biological entities in a region of interest independently at each magnification level in a whole-slide image. An entity graph is generated for each magnification level, the entity graph comprising nodes, representing respective biological entities detected at that magnification level, interconnected by edges representing interactions between entities. The method also includes, for each region of interest, generating a hierarchical graph, defining a hierarchy of the entity graphs for that region, in which nodes of different entity graphs are interconnected by hierarchical edges representing hierarchical relations between nodes of the entity graphs.

Abstract: A system and process for extracting information from filled form images is described. In one example, the claimed invention first extracts textual information and the hierarchy in a blank form. This information is then used to extract and understand the content of filled forms. In this way, the system does not have to analyze from the beginning each filled form. The system is designed so that it remains as generic as possible. The number of hard-coded rules in the whole pipeline was minimized to offer an adaptive solution able to address the largest number of forms, with various structures and typography. The system is also created to be integrated as a built-in function in a larger pipeline. The form understanding pipeline could be the starting point of any advanced Natural Language Processing application.

Abstract: A system and process for extracting information from filled form images is described. In one example the claimed invention first extracts textual information and the hierarchy in a blank form. This information is then used to extract and understand the content of filled forms. In this way, the system does not have to analyze from the beginning each filled form. The system is designed so that it remains as generic as possible. The number of hard coded rules in the whole pipeline was minimized to offer an adaptive solution able to address the largest number of forms, with various structures and typography. The system is also created to be integrated as a built-in function in a larger pipeline. The form understanding pipeline could be the starting point of any advanced Natural Language Processing application.

Abstract: A system and process for extracting information from filled form images is described. In one example the claimed invention first extracts textual information and the hierarchy in a blank form. This information is then used to extract and understand the content of filled forms. In this way, the system does not have to analyze from the beginning each filled form. The system is designed so that it remains as generic as possible. The number of hard coded rules in the whole pipeline was minimized to offer an adaptive solution able to address the largest number of forms, with various structures and typography. The system is also created to be integrated as a built-in function in a larger pipeline. The form understanding pipeline could be the starting point of any advanced Natural Language Processing application.

Abstract: A computer-implemented method for enforcing an idempotent-constrained characteristic during training of a neural network may be provided. The method comprises training of a neural network by minimizing a loss function, wherein the loss function comprises an additional term imposing an idempotence-based regularization to the neural network during the training.

Abstract: Computer-implemented methods are provided for generating a data structure representing tabular information in a scanned image. Such a method can include storing image data representing a scanned image of a table, processing the image data to identify positions of characters and lines in the image, and mapping locations in the image of information cells, each containing a set of the characters, in dependence on said positions. The method can also include, for each cell, determining cell attribute values, dependent on the cell locations, for a predefined set of cell attributes, and supplying the attribute values as inputs to a machine-learning model trained to pre-classify cells as header cells or data cells in dependence on cell attribute values.

Abstract: A system and process for extracting information from filled form images is described. In one example the claimed invention first extracts textual information and the hierarchy in a blank form. This information is then used to extract and understand the content of filled forms. In this way, the system does not have to analyze from the beginning each filled form. The system is designed so that it remains as generic as possible. The number of hard coded rules in the whole pipeline was minimized to offer an adaptive solution able to address the largest number of forms, with various structures and typography. The system is also created to be integrated as a built-in function in a larger pipeline. The form understanding pipeline could be the starting point of any advanced Natural Language Processing application.

Abstract: A system and process for extracting information from filled form images is described. In one example the claimed invention first extracts textual information and the hierarchy in a blank form. This information is then used to extract and understand the content of filled forms. In this way, the system does not have to analyze from the beginning each filled form. The system is designed so that it remains as generic as possible. The number of hard coded rules in the whole pipeline was minimized to offer an adaptive solution able to address the largest number of forms, with various structures and typography. The system is also created to be integrated as a built-in function in a larger pipeline. The form understanding pipeline could be the starting point of any advanced Natural Language Processing application.

Abstract: Computer-implemented methods are provided for generating a data structure representing tabular information in a scanned image. Such a method can include storing image data representing a scanned image of a table, processing the image data to identify positions of characters and lines in the image, and mapping locations in the image of information cells, each containing a set of the characters, in dependence on said positions. The method can also include, for each cell, determining cell attribute values, dependent on the cell locations, for a predefined set of cell attributes, and supplying the attribute values as inputs to a machine-learning model trained to pre-classify cells as header cells or data cells in dependence on cell attribute values.

Abstract: A computer-implemented method for modifying patterns in datasets using a generative adversarial network may be provided. The method comprises providing pairs of data samples. The pairs comprise each a base data sample and a modified data sample. Thereby, the modified pattern is determined by applying random modifications to the base data sample. Additionally, the method comprises training of the generator for building a model of the generator using an adversarial training method and using the pairs of data samples as input, wherein the discriminator receives as input dataset pairs of datasets, the dataset pairs comprising each a prediction output of the generator based on a base data sample and the corresponding modified data sample, thereby optimizing a joint loss function for the generator and the discriminator, and predicting an output dataset for unknown data samples as input for the generator without the discriminator.

Abstract: A method of determining a hierarchy of a blank template using an image of the blank template and using the determined hierarchy for providing labels and field values of text lines of a filled form document.

Abstract: A system and process for extracting information from filled form images is described. In one example the claimed invention first extracts textual information and the hierarchy in a blank form. This information is then used to extract and understand the content of filled forms. In this way, the system does not have to analyze from the beginning each filled form. The system is designed so that it remains as generic as possible. The number of hard coded rules in the whole pipeline was minimized to offer an adaptive solution able to address the largest number of forms, with various structures and typography. The system is also created to be integrated as a built-in function in a larger pipeline. The form understanding pipeline could be the starting point of any advanced Natural Language Processing application.

Abstract: A system and process r extracting information from filled form images is described. In one example the claimed invention first extracts textual information and the hierarchy in a blank form. This information is then used to extract and understand the content of filled forms. In this way, the system does not have to analyze from the beginning each filled form. The system is designed so that it remains as generic as possible. The number of hard coded rules in the whole pipeline was minimized to offer an adaptive solution able to address the largest number of forms, with various structures and typography. The system is also created to be integrated as a built-in function in a larger pipeline. The form understanding pipeline could be the starting point of any advanced Natural Language Processing application.

Abstract: A method of determining a hierarchy of a blank template using an image of the blank template and using the determined hierarchy for providing labels and field values of text lines of a filled form document.