Abstract: A computer implemented method includes transforming a set of received samples from a set of data into a multiplexed graph, by creating a plurality of planes, each plane having the set of nodes and the set of edges. Each set of edges is associated with a given relation type from the set of relation types. Message passing walks are alternated within and across the plurality of planes of the multiplexed graph using a graph neural network (GNN) layer. The GNN layer has a plurality of units where each unit outputs an aggregation of two parallel sub-units. Sub-units include a typed GNN layer that allows different permutations of connectivity patterns between intra-planar and inter-planar nodes. A task-specific supervision is used to train a set of weights of the GNN for the machine learning task.

Abstract: Computer-implemented methods are provided for generating machine learning model for multimodal data inference tasks. Such a method includes, for each sample in a training dataset of multimodal data samples, encoding the sample to produce a compressed vector representation of the sample in a k-dimensional latent space, and perturbing features of the sample to identify, for each dimension of the latent space, a set of active features perturbation of each of which produces more than a threshold change in the vector representation in that dimension. The method further comprises generating a sample graph having nodes interconnected by edges, wherein the nodes comprise nodes representing respective said features of the sample and edges interconnecting nodes indicate the active features for each dimension. The sample graph is then used to train a graph neural network model to perform the multimodal data inference task. Multimodal data inference systems employing such models are also provided.