Abstract: A computer-implemented method includes receiving first inputs associated with a first modality and second inputs associated with a second modality; processing the received first and second inputs with convolutional neural networks (CNN), wherein a set of first weights are used to handle the first inputs and a second set of weights are used to handle the second inputs; determining a loss for each of the first and the second inputs based on a loss function that applies the first set of weights, the second set of weights, and a presence of a co-occurrence; generating a shared feature space as an output of the CNNs, wherein a distance between cells associated with the first inputs and the second inputs in the shared feature space is determined based on the loss associated with each of the first inputs and the second inputs; and based on the shared feature space, providing an output.

Abstract: A computer-implemented method includes receiving first inputs associated with a first modality and second inputs associated with a second modality; processing the received first and second inputs with convolutional neural networks (CNN), wherein a set of first weights are used to handle the first inputs and a second set of weights are used to handle the second inputs; determining a loss for each of the first and the second inputs based on a loss function that applies the first set of weights, the second set of weights, and a presence of a co-occurrence; generating a shared feature space as an output of the CNNs, wherein a distance between cells associated with the first inputs and the second inputs in the shared feature space is determined based on the loss associated with each of the first inputs and the second inputs; and based on the shared feature space, providing an output.