Abstract: A method for training a machine learning model using self-contrastive decorrelation is provided. The method comprises training a machine learning model by receiving a sentence including text, performing a first encoding operation on the sentence, performing a second encoding operation on the sentence, mapping the first vector representation on which a first augmentation operation is performed to a first high dimensional vector representation and the second vector representation on which a first augmentation operation is performed to a second high dimensional vector representation, generating a correlation matrix using the first high dimensional vector representation and the second high dimensional vector representation, and performing a decorrelation operation on the correlation matrix.

Abstract: Disclosed herein are system, method, and computer program product embodiments for similarity scoring of sentences, while restricting distances between tokenized pairs in the sentences. An embodiment operates by determining a similarity of tokens between a first sequence of tokens and a second sequence of tokens to generate token pairs, determining a distance of relative positioning of token pairs in the first tokenized sequence and the second tokenized sequence and generating a score value that indicates the degree to which the first sentence matches the second sentence based on restricting matches to a maximum value of the distance of relative positions of the token pairs in the first tokenized sequence and the second tokenized sequence.

Abstract: A method for training a machine learning model using self-contrastive decorrelation is provided. The method comprises training a machine learning model by receiving a sentence including text, performing a first encoding operation on the sentence, performing a second encoding operation on the sentence, mapping the first vector representation on which a first augmentation operation is performed to a first high dimensional vector representation and the second vector representation on which a first augmentation operation is performed to a second high dimensional vector representation, generating a correlation matrix using the first high dimensional vector representation and the second high dimensional vector representation, and performing a decorrelation operation on the correlation matrix.

Abstract: A method for training a machine learning model using self-contrastive decorrelation is provided. The method comprises training a machine learning model by receiving a sentence including text, performing a first encoding operation on the sentence, performing a second encoding operation on the sentence, mapping the first vector representation on which a first augmentation operation is performed to a first high dimensional vector representation and the second vector representation on which a first augmentation operation is performed to a second high dimensional vector representation, generating a correlation matrix using the first high dimensional vector representation and the second high dimensional vector representation, and performing a decorrelation operation on the correlation matrix.

Abstract: A method comprises generating vectors of sentences, the sentences including text, performing an augmentation operation on a first vector of the vectors and a second vector of the vectors, the augmentation operation comprising: merging the first vector with the second vector such that at least a first element of the first vector is combined with a second element of the second vector, wherein the first element is representative of a first sentence of the plurality of sentences and the second element represents a second sentence of the plurality of sentences, inputting, the first vector that is merged with the second vector, into a first layer, and generating a third vector that is based on the first vector that is merged with the second vector, the third vector including a third element that represents the first sentence and the second sentence.

Abstract: A method and system for generating inputs specific to a multimodal learning based machine learning model. The generating of inputs specific to the multimodal learning based machine learning model from training dataset comprises image data, the generating including: determining a plurality of permutations based on the one or more images included in the image data, and applying orthogonal super-positioning relative to the plurality of permutations. The method further comprises providing the inputs that are generated, based on the orthogonal super-positioning, into the multimodal learning based machine learning model, and generating, by the multimodal learning based machine learning model, a prediction specific to at least one image of the one or more images.

Abstract: Disclosed herein are various embodiments for training and enriching a natural language processing system. An embodiment operates by identifying a natural language processor (NLP) trained on a first set of documents, wherein the NLP is trained to perform a set of functionality based on the first set of documents. An industry, set of words corresponding to the industry, and set of sentences including at least a subset of the set of words in which the NLP is to be configured to perform the set of functionality are identified. A set of sentences that exceed a similarity threshold are identified. The NLP is trained with the subset of the set of sentences that exceed the similarity threshold, wherein the trained NLP with the subset is configured to perform the set of functionality within the industry with a greater accuracy than NLP trained on only the first set of documents.

Abstract: According to an aspect, there is provided a method that includes receiving a first image and a second image as inputs for contrastive self-supervised learning; applying a high dynamic range augmentation to the first image to generate a first pair of views; applying the high dynamic range augmentation to the second image to generate a second pair of views; applying a first convolutional neural network to the first pair of views to output a first pair of encoded representations; applying a second convolutional neural network to the second pair of views to output a second pair of encoded representations; projecting the first pair of encoded representations to form first projected representations; projecting the second pair of encoded representations to form second projected representations; and training a machine learning model using the high dynamic range augmentations and an objective function that provides contrastive self-supervised learning.

Abstract: A method for text-image integration is provided. The method may include receiving a question related to pairable data comprising text data and image data. Embeddings are generated from the text tokens and image encodings. Embeddings are generated from the text tokens and image encodings. The embeddings include text embeddings and image embeddings. A spectral conversion of the text embeddings and the image embeddings is performed to generate spectral data. The spectral data is processed to extract text-image features. The text-image features are processed to generate inferred answers to the question.

Abstract: Disclosed herein are system, method, and computer program product embodiments for utilizing non-RAM memory to implement machine learning configured with a meta-learning training set (small dataset), to create a common-sense predictive language model, thus boosting the performance for downstream tasks. An embodiment operates by receiving a base sentence and perturbation sentences as an input and tokenizing the input to generate a sequence of tokens. Tokens of the semantic perturbation sentences are embedded with tokens of the base sentence as contextually similar tokens pairs to generate training data and classified to capture relationships of the base sentence and the perturbation sentences to generate a classification, which is used to train a language model.

Abstract: Disclosed herein are various embodiments for pronoun-based natural language processing. An embodiment operates by receiving a plurality of text-based sentences each comprising a plurality of words, and each text-based sentence including a pronoun. A plurality of candidate nouns are identified amongst the plurality of words. A trigger word is identified from the plurality of words, wherein the trigger word is associated with both the pronoun and one of the plurality of candidate nouns. A score for each of the candidate nouns is received based on a relationship with the trigger word. The candidate noun with a highest score is selected as being associated with the pronoun.

Abstract: A machine learning model may be trained based on a training set including training images depicting various base objects. Each training images may be associated with a ground-truth segmentation corresponding to one or more pixel-wise labels. The machine learning model may be trained to learn base class prototypes corresponding to segmentations of classes of similar base objects. The machine learning model may be further trained based on a support image depicting a novel object. The support image may be associated with an image-level label corresponding to the novel object. The machine learning model may be trained to learn, based on a base class prototype identified as being similar to the support image, a novel class prototype corresponding to the novel object. The trained machine learning model to may be applied to segment a query image. Related systems and computer program products are also provided.

Abstract: In an example embodiment, a self-supervised learning task is used for training commonsense-aware representations in a minimally supervised fashion and a pair level mutual-exclusive loss is used to enforce commonsense knowledge during representation learning. This helps to exploit the mutual-exclusive nature of the training samples of commonsense reasoning corpora. Given two pieces of input where the only difference between them are trigger pieces of data, it may be postulated that the pairwise pronoun disambiguation is mutually exclusive. This idea is formulated using a contrastive loss and then this is used to update the language model.

Abstract: A machine learning model may be trained on a first task of puzzle solving before being tuned on a second task of image analysis. The training of the machine learning model may be self-supervised whereas the tuning of the machine learning model may be supervised. The training data may include a puzzle generated to include multiple imaging modalities. The puzzle may be generated by shuffling a position of the pieces forming an original image. The machine learning model may be trained to perform the first task by reassembling the pieces in the puzzle to generate a reconstruction of the original image. Upon being trained to perform the first task and tuned to perform the second task, the machine learning model may be deployed to perform the second task. The second task may be an image segmentation task such as tumor segmentation and a regression task such as survival prediction.

Abstract: Aspects of the current subject matter are directed to a variational encoder that takes into account group characteristics of data elements of a dataset. For example, a prior adjusted variational autoencoder takes into account that not all attributes in the dataset naturally follow a normal Gaussian distribution N(0,1). To illustrate by way of an example, data from the dataset may be separated into groups in which elements in a group share group characteristics; for each group, a group representation N(mu_g, sigma_g) is calculated. And, for example, other attributes of data in the dataset do not depend on the group, and the associated data elements continue to follow the normal Gaussian distribution N(0,1). The representation may introduce a flexibility in which encodings of group-related attributes will be encoded close together in the content part instead of being close to an arbitrarily chosen point.

Abstract: A method may include training a machine learning model to perform a first task before training the machine learning model to perform the second task. The machine learning model includes a generator network and a discriminator network. The training includes training, based on a first training sample associated with the first task, the discriminator network to perform the first task. The generator network may be trained to generate a first synthetic training sample emulating the first training sample. The discriminator network trained to perform the first task may be reinitialized in order for the discriminator network to be trained, based on a second training sample, to perform the second task. The reinitialized discriminator network may be further retrained, based on the first synthetic training sample, to perform the first task. Related systems and articles of manufacture, including computer program products, are also provided.

Abstract: Aspects of the current subject matter are directed to a system in which knowledge graphs are incorporated with visual question answering. A knowledge graph is integrated into a visual question answering system to provide additional knowledge from one or more sources to answer a question about an image. Aspects of the current subject matter are directed to a neural network approach that combines methods of image feature extraction and questions processing with a neural network, such as a graph neural network, that operates on knowledge graphs. The graph neural network takes input vector representations of the nodes as inputs and combines them according to their relationships into question-specific representations. The question-specific representations are then processed with the image features and the question features to generate an answer.

Abstract: Disclosed herein are various embodiments for pronoun-based natural language processing. An embodiment operates by receiving a plurality of text-based sentences each comprising a plurality of words, and each text-based sentence including a pronoun. A plurality of candidate nouns are identified amongst the plurality of words. A trigger word is identified from the plurality of words, wherein the trigger word is associated with both the pronoun and one of the plurality of candidate nouns. A score for each of the candidate nouns is received based on a relationship with the trigger word.

Abstract: Disclosed herein are various embodiments for training and enriching a natural language processing system. An embodiment operates by identifying a natural language processor (NLP) trained on a first set of documents, wherein the NLP is trained to perform a set of functionality based on the first set of documents. An industry, set of words corresponding to the industry, and set of sentences including at least a subset of the set of words in which the NLP is to be configured to perform the set of functionality are identified. A set of sentences that exceed a similarity threshold are identified. The NLP is trained with the subset of the set of sentences that exceed the similarity threshold, wherein the trained NLP with the subset is configured to perform the set of functionality within the industry with a greater accuracy than NLP trained on only the first set of documents.

Abstract: Disclosed herein are system, method, and computer program product embodiments for similarity scoring of sentences, while restricting distances between tokenized pairs in the sentences. An embodiment operates by determining a similarity of tokens between a first sequence of tokens and a second sequence of tokens to generate token pairs, determining a distance of relative positioning of token pairs in the first tokenized sequence and the second tokenized sequence and generating a score value that indicates the degree to which the first sentence matches the second sentence based on restricting matches to a maximum value of the distance of relative positions of the token pairs in the first tokenized sequence and the second tokenized sequence.