Abstract: Methods, systems, and computer program products may simplify Transformer machine learning models for sequential recommendation via a softmax-free gated attention mechanism and/or may use a gated unit to further sparsify attentions, which may simplify attention distributions and reduce negative impacts of noisy items.

Abstract: A computer-implemented method for debiasing vectorized language representations can include identifying two (or more) pairs of concepts for which debiasing is desired, computing a mean vector for each concept, determining a center point for a rotation operation to orthogonalize based on the mean vectors, and shifting the vectors to the center point before performing a rectification operation (which can be a graded rotation), after which the vectors can be shifted back from the center point. If desired, the process can be performed iteratively.

Abstract: Methods, systems, and computer program products are provided for encoding feature interactions based on tabular data. An exemplary method includes receiving a dataset in a tabular format including a plurality of rows and a plurality of columns. Each column is indexed to generate a position embedding matrix. Each column is grouped based on at least one tree model to generate a domain embedding matrix. An input vector is generated based on the dataset, the position embedding matrix, and the domain embedding matrix. The input vector is inputted into a first multilayer perceptron (MLP) model to generate a first output vector, which is transposed to generate a transposed vector. The transposed vector is inputted into a second MLP model to generate a second output vector, which is inputted into at least one classifier model to generate at least one prediction.

Abstract: Systems, methods, and computer program products are provided for saving memory during training of knowledge graph neural networks. The method includes receiving a training dataset including a first set of knowledge graph embeddings associated with a plurality of entities for a first layer of a knowledge graph, inputting the training dataset into a knowledge graph neural network to generate at least one further set of knowledge graph embeddings associated with the plurality of entities for at least one further layer of the knowledge graph, quantizing the at least one further set of knowledge graph embeddings to provide at least one set of quantized knowledge graph embeddings, storing the at least one set of quantized knowledge graph embeddings in a memory, and dequantizing the at least one set of quantized knowledge graph embeddings to provide at least one set of dequantized knowledge graph embeddings.

Abstract: Methods, systems, and computer program products may formulate an iterative data mix up problem into a Markov decision process (MDP) with a tailored reward signal to guide a learning process. To solve the MDP, a deep deterministic actor-critic framework may be modified to adapt a discrete-continuous decision space for training a data augmentation policy.

Abstract: Methods, systems, and computer program products may formulate an iterative data mix up problem into a Markov decision process (MDP) with a tailored reward signal to guide a learning process. To solve the MDP, a deep deterministic actor-critic framework may be modified to adapt a discrete-continuous decision space for training a data augmentation policy.

Abstract: Liquid alkylated N-phenyl-?-naphthylamine (PANA) compositions are disclosed containing a high concentration of mono-alkylated PANA and low levels of di-alkylated and less than 1% by weight of unsubstituted PANA. The novel compositions may be prepared by controlled alkylation of PANA with propylene oligomers followed by subsequent alkylation with at least one second olefin.

Abstract: Systems, methods, and computer program products for determining long-range dependencies using a non-local graph neural network (GNN): receive a dataset comprising historical data; generate at least one layer of a graph neural network by generating graph convolutions to compute node embeddings for a plurality of nodes of the dataset, the graph convolutions generated by aggregating node data from a first node of the dataset and node data from at least one second node comprising a neighbor node of the first node; cluster the node embeddings to form a plurality of centroids; determine an attention operator for at least one node-centroid pairing, the at least one node-centroid pairing comprising the first node and a first centroid; and generate relational data corresponding to a relation between the first node and at least one third node comprising a non-neighbor node of the first node using the attention operator.

Abstract: Described are a system, method, and computer program product for denoising sequential machine learning models. The method includes receiving data associated with a plurality of sequences and training a sequential machine learning model based on the data associated with the plurality of sequences to produce a trained sequential machine learning model. Training the sequential machine learning model includes denoising a plurality of sequential dependencies between items in the plurality of sequences using at least one trainable binary mask. The method also includes generating an output of the trained sequential machine learning model based on the denoised sequential dependencies. The method further includes generating a prediction of an item associated with a sequence of items based on the output of the trained sequential machine learning model.

Abstract: Methods, systems, and computer program products are provided for embedding learning to provide uniformity and orthogonality of embeddings. A method may include receiving a dataset that includes a plurality of data points including a first plurality of data points having a first classification and a second plurality of data points having a second classification, generating a first normalized class mean vector of the first plurality of data instances having the first classification, generating a second normalized class mean vector of the second plurality of data instances having the second classification, performing a class rectification operation on the first plurality of data instances having the first classification and the second plurality of data instances having a second classification, and generating embeddings of the dataset based on original embedding space projections of the dataset.

Abstract: A method performed by a server computer is disclosed. The method comprises generating a binary compositional code matrix from an input matrix. The binary compositional code matrix is then converted into an integer code matrix. Each row of the integer code matrix is input into a decoder, including plurality of codebooks, to output a summed vector for each row. The method then includes inputting a derivative of each summed vector into a downstream machine learning model to output a prediction.

Abstract: A computer obtains node embeddings, node periodicity classifications, edge embeddings, and edge periodicity classifications for each time of a time period. The computer determines subgraph embeddings based on a subgraph of the graph, times in the time period, the node embeddings for nodes in the subgraph, the edge embeddings for edges in the subgraph, the node periodicity classifications for the nodes in the subgraph, and the edge periodicity classifications for the edges in the subgraph. The computer translates each subgraph embedding of the subgraph embeddings for each time of the time period into projected subgraph embeddings. For the subgraph, the computer aggregates the plurality of projected subgraph embeddings into an aggregated subgraph embedding. The computer determines if the subgraph is periodic based upon at least the aggregated subgraph embedding.

Abstract: Methods, systems, and computer program products may formulate an iterative data mix up problem into a Markov decision process (MDP) with a tailored reward signal to guide a learning process. To solve the MDP, a deep deterministic actor-critic framework may be modified to adapt a discrete-continuous decision space for training a data augmentation policy.

Abstract: Methods, systems, and computer program products are provided for optimizing training loss of a graph neural network machine learning model using bi-level optimization. An example method includes receiving a training dataset comprising graph data associated with a graph, training a graph neural network (GNN) machine learning model using a loss equation according to a bi-level optimization problem and based on the training dataset, where training the GNN machine learning model using the loss equation according to the bi-level optimization problem includes determining a solution to an inner loss problem and a solution to an outer loss problem, and providing a trained GNN machine learning model based on training the GNN machine learning model.

Abstract: Liquid alkylated N-phenyl-a-naphthylamine (PANA) compositions are disclosed containing a high concentration of mono-alkylated PANA and low levels of di-alkylated and less than 1% by weight of unsubstituted PANA. The novel compositions may be prepared by controlled alkylation of PANA with propylene oligomers followed by subsequent alkylation with at least one second olefin.

Abstract: A method includes determining a set of regions for each of a first plurality of images of a first item type, a second plurality of images of a second item type, and a third plurality of images of a third item type. The method also includes for each region in each set of regions of the images, generating, by the processing computer, a vector representing the region, and then generating a plurality of aggregated messages using the vectors corresponding to combinations of images of different types of items, the images being from the first, second, and third plurality of images. Then, unified embeddings are generated for the images in the first, second, and third plurality of images, respectively, using aggregated messages in the plurality of aggregated messages. Matching scores associated with combinations of the images are created using the unified embeddings and a machine learning model.

Abstract: Provided is a system for detecting an anomaly in a multivariate time series that includes at least one processor programmed or configured to receive a dataset of a plurality of data instances, wherein each data instance comprises a time series of data points, determine a set of target data instances based on the dataset, determine a set of historical data instances based on the dataset, generate, based on the set of target data instances, a true value matrix, a true frequency matrix, and a true correlation matrix, generate a forecast value matrix, a forecast frequency matrix, and a forecast correlation matrix based on the set of target data instances and the set of historical data instances, determine an amount of forecasting error, and determine whether the amount of forecasting error corresponds to an anomalous event associated with the dataset of data instances. Methods and computer program products are also provided.

Abstract: A method includes receiving a first data set comprising embeddings of first and second types, generating a fixed adjacency matrix from the first dataset, and applying a first stochastic binary mask to the fixed adjacency matrix to obtain a first subgraph of the fixed adjacency matrix. The method also includes processing the first subgraph through a first layer of a graph convolutional network (GCN) to obtain a first embedding matrix, and applying a second stochastic binary mask to the fixed adjacency matrix to obtain a second subgraph of the fixed adjacency matrix. The method includes processing the first embedding matrix and the second subgraph through a second layer of the GCN to obtain a second embedding matrix, and then determining a plurality of gradients of a loss function, and modifying the first stochastic binary mask and the second stochastic binary mask using at least one of the plurality of gradients.

Abstract: Described are a method, system, and computer program product for generating robust graph neural networks using universal adversarial training. The method includes receiving a graph neural network (GNN) model and a bipartite graph including an adjacency matrix, initializing model parameters of the GNN model, initializing perturbation parameters, and sampling a subgraph of a complementary graph based on the bipartite graph. The method further includes repeating until convergence of the model parameters: drawing a random variable from a uniform distribution; generating a universal perturbation matrix based on the subgraph, the random variable, and the perturbation parameters; determining Bayesian Personalized Ranking (BPR) loss by inputting the bipartite graph and the universal perturbation matrix to the GNN model; updating the perturbation parameters based on stochastic gradient ascent; and updating the model parameters based on stochastic gradient descent.

Abstract: A method includes receiving a first data set comprising embeddings of first and second types, generating a fixed adjacency matrix from the first dataset, and applying a first stochastic binary mask to the fixed adjacency matrix to obtain a first subgraph of the fixed adjacency matrix. The method also includes processing the first subgraph through a first layer of a graph convolutional network (GCN) to obtain a first embedding matrix, and applying a second stochastic binary mask to the fixed adjacency matrix to obtain a second subgraph of the fixed adjacency matrix. The method includes processing the first embedding matrix and the second subgraph through a second layer of the GCN to obtain a second embedding matrix, and then determining a plurality of gradients of a loss function, and modifying the first stochastic binary mask and the second stochastic binary mask using at least one of the plurality of gradients.