Abstract: Described are a system, method, and computer program product for debiasing embedding vectors of machine learning models. The method includes receiving embedding vectors and generating two clusters thereof. The method includes determining a first mean vector of the first cluster and a second mean vector of the second cluster. The method includes determining a bias associated with each of a plurality of first candidate vectors and replacing the first mean vector with a first candidate vector based on the bias. The method includes determining a bias associated with each of a plurality of second candidate vectors and replacing the second mean vector with a second candidate vector based on the bias. The method includes repeatedly replacing the first and second mean vectors until an extremum of the bias score is reached, and debiasing the embedding vectors by linear projection using a direction defined by the first and second mean vectors.

Abstract: Systems, methods, and computer program products train a residual neural network including a first fully connected layer, a first recurrent neural network layer, and at least one skip connection for anomaly detection. The at least one skip connection directly connects at least one of (i) an output of the first fully connected layer to a first other layer downstream of the first recurrent neural network layer in the residual neural network and (ii) an output of the first recurrent neural network layer to a second other layer downstream of a second recurrent neural network layer in the residual neural network.

Abstract: Provided are methods for generating a multitask machine learning model based on time series data, that may include receiving input time series data associated with an input time series of data points, calculating a pairwise distance between the input time series and a plurality of time series templates, providing the pairwise distance as a first input to a building block of a residual neural network, where the residual neural network has a plurality of multi-dimensional convolutional layers; generating a first output of the first building block of the residual neural network based on the first input, generating a final output of the residual neural network based on the first output, and generating a first output of a multitask machine learning model using a first output layer and a second output of the multitask machine learning model using a second output layer. Systems and computer program products are also disclosed.

Abstract: Provided are methods for enhancing a distribution of graph feature embeddings in an embedding space to improve discrimination of graph features by a graph neural network (GNN) that may include receiving a dataset comprising graph data associated with a graph, calculating a distance between a first set of node embeddings and a second set of node embeddings, determining a measure of uniformity for the dataset, determining a plurality of groups of node embeddings, determining a measure of alignment for the plurality of groups of node embeddings, generating a set of graph features based on the measure of uniformity, the measure of alignment, and the distance, and training the GNN based on the set of graph features to provide a trained GNN. Systems and computer program products are also disclosed.

Abstract: Systems, methods, and computer program products are provided for saving memory during training of knowledge graph neural networks. The method includes receiving a training dataset including a first set of knowledge graph embeddings associated with a plurality of entities for a first layer of a knowledge graph, inputting the training dataset into a knowledge graph neural network to generate at least one further set of knowledge graph embeddings associated with the plurality of entities for at least one further layer of the knowledge graph, quantizing the at least one further set of knowledge graph embeddings to provide at least one set of quantized knowledge graph embeddings, storing the at least one set of quantized knowledge graph embeddings in a memory, and dequantizing the at least one set of quantized knowledge graph embeddings to provide at least one set of dequantized knowledge graph embeddings.

Abstract: Provided are systems for analyzing a relational database using embedding learning that may include at least one processor programmed or configured to generate one or more entity-relation matrices from a relational database and perform, for each entity-relation matrix of the one or more entity-relation matrices, an embedding learning process on an embedding associated with an entity. When performing the embedding learning process on the embedding associated with the entity, the at least one processor is programmed or configured to generate an updated embedding associated with the entity. Computer-implemented methods and computer program products are also provided.

Abstract: Systems, methods, and computer program products for multi-head posterior based pre-trained model evaluation are provided. The system includes at least one processor configured to: generate an embedding dataset based on a pre-trained model, the embedding dataset including a plurality of embeddings representing a plurality of entities; cluster each entity of the plurality of entities based on a feature dataset, resulting in a plurality of clusters; and generate a metric for the pre-trained model based on a posterior probability of each entity of the plurality of entities and the plurality of clusters.

Abstract: Systems, methods, and computer program products are provided for saving memory during training of knowledge graph neural networks. The method includes receiving a training dataset including a first set of knowledge graph embeddings associated with a plurality of entities for a first layer of a knowledge graph, inputting the training dataset into a knowledge graph neural network to generate at least one further set of knowledge graph embeddings associated with the plurality of entities for at least one further layer of the knowledge graph, quantizing the at least one further set of knowledge graph embeddings to provide at least one set of quantized knowledge graph embeddings, storing the at least one set of quantized knowledge graph embeddings in a memory, and dequantizing the at least one set of quantized knowledge graph embeddings to provide at least one set of dequantized knowledge graph embeddings.

Abstract: A method is disclosed. The method comprises determining a time series, a subsequence length. The length of the time series may then be determined, and an initial matrix profile may then be computed. The method may then form a processed matrix profile for a first subsequence of the subsequence length by applying the first subsequence to the initial matrix profile. A second subsequence may then be determined from the processed matrix profile. The method may then include comparing the second subsequence to other subsequences in a dictionary and adding it to the dictionary. The subsequences in the dictionary may be used to generate a plurality of subsequence matrix profiles. The method may then include forming an approximate matrix profile using the plurality of subsequence matrix profiles and then determining one or more anomalies in the time series or another time series using the approximate matrix profile.

Abstract: Provided is a method for normalizing embeddings for cross-embedding alignment. The method may include applying mean centering to the at least one embedding set, applying spectral normalization to the at least one embedding set, and/or applying length normalization to the at least one embedding set. Spectral normalization may include decomposing the at least one embedding set, determining an average singular value of the at least one embedding set, determining a respective substitute singular value for each respective singular value of a diagonal matrix, and/or replacing the at least one embedding set with a product of the at least one embedding set, a right singular vector, and an inverse of the substitute diagonal matrix. The mean centering, spectral normalization, and/or length normalization may be iteratively repeated for a configurable number of iterations. A system and computer program product are also disclosed.

Abstract: A system for programmable control of micro-objects, such as droplets or particles, can include a microfluidic chip, a membrane displacement trap (MDT) actuation system, a pump connected to the microfluid chip, a detection system, and a control system. The microfluidic chip can have a microfluidic network with a main channel and a plurality of MDTs fluidically coupled to the main channel. The MDT actuation system can selectively actuate the plurality of MDTs, and the pump can pump a fluid into the microfluidic network. The detection system can detect a position of the micro-objects within the microfluidic chip. The control system can control the MDT actuation system and the pump to provide an operation on at least one of the micro-objects based on data received from the detection system. The operation can include generating, capturing, splitting, releasing, and/or merging of the at least one of the micro-objects.

Abstract: Provided are methods that include receiving interaction data associated with a plurality of interactions, the interaction data including interaction records that include a plurality of fields including a static field and a dynamic field, generating a static interaction embedding representation based on static field data associated with the static field and a first transformer model, generating a plurality of dynamic interaction embedding representations based on dynamic field data associated with the dynamic field of a sequence of interaction records and a second transformer model, generating a first intermediate input and a plurality of second intermediate inputs, generating a static sequence embedding representation and dynamic sequence embedding representations based on a third transformer model, and generating at least one prediction based on inputting the static sequence embedding representation and the plurality of dynamic sequence embedding representations to a machine learning model.

Abstract: Provided are methods, systems, and computer program products for unsupervised alignment of embedding spaces. A method may include receiving a first embedding matrix and a second embedding matrix. The first embedding matrix may include a plurality of source points and the second embedding matrix may include a plurality of target points. An initial permutation matrix and an initial orthogonal matrix may be initialized. A permutation matrix may be determined based on the initial permutation matrix, the first embedding matrix, and the second embedding matrix. An orthogonal matrix may be determined based on the initial orthogonal matrix, the first embedding matrix, the permutation matrix, and the second embedding matrix. For each step of a target number of steps, the following may be repeated: updating the permutation matrix based on a quantized 2-Wasserstein distance, and updating the orthogonal matrix based on a gradient descent and a Procrustes problem.

Abstract: Methods, systems, and computer program products may simplify Transformer machine learning models for sequential recommendation via a softmax-free gated attention mechanism and/or may use a gated unit to further sparsify attentions, which may simplify attention distributions and reduce negative impacts of noisy items.

Abstract: A computer-implemented method for debiasing vectorized language representations can include identifying two (or more) pairs of concepts for which debiasing is desired, computing a mean vector for each concept, determining a center point for a rotation operation to orthogonalize based on the mean vectors, and shifting the vectors to the center point before performing a rectification operation (which can be a graded rotation), after which the vectors can be shifted back from the center point. If desired, the process can be performed iteratively.

Abstract: Provided is a method for normalizing embeddings for cross-embedding alignment. The method may include applying mean centering to the at least one embedding set, applying spectral normalization to the at least one embedding set, and/or applying length normalization to the at least one embedding set. Spectral normalization may include decomposing the at least one embedding set, determining an average singular value of the at least one embedding set, determining a respective substitute singular value for each respective singular value of a diagonal matrix, and/or replacing the at least one embedding set with a product of the at least one embedding set, a right singular vector, and an inverse of the substitute diagonal matrix. The mean centering, spectral normalization, and/or length normalization may be iteratively repeated for a configurable number of iterations. A system and computer program product are also disclosed.

Abstract: Methods, systems, and computer program products are provided for content-based time series retrieval. An example system includes at least one processor configured to: obtain, from at least one database, a plurality of known time series; for each known time series of the plurality of known time series: compute a pairwise distance matrix between that known time series and each learned template of a plurality of learned templates to generate a plurality of pairwise distance matrices; stack the plurality of pairwise distance matrices together to generate a tensor; and process, with the residual network, the tensor, wherein the residual network receives, as input, the tensor, and provides, as output, a feature vector for that known time series; and provide the feature vector for each known time series of the plurality of known time series.

Abstract: Described are a system, method, and computer program product for dynamic node classification in temporal-based machine learning classification models. The method includes receiving graph data of a discrete time dynamic graph including graph snapshots, and node classifications associated with all nodes in the discrete time dynamic graph. The method includes converting the discrete time dynamic graph to a time-augmented spatio-temporal graph and generating an adjacency matrix based on a temporal walk of the time-augmented spatio-temporal graph. The method includes generating an adaptive information transition matrix based on the adjacency matrix and determining feature vectors based on the nodes and the node attribute matrix of each graph snapshot.

Abstract: Methods, systems, and computer program products are provided for coordinated analysis of output scores and input features of machine learning models in different environments. An example method includes receiving a plurality of first data records and a plurality of second data records. A first plot is generated based on a first score generated by a machine learning model for each first data record and a second score generated by the machine learning model for each second data record. The first plot is displayed. A plurality of second plots associated with at least a subset of the plurality of features are generated. Each respective second plot is generated based on a respective first field associated with a respective feature from the first data records and a respective second field associated with the respective feature from the second data records. The second plots are displayed.