Abstract: Described are a system, method, and computer program product for debiasing embedding vectors of machine learning models. The method includes receiving embedding vectors and generating two clusters thereof. The method includes determining a first mean vector of the first cluster and a second mean vector of the second cluster. The method includes determining a bias associated with each of a plurality of first candidate vectors and replacing the first mean vector with a first candidate vector based on the bias. The method includes determining a bias associated with each of a plurality of second candidate vectors and replacing the second mean vector with a second candidate vector based on the bias. The method includes repeatedly replacing the first and second mean vectors until an extremum of the bias score is reached, and debiasing the embedding vectors by linear projection using a direction defined by the first and second mean vectors.

Abstract: Methods, systems, and computer program products are provided for analyzing and/or improving transformer models. A method may include receiving a trained transformer model. The trained transformer model may include at least one multi-head self-attention layer including a plurality of attention heads. At least one sample may be received. The sample(s) may be inputted to the trained transformer model to generate at least one layer output of the multi-head self-attention layer(s) and at least one model output of the trained transformer model. Each respective attention head may be pruned, and the sample(s) may be inputted to the trained transformer model with the respective attention head pruned to generate at least one respective pruned layer output and at least one respective pruned model output. At least one importance metric may be determined for each respective attention head based on at least two of the aforementioned outputs.

Abstract: Methods and systems for anonymizing time-series data are disclosed. An anonymizing computer can generate an anonymized sequence of time-series data that can share many useful properties, patterns, or characteristics with a private sequence of time-series data, without revealing sensitive or private information about the private sequence of time-series data. This may enable data researchers and scientists to study the anonymized sequence of time-series data in lieu of the private sequence of time-series data, thereby preserving the privacy of data subjects (e.g., people) corresponding to the private sequence of time-series data. The anonymized sequence of time-series data can be generated using an iterative optimization process that can involve updating the anonymized sequence of time-series data to minimize a loss value. The loss value can correspond to both the utility and privacy of the anonymized sequence of time-series data.

Abstract: Systems, methods, and computer program products are provided for machine learning model visualization and optimized deployment. An example system includes at least one processor configured to generate a first set of candidate code objects using a first machine learning model, determine, for each candidate code object of the first set of candidate code objects, at least one value of at least one performance parameter, and deploy a selected code object of the first set of candidate code objects in a production environment based at least partly on the selected code object having an optimized value of the at least one performance parameter.

Abstract: Provided is a system, method, and computer program product for adversarial training for collaborative filtering. The system includes at least one processor programmed or configured to train a collaborative filtering model through a plurality of iterations by inputting a training data set including adversarial perturbations, update weights of the collaborative filtering model after each iteration or subset of iterations of the plurality of iterations, determine a training loss between each consecutive iteration or subset of iterations of the plurality of iterations, resulting in a plurality of training losses, and determine a loss function for the collaborative filtering model by minimizing a difference between an initial training loss and the plurality of training losses for the plurality of iterations.

Abstract: Methods, systems, and computer program products are provided for content-based time series retrieval. An example system includes at least one processor configured to: obtain, from at least one database, a plurality of known time series; for each known time series of the plurality of known time series: compute a pairwise distance matrix between that known time series and each learned template of a plurality of learned templates to generate a plurality of pairwise distance matrices; stack the plurality of pairwise distance matrices together to generate a tensor; and process, with the residual network, the tensor, wherein the residual network receives, as input, the tensor, and provides, as output, a feature vector for that known time series; and provide the feature vector for each known time series of the plurality of known time series.

Abstract: A method performed by a computer is disclosed. The method comprises receiving interaction data between electronic devices of a plurality of entities. The interaction data is used to form an entity interaction vector containing a number of interactions between the electronic devices of a chosen entity and an entity time series containing a plurality of metrics per unit time of the interactions. An interaction encoder of the computer can generate an interaction hidden representation of the entity interaction vector using embeddings of the plurality of entities. A temporal encoder of the computer can generate a temporal hidden representation of the entity time series. The interaction hidden representation and the temporal hidden representation can be used to generate a predicted scale and a shape estimation of a target interaction metric. The computer can then generate an estimated interaction metric of a time period using the predicted scale and the shape estimation.

Abstract: Methods, systems, and computer program products for traffic flow forecasting: obtain a graph representing a traffic network; obtain historical time-series traffic data associated with historical traffic conditions at a number of historical time steps in the traffic network; process, with each spatial-temporal graph sandwich transformer of at least one spatial-temporal graph sandwich transformer, the graph and the historical time-series traffic data to generate a sandwich transformer output, wherein each spatial-temporal graph sandwich transformer includes a top temporal transformer, a spatial transformer that receives, as an input, an output of the top temporal transformer and the graph, and a bottom temporal transformer that receives, as an input, an output of the spatial transformer; and generate, based on the sandwich transformer output from each spatial-temporal graph sandwich transformer, predicted time-series traffic data associated with predicted traffic conditions at a number of next time steps in the

Abstract: Described are a system, method, and computer program product for debiasing embedding vectors of machine learning models. The method includes receiving embedding vectors and generating two clusters thereof. The method includes determining a first mean vector of the first cluster and a second mean vector of the second cluster. The method includes determining a bias associated with each of a plurality of first candidate vectors and replacing the first mean vector with a first candidate vector based on the bias. The method includes determining a bias associated with each of a plurality of second candidate vectors and replacing the second mean vector with a second candidate vector based on the bias. The method includes repeatedly replacing the first and second mean vectors until an extremum of the bias score is reached, and debiasing the embedding vectors by linear projection using a direction defined by the first and second mean vectors.

Abstract: Systems, methods, and computer program products train a residual neural network including a first fully connected layer, a first recurrent neural network layer, and at least one skip connection for anomaly detection. The at least one skip connection directly connects at least one of (i) an output of the first fully connected layer to a first other layer downstream of the first recurrent neural network layer in the residual neural network and (ii) an output of the first recurrent neural network layer to a second other layer downstream of a second recurrent neural network layer in the residual neural network.

Abstract: Provided are methods for generating a multitask machine learning model based on time series data, that may include receiving input time series data associated with an input time series of data points, calculating a pairwise distance between the input time series and a plurality of time series templates, providing the pairwise distance as a first input to a building block of a residual neural network, where the residual neural network has a plurality of multi-dimensional convolutional layers; generating a first output of the first building block of the residual neural network based on the first input, generating a final output of the residual neural network based on the first output, and generating a first output of a multitask machine learning model using a first output layer and a second output of the multitask machine learning model using a second output layer. Systems and computer program products are also disclosed.

Abstract: Provided are methods for enhancing a distribution of graph feature embeddings in an embedding space to improve discrimination of graph features by a graph neural network (GNN) that may include receiving a dataset comprising graph data associated with a graph, calculating a distance between a first set of node embeddings and a second set of node embeddings, determining a measure of uniformity for the dataset, determining a plurality of groups of node embeddings, determining a measure of alignment for the plurality of groups of node embeddings, generating a set of graph features based on the measure of uniformity, the measure of alignment, and the distance, and training the GNN based on the set of graph features to provide a trained GNN. Systems and computer program products are also disclosed.

Abstract: Systems, methods, and computer program products are provided for saving memory during training of knowledge graph neural networks. The method includes receiving a training dataset including a first set of knowledge graph embeddings associated with a plurality of entities for a first layer of a knowledge graph, inputting the training dataset into a knowledge graph neural network to generate at least one further set of knowledge graph embeddings associated with the plurality of entities for at least one further layer of the knowledge graph, quantizing the at least one further set of knowledge graph embeddings to provide at least one set of quantized knowledge graph embeddings, storing the at least one set of quantized knowledge graph embeddings in a memory, and dequantizing the at least one set of quantized knowledge graph embeddings to provide at least one set of dequantized knowledge graph embeddings.

Abstract: Provided are systems for analyzing a relational database using embedding learning that may include at least one processor programmed or configured to generate one or more entity-relation matrices from a relational database and perform, for each entity-relation matrix of the one or more entity-relation matrices, an embedding learning process on an embedding associated with an entity. When performing the embedding learning process on the embedding associated with the entity, the at least one processor is programmed or configured to generate an updated embedding associated with the entity. Computer-implemented methods and computer program products are also provided.

Abstract: Systems, methods, and computer program products for multi-head posterior based pre-trained model evaluation are provided. The system includes at least one processor configured to: generate an embedding dataset based on a pre-trained model, the embedding dataset including a plurality of embeddings representing a plurality of entities; cluster each entity of the plurality of entities based on a feature dataset, resulting in a plurality of clusters; and generate a metric for the pre-trained model based on a posterior probability of each entity of the plurality of entities and the plurality of clusters.

Abstract: Systems, methods, and computer program products are provided for saving memory during training of knowledge graph neural networks. The method includes receiving a training dataset including a first set of knowledge graph embeddings associated with a plurality of entities for a first layer of a knowledge graph, inputting the training dataset into a knowledge graph neural network to generate at least one further set of knowledge graph embeddings associated with the plurality of entities for at least one further layer of the knowledge graph, quantizing the at least one further set of knowledge graph embeddings to provide at least one set of quantized knowledge graph embeddings, storing the at least one set of quantized knowledge graph embeddings in a memory, and dequantizing the at least one set of quantized knowledge graph embeddings to provide at least one set of dequantized knowledge graph embeddings.

Abstract: A method is disclosed. The method comprises determining a time series, a subsequence length. The length of the time series may then be determined, and an initial matrix profile may then be computed. The method may then form a processed matrix profile for a first subsequence of the subsequence length by applying the first subsequence to the initial matrix profile. A second subsequence may then be determined from the processed matrix profile. The method may then include comparing the second subsequence to other subsequences in a dictionary and adding it to the dictionary. The subsequences in the dictionary may be used to generate a plurality of subsequence matrix profiles. The method may then include forming an approximate matrix profile using the plurality of subsequence matrix profiles and then determining one or more anomalies in the time series or another time series using the approximate matrix profile.

Abstract: A system for programmable control of micro-objects, such as droplets or particles, can include a microfluidic chip, a membrane displacement trap (MDT) actuation system, a pump connected to the microfluid chip, a detection system, and a control system. The microfluidic chip can have a microfluidic network with a main channel and a plurality of MDTs fluidically coupled to the main channel. The MDT actuation system can selectively actuate the plurality of MDTs, and the pump can pump a fluid into the microfluidic network. The detection system can detect a position of the micro-objects within the microfluidic chip. The control system can control the MDT actuation system and the pump to provide an operation on at least one of the micro-objects based on data received from the detection system. The operation can include generating, capturing, splitting, releasing, and/or merging of the at least one of the micro-objects.

Abstract: Provided is a method for normalizing embeddings for cross-embedding alignment. The method may include applying mean centering to the at least one embedding set, applying spectral normalization to the at least one embedding set, and/or applying length normalization to the at least one embedding set. Spectral normalization may include decomposing the at least one embedding set, determining an average singular value of the at least one embedding set, determining a respective substitute singular value for each respective singular value of a diagonal matrix, and/or replacing the at least one embedding set with a product of the at least one embedding set, a right singular vector, and an inverse of the substitute diagonal matrix. The mean centering, spectral normalization, and/or length normalization may be iteratively repeated for a configurable number of iterations. A system and computer program product are also disclosed.