Abstract: A method for performing contrastive learning for graph tasks and datasets by employing an information-aware graph contrastive learning framework is presented. The method includes obtaining two semantically similar views of a graph coupled with a label for training by employing a view augmentation component, feeding the two semantically similar views into respective encoder networks to extract latent representations preserving both structure and attribute information in the two views, optimizing a contrastive loss based on a contrastive mode by maximizing feature consistency between the latent representations, training a neural network with the optimized contrastive loss, and predicting a new graph label or a new node label in the graph.

Abstract: A method for performing contrastive learning for graph tasks and datasets by employing an information-aware graph contrastive learning framework is presented. The method includes obtaining two semantically similar views of a graph coupled with a label for training by employing a view augmentation component, feeding the two semantically similar views into respective encoder networks to extract latent representations preserving both structure and attribute information in the two views, optimizing a contrastive loss based on a contrastive mode by maximizing feature consistency between the latent representations, training a neural network with the optimized contrastive loss, and predicting a new graph label or a new node label in the graph.

Abstract: A method for performing contrastive learning for graph tasks and datasets by employing an information-aware graph contrastive learning framework is presented. The method includes obtaining two semantically similar views of a graph coupled with a label for training by employing a view augmentation component, feeding the two semantically similar views into respective encoder networks to extract latent representations preserving both structure and attribute information in the two views, optimizing a contrastive loss based on a contrastive mode by maximizing feature consistency between the latent representations, training a neural network with the optimized contrastive loss, and predicting a new graph label or a new node label in the graph.

Abstract: A method for executing a multi-task deep learning model for learning trends in multivariate time series is presented. The method includes collecting multi-variate time series data from a plurality of sensors, jointly learning both local and global contextual features for predicting a trend of the multivariate time series by employing a tensorized long short-term memory (LSTM) with adaptive shared memory (TLASM) to learn historical dependency of historical trends, and employing a multi-task one-dimensional convolutional neural network (1dCNN) to extract salient features from local raw time series data to model a short-term dependency between local time series data and subsequent trends.

Abstract: A method for executing a multi-task deep learning model for learning trends in multivariate time series is presented. The method includes collecting multi-variate time series data from a plurality of sensors, jointly learning both local and global contextual features for predicting a trend of the multivariate time series by employing a tensorized long short-term memory (LSTM) with adaptive shared memory (TLASM) to learn historical dependency of historical trends, and employing a multi-task one-dimensional convolutional neural network (1dCNN) to extract salient features from local raw time series data to model a short-term dependency between local time series data and subsequent trends.

Abstract: A method for performing contrastive learning for graph tasks and datasets by employing an information-aware graph contrastive learning framework is presented. The method includes obtaining two semantically similar views of a graph coupled with a label for training by employing a view augmentation component, feeding the two semantically similar views into respective encoder networks to extract latent representations preserving both structure and attribute information in the two views, optimizing a contrastive loss based on a contrastive mode by maximizing feature consistency between the latent representations, training a neural network with the optimized contrastive loss, and predicting a new graph label or a new node label in the graph.

Abstract: Systems and methods for implementing a spatial and temporal attention-based gated recurrent unit (GRU) for node classification over temporal attributed graphs are provided. The method includes computing, using a GRU, embeddings of nodes at different snapshots. The method includes performing weighted sum pooling of neighborhood nodes for each node. The method further includes concatenating feature vectors for each node. Final temporal network embedding vectors are generated based on the feature vectors for each node. The method also includes applying a classification model based on the final temporal network embedding vectors to the plurality of nodes to determine temporal attributed graphs with classified nodes.

Abstract: Embodiments disclosed herein allowed neural networks to be pruned. The inputs and outputs generated by a reference neural network are used to prune the reference neural network. The pruned neural network may have a subset of the weights that are in the reference neural network.

Abstract: Embodiments disclose bank-balanced-sparse activation neural network models and methods to generate the bank-balanced-sparse activation neural network models. According to one embodiment, a neural network sparsification engine determines a first deep neural network (DNN) model having two or more hidden layers. The engine determines a bank size, a bank layout, and a target sparsity. The engine segments the activation feature maps into a plurality of banks based on the bank size and the bank layout. The engine generates a second DNN model by increasing a sparsity for each bank of activation feature map based on the target sparsity, wherein the second DNN model is used for inferencing.

Abstract: A method for executing a multi-task deep learning model for learning trends in multivariate time series is presented. The method includes collecting multi-variate time series data from a plurality of sensors, jointly learning both local and global contextual features for predicting a trend of the multivariate time series by employing a tensorized long short-term memory (LSTM) with adaptive shared memory (TLASM) to learn historical dependency of historical trends, and employing a multi-task one-dimensional convolutional neural network (1dCNN) to extract salient features from local raw time series data to model a short-term dependency between local time series data and subsequent trends.

Abstract: A method for unsupervised multivariate time series trend detection for group behavior analysis is presented. The method includes collecting multi-variate time series data from a plurality of sensors, learning piecewise linear trends jointly for all of the multi-variate time series data, dividing the multi-variate time series data into a plurality of time segments, counting a number of up/down trends in each of the plurality of time segments, for a training phase, employing a cumulative sum (CUSUM), and, for a testing phase, monitoring the CUSUM for trend changes.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for knowledge-preserving sparse pruning on neural networks are described. An exemplary method includes obtaining a pre-trained machine learning model trained based on a plurality of general-purpose training data; training a task-specific machine learning model by tuning the pre-trained machine learning model based on a plurality of task-specific training data corresponding to a task; constructing a student network based on the task-specific machine learning model; simultaneously performing (1) knowledge distillation from the trained task-specific machine learning model as a teacher network to the student network and (2) network pruning on the student network; and obtaining the trained student network for serving the task.

Abstract: Methods and systems for training a neural network model include training a modular neural network model, which has a shared encoder and one or more task-specific decoders, including training one or more policy networks that control connections between the shared encoder and the one or more task-specific decoders in accordance with multiple tasks. A multitask neural network model is trained for the multiple tasks, with an output of the modular neural network model and the multitask neural network model being combined to form a final output.

Abstract: A method for unsupervised multivariate time series trend detection for group behavior analysis is presented. The method includes collecting multi-variate time series data from a plurality of sensors, learning piecewise linear trends jointly for all of the multi-variate time series data, dividing the multi-variate time series data into a plurality of time segments, counting a number of up/down trends in each of the plurality of time segments, for a training phase, employing a cumulative sum (CUSUM), and, for a testing phase, monitoring the CUSUM for trend changes.

Abstract: A method for executing a multi-task deep learning model for learning trends in multivariate time series is presented. The method includes collecting multi-variate time series data from a plurality of sensors, jointly learning both local and global contextual features for predicting a trend of the multivariate time series by employing a tensorized long short-term memory (LSTM) with adaptive shared memory (TLASM) to learn historical dependency of historical trends, and employing a multi-task one-dimensional convolutional neural network (1dCNN) to extract salient features from local raw time series data to model a short-term dependency between local time series data and subsequent trends.

Abstract: Methods and systems for detecting anomalous behavior in a network include identifying topological state information in a dynamic network using a first neural network. Attribute state information in the dynamic network is identified, based on a partial labeling of nodes in the dynamic network, using a second neural network. The topological state information and the attribute state information are concatenated. Labels for unlabeled nodes in the dynamic network are predicted using a multi-factor attention, based on the concatenated state information. A security action is performed responsive to a determination that at least one node in the dynamic network is anomalous.

Abstract: Systems and methods for implementing a spatial and temporal attention-based gated recurrent unit (GRU) for node classification over temporal attributed graphs are provided. The method includes computing, using a GRU, embeddings of nodes at different snapshots. The method includes performing weighted sum pooling of neighborhood nodes for each node. The method further includes concatenating feature vectors for each node. Final temporal network embedding vectors are generated based on the feature vectors for each node. The method also includes applying a classification model based on the final temporal network embedding vectors to the plurality of nodes to determine temporal attributed graphs with classified nodes.

Abstract: Systems and methods are provided for detecting anomaly precursor events. The methods include organizing time series data into an input data structure that maintains an association between instances identified in the time series data and respective sensors. Additionally, the methods include calculating an instance attention value for each instance of at least one instance; calculating a sensor attention value for each sensor of the respective sensors; and identifying correlations between multiple sensors of the respective sensors based on the instance attention value and sensor attention value to identify a precursor event candidate based on a relationship between the instances and the respective sensors. Also, the method includes identifying an impending anomaly candidate from a database of historical anomalies based on the precursor event candidate.