Abstract: Methods and systems for anomaly detection include encoding a multivariate time series and a multi-type event sequence using respective transformers and an aggregation network to generate a feature vector. Anomaly detection is performed using the feature vector to identify an anomaly within a system. A corrective action is performed responsive to the anomaly to correct or mitigate an effect of the anomaly. The detected anomaly can be used in a healthcare context to support decision making by medical professionals with respect to the treatment of a patient. The encoding may include machine learning models to implement the transformers and the aggregation network using deep learning.

Abstract: Methods and systems for training a model include distinguishing hidden states of a monitored system based on condition information. An encoder and decoder are generated for each respective hidden state using forward and backward autoencoder losses. A hybrid hidden state is determined for an input sequence based on the hidden states. The input sequence is reconstructed using the encoders and decoders and the hybrid hidden state. Parameters of the encoders and decoders are updated based on a reconstruction loss.

Abstract: A computer-implemented method for simulating vehicle data and improving driving scenario detection is provided.

Abstract: A computer-implemented method for employing a time-series-to-text generation model to generate accurate description texts is provided. The method includes passing time series data through a time series encoder and a multilayer perceptron (MLP) classifier to obtain predicted concept labels, converting the predicted concept labels, by a serializer, to a text token sequence by concatenating an aspect term and an option term of every aspect, inputting the text token sequence into a pretrained language model including a bidirectional encoder and an autoregressive decoder, and using adapter layers to fine-tune the pretrained language model to generate description texts.

Abstract: A method for learning a self-explainable imitator by discovering causal relationships between states and actions is presented. The method includes obtaining, via an acquisition component, demonstrations of a target task from experts for training a model to generate a learned policy, training the model, via a learning component, the learning component computing actions to be taken with respect to states, generating, via a dynamic causal discovery component, dynamic causal graphs for each environment state, encoding, via a causal encoding component, discovered causal relationships by updating state variable embeddings, and outputting, via an output component, the learned policy including trajectories similar to the demonstrations from the experts.

Abstract: A method for learning a self-explainable imitator by discovering causal relationships between states and actions is presented. The method includes obtaining, via an acquisition component, demonstrations of a target task from experts for training a model to generate a learned policy, training the model, via a learning component, the learning component computing actions to be taken with respect to states, generating, via a dynamic causal discovery component, dynamic causal graphs for each environment state, encoding, via a causal encoding component, discovered causal relationships by updating state variable embeddings, and outputting, via an output component, the learned policy including trajectories similar to the demonstrations from the experts.

Abstract: A method for learning a self-explainable imitator by discovering causal relationships between states and actions is presented. The method includes obtaining, via an acquisition component, demonstrations of a target task from experts for training a model to generate a learned policy, training the model, via a learning component, the learning component computing actions to be taken with respect to states, generating, via a dynamic causal discovery component, dynamic causal graphs for each environment state, encoding, via a causal encoding component, discovered causal relationships by updating state variable embeddings, and outputting, via an output component, the learned policy including trajectories similar to the demonstrations from the experts.

Abstract: Methods and systems for event detection include training a joint neural network model with respective neural networks for audio data and video data relating to a same scene. The joint neural network model is configured to output a belief value, a disbelief value, and an uncertainty value. It is determined that an event has occurred based on the belief value, the disbelief value, and the uncertainty value.

Abstract: A method for detecting pod and node candidates from cloud computing systems representing potential root causes of failure or fault activities is presented. The method includes collecting, by a monitoring agent, multi-modality data including key performance indicator (KPI) data, metrics data, and log data, employing a feature extractor and representation learner to convert the log data to time series data, applying a metric prioritizer based on extreme value theory to prioritize metrics for root cause analysis and learn an importance of different metrics, ranking root causes of failure or fault activities by using a hierarchical graph neural network, and generating one or more root cause reports outlining the potential root causes of failure or fault activities.

Abstract: A method for detecting an origin of a computer attack given a detection point based on multi-modality data is presented. The method includes monitoring a plurality of hosts in different enterprise system entities to audit log data and metrics data, generating causal dependency graphs to learn statistical causal relationships between the different enterprise system entities based on the log data and the metrics data, detecting a computer attack by pinpointing attack detection points, backtracking from the attack detection points by employing the causal dependency graphs to locate an origin of the computer attack, and analyzing computer attack data resulting from the backtracking to prevent present and future computer attacks.

Abstract: A method for employing root cause analysis is presented. The method includes embedding, by an embedding layer, a sequence of events into a low-dimension space, employing a feature extractor and representation learner to convert log data from the sequence of events to time series data, the feature extractor including an auto-encoder model and a language model, and detecting root causes of failure or fault activities from the time series data.

Abstract: A method for system metric prediction and influential events identification by concurrently employing metric logs and event logs is presented. The method includes concurrently modeling multivariate metric series and individual events in event series by a multi-stream recurrent neural network (RNN) to improve prediction of future metrics, where the multi-stream RNN includes a series of RNNs, one RNN for each metric and one RNN for each event sequence and modeling causality relations between the multivariate metric series and the individual events in the event series by employing an attention mechanism to identify target events most responsible for fluctuations of one or more target metrics.

Abstract: Methods and systems of training and using a neural network model include training a time series embedding model and a text embedding model with unsupervised clustering to translate time series and text, respectively, to a shared latent space. The time series embedding model and the text embedding model are further trained using semi-supervised clustering that samples training data pairs of time series information and associated text for annotation.

Abstract: Methods and systems for anomaly detection include determining whether a system is in a stable state or a dynamic state based on input data from one or more sensors in the system, using reconstruction errors from a respective stable model and dynamic model. It is determined that the input data represents anomalous operation of the system, responsive to a determination that the system is in a stable state, using the reconstruction errors. A corrective operation is performed on the system responsive to a determination that the input data represents anomalous operation of the system.

Abstract: A method for multivariate time series prediction is provided. Each time series from among a batch of multiple driving time series and a target time series is decomposed into a raw component, a shape component, and a trend component. For each decomposed component, select a driving time series relevant thereto from the batch and obtain hidden features of the selected driving time series, by applying the batch to an input attention-based encoder of an Ensemble of Clustered dual-stage attention-based Recurrent Neural Networks (EC-DARNNS). Automatically cluster the hidden features in a hidden space using a temporal attention-based decoder of the EC-DARNNS. Each Clustered dual-stage attention-based RNN in the Ensemble is dedicated and applied to a respective one of the decomposed components. Predict a respective value of one or more future time steps for the target series based on respective prediction outputs for each of the decomposed components by the EC-DARNNS.

Abstract: Systems and methods for defect detection for vehicle operations, including collecting a multiple modality input data stream from a plurality of different types of vehicle sensors, extracting one or more features from the input data stream using a grid-based feature extractor, and retrieving spatial attributes of objects positioned in any of a plurality of cells of the grid-based feature extractor. One or more anomalies are detected based on residual scores generated by each of cross attention-based anomaly detection and time-series-based anomaly detection. One or more defects are identified based on a generated overall defect score determined by integrating the residual scores for the cross attention-based anomaly detection and the time-series based anomaly detection being above a predetermined defect score threshold. Operation of the vehicle is controlled based on the one or more defects identified.

Abstract: Systems and methods for data fusion and analysis of vehicle sensor data, including receiving a multiple modality input data stream from a plurality of different types of vehicle sensors, determining latent features by extracting modality-specific features from the input data stream, and aligning a distribution of the latent features of different modalities by feature-level data fusion. Classification probabilities can be determined for the latent features using a fused modality scene classifier. A tree-organized neural network can be trained to determine path probabilities and issue driving pattern judgments, with the tree-organized neural network including a soft tree model and a hard decision leaf. One or more driving pattern judgments can be issued based on a probability of possible driving patterns derived from the modality-specific features.

Abstract: A computer-implemented method for multi-model representation learning is provided. The method includes encoding, by a trained time series (TS) encoder, an input TS segment into a TS-shared latent representation and a TS-private latent representation. The method further includes generating, by a trained text generator, a natural language text that explains the input TS segment, responsive to the TS-shared latent representation, the TS-private latent representation, and a text-private latent representation.

Abstract: Systems and methods for updating a classification model of a neural network. The methods include selecting, as a set of landmarks, a limited number of data from a set of historical data used to train a classification model. Additionally, the methods generate new training data from recently collected data. Further, the methods update the classification model with the new training data and the set of landmarks to obtain an updated classification model having a loss function configured to capture similarities in the new training data and remember similarities in the historical data represented by the set of landmarks within a predefined tolerance.