Abstract: Systems and methods for training a time-series-language (TSLa) model adapted for domain-specific tasks. An encoder-decoder neural network can be trained to tokenize time-series data to obtain a discrete-to-language embedding space. The TSLa model can learn a linear mapping function by concatenating token embeddings from the discrete-to-language embedding space with positional encoding to obtain mixed-modality token sequences. Token augmentation can transform the tokens from the mixed-modality token sequences with to obtain augmented tokens. The augmented tokens can train the TSLa model using a computed token likelihood to predict next tokens for the mixed-modality token sequences to obtain a trained TSLa model. A domain-specific dataset can fine-tune the trained TSLa model to adapt the trained TSLa model to perform a domain-specific task.

Abstract: Systems and methods for data fusion and analysis of vehicle sensor data, including receiving a multiple modality input data stream from a plurality of different types of vehicle sensors, determining latent features by extracting modality-specific features from the input data stream, and aligning a distribution of the latent features of different modalities by feature-level data fusion. Classification probabilities can be determined for the latent features using a fused modality scene classifier. A tree-organized neural network can be trained to determine path probabilities and issue driving pattern judgments, with the tree-organized neural network including a soft tree model and a hard decision leaf. One or more driving pattern judgments can be issued based on a probability of possible driving patterns derived from the modality-specific features.

Abstract: Systems and methods for log representation learning for automated system maintenance. An optimized parser can transform collected system logs into log templates. A tokenizer can tokenize the log templates partitioned into time windows to obtain log template tokens. The log template tokens can train a language model (LM) with deep learning to obtain a trained LM. The trained LM can detect anomalies from system logs to obtain detected anomalies. A corrective action can be performed on a monitored entity based on the detected anomalies.

Abstract: A method classifies missing labels. The method computes, using a neural network model trained on training data, rank-based statistics of a feature of a time series segment to attempt to select two candidate labels from the training data that the segment most likely belongs to. The method classifies the segment using k-NN-based classification applied to the training data, responsive to the two candidate labels being present in the training data. The method classifies the segment by hypothesis testing, responsive to only one candidate label being present in the training data. The method classifies the segment into a class with higher values of the rank-based statistics from among a plurality of classes with different values of the rank-based statistics, responsive to no candidate labels being present in the training data. The method corrects a prediction by an applicable one of the classifying steps by majority voting with time windows.

Abstract: A computer-implemented method for multi-model representation learning is provided. The method includes encoding, by a trained time series (TS) encoder, an input TS segment into a TS-shared latent representation and a TS-private latent representation. The method further includes generating, by a trained text generator, a natural language text that explains the input TS segment, responsive to the TS-shared latent representation, the TS-private latent representation, and a text-private latent representation.

Abstract: Systems and methods are provided for incident analysis in Cyber-Physical Systems (CPS) using a Temporal Graph-based Incident Analysis System (TGIAS) and/or Transition Based Categorical Anomaly Detection (TCAD). Dynamically gathered multimodal data from a distributed network of sensors across the CPS are preprocessed to identify abnormal sensor readings indicative of potential incidents, and a multi-layered incident timeline graph, representing abnormal sensor readings, relationships to specific CPS components, and temporal sequencing of events is constructed. Severity scores are calculated, and severity rankings are assigned to identified anomalies based on a composite index including impact on CPS operation, comparison with historical incident data, and predictive risk assessments.

Abstract: Systems and methods are provided for incident analysis in Cyber-Physical Systems (CPS) using a Temporal Graph-based Incident Analysis System (TGIAS) and/or Transition Based Categorical Anomaly Detection (TCAD). Dynamically gathered multimodal data from a distributed network of sensors across the CPS are preprocessed to identify abnormal sensor readings indicative of potential incidents, and a multi-layered incident timeline graph, representing abnormal sensor readings, relationships to specific CPS components, and temporal sequencing of events is constructed. Severity scores are calculated, and severity rankings are assigned to identified anomalies based on a composite index including impact on CPS operation, comparison with historical incident data, and predictive risk assessments.

Abstract: Methods and systems for anomaly detection include encoding a time series with a time series encoder and encoding an event sequence with an event sequence encoder. A latent code is generated from outputs of the time series encoder and the event sequence encoder. The time series is reconstructed from the latent code using a time series decoder. The event sequence is reconstructed from the latent code using an event sequence decoder. An anomaly score is determined based on a reconstruction loss of the reconstructed time series and a reconstruction loss of the reconstructed event sequence. An action is performed responsive to the anomaly score.

Abstract: Methods and systems for anomaly detection include encoding a multivariate time series and a multi-type event sequence using respective transformers and an aggregation network to generate a feature vector. Anomaly detection is performed using the feature vector to identify an anomaly within a system. A corrective action is performed responsive to the anomaly to correct or mitigate an effect of the anomaly. The detected anomaly can be used in a healthcare context to support decision making by medical professionals with respect to the treatment of a patient. The encoding may include machine learning models to implement the transformers and the aggregation network using deep learning.

Abstract: Methods and systems for event prediction include encoding a multivariate time series and a multi-type event sequence using respective transformers and an aggregation network to generate a feature vector. Event prediction is performed using the feature vector to identify a next event to occur within a system. A corrective action is performed responsive to the next event to prevent or mitigate an effect of the next event. The predicted next event can be used in a healthcare context to support decision making by medical professionals with respect to the treatment of a patient. The encoding may include machine learning models to implement the transformers and the aggregation network using deep learning.

Abstract: Methods and systems for defect detection include determining a first residual score by comparing a first predicted system state, determined according to previously measured environment data, to an actual system state. A second residual score is determined by comparing a second predicted system state, determined according to previously measured system state data, to the actual system state. A defect score is generated based on a difference between the first residual score and the second residual score. An automatic action is performed responsive to a determination that the defect score indicates a defect in system behavior.

Abstract: Methods and systems for anomaly detection include encoding a multivariate time series and a multi-type event sequence using respective transformers and an aggregation network to generate a feature vector. Anomaly detection is performed using the feature vector to identify an anomaly within a system. A corrective action is performed responsive to the anomaly to correct or mitigate an effect of the anomaly. The detected anomaly can be used in a healthcare context to support decision making by medical professionals with respect to the treatment of a patient. The encoding may include machine learning models to implement the transformers and the aggregation network using deep learning.

Abstract: Methods and systems for training a model include distinguishing hidden states of a monitored system based on condition information. An encoder and decoder are generated for each respective hidden state using forward and backward autoencoder losses. A hybrid hidden state is determined for an input sequence based on the hidden states. The input sequence is reconstructed using the encoders and decoders and the hybrid hidden state. Parameters of the encoders and decoders are updated based on a reconstruction loss.

Abstract: A computer-implemented method for simulating vehicle data and improving driving scenario detection is provided.

Abstract: A computer-implemented method for employing a time-series-to-text generation model to generate accurate description texts is provided. The method includes passing time series data through a time series encoder and a multilayer perceptron (MLP) classifier to obtain predicted concept labels, converting the predicted concept labels, by a serializer, to a text token sequence by concatenating an aspect term and an option term of every aspect, inputting the text token sequence into a pretrained language model including a bidirectional encoder and an autoregressive decoder, and using adapter layers to fine-tune the pretrained language model to generate description texts.

Abstract: A method for learning a self-explainable imitator by discovering causal relationships between states and actions is presented. The method includes obtaining, via an acquisition component, demonstrations of a target task from experts for training a model to generate a learned policy, training the model, via a learning component, the learning component computing actions to be taken with respect to states, generating, via a dynamic causal discovery component, dynamic causal graphs for each environment state, encoding, via a causal encoding component, discovered causal relationships by updating state variable embeddings, and outputting, via an output component, the learned policy including trajectories similar to the demonstrations from the experts.

Abstract: A method for learning a self-explainable imitator by discovering causal relationships between states and actions is presented. The method includes obtaining, via an acquisition component, demonstrations of a target task from experts for training a model to generate a learned policy, training the model, via a learning component, the learning component computing actions to be taken with respect to states, generating, via a dynamic causal discovery component, dynamic causal graphs for each environment state, encoding, via a causal encoding component, discovered causal relationships by updating state variable embeddings, and outputting, via an output component, the learned policy including trajectories similar to the demonstrations from the experts.

Abstract: A method for learning a self-explainable imitator by discovering causal relationships between states and actions is presented. The method includes obtaining, via an acquisition component, demonstrations of a target task from experts for training a model to generate a learned policy, training the model, via a learning component, the learning component computing actions to be taken with respect to states, generating, via a dynamic causal discovery component, dynamic causal graphs for each environment state, encoding, via a causal encoding component, discovered causal relationships by updating state variable embeddings, and outputting, via an output component, the learned policy including trajectories similar to the demonstrations from the experts.

Abstract: Methods and systems for event detection include training a joint neural network model with respective neural networks for audio data and video data relating to a same scene. The joint neural network model is configured to output a belief value, a disbelief value, and an uncertainty value. It is determined that an event has occurred based on the belief value, the disbelief value, and the uncertainty value.

Abstract: A method for detecting pod and node candidates from cloud computing systems representing potential root causes of failure or fault activities is presented. The method includes collecting, by a monitoring agent, multi-modality data including key performance indicator (KPI) data, metrics data, and log data, employing a feature extractor and representation learner to convert the log data to time series data, applying a metric prioritizer based on extreme value theory to prioritize metrics for root cause analysis and learn an importance of different metrics, ranking root causes of failure or fault activities by using a hierarchical graph neural network, and generating one or more root cause reports outlining the potential root causes of failure or fault activities.