Abstract: Methods and systems for detecting and correcting anomalous behavior include generating a joint binary embedding of each of a set of historical time series sequences. A joint binary embedding of a recent time series sequence is generated. A ranked list of the plurality of historical time series sequences is generated according to respective similarities of each historical time series sequence to the recent time series sequence based on the respective joint binary embeddings of each. Anomalous behavior of a system associated with the recent time series sequence is determined according to a label of a top-ranked historical time series sequence in the ranked list. A corrective action is performed to correct the anomalous behavior.

Abstract: A power generator system with anomaly detection and methods for detecting anomalies include a power generator that includes one or more physical components configured to provide electrical power. Sensors are configured to make measurements of a state of respective physical components, outputting respective time series of said measurements. A monitoring system includes a fitting module configured to determine a predictive model for each pair of a set of time series, an anomaly detection module configured to compare new values of each pair of time series to values predicted by the respective predictive model to determine if the respective predictive model is broken and to determine a number of broken predictive model, and an alert module configured to generate an anomaly alert if the number of broken predictive models exceeds a threshold.

Abstract: Systems and methods are disclosed for operating a machine, by receiving training data from one or more sensors; training a machine learning module with the training data by: partitioning a data matrix into smaller submatrices to process in parallel and optimized for each processing node; for each submatrix, performing a greedy search for rank-one solutions; using alternating direction method of multipliers (ADMM) to ensure consistency over different data blocks; and controlling one or more actuators using live data and the learned module during operation.

Abstract: A computer-implemented method for employing deep learning for time series representation and retrieval is presented. The method includes retrieving multivariate time series segments from a plurality of sensors, storing the multivariate time series segments in a multivariate time series database constructed by a sliding window over a raw time series of data, applying an input attention based recurrent neural network to extract real value features and corresponding hash codes, executing similarity measurements by an objective function, given a query, obtaining a relevant time series segment from the multivariate time series segments retrieved from the plurality of sensors, and generating an output including a visual representation of the relevant time series segment on a user interface.

Abstract: Systems and devices including an imaging sensor to capture video sequences in an environment having safety concerns therein. The systems and devices further including a processor to generate driving series based on observations from the video sequences, and generate predictions of future events based on the observations using a dual-stage attention-based recurrent neural network (DA-RNN). The DA-RNN includes an input attention mechanism to extract relevant driving series, an encoder to encode the extracted relevant driving series into hidden states, a temporal attention mechanism to extract relevant hidden states, and a decoder to decode the relevant hidden states. The processor further generates a signal for initiating an action to machines to mitigate harm to items.

Abstract: Systems and methods for time series prediction are described. The systems and methods include encoding driving series into encoded hidden states, the encoding including adaptively prioritizing driving series at each timestamp using input attention, the driving series including data sequences collected from sensors. The systems and methods further includes decoding the encoded hidden states to generate a predicting model, the decoding including adaptively prioritizing encoded hidden states using temporal attention. The systems and methods further include generating predictions of future events using the predicting model based on the data sequences. The systems and methods further include generating signals for initiating an action to devices based on the predictions.

Abstract: Systems and devices including an imaging sensor to capture video sequences in an environment having safety concerns therein. The systems and devices further including a processor to generate driving series based on observations from the video sequences, and generate predictions of future events based on the observations using a dual-stage attention-based recurrent neural network (DA-RNN). The DA-RNN includes an input attention mechanism to extract relevant driving series, an encoder to encode the extracted relevant driving series into hidden states, a temporal attention mechanism to extract relevant hidden states, and a decoder to decode the relevant hidden states. The processor further generates a signal for initiating an action to machines to mitigate harm to items.

Abstract: A power generator system with anomaly detection and methods for detecting anomalies include a power generator that includes one or more physical components configured to provide electrical power. Sensors are configured to make measurements of a state of respective physical components, outputting respective time series of said measurements. A monitoring system includes a fitting module configured to determine a predictive model for each pair of a set of time series, an anomaly detection module configured to compare new values of each pair of time series to values predicted by the respective predictive model to determine if the respective predictive model is broken and to determine a number of broken predictive model, and an alert module configured to generate an anomaly alert if the number of broken predictive models exceeds a threshold.

Abstract: Methods and systems for detecting anomalies include determining a predictive model for each pair of a set of time series, each time series being associated with a component of a system. New values of each pair of time series are compared to values predicted by the respective predictive model to determine if the respective predictive model is broken. A number of broken predictive models is determined. An anomaly alert is generated if the number of broken predictive models exceeds a threshold.

Abstract: A video camera is provided for video-based anomaly detection that includes at least one imaging sensor configured to capture video sequences in a workplace environment having a plurality of machines therein. The video camera further includes a processor. The processor is configured to generate one or more predictions of an impending anomaly affecting at least one item selected from the group consisting of (i) at least one of the plurality of machines and (ii) at least one operator of the at least one of the plurality of machines, using a Deep High-Order Convolutional Neural Network (DHOCNN)-based model applied to the video sequences. The DHOCNN-based model has a one-class SVM as a loss layer of the model. The processor is further configured to generate a signal for initiating an action to the at least one of the plurality of machines to mitigate expected harm to the at least one item.

Abstract: A computer-implemented method and system are provided for video-based anomaly detection. The method includes forming, by a processor, a Deep High-Order Convolutional Neural Network (DHOCNN)-based model having a one-class Support Vector Machine (SVM) as a loss layer of the DHOCNN-based model. An objective of the SVM is configured to perform the video-based anomaly detection. The method further includes generating, by the processor, one or more predictions of an impending anomaly based on the high-order deep learning based model applied to an input image. The method also includes initiating, by the processor, an action to a hardware device to mitigate expected harm to at least one item selected from the group consisting of the hardware device, another hardware device related to the hardware device, and a person related to the hardware device.

Abstract: A video camera is provided for video-based anomaly detection that includes at least one imaging sensor configured to capture video sequences in a workplace environment having a plurality of machines therein. The video camera further includes a processor. The processor is configured to generate one or more predictions of an impending anomaly affecting at least one item selected from the group consisting of (i) at least one of the plurality of machines and (ii) at least one operator of the at least one of the plurality of machines, using a Deep High-Order Convolutional Neural Network (DHOCNN)-based model applied to the video sequences. The DHOCNN-based model has a one-class SVM as a loss layer of the model. The processor is further configured to generate a signal for initiating an action to the at least one of the plurality of machines to mitigate expected harm to the at least one item.

Abstract: A computer-implemented method and a system are provided for, in turn, providing driver assistance for a vehicle. The method includes forming, by a processor, a deep High-Order Long Short-Term Memory (HOLSTM)-based model by applying, to a HOLSTM, high-order interactions captured between global pattern distribution probabilities and local feature representations of an input sensor signal vector at each of a plurality of time steps. The input sensor signal vector is formed from multiple time series. Each of the multiple time series corresponds to a different one of a plurality of driving related sensors. The method further includes generating, by the processor, one or more predictions of impending dangerous conditions related to driving the vehicle based on the deep HOLSTM-based model. The method also includes informing, by an operator-perceptable warning device, an operator of the vehicle of the one or more predictions of impending dangerous conditions.

Abstract: Systems and methods are disclosed for operating a machine, by receiving training data from one or more sensors; training a machine learning module with the training data by: partitioning a data matrix into smaller submatrices to process in parallel and optimized for each processing node; for each submatrix, performing a greedy search for rank-one solutions; using alternating direction method of multipliers (ADMM) to ensure consistency over different data blocks; and controlling one or more actuators using live data and the learned module during operation.