Abstract: Methods and systems for detecting anomalous events include detecting anomalous events in monitored system data. An event correlation graph is generated based on the monitored system data that characterizes the tendency of processes to access system targets. Kill chains are generated that connect malicious events over a span of time from the event correlation graph that characterize events in an attack path over time by sorting events according to a maliciousness value and determining at least one sub-graph within the event correlation graph with an above-threshold maliciousness rank. A security management action is performed based on the kill chains.

Abstract: Systems and methods for anomaly detection in complex physical systems, including extracting features representative of a temporal evolution of the complex physical system, and analyzing the extracted features by deriving vector trajectories using sliding window segmentation of time series, applying a linear test to determine whether the vector trajectories are linear, and performing subspace decomposition on the vector trajectory based on the linear test. A system evolution model is generated from an ensemble of models, and a fitness score is determined by analyzing different data properties of the system based on specific data dependency relationships. An alarm is generated if the fitness score exceeds a predetermined number of threshold violations for the different data properties.

Abstract: Methods are provided for both single modal and multimodal fault diagnosis. In a method, a fault fingerprint is constructed based on a fault event using an invariant model. A similarity matrix between the fault fingerprint and one or more historical representative fingerprints are derived using dynamic time warping and at least one convolution. A feature vector in a feature subspace for the fault fingerprint is generated. The feature vector includes at least one status of at least one system component during the fault event. A corrective action correlated to the fault fingerprint is determined. The corrective action is initiated on a hardware device to mitigate expected harm to at least one item selected from the group consisting of the hardware device, another hardware device related to the hardware device, and a person related to the hardware device.

Abstract: Methods and systems for reporting anomalous events include building a process graph that models states of process-level events in a network. A topology graph is built that models source and destination relationships between connection events in the network. A set of alerts is clustered based on the process graph and the topology graph. Clustered alerts that exceed a threshold level of trustworthiness are reported.

Abstract: Methods and systems for reporting anomalous events include intra-host clustering a set of alerts based on a process graph that models states of process-level events in a network. Hidden relationship clustering is performed on the intra-host clustered alerts based on hidden relationships between alerts in respective clusters. Inter-host clustering is performed on the hidden relationship clustered alerts based on a topology graph that models source and destination relationships between connection events in the network. Inter-host clustered alerts that exceed a threshold level of trustworthiness are reported.

Abstract: A computer-implemented method for real-time detecting of abnormal network connections is presented. The computer-implemented method includes collecting network connection events from at least one agent connected to a network, recording, via a topology graph, normal states of network connections among hosts in the network, and recording, via a port graph, relationships established between host and destination ports of all network connections.

Abstract: Methods and systems for detecting anomalous network activity include determining whether a network event exists within an existing topology graph and port graph. A connection probability for the network event is determined if the network does not exist within the existing topology graph and port graph. The network event is identified as abnormal if the connection probability is below a threshold.

Abstract: Systems and methods are disclosed for detecting periodic event behaviors from machine generated logging by: capturing heterogeneous log messages, each log message including a time stamp and text content with one or more fields; recognizing log formats from log messages; transforming the text content into a set of time series data, one time series for each log format; during a training phase, analyzing the set of time series data and building a category model for each periodic event type in heterogeneous logs; and during live operation, applying the category model to a stream of time series data from live heterogeneous log messages and generating a flag on a time series data point violating the category model and generating an alarm report for the corresponding log message.

Abstract: Systems and methods are provided for optimizing system output in production systems, comprising. The method includes separating, by a processor, one or more initial input variables into a plurality of output variables, the output variables including environmental variables and system response variables. The method also includes building, using the processor, a nonparametric estimation that determines a relationship between one or more initial control variables and the system response variables, and estimating a global input-output mapping function, using the determined relationship, and a range of the environmental variables. The method further includes generating one or more optimal control variables from the initial control variables by maximizing the input-output mapping function and the range of the environmental variables. The method additionally includes incorporating one or more of the optimal control variables into a production system to increase production output of the production system.

Abstract: This invention provides a new mechanism for “Hot-Tracing” using a novel placeholder mechanism and binary rewriting techniques, which leverages existing compiler flags in order to enable light-weight and highly flexible dynamic instrumentation. Broadly, I-Probe can be divided in 2 distinct workflows—1. Pre-processing (ColdPatch), and 2. Hot Tracing. The first phase is a pre-processing mechanism to prepare the binary for phase 2. The second phase is the actual hot-tracing mechanism, which allows users to dynamically instrument functions (more specifically symbols) of their choice.

Abstract: A method and system are provided. The method includes performing, by a logs-to-time-series converter, a logs-to-time-series conversion by transforming a plurality of heterogeneous logs into a set of time series. Each of the heterogeneous logs includes a time stamp and text portion with one or more fields. The method further includes performing, by a time-series-to-sequential-pattern converter, a time-series-to-sequential-pattern conversion by mining invariant relationships between the set of time series, and discovering sequential message patterns and association rules in the plurality of heterogeneous logs using the invariant relationships. The method also includes executing, by a processor, a set of log management applications, based on the sequential message patterns and the association rules.

Abstract: A system and method for analysis of complex systems which includes determining model parameters based on time series data, further including profiling a plurality of types of data properties to discover complex data properties and dependencies; classifying the data dependencies into predetermined categories for analysis; and generating a plurality of models based on the discovered properties and dependencies. The system and method may analyze, using a processor, the generated models based on a fitness score determined for each model to generate a status report for each model; integrate the status reports for each model to determine an anomaly score for the generated models; and generate an alarm when the anomaly score exceeds a predefined threshold.

Abstract: A method and system determines capacity needs of components in a distributed computer system. In the method and system, a pair-wise invariant network is determined from collected flow intensity measurements. The network includes at least two separate and unconnected pair-wise invariant subnetworks, each of the subnetworks including two of the flow intensity measurements connected by a pairwise invariant, each of the pair-wise invariants characterizing a constant relationship between their two connected flow intensity measurements. At least one overlay invariant is determined from the pair-wise invariant network and from the collected flow intensity measurements using a minimal redundancy least regression process. The capacity needs of the components are determined using the pair-wise and overlay invariants.

Abstract: A system for automatically instrumenting and tracing an application program and related software components achieves a correlated tracing of the program execution. It includes tracing of endpoints that are the set of functions in the program execution path that the developers are interested. The tracing endpoints and related events become the total set of functions to be traced in the program (called instrument points). This invention automatically analyzes the program and generates such instrumentation points to enable correlated tracing. The generated set of instrumentation points addresses common questions that developers ask when they use monitoring tools.

Abstract: A method includes generating a normal trace in a training stage for the monitored software systems and a monitored trace in the deployment stage for anomaly detection, applying resource transfer functions to traces to convert them to resource features, and system call categorization to traces to convert them to program behavior features, performing anomaly detection in a global scope using the derived resource features and program behavior features, in case the system finds no anomaly, generating no anomaly report, in case the anomaly is found, including the result in an anomaly report; and performing conditional anomaly detection.

Abstract: A computer implemented method for temporal ranking in invariant networks includes considering an invariant network and a set of broken invariants in the invariant network, assuming, for each time point inside a window W, that each metric with broken invariants is affected by a fault at that time point, computing an expected pattern for each invariant of a metric with assumed fault, said pattern indicative of time points at which an invariant will be broken given that its associated metric was affected by a fault at time t, comparing the expected pattern with the pattern observed over the time window W; and determining a temporal score based on a match from the prior comparing.

Abstract: A computer implemented method provides efficient monitoring and analysis of a program's memory objects in the operation stage. The invention can visualize and analyze a monitored program's data status with improved semantic information without requiring source code at runtime. The invention can provide higher quality of system management, performance debugging, and root-cause error analysis of enterprise software in the production stage.

Abstract: A method and system are provided for heterogeneous log analysis. The method includes performing hierarchical log clustering on heterogeneous logs to generate a log cluster hierarchy for the heterogeneous logs. The method further includes performing, by a log pattern recognizer device having a processor, log pattern recognition on the log cluster hierarchy to generate log pattern representations. The method also includes performing log field analysis on the log pattern representations to generate log field statistics. The method additionally includes performing log indexing on the log pattern representations to generate log indexes.

Abstract: The invention efficiently provides user code information for kernel level tracing approaches. It applies an advanced variation of stack walking called multi-mode stack walking to the entire system level and generates the unified trace where the user code and kernel events are integrated. The invention uses runtime stack information and internal kernel data structures. Therefore, source code for user level code and libraries are not required for inspection. The invention introduces the mechanism to narrow down the monitoring focus to specific application software and improve monitoring performance.

Abstract: A method for request profiling in service systems with kernel events includes collecting kernel events traces from a target system, the kernel event traces being obtainable from individual service machines by instrumenting core kernel functions, analyzing kernel event traces for constructing end-to-end request profiling traces consisting of kernel events belonging to service processes, and categorizing request traces responsive to the analyzing with the constructing including grouping requests based on marking kernel events used in the analyzing.