Abstract: Systems and methods are provided for optimizing system output in production systems, comprising. The method includes separating, by a processor, one or more initial input variables into a plurality of output variables, the output variables including environmental variables and system response variables. The method also includes building, using the processor, a nonparametric estimation that determines a relationship between one or more initial control variables and the system response variables, and estimating a global input-output mapping function, using the determined relationship, and a range of the environmental variables. The method further includes generating one or more optimal control variables from the initial control variables by maximizing the input-output mapping function and the range of the environmental variables. The method additionally includes incorporating one or more of the optimal control variables into a production system to increase production output of the production system.

Abstract: The invention is directed to a computer implemented method and a system that implements an application performance profiler with hardware performance event information. The profiler provides dynamic tracing of application programs, and offers fine-grained hardware performance event profiling at function levels. To control the perturbation on target applications, the profiler also includes a control mechanism to constraint the function profiling overhead within a budget configured by users.

Abstract: A method and system are provided for heterogeneous log analysis. The method includes performing hierarchical log clustering on heterogeneous logs to generate a log cluster hierarchy for the heterogeneous logs. The method further includes performing, by a log pattern recognizer device having a processor, log pattern recognition on the log cluster hierarchy to generate log pattern representations. The method also includes performing log field analysis on the log pattern representations to generate log field statistics. The method additionally includes performing log indexing on the log pattern representations to generate log indexes.

Abstract: Methods and systems for network management include performing path regression to determine an end-to-end path across physical links for each data flow in a network. A per-flow utilization of each physical link in the network is estimated based on the determined end-to-end paths. A management action is performed in the network based on the estimated per-flow utilization.

Abstract: Methods and systems for profiling requests include generating request units based on collected kernel events that include complete request units and half-open request units. The generated request units are sequenced based on a causality relationship set that describes causality relationships between kernel events.

Abstract: A computer-implemented method for automatically analyzing log contents received via a network and detecting content-level anomalies is presented. The computer-implemented method includes building a statistical model based on contents of a set of training logs and detecting, based on the set of training logs, content-level anomalies for a set of testing logs. The method further includes maintaining an index and metadata, generating attributes for fields, editing model capability to incorporate user domain knowledge, detecting anomalies using field attributes, and improving anomaly quality by using user feedback.

Abstract: A computer-implemented method, computer program product, and computer processing system are provided. The method includes preprocessing, by a processor, a set of heterogeneous logs by splitting each of the logs into tokens to obtain preprocessed logs. Each of the logs in the set is associated with a timestamp and textual content in one or more fields. The method further includes generating, by the processor, a set of regular expressions from the preprocessed logs. The method also includes performing, by the processor, an unsupervised parsing operation by applying the regular expressions to the preprocessed logs to obtain a set of parsed logs and a set of unparsed logs, if any. The method additionally includes storing, by the processor, the set of parsed logs in a log analytics database and the set of unparsed logs in a debugging database.

Abstract: A method is provided that includes transforming training data into a neural network based learning model using a set of temporal graphs derived from the training data. The method includes performing model learning on the learning model by automatically adjusting learning model parameters based on the set of the temporal graphs to minimize differences between a predetermined ground-truth ranking list and a learning model output ranking list. The method includes transforming testing data into a neural network based inference model using another set of temporal graphs derived from the testing data. The method includes performing model inference by applying the inference and learning models to test data to extract context features for alerts in the test data and calculate a ranking list for the alerts based on the extracted context features. Top-ranked alerts are identified as critical alerts. Each alert represents an anomaly in the test data.

Abstract: Systems and methods are disclosed for processing a stream of logged data by: creating one or more models from a set of training logs during a training phase; receiving testing data in real-time and generating anomalies using the models created during the training phase; updating the one or more models during real-time processing of a live stream of logs; and detecting a log anomaly from the live stream of logs.

Abstract: Methods and systems for detecting a system fault include determining a network of broken correlations for a current timestamp, relative to a predicted set of correlations, based on a current set of sensor data. The network of broken correlations for the current timestamp is compared to networks of broken correlations for previous timestamps to determine a fault propagation pattern. It is determined whether a fault has occurred based on the fault propagation pattern. A system management action is performed if a fault has occurred.

Abstract: Systems and methods are disclosed for handling log data from one or more applications, sensors or instruments by receiving heterogeneous logs from arbitrary/unknown systems or applications; generating regular expression patterns from the heterogeneous log sources using machine learning and extracting a log pattern therefrom; generating models and profiles from training logs based on different conditions and updating a global model database storing all models generated over time; tokenizing raw log messages from one or more applications, sensors or instruments running a production system; transforming incoming tokenized streams are into data-objects for anomaly detection and forwarding of log messages to various anomaly detectors; and generating an anomaly alert from the one or more applications, sensors or instruments running a production system.

Abstract: A computer-implemented method for simultaneous metric learning and variable selection in non-linear regression is presented. The computer-implemented method includes introducing a dataset and a target variable, creating a univariate neighborhood probability map for each reference point of the dataset, and determining a pairwise distance between each reference point and other points within the dataset. The computer-implemented method further includes computing a Hessian matrix of a quadratic programming (QP) problem, performing optimization of the QP problem, re-weighing data derived from the optimization of the QP problem, and performing non-linear regression on the re-weighed data.

Abstract: A computer-implemented method for diagnosing system faults by fine-grained causal anomaly inference is presented. The computer-implemented method includes identifying functional modules impacted by causal anomalies and backtracking causal anomalies in impaired functional modules by a low-rank network diffusion model. An invariant network and a broken network are inputted into the system, the invariant network and the broken network being jointly clustered to learn a degree of broken severities of different clusters as a result of fault propagations.

Abstract: A heterogeneous log pattern editing recommendation system and computer-implemented method are provided. The system has a processor configured to identify, from heterogeneous logs, patterns including variable fields and constant fields. The processor is also configured to extract a category feature, a cardinality feature, and a before-after n-gram feature by tokenizing the variable fields in the identified patterns. The processor is additionally configured to generate target similarity scores between target fields to be potentially edited and other fields from among the variable fields in the heterogeneous logs using pattern editing operations based on the extracted category feature, the extracted cardinality feature, and the extracted before-after n-gram feature. The processor is further configured to recommend, to a user, log pattern edits for at least one of the target fields based on the target similarity scores between the target fields in the heterogeneous logs.

Abstract: Systems and devices including an imaging sensor to capture video sequences in an environment having safety concerns therein. The systems and devices further including a processor to generate driving series based on observations from the video sequences, and generate predictions of future events based on the observations using a dual-stage attention-based recurrent neural network (DA-RNN). The DA-RNN includes an input attention mechanism to extract relevant driving series, an encoder to encode the extracted relevant driving series into hidden states, a temporal attention mechanism to extract relevant hidden states, and a decoder to decode the relevant hidden states. The processor further generates a signal for initiating an action to machines to mitigate harm to items.

Abstract: Systems and methods for time series prediction are described. The systems and methods include encoding driving series into encoded hidden states, the encoding including adaptively prioritizing driving series at each timestamp using input attention, the driving series including data sequences collected from sensors. The systems and methods further includes decoding the encoded hidden states to generate a predicting model, the decoding including adaptively prioritizing encoded hidden states using temporal attention. The systems and methods further include generating predictions of future events using the predicting model based on the data sequences. The systems and methods further include generating signals for initiating an action to devices based on the predictions.

Abstract: Automated security systems and methods include a set monitored systems, each having one or more corresponding monitors configured to record system state information. A progressive software behavioral query language (PROBEQL) database is configured to store the system state information from the monitored systems. A query optimizing module is configured to optimize a database query for parallel execution using spatial and temporal information relating to elements in the PROBEQL database. The optimized database query is split into sub-queries with sub-queries being divided spatially according to host and temporally according to time window. A parallel execution module is configured to execute the sub-queries on the PROBEQL database in parallel. A results module is configured to output progressive results of the database query. A security control system is configured to perform a security control action in accordance with the progressive results.

Abstract: Methods and systems for log management include pre-processing heterogeneous logs and performing a log management action on the pre-processed plurality of heterogeneous logs. Pre-processing the logs includes performing a fixed tokenization of the heterogeneous logs based on a predefined set of symbols, performing a flexible tokenization of the heterogeneous logs based on a user-defined set of rules, converting timestamps in the heterogeneous logs to a single target timestamp format, and performing structural log tokenization of the heterogeneous logs based on user-defined structural information.

Abstract: Methods for querying a database and database systems include optimizing a database query for parallel execution using spatial and temporal information relating to elements in the database, the optimized database query being split into sub-queries with sub-queries being divided spatially according to host and temporally according to time window. The sub-queries are executed in parallel. The results of the database query are outputted progressively.

Abstract: Systems and methods for detection and prevention of Return-Oriented-Programming (ROP) attacks in one or more applications, including an attack detection device and a stack inspection device for performing stack inspection to detect ROP gadgets in a stack. The stack inspection includes stack walking from a stack frame at a top of the stack toward a bottom of the stack to detect one or more failure conditions, determining whether a valid stack frame and return code address is present; and determining a failure condition type if no valid stack frame and return code is present, with Type III failure conditions indicating an ROP attack. The ROP attack is contained using a containment device, and the ROP gadgets detected in the stack during the ROP attack are analyzed using an attack analysis device.