Abstract: The current document is directed to methods and systems that process, classify, efficiently store, and display large volumes of event messages generated in modern computing systems. In a disclosed implementation, received event messages are assigned to event-message clusters based on non-parameter tokens identified within the event messages. A parsing function is generated for each cluster that is used to extract data from incoming event messages and to prepare event records from event messages that more efficiently and accessible store event information. The parsing functions also provide an alternative basis for assignment of event messages to clusters. Event types associated with the clusters are used for gathering information from various information sources with which to automatically annotate event messages displayed to system administrators, maintenance personnel, and other users of event messages.

Abstract: The current document is directed to methods and systems for detecting the occurrences of abnormal events and operational behaviors within the distributed computer system. The currently described methods and systems continuously collect metric data from various metric-data sources, generate a sequence of metric-data observations, each metric-data observation comprising a set of temporally aligned metric data, and employ principle-component analysis to transform the metric-data observations to facilitate reduction of the dimensionality of the metric-data observations.

Abstract: Automated methods and systems to determine a baseline event-type distribution of an event source and use the baseline event type distribution to detect changes in the behavior of the event source are described. In one implementation, blocks of event messages generated by the event source are collected and an event-type distribution is computed for each of block of event messages. Candidate baseline event-type distributions are determined from the event-type distributions. The candidate baseline event-type distribution has the largest entropy of the event-type distributions. A normal discrepancy radius of the event-type distributions is computed from the baseline event-type distribution and the event-type distributions. A block of run-time event messages generated by the event source is collected. A run-time event-type distribution is computed from the block of run-time event messages.

Abstract: Methods and systems adjust resources and monitoring configuration of a monitoring tool and auxiliary servers of a distributed computing system. Any new virtual object installed in the distributed computing system or change in functionality of an existing virtual object of the distributed computing system is identified. Expected configuration rules and resource allocation rules are determined for the virtual object. The expected configured rules are used to adjust a monitoring configuration of the monitoring tool. The resource allocation rules are used to adjust the infrastructure resources available to the monitoring tool. When the virtual object creates a change in demand for auxiliary services, corresponding auxiliary servers are installed, removed, or changed in the distributed computing system. Resource allocated rules and expected configuration rules are determined for the auxiliary servers and used to adjust the infrastructure resources and monitoring configuration of the monitoring tool.

Abstract: The current document is directed to methods and systems that process, classify, efficiently store, and display large volumes of event messages generated in modern computing systems. In a disclosed implementation, received event messages are assigned to event-message clusters based on non-parameter tokens identified within the event messages. A parsing function is generated for each cluster that is used to extract data from incoming event messages and to prepare event records from event messages that more efficiently and accessible store event information. The parsing functions also provide an alternative basis for assignment of event messages to clusters. Event types associated with the clusters are used for gathering information from various information sources with which to automatically annotate event messages displayed to system administrators, maintenance personnel, and other users of event messages.

Abstract: Methods and system described herein are directed to identifying anomalous behaving components of a distributed computing system. Methods and system collect log messages generated by a set of event log source running in the distributed computing system within an observation time window. Frequencies of various types of event messages generated within the observation time window are determined for each of the log sources. A similarity value is calculated for each pair of event sources. The similarity values are used to identify similar clusters of event sources of the distributed computing system for various management purposes. Components of the distributed computing system that are used to host the event source outliers may be identified as potentially having problems or may be an indication of future problems.

Abstract: The current document is directed to automated methods and systems that employ unsupervised-machine-learning approaches as well as rule-based systems to discover distributed applications within distributed-computing environments. These automated methods and systems provide a basis for higher-level distributed-application administration and management tools and subsystems that provide distributed-application-level user interfaces and operations. In one implementation, the currently disclosed methods and systems employ agents within virtual machines that execute routines and programs and that together comprise a distributed application to continuously furnish information about the virtual machines to a pipeline of stream processors that collect and filter the information to provide for periodic application-discovery.

Abstract: Automated methods and systems to determine a baseline event-type distribution of an event source and use the baseline event type distribution to detect changes in the behavior of the event source are described. In one implementation, blocks of event messages generated by the event source are collected and an event-type distribution is computed for each of block of event messages. Candidate baseline event-type distributions are determined from the event-type distributions. The candidate baseline event-type distribution has the largest entropy of the event-type distributions. A normal discrepancy radius of the event-type distributions is computed from the baseline event-type distribution and the event-type distributions. A block of run-time event messages generated by the event source is collected. A run-time event-type distribution is computed from the block of run-time event messages.

Abstract: Methods and systems are directed to detecting and classifying changes in a distributed computing system. Divergence value are computed from distributions of different types of event messages generated in time intervals of a sliding time window. Each divergence value is a measure of change in types of events generated in each time interval. When a divergence value, or a rate of change in divergence values, exceeds a threshold, the time interval associated with the threshold violation is used to determine a change point in the operation of the distributed computing system. Based on the change point, a start time of the change is determined. The change is classified based on various previously classified change points in the disturbed computing system. A recommendation may be generated to address the change based on the classification of the change.

Abstract: This disclosure is directed to tagging tokens or sequences of tokens in log messages generated by a logging source. Event types of log messages in a block of log messages are collected. A series of tagging operations are applied to each log message in the block. For each tagging operation, event types that are qualified to receive the corresponding tag are identified. When a log message is received, the event type is determined and compared with the event types of the block in order to identify a matching event type. The series of tagging operations are applied to the log message to generate a tagged log message with the restriction that each tagging operation only applies a tag to token or sequences of tokens when the event type is qualified to receive the tag. The tagged log message is stored in a data-storage device.

Abstract: Automated methods and systems to determine a baseline event-type distribution of an event source and use the baseline event type distribution to detect changes in the behavior of the event source are described. In one implementation, blocks of event messages generated by the event source are collected and an event-type distribution is computed for each of block of event messages. Candidate baseline event-type distributions are determined from the event-type distributions. The candidate baseline event-type distribution has the largest entropy of the event-type distributions. A normal discrepancy radius of the event-type distributions is computed from the baseline event-type distribution and the event-type distributions. A block of run-time event messages generated by the event source is collected. A run-time event-type distribution is computed from the block of run-time event messages.

Abstract: The current document is directed to systems, and methods incorporated within the systems, that execute queries against log-file entries. A monitoring subsystem within a distributed computer system uses query results during analysis of log-file entries in order to detect changes in the state of the distributed computer system, identify problems or potential problems, and predict and forecast system characteristics. Because of the large numbers of log-file-entry containers that may need to be opened and processed in order to execute a single query, and because opening and reading through the entries in a log-file-entry container is a computationally expensive and time-consuming operation, the currently disclosed systems employ event-type metadata associated with log-file-entry containers to avoid opening and reading through the log-file entries of log-file-entry containers that do not contain log-file entries with event types relevant to the query.

Abstract: Methods and systems to narrow a search for potential sources of problems in a distributed computing system are described. A volatile event type of event messages recorded in an event-log file is identified. The volatile event type is an event type that may have unexpectedly increased in frequency over an observation time window. An historical period of time may be selected to search for potential sources of the volatile event type. Frequencies of event messages in the event-log file with the same event type as the volatile event type are determined for time intervals of the historical period of time. A time interval of the historical period of time with a largest increase in frequency of event messages is identified. A list of event messages of the event-log file in a selected sub-time interval of the sub-time intervals of the time interval are displayed in a graphical user interface.

Abstract: The current document is directed to methods and systems for processing, classifying, and efficiently storing large volumes of event messages generated in modern computing systems. In a disclosed implementation, received event messages are normalized to identify non-parameter tokens within the event messages. The non-parameter event tokens are used to compute a metric for each event message. The metrics are used, in turn, to identify a type-associated cluster to which to assign each received event message. The type-associated clusters are created dynamically as streams of event messages are processed. The type-associated clusters may be dynamically split and merged to refine event-message typing.

Abstract: Methods and systems adjust resources and monitoring configuration of a monitoring tool and auxiliary servers of a distributed computing system. Any new virtual object installed in the distributed computing system or change in functionality of an existing virtual object of the distributed computing system is identified. Expected configuration rules and resource allocation rules are determined for the virtual object. The expected configured rules are used to adjust a monitoring configuration of the monitoring tool. The resource allocation rules are used to adjust the infrastructure resources available to the monitoring tool. When the virtual object creates a change in demand for auxiliary services, corresponding auxiliary servers are installed, removed, or changed in the distributed computing system. Resource allocated rules and expected configuration rules are determined for the auxiliary servers and used to adjust the infrastructure resources and monitoring configuration of the monitoring tool.

Abstract: Methods and systems automatically adjusting resources and monitoring configurations of objects of a distributed computing system in response to changes to application programs. Methods search event messages for information indicating a change in execution of an object. The information is used to determine resource allocation rules of infrastructure resources by and a monitoring configuration for the object. Expected impacts on the infrastructure resource are determined from the rules. When an expected impact is greater than an associated impact threshold, use of the infrastructure resources may be adjusted to accommodate the changes. The adjustments include scaling up or down the infrastructure resources. When the object is a virtual object, the virtual object may be migrated from one server computer to another server computer within the distributed computer system. The monitoring configuration is used to adjust tools that monitor the objects of the distributed computing system.

Abstract: Methods and systems are directed to detecting and classifying changes in a distributed computing system. Divergence value are computed from distributions of different types of event messages generated in time intervals of a sliding time window. Each divergence value is a measure of change in types of events generated in each time interval. When a divergence value, or a rate of change in divergence values, exceeds a threshold, the time interval associated with the threshold violation is used to determine a change point in the operation of the distributed computing system. Based on the change point, a start time of the change is determined. The change is classified based on various previously classified change points in the disturbed computing system. A recommendation may be generated to address the change based on the classification of the change.

Abstract: The current document is directed to methods and systems for processing, classifying, and efficiently storing large volumes of event messages generated in modern computing systems. In a disclosed implementation, received event messages are assigned to event-message clusters based on non-parameter tokens identified within the event messages. A parsing function is generated for each cluster that is used to extract data from incoming event messages and to prepare event records from event messages that more efficiently and accessible store event information. The parsing functions also provide an alternative basis for assignment of event massages to clusters.

Abstract: Methods and systems that detect computer system anomalies based on log file sampling are described. Computers systems generate log files that record various types of operating system and software run events in event messages. For each computer system, a sample of event messages are collected in a first time interval and a sample of event messages are collected in a recent second time interval. Methods calculate a difference between the event messages collected in the first and second time intervals. When the difference is greater than a threshold, an alert is generated. The process of repeatedly collecting a sample of event messages in a recent time interval, calculating a difference between the event messages collected in the recent and previous time intervals, comparing the difference to the threshold, and generating an alert when the threshold is violated may be executed for each computer system of a cluster of computer systems.

Abstract: Methods and systems to identify log write instructions of a source code as potential sources of an event message of interest are described. Methods identify non-parametric tokens, such as text strings and natural language words and phrases, of an event message of interest. Candidate log write instructions and associated line numbers in a source code are identified. Non-parametric tokens of each event message of the one or more candidate log write instructions are determined. A confidence score is calculated for each candidate log write instruction based the number of non-parametric tokens the event message of interest and event message of the candidate log write instruction have in common. The candidate log write instructions are rank ordered based on the corresponding one or more confidence scores and the rank ordered candidate log write instructions and associated line numbers of the source code may be displayed in a graphical user interface.