Abstract: Methods and systems described herein are directed to aggregating and querying log messages. Methods and systems determine event types of log message generated by event sources of the distributed computing system. The event types are aggregated into aggregated records for a shortest time unit and event types are aggregated into aggregated records for longer time units based on the aggregated records associated with the shortest time unit. In response to a query regarding occurrences of an event type in a query time interval, the query time interval is split into subintervals with time lengths that range from the shortest time unit to a longest time unit that lie within the query time interval. The method determines a total event count of occurrences of the event type in the query time interval based on the aggregated records with time stamps in the subintervals. The event count in the query time interval may be used to detect abnormal behavior of the event sources.

Abstract: Methods and systems described herein are directed to storing and querying log messages using log message bifurcation. Each log message that exceeds a time limit for storage in a live storage database is bifurcated into a log pattern that is stored in a log-pattern database and a metric object that records variable information of the log message in a time-series metric database. The log-pattern database and the time-series metric database together provide long-term log storage for the information contained in log messages that are too old to be stored in a live storage database. A request for access to log messages stored in the long-term log storage is completed by reconstructing the requested log messages from log patterns stored in the log-pattern database and metric objects stored in the time-series metric database.

Abstract: Methods and systems described herein are directed to storing and querying log messages using log message bifurcation. Each log message that exceeds a time limit for storage in a live storage database is bifurcated into a log pattern that is stored in a log-patter database and a metric object that records variable information of the log message in a time-series metric database. The log-pattern database and the time-series metric database: together provide long-term log storage for the information contained in log messages that are too old to be stored in a live storage database. A request for access to log messages stored in the long-term log storage is completed by reconstructing the requested log messages from log patterns stored in the log-pattern database and metric objects stored in the time-series metric database.

Abstract: Methods and systems described herein are directed to aggregating and querying log messages. Methods and systems determine event types of log message generated by event sources of the distributed computing system. The event types are aggregated into aggregated records for a shortest time unit and event types are aggregated into aggregated records for longer time units based on the aggregated records associated with the shortest time unit. In response to a query regarding occurrences of an event type in a query time interval, the query time interval is split into subintervals with time lengths that range from the shortest time unit to a longest time unit that lie within the query time interval. The method determines a total event count of occurrences of the event type in the query time interval based on the aggregated records with time stamps in the subintervals. The event count in the query time interval may be used to detect abnormal behavior of the event sources.