Abstract: Querying of time-aware metrics time series includes receiving a query, the query comprising a set of query metadata and a query time range. It further includes, based at least in part on the set of query metadata and the query time range, selecting a time series from a plurality of metrics time series. Each metrics time series in the plurality of metrics time series is associated with a set of metadata and an active interval of time. A set of metadata associated with the selected time series matches the set of query metadata, and an active interval of time associated with the selected metrics time series intersects with the query time range. The selected metrics time series is returned.

Abstract: A technique for logs to metrics synthesis is disclosed. A log message is received. It is determined that the received log message should be translated into a metrics data point. In response to determining that the received log message should be translated into a metrics data point, the metrics data point is generated using the received log message, the generated metrics data point comprising a timestamp, a metric name, a metric value, and a set of metadata key-value pairs. A time series in which to insert the metrics data point generated using the received log message is identified. The generated metrics data point is inserted into the identified time series.

Abstract: Obfuscating data is disclosed. A processor identifies structured information in log data. The structured information is transformed in a manner that preserves the structure to form transformed raw data. The transformed raw data is sent to a remote analysis engine. The remote analysis engine receives a query and responds to the query by providing as results at least a portion of the transformed raw data. A processor is configured to de-transform the transformed raw data.

Abstract: One variation of a method for predicting and characterizing cyber attacks includes: receiving, from a sensor implementing deep packet inspection to detect anomalous behaviors on the network, a first signal specifying a first anomalous behavior of a first asset on the network at a first time; representing the first signal in a first vector representing frequencies of anomalous behaviors—in a set of behavior types—of the first asset within a first time window; calculating a first malicious score representing proximity of the first vector to malicious vectors defining sets of behaviors representative of security threats; calculating a first benign score representing proximity of the first vector to a benign vector representing an innocuous set of behaviors; and in response to the first malicious score exceeding the first benign score and a malicious threshold score, issuing a first alert to investigate the network for a security threat.

Abstract: The automatic selection and usage of a parser is disclosed. Raw data is obtained from a first remote device. At least a portion of the raw data is evaluated using a plurality of rules. A confidence measure is determined for at least some of the rules. An indication that the raw data pertains to a source is provided as output when the confidence measure exceeds a threshold.

Abstract: Systems and methods for tokenization of log records for efficient data storage, log querying, and log data analytics can utilize a trie pattern conversion of the log files, storing trie data pattern IDs, free parameters, and metadata instead of the entire log record. New trie patterns can be discovered automatically by counting the occurrences of tokens matching wildcards for existing patterns.

Abstract: One variation of a method for detecting a cyber attack includes: recording representations of network events occurring on a network over a period of time to a network accounting log; writing metadata values of network events in the accounting log to a compressed log file; in response to receipt of a new threat intelligence representing a newly-identified security threat identified after the period of time, querying the compressed log file for a set of metadata values of a threat element defined in the new threat intelligence; in response to detecting the set of metadata values of the threat element in the compressed log file, querying the network accounting log for a set of threat elements defined in the new threat intelligence; and in response to detecting the set of threat elements in the network accounting log, issuing an alert to respond to the newly-identified security threat on the network.

Abstract: One variation of a method for predicting security risks of assets on a computer network includes: over a first period of time, detecting an asset connected to the computer network and a first set of behaviors exhibited by the asset; associating the asset with a first set of assets based on similarity of the first set of behaviors to behaviors characteristic of the first set of assets; over a second period of time succeeding the first period of time, detecting the asset connected to the computer network and a second set of behaviors exhibited by the asset; detecting deviation of the asset from the first set of assets based on differences between the second set of behaviors and behaviors characteristic of the first set of assets; and generating a security alert for the asset in response to deviation of the asset from the first set of assets.

Abstract: A processing device receives a query comprising a first field value and a time period and performs a first search of a data store using the first field value to identify a plurality of events having the time period and a field that comprises the first field value. The processing device determines a first subset of the plurality of events associated with a first context definition and determines a plurality of fields specified in the first context definition. The processing device determines, for events in the first subset, field values of one or more fields specified in the first context definition. The processing device generates a report based on the field values of the one or more fields specified in the first context definition from the events in the first subset. The processing device generates a response to the query that comprises at least a portion of the report.

Abstract: Single-click delta analysis is disclosed. A user query of status information collected from one or more monitored devices is received from a user. In response to receiving an indication from the user to determine a variance between different portions of the collected status information, a target query and a baseline query are generated using the user query. The generated target query and the generated baseline query are performed, respectively, against data in a data store including the status information collected from the one or more monitored devices. A target set of status information results and a baseline set of status information results are obtained in response to performing, respectively, the generated target query and the generated baseline query. The obtained target and baseline sets of results are combined. Output indicative of a variance between the target and baseline sets of status information results is provided based at least in part on the combining.

Abstract: Automatically generating a parser is disclosed. Raw data is received from a first remote device. A determination that the raw data does not, within a predefined confidence measure, conform to any rules included in a set of rules is made. A clustering function is performed on the raw data. At least one parser rule is generated based on the clustering.

Abstract: A processing device receives a first query comprising a first field value and a first time period. The processing device performs a first search of a data store to identify a first plurality of events having the first time period and at least one field that comprises the first field value. The processing device generates a first search object comprising the first field value. The processing device generates a search event comprising the first field value and a reference to the first search object. An event entry for the first search event is then written to the data store. Future searches may return both the first search event and other events.

Abstract: Obfuscating data is disclosed. A processor identifies structured information in log data. The structured information is transformed in a manner that preserves the structure to form transformed raw data. The transformed raw data is sent to a remote analysis engine. The remote analysis engine receives a query and responds to the query by providing as results at least a portion of the transformed raw data. A processor is configured to de-transform the transformed raw data.

Abstract: Analyzing log data, such as security log data and machine data, is disclosed. A baseline is built for a set of machine data. The baseline is built at least in part by determining a plurality of signature profiles for a plurality of respective time slices. An occurrence of an anomaly associated with the source of the machine data is determined. The occurrence is determined at least in part by determining that received machine data does not conform to the baseline within a threshold.

Abstract: A processing device receives a query comprising a first field value and a time period. The processing device performs a first search of a data store using the first field value to identify a first plurality of events having the time period and a field that comprises the first field value. The processing device determines, for one of the plurality of events, a second field value of a second field that is specified in a first context definition, the second field having an assigned field type. The processing device automatically performs a second search of the data store using the additional field value to identify a second plurality of events having the time period and the additional field value. Information from the first plurality of events and the second plurality of events is aggregated, and a response to the query is generated that comprises the aggregated information.

Abstract: A processing device receives a plurality of discrete log entries from a first data store and generates an event for each discrete log entry that satisfies a criterion. To generate an event the processing device determines a source type associated with a discrete log entry, parses the discrete log entry based on the source type, determines a plurality of fields of the discrete log entry, identifies a subset of the plurality of fields, wherein one or more fields in the subset are to be used as link keys for linking together events, and assigns a field type to each field in the subset of the plurality of fields. The processing device additionally writes a plurality of event entries for the event into a second data store. A separate event entry is written for each field of the subset of the plurality of fields having an assigned field type.

Abstract: Data collection and transmission is disclosed. A server is configured to receive, from a remote device, a message including raw information, and to parse at least a portion of the received raw information. The raw information is received by the system from an information reporting module interface of the remote device. The information reporting module of the remote device is configured to receive information from at least one separately installed information reporting module. A client device includes an information reporting module interface and a server interface. The client device is configured to receive configuration information from a remote server.

Abstract: Analyzing log data, such as security log data and event data, is disclosed. Log data is obtained. Portions of the log data are clustered into clusters of similar data portions. A signature for each cluster is generated. Comparison of subsequent log data with the signature indicates whether the subsequent log data belongs in the cluster.

Abstract: Obfuscating data is disclosed. A processor identifies structured information in log data. The structured information is transformed in a manner that preserves the structure to form transformed raw data. The transformed raw data is sent to a remote analysis engine. The remote analysis engine receives a query and responds to the query by providing as results at least a portion of the transformed raw data. A processor is configured to de-transform the transformed raw data.

Abstract: Automatically generating a parser is disclosed. Raw data is received from a first remote device. A determination that the raw data does not, within a predefined confidence measure, conform to any rules included in a set of rules is made. A clustering function is performed on the raw data. At least one parser rule is generated based on the clustering.