Abstract: Methods, systems, and computer programs are presented for automatic evaluation of security incidents. One method includes receiving a resolution status, for a set of insights, indicating if each insight was a true or a false positive. A global training set, comprising the resolution status for the insights, is generated, and a local training set with a subset of the insights associated with a first user. A machine-learning (ML) program is trained, using the global training set, to obtain a global model, and using the local training set to obtain a local model for the first user. When a new insight for the first user is detected, a global score is obtained using the global model, and a local score is obtained using the local model. A confidence score, calculated based on the global and local scores, is presented as an indication of an estimated severity of the new insight.

Abstract: Techniques are presented for recommending queries to search log information. The system provides useful insights and recommendations based on user needs and queries by utilizing the user context, with information about the user activities (e.g., recent alerts) and the user configuration in the system (e.g., applications configured by the user), to provide recommendations. There may not be enough context for a new user to provide good recommendations, so the system determines the context based on the activities of other users, such as more experienced users or users investigating the same type of problem. Based on the context, the user recommends natural language queries (NLQ) or system queries to accelerate the search process and assist the user during an investigation. Further, NLQs may be converted to complex search queries that use the search query language, and the NLQs may also be used as part of the context for the subsequent recommendations.

Abstract: A processing device receives a first query comprising a first field value and a first time period. The processing device performs a first search of a data store to identify a first plurality of events having the first time period and at least one field that comprises the first field value. The processing device generates a first search object comprising the first field value. The processing device generates a search event comprising the first field value and a reference to the first search object. An event entry for the first search event is then written to the data store. Future searches may return both the first search event and other events.

Abstract: Automatic partitioning is disclosed. A set of previously run queries is obtained. The set of previously run queries is analyzed to determine one or more query fragments from the set of previously run queries. One or more partitions are generated at least in part by using the obtained query fragments.

Abstract: A new cybersecurity incident is registered at a security incident response platform. At a playbook generation system, details are received of the new cybersecurity incident from the security incident response platform. At least some of the details correspond to a set of features of the new cybersecurity incident. A set or subset of nearest neighbors of the new cybersecurity incident is localized in a feature space. The nearest neighbors of the new cybersecurity incident are other cybersecurity incidents having a distance from the new cybersecurity incident within the feature space that is defined by differences in features of the nearest neighbors with respect to the set of features of the new cybersecurity incident. A custom playbook is created for responding to the new cybersecurity incident having prescriptive procedures based on occurrences of prescriptive procedures previously employed in response to the nearest neighbor cybersecurity incidents.

Abstract: Methods, systems, and computer programs are presented to generate response information for an alert. One method includes an operation for detecting an alert based on incoming log data or metric data and for calculating information for panels to be presented on a response-alert page. Calculating the information includes calculating first performance values for a period associated with the alert, calculating second performance values for a background period where the alert condition was not present, and calculating a difference between the first performance values and the second performance values. Further, the method includes an operation for selecting, based on the difference, relevant performance values for presentation in one of the panels. The response-alert page is presented with at least one of the panels based on the selected relevant performance values.

Abstract: Clustering structured log data by key-values includes receiving, via a user interface, a request to apply an operator to cluster log messages according to values for keys associated with the request. At least a portion of each log message comprises structured machine data including a set of key-value pairs. The method further includes receiving a log message and determining whether to include the log message in a cluster based at least in part on an evaluation of values in the structured machine data of the log message for the keys associated with the request. The cluster is included in a set of clusters. Each cluster in the set is associated with a different combination of values for the keys associated with the request. The method further includes providing, via the user interface, information associated with the cluster.

Abstract: A method and apparatus for controlling digital evidence comprising creating a case record comprising information about an investigative case, electronically storing at least one piece of digital evidence into memory, and associating the stored at least one piece of evidence with the case record.

Abstract: Data enrichment and augmentation is disclosed. Machine data comprising at least one of a log message and a metrics data point is received. The received machine data comprises an identifier of an instance of a virtual machine. Based at least in part on the identifier of the instance of the virtual machine, a query for tags associated with the instance of the virtual machine is performed. At least one key-value pair is generated based at least in part on tags received in response to the query performed based at least in part on the identifier of the instance of the virtual machine. The received machine data is augmented with the at least one key-value pair generated based at least in part on the tags received in response to the query based at least in part on the identifier of the instance of the virtual machine.

Abstract: Single-click delta analysis is disclosed. A user query of status information collected from one or more monitored devices is received from a user. In response to receiving an indication from the user to determine a variance between different portions of the collected status information, a target query and a baseline query are generated using the user query. The generated target query and the generated baseline query are performed, respectively, against data in a data store including the status information collected from the one or more monitored devices. A target set of status information results and a baseline set of status information results are obtained in response to performing, respectively, the generated target query and the generated baseline query. The obtained target and baseline sets of results are combined. Output indicative of a variance between the target and baseline sets of status information results is provided based at least in part on the combining.

Abstract: Systems and methods for tokenization of log records for efficient data storage, log querying, and log data analytics can utilize a trie pattern conversion of the log files, storing trie data pattern IDs, free parameters, and metadata instead of the entire log record. New trie patterns can be discovered automatically by counting the occurrences of tokens matching wildcards for existing patterns.

Abstract: A method includes defining a set of context types; defining a set of source types, each comprising context types; defining, for each source type, and for each context type included in the events from data sources having the source type, a context definition comprising a set of fields, in events from the data sources, that are associated with the context type; receiving a query comprising a first field value and a time period; retrieving a plurality of events that include the first field value and the time period; for each retrieved event, and for each context definition defined for a source type and a context type of a data source from which the retrieved event originated, determining field values of fields in the set of fields of the context definition; aggregating, for each context type, determined field values from the events; and generating an output.

Abstract: Querying of time-aware metrics time series includes receiving a query, the query comprising a set of query metadata and a query time range. It further includes, based at least in part on the set of query metadata and the query time range, selecting a time series from a plurality of metrics time series. Each metrics time series in the plurality of metrics time series is associated with a set of metadata and an active interval of time. A set of metadata associated with the selected time series matches the set of query metadata, and an active interval of time associated with the selected metrics time series intersects with the query time range. The selected metrics time series is returned.

Abstract: Key name synthesis is disclosed. A metrics data point is received. Based at least in part on a translation statement, at least a portion of the received metrics data point is associated with a key specified by the translation statement such that the specified key and the associated at least portion of the received metrics data point form a key-value pair. The key-value pair is associated with the received metrics data point.

Abstract: Analyzing log data, such as security log data and machine data, is disclosed. A baseline is built for a set of machine data. The baseline is built at least in part by determining a plurality of signature profiles for a plurality of respective time slices. An occurrence of an anomaly associated with the source of the machine data is determined. The occurrence is determined at least in part by determining that received machine data does not conform to the baseline within a threshold.

Abstract: Clustering structured log data by key schema includes receiving a raw log message. At least a portion of the raw log message comprises structured machine data including a set of key-value pairs. It further includes receiving a map of keys to values. It further includes using the received map of keys to values to determine a key schema of the structured machine data. The key schema is associated with a corresponding cluster. It further includes associating the raw log message with the cluster corresponding to the determined key schema.

Abstract: A processing device receives a first query comprising a first field value and a first time period. The processing device performs a first search of a data store to identify a first plurality of events having the first time period and at least one field that comprises the first field value. The processing device generates a first search object comprising the first field value. The processing device generates a search event comprising the first field value and a reference to the first search object. An event entry for the first search event is then written to the data store. Future searches may return both the first search event and other events.

Abstract: A new cybersecurity incident is registered at a security incident response platform. At a playbook generation system, details are received of the new cybersecurity incident from the security incident response platform. At least some of the details correspond to a set of features of the new cybersecurity incident. A set or subset of nearest neighbors of the new cybersecurity incident is localized in a feature space. The nearest neighbors of the new cybersecurity incident are other cybersecurity incidents having a distance from the new cybersecurity incident within the feature space that is defined by differences in features of the nearest neighbors with respect to the set of features of the new cybersecurity incident. A custom playbook is created for responding to the new cybersecurity incident having prescriptive procedures based on occurrences of prescriptive procedures previously employed in response to the nearest neighbor cybersecurity incidents.

Abstract: A cybersecurity incident is registered at a security incident response platform. At a playbook generation system, details are received of the cybersecurity incident from the security incident response platform. At least some of the details correspond to a set of features of the cybersecurity incident. A set or subset of nearest neighbors of the cybersecurity incident is localized in a feature space. The nearest neighbors of the cybersecurity incident are other cybersecurity incidents having a distance from the cybersecurity incident within the feature space that is defined by differences in features of the nearest neighbors with respect to the set of features of the cybersecurity incident. A playbook is created for responding to the cybersecurity incident having prescriptive procedures based on occurrences of prescriptive procedures previously employed in response to the nearest neighbor cybersecurity incidents.

Abstract: A method and apparatus for controlling digital evidence comprising creating a case record comprising information about an investigative case, electronically storing at least one piece of digital evidence into memory, and associating the stored at least one piece of evidence with the case record.