Abstract: A computerized method is disclosed for automated handling of data ingestion anomalies. The method features operations of detecting a data ingestion anomaly and determining a cause for the data ingestion anomaly. The causal determination may be conducted by at least (i) determining features of an anomalous data ingestion volume, (ii) training a second data model, after a first data model being used to detect the data ingestion anomaly, with data sets consistent with the determined features, (iii) applying the second data model to predict whether a data ingestion sub-volume is anomalous, (iv) obtaining system state information during ingestion of the anomalous data ingestion sub-volume, and (v) determining the cause of the anomalous data ingestion volume based on the system state information.

Abstract: The disclosed embodiments provide a method and system for processing network data. During operation, the system obtains, at a remote capture agent, configuration information for the remote capture agent from a configuration server over a network. Next, the system uses the configuration information to configure the generation of event data from network data obtained from network packets at the remote capture agent. The system then uses the configuration information to configure transformation of the event data or the network data into transformed event data at the remote capture agent.

Abstract: Disclosed is a data fabric service system that can be implemented in a distributed computer network, such as a data intake and query system. The data index and query system can receive a search query and define a search scheme for applying the search query on distributed data storage systems including internal data storage and external data storage. The data index and query system may provide a portion of the search scheme to a search service of the data fabric service system, which can cause worker nodes of the data fabric service system to perform various functions—including applying the search query to the external data storage based on the portion of the search scheme in order to obtain search results.

Abstract: Systems and methods are described for extracting data fields from logs ingested in a data processing pipeline or otherwise stored. For example, a log can be applied as an input to an artificial intelligence model trained to infer a log sourcetype of logs, and the artificial intelligence model can output an inferred log sourcetype of the log. The inferred log sourcetype can be used to select another artificial intelligence model trained to extract data fields from logs having the inferred log sourcetype, and the log can then be applied as an input to the other artificial intelligence model. The other artificial intelligence model may then output one or more data fields extracted from the log.

Abstract: Techniques, which may be embodied herein as systems, computing devices, methods, algorithms, software, code, computer readable media, or the like, are described herein for comparing a set of metrics generated during a simulated user interaction with a website to metrics generated by observing real user interactions with the website. Simulated user interactions with a website can be used to diagnose a website's performance issues, but it can be difficult to determine whether the simulated interactions reflect the experience of real users. In addition, the simulated user interactions can be challenging to contextualize because the number of observed real user interactions may significantly outnumber the simulated interactions. A graphical user interface can help with the interpretation of these website interactions by using the real user interactions to properly contextualize the simulated results.

Abstract: Systems and methods are described for scheduling a query for execution. The system receives and parses a query to identify one or more portions of the query. The system determines a resource allocation for each portion of the query, and determines an availability of compute resources for the different portions of the query. Based on the resource allocation and the availability of compute resources, the system schedules the query.

Abstract: The disclosed embodiments provide a method and system for processing network data. During operation, the system obtains one or more event streams from one or more remote capture agents over one or more networks, wherein the one or more event streams include event data generated from network packets captured by the one or more remote capture agents. Next, the system applies one or more transformations to the one or more event streams to obtain transformed event data from the event data. The system then enables querying of the transformed event data.

Abstract: A computerized method is disclosed that includes operations of obtaining network traffic data between a source device and a destination device, applying a set of one or more security rules to a plurality of metrics of the network traffic data to obtain a subset of network traffic metrics, applying a first trained machine learning model to the subset of network traffic metrics to generate a feature vector through feature extraction of the subset of network traffic metrics, and evaluate the feature vector for a presence of beaconing and classify the subset of network traffic metrics, and responsive to the classifying of the subset of network traffic metrics, generating a flag for a system administrator. The plurality of metrics include at least one or more of packet size, packet transmission rate, or a ratio of (i) packet size for inbound packets and (ii) packet size for outbound packets.

Abstract: A computer-implemented method of configuring an anomalous behavior detector includes updating a distribution used for modeling anomalous behavior in telemetry data with information associated with observed anomalous behavior to generate an updated distribution representative of the observed anomalous behavior where, prior to the updating, the distribution is representative of theoretical anomalous behavior. The method further includes computing a threshold for a detector operable to alert on anomalous activity using the updated distribution. The method also comprises computing a divergence between the live telemetry data monitored by the detector and the anomalous behavior modeled by the updated distribution. Responsive to a determination that the divergence is above a critical threshold, the method comprises enabling the detector to continue to monitor the live telemetry data in the application.

Abstract: Systems and methods are disclosed for providing a multi-component application, including a first and second component, and a first and second server. The first component may be implemented at the first server, while a second component may be implemented at a client device. An end user of a client device may request access to metadata stored on the second server that is utilized by the second component to implement the multi-component application. The end user may authenticate with the first component. The first component may then communicate with the second server to authenticate the end user to the second server, thereby granting the end user access to the second server without having to reauthenticate to the second server.

Abstract: Described herein are techniques for integrating external sensors to an edge device, such as for ingesting data into a data intake and query system. The edge device has an internal message broker for communicating with internal (e.g., preconfigured, recognized) sensors, and an external message broker for communicating with external (e.g., customer-configured, otherwise unrecognized) sensors. The external message broker provides access to customer configuration of external sensors, but is logically quarantined from the internal message broker to prevent unwanted customer access to internal configurations. The internal and external message brokers interface only via a bridging service that transforms external sensor data into data based on customer-configurable transformations. The transformed data can be handled by the edge device and/or downstream components (e.g., a data intake and query system) in the same manner as internal sensor data.

Abstract: Resegmenting chunks of data for load balancing is disclosed. A plurality of first chunks of data is received. The plurality of first chunks of data includes one or more entries that include raw data produced by a component of an information technology environment and that reflects activity in the information technology environment. The plurality of first chunks of data is resegmented into a plurality of second chunks of data based on a source type of the plurality of first chunks. A first subset of the plurality of second chunks of data is distributed to a first indexer of a set of indexers. An occurrence of a trigger event is determined, and in response to the trigger event, a second subset of the plurality of second chunks of data is distributed to a second indexer of the set of indexers.

Abstract: A method includes displaying events that correspond to search results of a search query, the events comprising data items of event attributes, the events displayed in a table. The table includes columns corresponding to an event attribute, rows corresponding events, cells populated data items, and interactive regions corresponding to at least one data item and selectable to add one or more commands to the search query. A reference event attribute is determined based on an analysis of a data object. A supplemental column corresponding to a supplemental event attribute is added to the table based on the reference event attribute. Supplemental interactive regions are added to the table and correspond to supplemental data items.

Abstract: A system generates a user interface that enables a user to modify time ranges associated with search-related statements of a data processing package. Via one or more user interactions with the user interface, the system may receive a modified time range for the statement. The modified time range may be appended to the data processing package to form an enriched data processing package. The system may communicate the enriched data processing package to a search service for execution. The system may display the results in the user interface.

Abstract: In embodiments of statistics chart row mode drill down, a first interface is displayed in a table format that includes columns and rows, where each row is associated with an event and each column includes field for a respective event. The rows can further include one or more aggregated metrics representing a number of events associated with a respective row. A row can be emphasized in the first interface and, in response a menu can be displayed with selectable options to transition to a second interface, where the data displayed by the second interface is based on an option selected from the menu.

Abstract: Embodiments described herein are directed to facilitating management and storage of configurations. In one embodiment, a request to provide configurations associated with a user-application pair is identified. Based on the user-application pair, a user-defined configuration cache is accessed to obtain user-defined configurations, an application-defined configuration cache is accessed to obtain application-defined configurations, and a system-defined configuration cache is accessed to obtain system-defined configurations. Thereafter, at least a portion of the user-defined configurations, the application-defined configurations, and the system-defined configurations are aggregated or merged to generate a set of configurations associated with the user-application pair. Such a set of configurations associated with the user-application pair are provided for use in performing a task.

Abstract: A search assistant engine is described that integrates with a data intake and query system and provides an intuitive user interface to assist a user in searching and evaluating indexed event data. Additionally, the search assistant engine provides logic to intelligently provide data to the user through the user interface such as determining fields of events likely to be of interest based on determining a mutual information score for each field and determining groups of related fields based on determining a mutual information score for each field grouping. Some implementations utilize machine learning techniques in certain analyses such as when clustering events and determining an event templates for each cluster. Additionally, the search assistant engine may import terms or characters from user interaction into predetermined search query templates to generate tailored search query for the user.

Abstract: A system is described that receives a query model of a query that includes one or more query commands. The query model includes a command model that corresponds to at least query command of the one or more query commands. The system uses the command model to generate an interactive action model summary and causes a user interface to display the query and the interactive action model summary in a query actions panel. A modification to the query in the user interface causes an update to the query actions panel and a modification to the action model summary causes an update to the at least one query command of the query.

Abstract: A computerized method is disclosed for grouping alerts through machine learning while implementing certain time constraints. The method includes receiving an alert to be assigned to any of a plurality of existing issues or to a newly created issue, the alert including a temporal field that includes a timestamp of an arrival time of the alert, wherein an issue is a grouping of one or more alerts, determining a subset of existing issues from the plurality of existing issues that each satisfy time constraints, wherein the time constraints correspond to (i) a time elapsed between a most recent alert of a first existing issue and a timestamp of the alert, or (ii) a maximum issue time length of the first existing issue, and deploying a trained machine learning model to assign the alert to either an existing issue of the subset of existing issues or a newly created issue.

Abstract: A process for providing requests to a management application in a multi-tenant environment is described herein. In embodiments, a broker client is deployed within a tenant execution environment executed by a server computer system. In embodiments, the broker client is configured to communicate with a broker responsible for managing the tenant execution environment based on configuration information. Furthermore, in various embodiments, request to perform operations associated with the tenant execution environment are transmitted to the broker client over a connection and the broker client provides the request to the broker for execution.