Abstract: Load balancing processes are performed in an observability pipeline system comprising a plurality of computing resources. In some aspects, the observability pipeline system defines a leader role and worker roles. A plurality of computing jobs each include computing tasks associated with event data. The leader role dispatches the computing tasks to the worker roles according to a least in-flight task dispatch criteria, which includes iteratively: identifying an available worker role; identifying one or more incomplete computing jobs; selecting, from the one or more incomplete computing jobs, a computing job that has the least number of in-flight computing tasks currently being executed in the observability pipeline system; identifying a next computing task from the selected computing job; and dispatching the next computing task to the available worker role. The worker roles execute the computing tasks by applying an observability pipeline process to the event data associated with the respective computing task.

Abstract: In the disclosed embodiments, a remote capture agent monitors network packets traversing a network interface of a computing device in an information technology environment. Network data is obtained from the network packets. The network data is modified based on configuration information obtained by the remote capture agent from a configuration server to obtain modified network data. Timestamped events are generated based on the modified network data, and the timestamped events are sent to another component on the network for subsequent processing.

Abstract: In some aspects, data collection functions are interposed to generate input data for an observability pipeline system. In some aspects, a data collection function is made available to an application running on a computer system, with the data collection function having the same name as an original function referenced by the application. In response to a call to the original function, the data collection function is executed and data is extracted from the application. The original function is then executed. A reporting thread of the application is executed; executing the reporting thread generates observability pipeline input data by formatting the extracted data and sends the observability pipeline input data from the computer system to an observability pipeline system.

Abstract: The disclosed embodiments provide a system for extracting custom content from network packets. During operation, the system receives a stream of packets. The system then parses packets in the stream to determine a protocol for each packet. Next, the system applies a custom-content-extraction rule to each packet associated with a target protocol to obtain the extracted content. Then, the system stores the extracted content in events in a data store to facilitate subsequent queries involving the extracted content.

Abstract: In some aspects, search functionality is provided in an observability pipeline system. In some implementations, a search method includes receiving a search query from a leader role in an observability pipeline system. The search query represents a request to search event data at a computer resource. An observability pipeline process is configured to perform a search according to the search query, and search results are obtained based on applying the observability pipeline process to the event data. The search results include events from the event data. Provenance information is obtained for each of the events. The provenance information for each event includes an identification of the computer resource and a link to the computer resource. Augmented search results are generated by associating the provenance information with the respective events, and the augmented search results are communicated to the leader role.

Abstract: In some aspects, search functionality is provided in an observability pipeline system. In some implementations, a method of searching remotely-stored data includes receiving a search query at a computer node residing at a node geolocation, the search query representing a request to search data stored at a storage geolocation; identifying compute geolocations each including computing resources, each of the compute geolocations being distinct from the node geolocation and the storage geolocation; obtaining latency data including latency values for the compute geolocations; selecting one of the compute geolocations based on the latency data; by operation of a coordinator agent on the computer node, initiating a dynamic computing resource at the selected compute geolocation, and receiving search results identified by the dynamic computing resource.

Abstract: In some aspects, search functionality is provided in an observability pipeline system. In some implementations, a search query is received at a computer node from a leader role of an observability pipeline system. The search query represents a request to search event data at the computer node and includes a first search operator that specifies a system state context criterion and a second search operator that specifies an event criterion. An observability pipeline process is configured according to the search query. Search results are generated based on applying the observability pipeline process at the computer node. A current system state of the computer node that matches the system state context criterion specified by the first search operator is determined; and a subset of event data on the computer node that matches the event criterion specified by the second search operator is identified. The search results including the subset of event data are sent to the leader role.

Abstract: In some aspects, an interactive graphical user interface displays search data for an observability pipeline system. In some aspects, a method includes obtaining search results including events identified based on searching data generated by an observability pipeline system. The method includes identifying time bins based on the search results; generating first histogram data based on the time bins and the events; and generating second histogram data based on the time bins and the events. The method includes generating a graphical user interface including a first histogram representing the first histogram data and including a first set of bins, and a second histogram representing the second histogram data and including a second set of bins. The method includes updating the graphical user interface to include a visual indication of a selected bin in the first histogram and a visual indication of a corresponding bin in the second histogram.

Abstract: Technologies are disclosed for providing a common information model. Features include: detecting a scheduled time for a key performance indicator reflecting how a service provided by one or more entities is performing, entity definition information recording the association between the entities and its machine data, service definition information associating the entities that provide the service, and the KPI being defined by a search query, including a field identifier specified in a data model, the KPI derives a value from the machine data; performing the query in response to said detecting, including: associating values in the machine data having disparate field names in accordance with disparate schemas with the field identifier specified in the data model, and processing the associated values as semantically equivalent data instances. In doing so, values having the same semantic (or related semantics) can be used together despite being associated with disparate field names from disparate schemas.

Abstract: Load balancing processes are performed in an observability pipeline system comprising a plurality of computing resources. In some aspects, the observability pipeline system defines a leader role and worker roles. A plurality of computing jobs each include computing tasks associated with event data. The leader role dispatches the computing tasks to the worker roles according to a least in-flight task dispatch criteria, which includes iteratively: identifying an available worker role; identifying one or more incomplete computing jobs; selecting, from the one or more incomplete computing jobs, a computing job that has the least number of in-flight computing tasks currently being executed in the observability pipeline system; identifying a next computing task from the selected computing job; and dispatching the next computing task to the available worker role. The worker roles execute the computing tasks by applying an observability pipeline process to the event data associated with the respective computing task.

Abstract: The disclosed embodiments include a method performed by a data intake and query system. The method includes populating each metric including a measure value, cataloging metadata in an in-memory metrics catalog, where the metadata is related to the metrics. The method further includes receiving a search query including search criteria, evaluating the search query by applying the search criteria to the metadata of the metrics catalog to obtain results that satisfy the search criteria, and causing display, on a display device, of the results or data indicative of the results.

Abstract: The disclosed embodiments provide a system for extracting custom content from network packets. During operation, the system receives a stream of packets. The system then parses packets in the stream to determine a protocol for each packet. Next, the system applies a custom-content-extraction rule to each packet associated with a target protocol to obtain the extracted content. Then, the system stores the extracted content in events in a data store to facilitate subsequent queries involving the extracted content.

Abstract: Techniques are disclosed for providing an aggregate key performance indicator (KPI) that spans multiple services and is based on a priority value. The techniques may enable a user to select KPIs and to adjust weights (e.g., importance) associated with the KPIs. The weight of a KPI may affect the influence a value of the KPI has on the calculation of an aggregate KPI value. The techniques may also include the ability to create a correlation search using the selected KPIs and weights so that a notification may be generated when the aggregate KPI value exceeds a threshold.

Abstract: Technologies are disclosed for providing a common information model. Features include: detecting a scheduled time for a key performance indicator reflecting how a service provided by one or more entities is performing, entity definition information recording the association between the entities and its machine data, service definition information associating the entities that provide the service, and the KPI being defined by a search query, including a field identifier specified in a data model, the KPI derives a value from the machine data; performing the query in response to said detecting, including: associating values in the machine data having disparate field names in accordance with disparate schemas with the field identifier specified in the data model, and processing the associated values as semantically equivalent data instances. In doing so, values having the same semantic (or related semantics) can be used together despite being associated with disparate field names from disparate schemas.

Abstract: The disclosed embodiments include a method performed by a data intake and query system. The method includes ingesting each metric including at least one key value and a measured value taken of a computing resource, and storing each metric in an index of a metrics store, where the index defines at least one dimension populated with the at least one key value and a measure populated with the measured value. The method further includes cataloging metadata in a metrics catalog, where the metadata is related to the metrics stored in the metrics store, performing an analysis of metrics data included in the metrics store and/or the metrics catalog to obtain results, and causing display of the results or an indication of the results on a display device.

Abstract: The disclosed embodiments provide a system for extracting custom content from network packets. During operation, the system receives a stream of packets. The system then parses packets in the stream to determine a protocol for each packet. Next, the system applies a custom-content-extraction rule to each packet associated with a target protocol to obtain the extracted content. Then, the system stores the extracted content in events in a data store to facilitate subsequent queries involving the extracted content.

Abstract: Raw data in distributed servers is divided into groups of data called buckets containing raw data that have timestamps that fall within a specific time range. When a bucket becomes inactive a server can archive the bucket to an external storage system. The external storage system containing archived data may be specified in a search query. Archived data from the external storage system is obtained, processed, and a search performed on the processed archived data using the search query.

Abstract: The disclosed embodiments include a method performed by a data intake and query system. The method includes populating each metric including a measure value, cataloging metadata in an in-memory metrics catalog, where the metadata is related to the metrics. The method further includes receiving a search query including search criteria, evaluating the search query by applying the search criteria to the metadata of the metrics catalog to obtain results that satisfy the search criteria, and causing display, on a display device, of the results or data indicative of the results.

Abstract: The disclosed embodiments include a method performed by a data intake and query system. The method includes populating each metric including a measure value, cataloging metadata in an in-memory metrics catalog, where the metadata is related to the metrics. The method further includes receiving a search query including search criteria, evaluating the search query by applying the search criteria to the metadata of the metrics catalog to obtain results that satisfy the search criteria, and causing display, on a display device, of the results or data indicative of the results.

Abstract: Techniques are disclosed for providing an aggregate key performance indicator (KPI) that spans multiple services and for providing adjustment to KPI factors via a GUI that enables an aggregate KPI to be configured with feedback that better characterizes the performance of the services. The GUI may enable selection of KPIs and adjustment to weights (e.g., importance) associated with the KPIs. The weight of a KPI may affect the influence a value of the KPI has on the calculation of an aggregate KPI value (e.g., score). The GUI may provide near real-time feedback concerning the effect the weights have on the aggregate KPI value by displaying the aggregate KPI value (e.g., score) and updating the aggregate KPI value as the weights are adjusted.