Abstract: In some aspects, an edge-based data collection system discovers, collects, processes, and forwards data in an observability pipeline system. In some implementations, an edge agent of the observability pipeline system runs on a computer node. The edge agent identifies processes running on the computer node; identifies files on the computer node that the processes have opened for writing; accesses log discovery parameters of the observability pipeline system; selects a plurality of files from the identified files according to the log discovery parameters; generates a list of discovered log files that includes a path and a name for each of the plurality of files; adds the list of discovered log files to a list of monitored log files to be monitored by the observability pipeline system; and then monitors the plurality of files to generate input for the observability pipeline system.

Abstract: In some aspects, search functionality is provided in an observability pipeline system. In some implementations, a search query is received at a computer node from a leader role of an observability pipeline system. The search query represents a request to search event data at the computer node and includes a first search operator that specifies a system state context criterion and a second search operator that specifies an event criterion. An observability pipeline process is configured according to the search query. Search results are generated based on applying the observability pipeline process at the computer node. A current system state of the computer node that matches the system state context criterion specified by the first search operator is determined; and a subset of event data on the computer node that matches the event criterion specified by the second search operator is identified. The search results including the subset of event data are sent to the leader role.

Abstract: In some aspects, search functionality is provided in an observability pipeline system. In some implementations, a method of searching remotely-stored data includes receiving a search query at a computer node residing at a node geolocation, the search query representing a request to search data stored at a storage geolocation; identifying compute geolocations each including computing resources, each of the compute geolocations being distinct from the node geolocation and the storage geolocation; obtaining latency data including latency values for the compute geolocations; selecting one of the compute geolocations based on the latency data; by operation of a coordinator agent on the computer node, initiating a dynamic computing resource at the selected compute geolocation, and receiving search results identified by the dynamic computing resource.

Abstract: In some aspects, search functionality is provided in an observability pipeline system. In some implementations, a search method includes receiving a search query from a leader role in an observability pipeline system. The search query represents a request to search event data at a computer resource. An observability pipeline process is configured to perform a search according to the search query, and search results are obtained based on applying the observability pipeline process to the event data. The search results include events from the event data. Provenance information is obtained for each of the events. The provenance information for each event includes an identification of the computer resource and a link to the computer resource. Augmented search results are generated by associating the provenance information with the respective events, and the augmented search results are communicated to the leader role.

Abstract: In some aspects, an interactive graphical user interface displays search data for an observability pipeline system. In some aspects, a method includes obtaining search results including events identified based on searching data generated by an observability pipeline system. The method includes identifying time bins based on the search results; generating first histogram data based on the time bins and the events; and generating second histogram data based on the time bins and the events. The method includes generating a graphical user interface including a first histogram representing the first histogram data and including a first set of bins, and a second histogram representing the second histogram data and including a second set of bins. The method includes updating the graphical user interface to include a visual indication of a selected bin in the first histogram and a visual indication of a corresponding bin in the second histogram.

Abstract: Load balancing processes are performed in an observability pipeline system comprising a plurality of computing resources. In some aspects, the observability pipeline system defines a leader role and worker roles. A plurality of computing jobs each include computing tasks associated with event data. The leader role dispatches the computing tasks to the worker roles according to a least in-flight task dispatch criteria, which includes iteratively: identifying an available worker role; identifying one or more incomplete computing jobs; selecting, from the one or more incomplete computing jobs, a computing job that has the least number of in-flight computing tasks currently being executed in the observability pipeline system; identifying a next computing task from the selected computing job; and dispatching the next computing task to the available worker role. The worker roles execute the computing tasks by applying an observability pipeline process to the event data associated with the respective computing task.

Abstract: In some aspects, an edge-based data collection system discovers, collects, processes, and forwards data in an observability pipeline system. In some implementations, an edge agent of the observability pipeline system runs on a computer node. The edge agent identifies processes running on the computer node; identifies files on the computer node that the processes have opened for writing; accesses log discovery parameters of the observability pipeline system; selects a plurality of files from the identified files according to the log discovery parameters; generates a list of discovered log files that includes a path and a name for each of the plurality of files; adds the list of discovered log files to a list of monitored log files to be monitored by the observability pipeline system; and then monitors the plurality of files to generate input for the observability pipeline system.

Abstract: Techniques and mechanisms are disclosed that enable collection of various types of data from cloud computing services and the generation of various dashboards and visualizations to view information about collections of cloud computing resources. A user can configure collection of data from one or more cloud computing services and view visualizations using an application platform referred to herein as a cloud computing management application. A cloud computing management application further may be configured to generate and cause display of interactive topology map representations of cloud computing resources based on the collected data, where an interactive topology map enables users to view an intuitive visualization of a collection of computing resources, efficiently cause performance of actions with respect to various resources displayed in the topology map, and analyze the collection of resources in ways that are not possible using conventional cloud computing service management consoles.

Abstract: Techniques and mechanisms are disclosed that enable collection of various types of data from cloud computing services and the generation of various dashboards and visualizations to view information about collections of cloud computing resources. A user can configure collection of data from one or more cloud computing services and view visualizations using an application platform referred to herein as a cloud computing management application. A cloud computing management application further may be configured to generate and cause display of interactive topology map representations of cloud computing resources based on the collected data, where an interactive topology map enables users to view an intuitive visualization of a collection of computing resources, efficiently cause performance of actions with respect to various resources displayed in the topology map, and analyze the collection of resources in ways that are not possible using conventional cloud computing service management consoles.

Abstract: Techniques and mechanisms are disclosed that enable collection of various types of data from cloud computing services and the generation of various dashboards and visualizations to view information about collections of cloud computing resources. A user can configure collection of data from one or more cloud computing services and view visualizations using an application platform referred to herein as a cloud computing management application. A cloud computing management application further may be configured to generate and cause display of interactive topology map representations of cloud computing resources based on the collected data, where an interactive topology map enables users to view an intuitive visualization of a collection of computing resources, efficiently cause performance of actions with respect to various resources displayed in the topology map, and analyze the collection of resources in ways that are not possible using conventional cloud computing service management consoles.

Abstract: Techniques and mechanisms are disclosed that enable collection of various types of data from cloud computing services and the generation of various dashboards and visualizations to view information about collections of cloud computing resources. A user can configure collection of data from one or more cloud computing services and view visualizations using an application platform referred to herein as a cloud computing management application. A cloud computing management application further may be configured to generate and cause display of interactive topology map representations of cloud computing resources based on the collected data, where an interactive topology map enables users to view an intuitive visualization of a collection of computing resources, efficiently cause performance of actions with respect to various resources displayed in the topology map, and analyze the collection of resources in ways that are not possible using conventional cloud computing service management consoles.

Abstract: Techniques and mechanisms are disclosed that enable a data collection system to adaptively control collection of data from one or more external data sources. At a high level, adaptively controlling collection of data from external data sources may include collecting performance information related to one or more data collection nodes and, in response to analyzing the collected performance information, adapting rates at which the data collection nodes send data collection requests to external data sources. Data collection performance information generally may include, but is not limited to, network traffic data, error messages generated by external data sources and/or data collection nodes, computing device performance information, and any other types of information related to a data collection node's ability to collect data from external data sources.

Abstract: Techniques and mechanisms are disclosed that enable a data collection system to adaptively control collection of data from one or more external data sources. At a high level, adaptively controlling collection of data from external data sources may include collecting performance information related to one or more data collection nodes and, in response to analyzing the collected performance information, adapting rates at which the data collection nodes send data collection requests to external data sources. Data collection performance information generally may include, but is not limited to, network traffic data, error messages generated by external data sources and/or data collection nodes, computing device performance information, and any other types of information related to a data collection node's ability to collect data from external data sources.

Abstract: Techniques and mechanisms are disclosed that enable a data collection system to adaptively control collection of data from one or more external data sources. At a high level, adaptively controlling collection of data from external data sources may include collecting performance information related to one or more data collection nodes and, in response to analyzing the collected performance information, adapting rates at which the data collection nodes send data collection requests to external data sources. Data collection performance information generally may include, but is not limited to, network traffic data, error messages generated by external data sources and/or data collection nodes, computing device performance information, and any other types of information related to a data collection node's ability to collect data from external data sources.

Abstract: Techniques and mechanisms are disclosed that enable collection of various types of data from cloud computing services and the generation of various dashboards and visualizations to view information about collections of cloud computing resources. A user can configure collection of data from one or more cloud computing services and view visualizations using an application platform referred to herein as a cloud computing management application. A cloud computing management application further may be configured to generate and cause display of interactive topology map representations of cloud computing resources based on the collected data, where an interactive topology map enables users to view an intuitive visualization of a collection of computing resources, efficiently cause performance of actions with respect to various resources displayed in the topology map, and analyze the collection of resources in ways that are not possible using conventional cloud computing service management consoles.

Abstract: Techniques and mechanisms are disclosed that enable a data collection system to adaptively control collection of data from one or more external data sources. At a high level, adaptively controlling collection of data from external data sources may include collecting performance information related to one or more data collection nodes and, in response to analyzing the collected performance information, adapting rates at which the data collection nodes send data collection requests to external data sources. Data collection performance information generally may include, but is not limited to, network traffic data, error messages generated by external data sources and/or data collection nodes, computing device performance information, and any other types of information related to a data collection node's ability to collect data from external data sources.