Abstract: Data intake and query system (DIQS) instances supporting applications including lower-tier, focused, work group oriented applications, are tailored to display the metrics for the needs of the user. An interface caused by operation of an entity monitoring system (EMS) operating in conjunction with the lower-tier DIQS displays the monitored entities as individual representations. The user selects a metric and a metric threshold. The EMS causes a display of an interface having a representation for each monitored entity. Each representation includes a metric value and indicates an entity status based on the metric value and the threshold. The user can dynamically change the threshold on the interface for easy visualization of aggregation of monitored entities to determine the performance of the infrastructure. The interface also provides the user with the ability to select an entity and click through to the entity analysis workspace for more detailed information.

Abstract: One or more lower-tier system monitoring components are installed and operated prior to installing a higher-tier system monitoring component. A lower-tier system may be an individual server, network device, or local area network. A higher-tier system may include an enterprise or organization wide network or service that includes at least a part of the lower-tier system. Once the higher-tier system monitoring component is installed, the higher-tier and lower-tier system monitoring components use an interface to operate with one another to form a single larger instance of an organization wide monitoring system. The combination of the higher-tier system monitoring component and the one or more lower-tier system operating components performs monitoring aspects of the overall information technology environment based at least in part on machine data produced and made searchable to provide monitoring results.

Abstract: A service monitoring system (SMS) produces key performance indicator (KPI) scores that indicate the performance of a service. To produce the KPI scores, the SMS may process the data for a large number of machine entities that perform the service. This data can be processed on a per-entity basis to produce a per-entity KPI score representing the contribution of a particular machine to the overall KPI. The per-entity KPI scores can be transformed to statistical representations which can be visualized as a distribution stream graph. The visualization may be presented with interactive aspects. Automatic entity definitions may also be generated based on content derived from the processed data.

Abstract: Automated discovery of relationships between entities within an IT environment. A technique is performed by a relationship module that performs a discovery search for entity relationships to produce a set of relationship search results. The relationship module then generates a set of relationship definitions from the set of relationship search results which are stored to a relationship collection in a data store. A technique for automatically updating entity and relationship definitions and removing outdated entity and relationship definitions stored to a data store. An update module automatically updates entity and relationship definitions at predetermined time intervals. The update history in each definition is also modified to reflect the update process. A retire module automatically removes outdated definitions using the update history in each definition.

Abstract: Systems and methods for rule-based data stream processing by data collection, indexing, and visualization systems. An example method includes: receiving, by the computer system, an input data stream comprising raw machine data; processing the raw machine data by a data processing pipeline that produces transformed machine data, wherein the data processing pipeline comprises an ordered plurality of pipeline stages, wherein a pipeline stage of the ordered plurality of pipeline stages applies a rule of a set of rules to an input of the pipeline stage, wherein the rule specifies an action to be performed on the input of the pipeline stage responsive to evaluating a conditional expression applied to the input of the pipeline stage, wherein the action generates an output of the pipeline stage, and wherein the rule is selected based on a source type associated with the input data stream; and supplying the transformed machine data to a data collection, indexing, and visualization system.

Abstract: Systems and methods are disclosed for processing and executing queries in a data intake and query system. The data intake and query system receives a query identifying a set of data to be processed and a manner of processing the set of data. The data intake and query system parses the query and uses a metadata catalog to dynamically identify configuration parameters of datasets and/or rules associated with the query. The identified configuration parameters are communicated to a query processing component of the data intake and query system for use in executing the query.

Abstract: Systems and methods are disclosed for annotating a metadata catalog in a data intake and query system based on a query received by the data intake and query system. The metadata catalog can store information about datasets associated with the data intake and query system, including dataset configuration records of the datasets, which can be used to process queries for execution by the data intake and query system. The data intake and query system can receive a query identifying a set of data to be processed and a manner of processing the set of data. The data intake and query system can parse the query to identify datasets and/or data fields associated with the query. Based on the identified datasets and/or fields, the data intake and query system can generate one or more annotations, and use the annotations to update the metadata catalog.

Abstract: Systems and methods are disclosed for flexibly applying a query term to heterogeneous data. A query system can receive a query that includes a data-determinant query term. As the system executes the query it can generate interim search results. As the system query processes the interim search results based on the query, it can apply the data-determinant query term to records of the interims search results based on the structure of the records.

Abstract: An example method of implementing a monitoring interface for an information technology environment comprises: identifying machine data reflecting activity in the information technology environment comprising a plurality of entities providing a service; executing a search query to derive, from the machine data, a value of a key performance indicator (KPI) reflecting performance of the service; and causing display of a monitoring interface including: an identifier of the service, a color coded indication of a state of the KPI, and a visual representation of time series data associated with the service.

Abstract: Systems and methods are disclosed for annotating a metadata catalog in a data intake and query system based on a detected change to the metadata catalog. The metadata catalog can store information about datasets associated with the data intake and query system, including dataset configuration records of the datasets, which can be used to process queries for execution by the data intake and query system. Based on a detected change to the metadata catalog, such as a manual or system annotation, the data intake and query system can generate additional annotations for the metadata catalog and use the annotations to update the metadata catalog.

Abstract: Systems and methods are disclosed for annotating a metadata catalog in a data intake and query system based on a query received by the data intake and query system. The metadata catalog can store information about datasets associated with the data intake and query system, including dataset configuration records of the datasets, which can be used to process queries for execution by the data intake and query system. The data intake and query system can receive a query identifying a set of data to be processed and a manner of processing the set of data. The data intake and query system can parse the query to identify datasets and/or data fields associated with the query. Based on the identified datasets and/or fields, the data intake and query system can generate one or more annotations, and use the annotations to update the metadata catalog.

Abstract: Systems and methods for rule-based data stream processing by data collection, indexing, and visualization systems. An example method includes: receiving, by the computer system, an input data stream comprising raw machine data; processing the raw machine data by a data processing pipeline that produces transformed machine data, wherein the data processing pipeline comprises an ordered plurality of pipeline stages, wherein a pipeline stage of the ordered plurality of pipeline stages applies a rule of a set of rules to an input of the pipeline stage, wherein the rule specifies an action to be performed on the input of the pipeline stage responsive to evaluating a conditional expression applied to the input of the pipeline stage, wherein the action generates an output of the pipeline stage, and wherein the rule is selected based on a source type associated with the input data stream; and supplying the transformed machine data to a data collection, indexing, and visualization system.

Abstract: One or more lower-tier system monitoring components are installed and operated prior to installing a higher-tier system monitoring component. A lower-tier system may be an individual server, network device, or local area network. A higher-tier system may include an enterprise or organization wide network or service that includes at least a part of the lower-tier system. Once the higher-tier system monitoring component is installed, the higher-tier and lower-tier system monitoring components use an interface to operate with one another to form a single larger instance of an organization wide monitoring system. The combination of the higher-tier system monitoring component and the one or more lower-tier system operating components performs monitoring aspects of the overall information technology environment based at least in part on machine data produced and made searchable to provide monitoring results.

Abstract: Systems and methods are disclosed for annotating a metadata catalog in a data intake and query system based on a query received by the data intake and query system. The metadata catalog can store information about datasets associated with the data intake and query system, including dataset configuration records of the datasets, which can be used to process queries for execution by the data intake and query system. The data intake and query system can receive a query identifying a set of data to be processed and a manner of processing the set of data. The data intake and query system can parse the query to identify datasets and/or data fields associated with the query. Based on the identified datasets and/or fields, the data intake and query system can generate one or more annotations, and use the annotations to update the metadata catalog.

Abstract: Automated discovery of relationships between entities within an IT environment. A technique is performed by a relationship module that performs a discovery search for entity relationships to produce a set of relationship search results. The relationship module then generates a set of relationship definitions from the set of relationship search results which are stored to a relationship collection in a data store. A technique for automatically updating entity and relationship definitions and removing outdated entity and relationship definitions stored to a data store. An update module automatically updates entity and relationship definitions at predetermined time intervals. The update history in each definition is also modified to reflect the update process. A retire module automatically removes outdated definitions using the update history in each definition.

Abstract: Systems and methods are disclosed for processing and executing queries in a data intake and query system. The data intake and query system receives a query identifying a set of data to be processed and a manner of processing the set of data. The data intake and query system parses the query and uses a metadata catalog to dynamically identify configuration parameters of datasets and/or rules associated with the query. The identified configuration parameters are communicated to a query processing component of the data intake and query system for use in executing the query.

Abstract: Data intake and query system (DIQS) instances supporting applications including lower-tier, focused, work group oriented applications, are tailored to display the metrics for the needs of the user. An interface caused by operation of an entity monitoring system (EMS) operating in conjunction with the lower-tier DIQS displays the monitored entities as individual representations. The user selects a metric and a metric threshold. The EMS causes a display of an interface having a representation for each monitored entity. Each representation includes a metric value and indicates an entity status based on the metric value and the threshold. The user can dynamically change the threshold on the interface for easy visualization of aggregation of monitored entities to determine the performance of the infrastructure. The interface also provides the user with the ability to select an entity and click through to the entity analysis workspace for more detailed information.

Abstract: One or more lower-tier system monitoring components are installed and operated prior to installing a higher-tier system monitoring component. A lower-tier system may be an individual server, network device, or local area network. A higher-tier system may include an enterprise or organization wide network or service that includes at least a part of the lower-tier system. Once the higher-tier system monitoring component is installed, the higher-tier and lower-tier system monitoring components use an interface to operate with one another to form a single larger instance of an organization wide monitoring system. The combination of the higher-tier system monitoring component and the one or more lower-tier system operating components performs monitoring aspects of the overall information technology environment based at least in part on machine data produced and made searchable to provide monitoring results.

Abstract: Systems and methods are disclosed for processing and executing queries in a data intake and query system. The data intake and query system receives a first query identifying a set of data to be processed and a manner of processing the set of data. The data intake and query system parses the first query and uses a dataset association records of a metadata catalog to dynamically identify one or more datasets associated with the query and generate a second query. The data intake and query system executes the second query.

Abstract: An automatic service monitor in an information-technology environment performs regular search queries against generated machine data to derive performance measurements. The information technology environment is defined in terms of services provided by entities, and the performance measurements are defined as key performance indicators (KPIs) of the services. Generated machine data used by the search queries pertain to the entities performing the service. Definitional information for the services, entities, and KPIs is administered by a user to control the operation of the service monitor. Various aspects of such definitional information as well as related performance measurement information may be presented in a unified console display tailored to, and organized around, a particular entity. The console display may serve as a central launch point by supporting user interaction to navigate to other specialized monitoring interfaces.