Abstract: Systems and methods are described for providing previews of deployment of data stream processing instructions sets, sometimes called pipelines, to a stream data processing system. Rather than deploying such an instruction set, which may cause detrimental side effects, previews can be facilitated by conversion of a data stream processing instructions set to a batch query that is applied to an existing data set. An output of the batch query can then be provided to an end user as a preview of output of the data stream processing instructions set, when implemented.

Abstract: Embodiments described herein are directed to efficient content management within a data-processing environment. In some embodiments, an indication of a data source is received via a user interface. A representation of a technology add-on associated with the data source is provided via the user interface. Thereafter, an indication to use the technology add-on associated with the data source is received. The technology add-on, or a portion thereof, may be installed in association with a data processor.

Abstract: Embodiments described herein are directed to efficient content management within a data-processing environment. In some embodiments, data is obtained at an ingest processor. The ingest processor may perform data processing in a cloud computing service and provide the processed data to another component for subsequent indexing or storage. At the ingest processor, advanced data processing is performed to process the data based on an advanced data processing pipeline executed on the ingest processor. The advanced data processing pipeline may use advanced data processing content to perform the advanced data processing. Thereafter, the processed data may be provided to a component of the cloud computing service.

Abstract: Embodiments described herein are directed to efficient content management within a data-processing environment. In some embodiments, data is obtained at an edge processor. The edge processor may perform data processing in proximity to generation of the data and provide the processed data to a centralized computing service for subsequent processing or storage. At the edge processor, basic data processing may be performed to process the data based on a basic data processing pipeline executed on the edge processor. The basic data processing pipeline may use basic data processing content to perform the basic data processing. The processed data may be provided to a component of the centralized computing service.

Abstract: Systems and methods are described for unified processing of indexed and streaming data. A system enables users to query indexed data or specify processing pipelines to be applied to streaming data. In some instances, a user may specify a query intended to be run against indexed data, but may specify criteria that includes not-yet-indexed data (e.g., a future time frame). The system may convert the query into a data processing pipeline applied to not-yet-indexed data, thus increasing the efficiency of the system. Similarly, in some instances, a user may specify a data processing pipeline to be applied to a data stream, but specify criteria including data items outside the data stream. For example, a user may wish to apply the pipeline retroactively, to data items that have already exited the data stream. The system can convert the pipeline into a query against indexed data to satisfy the users processing requirements.

Abstract: Systems and methods are described for unified processing of indexed and streaming data. A system enables users to query indexed data or specify processing pipelines to be applied to streaming data. In some instances, a user may specify a query intended to be run against indexed data, but may specify criteria that includes not-yet-indexed data (e.g., a future time frame). The system may convert the query into a data processing pipeline applied to not-yet-indexed data, thus increasing the efficiency of the system. Similarly, in some instances, a user may specify a data processing pipeline to be applied to a data stream, but specify criteria including data items outside the data stream. For example, a user may wish to apply the pipeline retroactively, to data items that have already exited the data stream. The system can convert the pipeline into a query against indexed data to satisfy the users processing requirements.

Abstract: Systems and methods are described for providing previews of deployment of data stream processing instructions sets, sometimes called pipelines, to a stream data processing system. Rather than deploying such an instruction set, which may cause detrimental side effects, previews can be facilitated by conversion of a data stream processing instructions set to a batch query that is applied to an existing data set. An output of the batch query can then be provided to an end user as a preview of output of the data stream processing instructions set, when implemented.

Abstract: Systems and methods are described for unified processing of indexed and streaming data. A system enables users to query indexed data or specify processing pipelines to be applied to streaming data. In some instances, a user may specify a query intended to be run against indexed data, but may specify criteria that includes not-yet-indexed data (e.g., a future time frame). The system may convert the query into a data processing pipeline applied to not-yet-indexed data, thus increasing the efficiency of the system. Similarly, in some instances, a user may specify a data processing pipeline to be applied to a data stream, but specify criteria including data items outside the data stream. For example, a user may wish to apply the pipeline retroactively, to data items that have already exited the data stream. The system can convert the pipeline into a query against indexed data to satisfy the users processing requirements.

Abstract: Systems and methods are disclosed for implementing dual textual/graphical programming interfaces for programming streaming data processing pipelines. A user interface is provided that enables a user to author a processing pipeline as a query in a query language, and to request conversion of that query language into a graph data structure representation of the pipeline, which can be visualized in the interface. The interface further enables modification of the graph via interaction with the visualization. On request, the modified graph can be converted back into the query language, with the querying being modified to reflect the modifications to the graph.

Abstract: Systems and methods are described for graphical user interfaces that enable users to graphically design nodes and interconnections of pipelines for preprocessing data later ingested into an indexing system. The preprocessing can include receiving messages published to a first publish-subscribe messaging system, the messages containing raw machine data generated by one or more components in an information technology environment, performing one or more processing operations on at least some of the messages to generate preprocessed messages, republishing the preprocessed messages to a second publish-subscribe messaging system, and providing to the indexing system, a subset of the messages from the second publish-subscribe messaging system. Nodes in the pipeline can specify the preprocessing operations, and interconnections in the pipeline can represent data flow through the nodes of the pipeline.

Abstract: Systems and methods are described for graphical user interfaces that enable users to graphically design nodes and interconnections of pipelines for preprocessing data later ingested into an indexing system. The preprocessing can include receiving messages published to a first publish-subscribe messaging system, the messages containing raw machine data generated by one or more components in an information technology environment, performing one or more processing operations on at least some of the messages to generate preprocessed messages, republishing the preprocessed messages to a second publish-subscribe messaging system, and providing to the indexing system, a subset of the messages from the second publish-subscribe messaging system. Nodes in the pipeline can specify the preprocessing operations, and interconnections in the pipeline can represent data flow through the nodes of the pipeline.

Abstract: Systems and methods are disclosed for implementing dual textual/graphical programming interfaces for programming streaming data processing pipelines. A user interface is provided that enables a user to author a processing pipeline as a query in a query language, and to request conversion of that query language into a graph data structure representation of the pipeline, which can be visualized in the interface. The interface further enables modification of the graph via interaction with the visualization. On request, the modified graph can be converted back into the query language, with the querying being modified to reflect the modifications to the graph. In some embodiments, the graph data structure representation and/or the query language representation of the pipeline can be converted into a third representation, such as an Abstract Syntax Tree, that is deployed to an intake system to implement the pipeline.

Abstract: Systems and methods are described for graphical user interfaces that enable users to graphically design nodes and interconnections of pipelines for preprocessing data later ingested into an indexing system. The preprocessing can include receiving messages published to a first publish-sub scribe messaging system, the messages containing raw machine data generated by one or more components in an information technology environment, performing one or more processing operations on at least some of the messages to generate preprocessed messages, republishing the preprocessed messages to a second publish-subscribe messaging system, and providing to the indexing system, a subset of the messages from the second publish-subscribe messaging system. Nodes in the pipeline can specify the preprocessing operations, and interconnections in the pipeline can represent data flow through the nodes of the pipeline.

Abstract: Systems and methods are described for unified processing of indexed and streaming data. A system enables users to query indexed data or specify processing pipelines to be applied to streaming data. In some instances, a user may specify a query intended to be run against indexed data, but may specify criteria that includes not-yet-indexed data (e.g., a future time frame). The system may convert the query into a data processing pipeline applied to not-yet-indexed data, thus increasing the efficiency of the system. Similarly, in some instances, a user may specify a data processing pipeline to be applied to a data stream, but specify criteria including data items outside the data stream. For example, a user may wish to apply the pipeline retroactively, to data items that have already exited the data stream. The system can convert the pipeline into a query against indexed data to satisfy the users processing requirements.

Abstract: Systems and methods are described for graphical user interfaces that enable users to graphically design nodes and interconnections of pipelines for preprocessing data later ingested into an indexing system. The preprocessing can include receiving messages published to a first publish-subscribe messaging system, the messages containing raw machine data generated by one or more components in an information technology environment, performing one or more processing operations on at least some of the messages to generate preprocessed messages, republishing the preprocessed messages to a second publish-subscribe messaging system, and providing to the indexing system, a subset of the messages from the second publish-subscribe messaging system. Nodes in the pipeline can specify the preprocessing operations, and interconnections in the pipeline can represent data flow through the nodes of the pipeline.

Abstract: Systems and methods are described for graphical user interfaces that enable users to graphically design nodes and interconnections of pipelines for preprocessing data later ingested into an indexing system. The preprocessing can include receiving messages published to a first publish-subscribe messaging system, the messages containing raw machine data generated by one or more components in an information technology environment, performing one or more processing operations on at least some of the messages to generate preprocessed messages, republishing the preprocessed messages to a second publish-subscribe messaging system, and providing to the indexing system, a subset of the messages from the second publish-subscribe messaging system. Nodes in the pipeline can specify the preprocessing operations, and interconnections in the pipeline can represent data flow through the nodes of the pipeline.

Abstract: Systems and methods are described for graphical user interfaces that enable users to graphically design nodes and interconnections of pipelines for preprocessing data later ingested into an indexing system. The preprocessing can include receiving messages published to a first publish-subscribe messaging system, the messages containing raw machine data generated by one or more components in an information technology environment, performing one or more processing operations on at least some of the messages to generate preprocessed messages, republishing the preprocessed messages to a second publish-subscribe messaging system, and providing to the indexing system, a subset of the messages from the second publish-subscribe messaging system. Nodes in the pipeline can specify the preprocessing operations, and interconnections in the pipeline can represent data flow through the nodes of the pipeline.