Abstract: Systems and methods are described for distributed processing a query in a first query language utilizing a query execution engine intended for single-device execution. While distributed processing provides numerous benefits over single-device processing, distributed query execution engines can be significantly more difficult to develop that single-device engines. Embodiments of this disclosure enable the use of a single-device engine to support distributed processing, by dividing a query into multiple stages, each of which can be executed by multiple, concurrent executions of a single-device engine. Between stages, data can be shuffled between executions of the engine, such that individual executions of the engine are provided with a complete set of records needed to implement an individual stage. Because single-device engines can be significantly less difficult to develop, use of the techniques described herein can enable a distributed system to rapidly support multiple query languages.

Abstract: Systems and methods are described for distributed processing a query in a first query language utilizing a query execution engine intended for single-device execution. While distributed processing provides numerous benefits over single-device processing, distributed query execution engines can be significantly more difficult to develop that single-device engines. Embodiments of this disclosure enable the use of a single-device engine to support distributed processing, by dividing a query into multiple stages, each of which can be executed by multiple, concurrent executions of a single-device engine. Between stages, data can be shuffled between executions of the engine, such that individual executions of the engine are provided with a complete set of records needed to implement an individual stage. Because single-device engines can be significantly less difficult to develop, use of the techniques described herein can enable a distributed system to rapidly support multiple query languages.

Abstract: Systems and methods are described for assigning a processing task from one component of a data intake and query system to a different component of the data intake and query system. As part of processing a query, the system can determine that a particular processing task is to be executed by a particular component of the data intake and query system. Based on the characteristics of the component that is to execute the processing task, the system can assign the task or a supplemental task to one or more other components of the data intake and query system.

Abstract: Systems and methods are described for partitioning and reducing records at ingest of a worker node. The worker node receives chunks of data from one or more indexers of a data intake and query system based on the execution of a query by the data intake and query system. The worker node assigns records to different record groups based on the content of the records. The system also assigns the record to a partition of a group of partitions. Record data of the records in a particular partition is combined. The system processes the partitions based on the query.

Abstract: Systems and methods are described for distributed processing a query in a first query language utilizing a query execution engine intended for single-device execution. While distributed processing provides numerous benefits over single-device processing, distributed query execution engines can be significantly more difficult to develop that single-device engines. Embodiments of this disclosure enable the use of a single-device engine to support distributed processing, by dividing a query into multiple stages, each of which can be executed by multiple, concurrent executions of a single-device engine. Between stages, data can be shuffled between executions of the engine, such that individual executions of the engine are provided with a complete set of records needed to implement an individual stage. Because single-device engines can be significantly less difficult to develop, use of the techniques described herein can enable a distributed system to rapidly support multiple query languages.

Abstract: Systems and methods are described for assigning a processing task from one component of a data intake and query system to a different component of the data intake and query system. As part of processing a query, the system can determine that a particular processing task is to be executed by a particular component of the data intake and query system. Based on the characteristics of the component that is to execute the processing task, the system can assign the task or a supplemental task to one or more other components of the data intake and query system.

Abstract: Systems and methods are described for partitioning and reducing records at ingest of a worker node. The worker node receives chunks of data from one or more indexers of a data intake and query system based on the execution of a query by the data intake and query system. The worker node assigns records to different record groups based on the content of the records. The system also assigns the record to a partition of a group of partitions. Record data of the records in a particular partition is combined. The system processes the partitions based on the query.

Abstract: Embodiments of the invention provide a method, system, and media for determining search results based on a query. One embodiment of the method includes receiving an initial query, inspecting an initial set of query-related information that is associated with the query, which is the fruit of analyzing aggregated user-interaction data, which includes information related to how users have previously interacted with former search results that were presented in response to the query. This information includes prior metadata associated with the former search results. Embodiments further include presenting an initial set of search results based on the initial set of query-related information, gathering current user-interaction data, and updating the initial set of query-related information based on the current user-interaction data. In this way, an embodiment of the invention helps, among other things, map a semantic meaning of a query to results that bring about a satisfying user experience.

Abstract: Embodiments of the invention provide a method, system, and media for determining search results based on a query. One embodiment of the method includes receiving an initial query, inspecting an initial set of query-related information that is associated with the query, which is the fruit of analyzing aggregated user-interaction data, which includes information related to how users have previously interacted with former search results that were presented in response to the query. This information includes prior metadata associated with the former search results. Embodiments further include presenting an initial set of search results based on the initial set of query-related information, gathering current user-interaction data, and updating the initial set of query-related information based on the current user-interaction data. In this way, an embodiment of the invention helps, among other things, map a semantic meaning of a query to results that bring about a satisfying user experience.