Abstract: A computer implemented processing service for efficient streaming of data input from one or more sources to one or more receivers is disclosed. The processing service includes a schema manager that receives commands from a tenant of the data stream processing service. The processing service includes a processing services gateway that validates the data and sends the data to a transformation processor. The transformation processor receives the data and parses the data into the one or more data formats in accordance with the instructions and sends the data in the one or more data formats to an egress service. The egress service outputs the data in the one or more data formats to the one or more receivers, each receiver receiving the data in a data pipeline in one of the one or more formats.

Abstract: A computer implemented processing service for efficient streaming of data input from one or more sources to one or more receivers is disclosed. The processing service includes a schema manager that receives commands from a tenant of the data stream processing service. The processing service includes a processing services gateway that validates the data and sends the data to a transformation processor. The transformation processor receives the data and parses the data into the one or more data formats in accordance with the instructions and sends the data in the one or more data formats to an egress service. The egress service outputs the data in the one or more data formats to the one or more receivers, each receiver receiving the data in a data pipeline in one of the one or more formats.

Abstract: Techniques for ingesting data streams to a distributed-computing system using a multi-directional data ingestion pipeline are provided. In one embodiment, a method for ingesting data streams includes, at a client gateway, receiving a plurality of messages; assigning the plurality of messages to one or more data streams; obtaining stream routing configurations; and identifying one or more receivers. The method further includes determining whether at least one of the one or more data streams is to be delivered to one or more receivers operating in the first computing environment; and if so, delivering the at least one of the one or more data streams to the one or more receivers operating in the first computing environment. The method further includes delivering the one or more data streams to a data ingress gateway operating in a second computing environment.

Abstract: The present disclosure relates generally to techniques for processing asynchronous queries across multiple data cores including secondary data cores stored in the secondary storage system in a distributed computing system. Data from secondary data cores are recovered to new data cores generated in the data plane in order to be accessible for searching. Using this technique, asynchronous queries are run in parallel allowing the client to dynamically manage the queries and receive notifications when results are available.

Abstract: Techniques for data ingestion by a distributed-computing system are provided. In one embodiment, data received from one or more data sources is processed at a management node of the distributed-computing system. The data is generated by one or more components of an information processing system external to the distributed-computing system. The data is stored at the management plane. The management plane selects, from a plurality of data cores, a data core to ingest the data. The plurality of data cores are stored across one or more data plane containers of a data plane of the distributed-computing system. The management plane processes the data to generate one or more event records corresponding to the data. The one or more event records are sent to the selected data core, which causes the data plane to store the one or more event records in the selected data core.

Abstract: The present disclosure relates generally to techniques for processing asynchronous queries across multiple data cores including secondary data cores stored in the secondary storage system in a distributed computing system. Secondary data cores are recovered in the data plane from secondary storage in order to be accessible for searching. Using this technique, asynchronous queries are run in parallel allowing the client to dynamically manage the queries and receive notifications when results are available.

Abstract: Techniques for data ingestion by a distributed-computing system are provided. In one embodiment, data received from one or more data sources is processed at a management node of the distributed-computing system. The data is generated by one or more components of an information processing system external to the distributed-computing system. The data is stored at the management plane. The management plane selects, from a plurality of data cores, a data core to ingest the data. The plurality of data cores are stored across one or more data plane containers of a data plane of the distributed-computing system. The management plane processes the data to generate one or more event records corresponding to the data. The one or more event records are sent to the selected data core, which causes the data plane to store the one or more event records in the selected data core.

Abstract: Disclosed is a system, method, and computer program product for analyzing sets of data in an efficient manner, such that analytics can be effectively performed over that data. Classification operations can be performed to generate groups of similar log records. This permits classification of the log records in a cohesive and informative manner.

Abstract: In a computer-implemented method for performing alert notification on streaming textual data, streaming textual data is received. A plurality of copies of the streaming textual data is generated, wherein a first copy of the streaming textual data is transmitted to an ingestion service for persistent storage at a data plane and a second copy of the streaming textual data is transmitted to an alert evaluation service for performing alert evaluation on the streaming textual data, such that the ingestion service and the alert evaluation service are performed independently. The streaming textual data is evaluated by the alert evaluation service to detect an alert. Responsive to detecting an alert at the alert evaluation service, a notification of the alert is generated.

Abstract: Techniques for splitting a time-range query into sub-queries for serial execution are provided. In one embodiment, a user query is received requesting items within a time range from a database. The time range is divided into a plurality of time periods within the time range. Sub-queries defining respective time periods of the plurality of time periods are generated from the user query, and a first sub-query is executed. The first sub-query defines a first time period of the plurality of time periods, where the first time period is a most-recent time period or a least-recent time period among the plurality of time periods. If it is determined that a number of items obtained from executing the first sub-query is greater than or equal to a predetermined result target, then the items obtained from executing the first sub-query are provided and subsequent sub-queries are not executed.

Abstract: A computer-implemented method of receiving and incrementally processing hierarchical data in a computing environment. Receiving hierarchical data within a computing environment. The computing environment including a plurality of interrelated components. Incrementally processing the hierarchical data to obtain processed portions. The incremental processing of the portions of hierarchical data able to be initiated with requiring receipt of the entirety of the hierarchical data. Maintaining an indexed representation of previously processed portions of the hierarchical data to prevent unnecessarily processing a same portion of the hierarchical data.

Abstract: Techniques for splitting a query into native query operations and post-processing operations are provided. In one embodiment, a user query including a plurality of search parameters is received. Expressions representing the plurality of search parameters in the user query are determined. A first subset and a second subset of the expressions are identified. Each expression of the first subset corresponds to one of a plurality of native query operations executable by a database platform, and each expression of the second subset corresponds to one of a plurality of predefined post-processing operations not executable by the database platform. A database query is generated based on one or more expressions of the first subset, and a post-processing query is generated based on each expression in the second subset. The database query is executed by the database platform, then the post-processing query is executed on the results from database platform.

Abstract: In a computer-implemented method for performing alert notification on streaming textual data, streaming textual data is received. A plurality of copies of the streaming textual data is generated, wherein a first copy of the streaming textual data is transmitted to an ingestion service for persistent storage at a data plane and a second copy of the streaming textual data is transmitted to an alert evaluation service for performing alert evaluation on the streaming textual data, such that the ingestion service and the alert evaluation service are performed independently. The streaming textual data is evaluated by the alert evaluation service to detect an alert. Responsive to detecting an alert at the alert evaluation service, a notification of the alert is generated.

Abstract: Techniques for data ingestion by a distributed-computing system are provided. In one embodiment, data received from one or more data sources is processed at a management node of the distributed-computing system. The data is generated by one or more components of an information processing system external to the distributed-computing system. The data is stored at the management plane. The management plane selects, from a plurality of data cores, a data core to ingest the data. The plurality of data cores are stored across one or more data plane containers of a data plane of the distributed-computing system. The management plane processes the data to generate one or more event records corresponding to the data. The one or more event records are sent to the selected data core, which causes the data plane to store the one or more event records in the selected data core.

Abstract: In a computer-implemented method for performing alert notification on streaming textual data, streaming textual data is received. A plurality of copies of the streaming textual data is generated, wherein a first copy of the streaming textual data is transmitted to an ingestion service for persistent storage at a data plane and a second copy of the streaming textual data is transmitted to an alert evaluation service for performing alert evaluation on the streaming textual data, such that the ingestion service and the alert evaluation service are performed independently. The streaming textual data is evaluated by the alert evaluation service to detect an alert. Responsive to detecting an alert at the alert evaluation service, a notification of the alert is generated.

Abstract: Disclosed is a system, method, and computer program product for analyzing sets of data in an efficient manner, such that analytics can be effectively performed over that data. Classification operations can be performed to generate groups of similar log records. This permits classification of the log records in a cohesive and informative manner.

Abstract: Techniques for data ingestion by a distributed-computing system are provided. In one embodiment, data received from one or more data sources is processed at a management node of the distributed-computing system. The data is generated by one or more components of an information processing system external to the distributed-computing system. The data is stored at the management plane. The management plane selects, from a plurality of data cores, a data core to ingest the data. The plurality of data cores are stored across one or more data plane containers of a data plane of the distributed-computing system. The management plane processes the data to generate one or more event records corresponding to the data. The one or more event records are sent to the selected data core, which causes the data plane to store the one or more event records in the selected data core.

Abstract: Techniques for splitting a time-range query into sub-queries for serial execution are provided. In one embodiment, a user query is received requesting items within a time range from a database. The time range is divided into a plurality of time periods within the time range. Sub-queries defining respective time periods of the plurality of time periods are generated from the user query, and a first sub-query is executed. The first sub-query defines a first time period of the plurality of time periods, where the first time period is a most-recent time period or a least-recent time period among the plurality of time periods. If it is determined that a number of items obtained from executing the first sub-query is greater than or equal to a predetermined result target, then the items obtained from executing the first sub-query are provided and subsequent sub-queries are not executed.

Abstract: Techniques for ingesting data streams to a distributed-computing system using a multi-directional data ingestion pipeline are provided. In one embodiment, a method for ingesting data streams includes, at a client gateway, receiving a plurality of messages; assigning the plurality of messages to one or more data streams; obtaining stream routing configurations; and identifying one or more receivers. The method further includes determining whether at least one of the one or more data streams is to be delivered to one or more receivers operating in the first computing environment; and if so, delivering the at least one of the one or more data streams to the one or more receivers operating in the first computing environment. The method further includes delivering the one or more data streams to a data ingress gateway operating in a second computing environment.

Abstract: The present disclosure relates generally to techniques for processing asynchronous queries across multiple data cores including secondary data cores stored in the secondary storage system in a distributed computing system. Secondary data cores are recovered in the data plane from secondary storage in order to be accessible for searching. Using this technique, asynchronous queries are run in parallel allowing the client to dynamically manage the queries and receive notifications when results are available.