Abstract: A database system receives a database language query specifying a grouping clause and one or more aggregate result values. The database system generates a pre-aggregation table based on the query. The database system generates an ingest query configured to generate data for populating the pre-aggregation table and a completion query configured to processes data stored in the pre-aggregation table. The database system periodically receives a set of input records and executes the ingest query to generate a set of pre-aggregation records. The set of pre-aggregation records are combined with the pre-aggregation table. The database system executes the completion query on the data stored in the pre-aggregation table to generate a result set and sends the generated result set to the requestor of the results.

Abstract: A system stores associates a database query with a name and stores versions of the database query. The system allows applications to execute a specific version of the database query by specifying the name of the database query and a version identifier. For example, the database query may be exposed as an endpoint of a REST API and invoked using calls to the REST API. This allows the applications to be designed without including the query definition within the code of the application. Continuous delivery of database query is performed by creating a new version of the database query that are tested using applications in a test environment. Once the new version of a database query meets test criteria, the new version of the database query is executed by applications running in production environment by sending API requests that identify the query name and the new version.

Abstract: A system stores associates a database query with a name and stores versions of the database query. The system allows applications to execute a specific version of the database query by specifying the name of the database query and a version identifier. For example, the database query may be exposed as an endpoint of a REST API and invoked using calls to the REST API. This allows the applications to be designed without including the query definition within the code of the application. Continuous delivery of database query is performed by creating a new version of the database query that are tested using applications in a test environment. Once the new version of a database query meets test criteria, the new version of the database query is executed by applications running in production environment by sending API requests that identify the query name and the new version.

Abstract: A real-time analytical database system allows queries on data obtained from a transactional system that is frequently updated by the transactional system. The real-time analytical database system receives change logs from the transactional system. The change logs describe operations performed by the transactional system. The real-time analytical database system stores a plurality of indexes in a key-value store, each index comprising key-value pairs. A key of a key-value pair is obtained by combining field data describing a field of a document. The real-time analytical database system determines modifications to the key-value pairs of the key-value store corresponding to the operations of the transactional system as represented by the received change logs. The real-time analytical database system receives and processes queries using the plurality of indexes.

Abstract: A search system processes queries for accessing information stored in documents. A document comprises fields. The search system stores a plurality of indexes in a key-value store. Each index comprises key-value pairs. A key of a key-value pair is obtained by combining field data describing a field of a document. The value of each field is stored as an individual key-value in the key-value store. The search system receives a query requesting information stored in documents and specifying a search criteria. The search system builds a key-expression based on the search criteria and uses one or more indexes to find key-value pairs matching the key-expression. The search system finds the requested information based on the matching key-value pairs and provides the requested information to the query source.

Abstract: Techniques for facilitating and accelerating log data processing are disclosed herein. The front-end clusters generate a large amount of log data in real time and transfer the log data to an aggregating cluster. When the aggregating cluster is not available, the front-clusters write the log data to local filers and send the data when the aggregating cluster recovers. The aggregating cluster is designed to aggregate incoming log data streams from different front-end servers and clusters. The aggregating cluster further sends the aggregated log data stream to centralized NFS filers or a data warehouse cluster. The local filers and the aggregating cluster stage the log data for access by applications, so that the applications do not wait until the data reach the centralized NFS filers or data warehouse cluster.

Abstract: Techniques for facilitating and accelerating log data processing by splitting data streams are disclosed herein. The front-end clusters generate large amount of log data in real time and transfer the log data to an aggregating cluster. The aggregating cluster is designed to aggregate incoming log data streams from different front-end servers and clusters. The aggregating cluster further splits the log data into a plurality of data streams so that the data streams are sent to a receiving application in parallel. In one embodiment, the log data are randomly split to ensure the log data are evenly distributed in the split data streams. In another embodiment, the application that receives the split data streams determines how to split the log data.

Abstract: Techniques for facilitating and accelerating log data processing are disclosed herein. The front-end clusters generate a large amount of log data in real time and transfer the log data to an aggregating cluster. When the aggregating cluster is not available, the front-clusters write the log data to local filers and send the data when the aggregating cluster recovers. The aggregating cluster is designed to aggregate incoming log data streams from different front-end servers and clusters. The aggregating cluster further sends the aggregated log data stream to centralized NFS filers or a data warehouse cluster. The local filers and the aggregating cluster stage the log data for access by applications, so that the applications do not wait until the data reach the centralized NFS filers or data warehouse cluster.

Abstract: Techniques for facilitating and accelerating log data processing are disclosed herein. The front-end clusters generate a large amount of log data in real time and transfer the log data to an aggregating cluster. When the aggregating cluster is not available, the front-clusters write the log data to local filers and send the data when the aggregating cluster recovers. The aggregating cluster is designed to aggregate incoming log data streams from different front-end servers and clusters. The aggregating cluster further sends the aggregated log data stream to centralized NFS filers or a data warehouse cluster. The local filers and the aggregating cluster stage the log data for access by applications, so that the applications do not wait until the data reach the centralized NFS filers or data warehouse cluster.

Abstract: Techniques for facilitating and accelerating log data processing are disclosed herein. The front-end clusters generate a large amount of log data in real time and transfer the log data to an aggregating cluster. When the aggregating cluster is not available, the front-clusters write the log data to local filers and send the data when the aggregating cluster recovers. The aggregating cluster is designed to aggregate incoming log data streams from different front-end servers and clusters. The aggregating cluster further sends the aggregated log data stream to centralized NFS filers or a data warehouse cluster. The local filers and the aggregating cluster stage the log data for access by applications, so that the applications do not wait until the data reach the centralized NFS filers or data warehouse cluster.

Abstract: A method of operation of a storage control system includes: configuring a state change policy on a data server, the state change policy including an online duration for a storage device; activating the storage device based on the state change policy; mounting the storage device based on the state change policy; and scheduling a filesystem maintenance task to be performed on the storage device based on the state change policy.