Abstract: A database system is operable to determine a query for execution that requires access to a set of records stored by the database system. A first proper subset of the set of records are accessed in conjunction with executing the query by reading exactly one of a set of multiple replicas of each record of the first proper subset of the set of records from the replication-based storage system. A second proper subset of the set of records are accessed in conjunction with executing the query by reading at least one redundancy-coded segment from the redundancy-coding based storage system. A final resultant for the query is generated by performing at least one query operation on the first proper subset of the set of records and the second proper subset of the set of records in conjunction with executing the query.

Abstract: A computing device of a database system includes a plurality of processing modules, a computing device operating system, and an application specific operating system. The application specific operating system includes at least one custom instruction set that configures operation of a configurable set of processing modules of the plurality of processing modules based on generating, for each processing module of the configurable set of processing modules, a corresponding configuration signal indicating a selected instruction set of either the computing device operating system or the application specific operating system. Each processing module of the configurable set of processing modules operates in accordance with the selected instruction set based on the corresponding configuration signal.

Abstract: A method includes generating a predictive model mapping batch size to estimated memory utilization based on a plurality of prior batch sizes applied for performing a corresponding plurality of prior data format conversions, and further based on a plurality of prior actual memory utilization occurring in performing the corresponding plurality of prior data format conversions. A next batch size is automatically selected for performance of a subsequent data format conversion based on the predictive model and a current memory availability. A batch of data for performance of the subsequent data format conversion is identified based on the batch of data having the next batch size. The subsequent data format conversion is performed the batch of data, and an actual amount of memory utilized in performing the subsequent data format conversion is determined. The predictive model is updated based on the next batch size and the actual amount of memory utilized.

Abstract: A query processing system operates by determining a query pricing scheme; receiving a query request from a requesting entity that indicates access to a subset of a plurality of records in a database system; generating query cost data based on the subset of the plurality of records by utilizing the query pricing scheme; and transmitting the query cost data to the requesting entity.

Abstract: A computing device comprises a plurality of nodes and a plurality of operating system layers. The plurality of operating system layers includes a local database operating system and a sub-system database operating system. The plurality of nodes utilize the local database operating system to execute at least one database operation independently of other ones of the plurality of nodes. The computing device utilizes the sub-system database operating system in conjunction with other ones of a plurality of computing devices of at least one sub-system to facilitate execution of at least one sub-system operation of the at least one sub-system.

Abstract: A method includes generating a plurality of parity blocks from a plurality of lines of data blocks. The plurality of lines of data blocks are stored in data sections of memory of a cluster of computing devices of the computing system by distributing storage of individual data blocks of the plurality of lines of data blocks among unique data sections of the cluster of computing devices. The plurality of parity blocks are stored in parity sections of memory of the cluster of computing devices by distributing storage of parity blocks of the plurality of parity blocks among unique parity sections of the cluster of computing devices.

Abstract: A method for execution by a temporary ingress storage system includes receiving a set of records to be processed for long-term storage. The set of records are temporarily stored in a set of memory resources of the temporary ingress storage system during a first temporal period. Execution of a query is facilitated by accessing a subset of the set of records from at least one memory resource of the set of memory resources during the first temporal period. The set of records are processed to generate a set of segments for long-term storage. Migration of the set of records from the temporary ingress storage system to a long-term storage system for during a second temporal period that begins after the first temporal period has elapsed by sending the set of records to the long-term storage system.

Abstract: A networked database management system (DBMS) is disclosed. In particular, the disclosed DBMS includes a plurality of nodes, one of which is elected as a designated leader. The designated leader is elected using a consensus algorithm, such as tabulated random votes, RAFT or PAXOS. The designated leader is responsible for managing open coding lines, and determining when to close an open coding line.

Abstract: A method for execution by a query processing system includes receiving a query request from a requesting entity. Query cost data is generated based on the query request by utilizing a query pricing scheme. Minimum query cost compliance data is generated for the query request based on determining whether the minimum query cost data complies with a minimum query cost rule. When the minimum query cost compliance data indicates the minimum query cost data complies with the minimum query cost rule a query result is generated based on facilitating execution of the query by executing at least one query function of the query against a database system, and the query result is transmitted to the requesting entity. When the minimum query cost compliance data indicates the query cost data does not comply with the minimum query cost rule, the query result is not transmitted to the requesting entity.

Abstract: A method for execution by a node of a database system includes receiving a first data block, determining data block processing instruction data for the first data block is not indicated in previously received data blocks, and adding the first data block to an unknown purpose data block cache. Prior to elapsing of a storage time window for storage of the first data block, at least one second data block is received that indicates data block processing instruction data for the first data block. The first data block is processed by applying the data block processing instruction data. A third data block is received and is added to the unknown purpose data block cache. The third data block is removed from the unknown purpose data block cache based on elapsing of a storage time window for storage of the third data block.

Abstract: A massively parallel database management system includes an index store and a payload store including a set of storage systems of different temperatures. Both the index store and the storage system each include a list of clusters. Each cluster includes a set of nodes with storage devices forming a group of segments. Nodes and clusters are connected over high speed links. Each cluster receives data and splits the data into data rows based on a predetermined size. The data rows are randomly and evenly distributed between all nodes of the cluster.

Abstract: A method for execution by a query processing system includes determining a query request that indicates a plurality of operators, where the plurality of operators includes at least one relational algebra operator and further includes at least one non-relational operator. A query operator execution flow is generated from the query request that indicates a serialized ordering of the plurality of operators. A query resultant of the query is generated by facilitating execution of the query via a set of nodes of a database system that each perform a plurality of operator executions in accordance with the query operator execution flow, where a subset of the set of nodes each execute at least one operator execution corresponding to the at least one non-relational operator in accordance with the execution of the query.

Abstract: A processing system of a node is operable to determine at least one query for execution, and determine a set of segments that include a set of rows required to execute the at least one query. At least one virtual segment in the set of segments is identified, and at least two data portions in the at least one virtual segment are identified. A locally rebuilt set of rows for the at least two data portions is generated by utilizing a recovery scheme. Query resultant data is generated based on performing execution of the at least one query by utilizing the locally rebuilt set of rows.

Abstract: A large highly parallel database management system includes thousands of nodes storing huge volume of data. The database management system includes multiple query optimizers for determining low cost execution plans for queries. The database management system is adapted to receive a data query. An execution plan generator component of the database management system generates an initial execution plan for the query. The initial execution plan is fed as input to more than one query optimizers. Each optimizer starts with the initial execution plan, generates alternative execution plans, and determines a satisfactory execution plan that incurs the lowest cost. The database management system compares the selected execution plans by the optimizers and selects one with the lowest cost. The multiple query optimizers run in parallel.

Abstract: A method for execution by a computer of a database management system includes obtaining a dataset that includes a set of data records, where the data set is associated with a set of data characteristics. The method further includes ranking analytical calculations of a list of analytical calculations that are able to be executed on the dataset based on the set of data characteristics and a set of analytical calculation characteristics associated with the list of analytical calculations to produce a ranked list of analytical calculations. The method further includes selecting an analytical calculation from the ranked list of analytical calculations based on the ranking. The method further includes executing the selected analytical calculation on the dataset to produce an analytical calculation result.

Abstract: A method for execution by a node of a computing device includes determining a plurality of queries for concurrent execution. A plurality of sets of segments required to execute the plurality of queries is determined, and a set of virtual segments in the plurality of sets of segments is determined. A subset of the set of virtual segments is be determined by identifying ones of the set of virtual segments that are required to execute multiple ones of plurality of queries. A locally rebuilt set of rows for each of the set of virtual segments is generated by utilizing a recovery scheme. For each one of the set of virtual segments included in the subset, in response to generating the locally rebuilt set of rows, concurrent partial execution of corresponding multiple ones of the plurality of queries is facilitated.

Abstract: A node is operable to determine to send at least on data block to a destination node. A second relay node of an initial route that includes exactly a predefined number of relay nodes for relaying at least one data block from the node to a destination node is identified. A set of intermediate node options is identified by identifying nodes of the database system capable of receiving the at least one data block from the node and further capable of sending the at least one data block to the second relay node. Intermediate node selection data is generated by selecting one intermediate node from the set of intermediate node options. The at least one data block is transmitted from the node to the one intermediate node via the first direct connection to facilitate sending of the at least one data block from the node to the destination node.

Abstract: A method for execution by a node includes determining first level assignment information indicating assignment to a first subset of levels of a plurality of levels and determining second level assignment information indicating assignment to a second subset of levels of the plurality of levels. The second level assignment information indicates a second compute sequence number that is different from a first compute sequence number indicated by the first level assignment information. A query indicating the first compute sequence number is determined for execution. Execution of at least one query operator of the query is performed in accordance with a level included in the first subset of levels based on determining to utilize the first level assignment information based on the query indicating the first compute sequence number.

Abstract: A large highly parallel database management system includes thousands of nodes storing huge volume of data. The database management system includes multiple query optimizers for determining low cost execution plans for queries. The database management system is adapted to receive a data query. An execution plan generator component of the database management system generates an initial execution plan for the query. The initial execution plan is fed as input to more than one query optimizers. Each optimizer starts with the initial execution plan, generates alternative execution plans, and determines a satisfactory execution plan that incurs the lowest cost. The database management system compares the selected execution plans by the optimizers and selects one with the lowest cost. The multiple query optimizers run in parallel.

Abstract: A method for managing data storage and retrieval and operated within a database management system includes retrieving a set of confidence data records from memory of the database management system, where a confidence data record of the set of confidence data records includes a set of data columns and one or more confidence columns populated with one or more data confidence values associated with data within one or more data columns of the set of data columns. The method further includes analyzing the set of confidence data records to produce a result, where the result includes a confidence level that is based on corresponding ones of the one or more data confidence values associated with the data within one or more data columns of the set of data columns.