Abstract: A method includes determining a data set for storage that includes a plurality of uncompressed data slabs in accordance with a serialized data slab ordering. A storage data set that includes a plurality of compressed data slabs is created based on the data set in accordance with the serialized data slab ordering. Each compressed data slab of the plurality of compressed data slabs is generated from at least one corresponding uncompressed data slab of the plurality of uncompressed data slabs that includes a plurality of values based on generating compressed data for each compressed data slab based on the at least one corresponding uncompressed data slab, and generating compression information for each compressed data slab. The storage data set is stored via a plurality of computing devices.

Abstract: A method for execution by a node of a computing device includes determining a plurality of queries for concurrent execution. A plurality of sets of segments required to execute the plurality of queries is determined, and a set of virtual segments in the plurality of sets of segments is determined. A subset of the set of virtual segments is be determined by identifying ones of the set of virtual segments that are required to execute multiple ones of plurality of queries. A locally rebuilt set of rows for each of the set of virtual segments is generated by utilizing a recovery scheme. For each one of the set of virtual segments included in the subset, in response to generating the locally rebuilt set of rows, concurrent partial execution of corresponding multiple ones of the plurality of queries is facilitated.

Abstract: A method for execution by a node of a database system includes receiving a first data block, determining data block processing instruction data for the first data block is not indicated in previously received data blocks, and adding the first data block to an unknown purpose data block cache. Prior to elapsing of a storage time window for storage of the first data block, at least one second data block is received that indicates data block processing instruction data for the first data block. The first data block is processed by applying the data block processing instruction data. A third data block is received and is added to the unknown purpose data block cache. The third data block is removed from the unknown purpose data block cache based on elapsing of a storage time window for storage of the third data block.

Abstract: A method for execution by a database system includes determining physical network topology data of a plurality of nodes of the database system utilized to facilitate query executions. Initial route selection data indicating a set of initial relay nodes for each of a plurality of pairs of the plurality of nodes is based on the physical network topology data by selecting one of a plurality of possible initial route options for the each of the plurality of pairs of the plurality of nodes. The initial route selection data is sent to the plurality of nodes. A query resultant is generated by facilitating execution of a query, via the plurality of nodes, by facilitating routing of data blocks between at least one of the plurality of pairs of the plurality of nodes via a set of intermediate nodes selected based on the initial route selection data in accordance with a dynamic data block routing scheme.

Abstract: A method for execution by a query processing system includes receiving a query request that indicates a query for execution by a database system. A plurality of query execution mode options for execution of the query via the database system can be determined. A plurality of execution success conditions corresponding to the plurality of query execution mode options can be determined. A plurality of resultant correctness guarantee data corresponding to the plurality of query execution mode options based on the plurality of execution success conditions can be generated. Query execution mode selection data can be generated by selecting a query execution mode from the plurality of query execution mode options based having resultant correctness guarantee data that compares favorably to determined resultant correctness requirement data. A resultant for the query can be generated by facilitating execution of the query in accordance with the selected execution mode.

Abstract: A database management system operates by: obtaining a dataset from a memory of the database management system, wherein the dataset includes a set of data records; determining a set of data characteristics of the dataset; determining a list of analytical calculations that are able to be executed on the dataset; determining an analytical calculation characteristic for each analytical calculation of the list of analytical calculations to produce a set of analytical calculation characteristics, wherein the analytical calculation characteristic indicates an estimated execution time to perform the analytical calculation; ranking each analytical calculation of the list of analytical calculations based on the set of data characteristics and the set of analytical calculation characteristics to produce a ranked list of analytical calculations; selecting an analytical calculation from the ranked list of analytical calculations based on the ranking; and executing the selected analytical calculation on the dataset to prod

Abstract: A method includes generating, by a processing entity of a computing system, a plurality of parity blocks from a plurality of lines of data blocks. A first number of parity blocks of the plurality of parity blocks is generated from a first line of data blocks of the plurality of lines of data blocks. The method further includes storing, by the processing entity, the plurality of lines of data blocks in data sections of memory of a cluster of computing devices of the computing system in accordance with a read/write balancing pattern and a restricted file system. The method further includes storing, by the processing entity, the plurality of parity blocks in parity sections of memory of the cluster of computing devices in accordance with the read/write balancing pattern and the restricted file system.

Abstract: A massively parallel database management system includes an index store and a payload store including a set of storage systems of different temperatures. Both the stores each include a list of clusters. Each cluster includes a set of nodes with storage devices forming a group of segments. Nodes and clusters are connected over high speed links. The list of clusters within the payload store includes clusters of different temperatures. The payload store transitions data of a segment group from a higher temperature to a segment group in a lower temperature cluster in parallel. A node moves data of a segment in the higher temperature cluster to a corresponding node's segment in the lower temperature cluster. Once the data is written in the destination segment in the lower temperature cluster, the source segment is freed to store other data. The temperatures include blazing, hot, warm and cold.

Abstract: A method includes receiving, by a first computing entity of a database system, a data set that is organized in rows and columns. The method further includes determining whether to partition the data set based on a parameter associated with the data set. When determining to partition the data set, the method includes determining partitioning parameters for the data set, and partitioning the data set into a plurality of data partitions in accordance with the partitioning parameters. The method further includes determining a first coding scheme for a first data partition and determining a first number of first raw data segments for a first segment group of the first partition based on the coding scheme. The method further includes dividing the first partition to produce the first number of first raw data segments for storage in the database system.

Abstract: A method includes receiving a data set that includes a plurality of data records, where a data record includes a first data field containing a first fixed length data value and a second data field containing a first variable length data value. The method further includes accessing a compression dictionary for the second data field, where a first entry of the compression dictionary includes a key field storing a first fixed length index value and a value field storing the first variable length data value, and where the key field has a smaller data size than the value field. The method further includes creating a storage data set based on the compression dictionary and sending the storage data set to a storage sub-system for storage, where the first variable length data value of the second data field of the data record is replaced with the first fixed length index value.

Abstract: A method for execution, by a first intermediate node of a plurality of nodes in a database management system, includes receiving a message, where the first intermediate node is limited to communication with a subset of nodes of the plurality of nodes, where the message: includes data that is being sent in accordance with a routing path, is a first size, and indicates a next node of the routing path, and where the subset of nodes includes the next node. The method continues by generating a revised message, wherein the revised message includes the data and has a second size. The method continues by determining whether there is at least one additional intermediate node after the next node in the routing path. When yes, determining an optimal route for forwarding the revised message via a node of the subset of nodes, and sending the revised message to the node.

Abstract: A method for execution by a virtual machine core includes retrieving a first pointer by accessing a first buffer of a plurality of buffers stored in allocated memory of a main memory based on assignment of the virtual machine core as a single consumer of the first buffer. First intermediate data in the allocated memory is accessed by utilizing the first pointer. Second intermediate data is generated by executing one of an ordered set of operations on the first intermediate data. The second intermediate data is written to the allocated memory. A second pointer is written to a second buffer of the plurality of buffers based on assignment of the virtual machine core as a single producer of the second buffer.

Abstract: A data input sub-system of a large scale application specific computing system receives a data set that includes a plurality of records, each with a plurality of data fields, and divides the data set into a plurality of data segments. The data input sub-system further restructures records of data segments based on a key field of the plurality of data fields to produce restructured data segments and generates storage instructions for storing the restructured data segments. A data storage and processing sub-system of the computing system interprets the storage instructions to determine resources to engage and stores the restructured data segments using engaged resources. A query and results sub-system of the computing system generates an initial query plan based on a data processing request, optimizes the initial query plan to produce an optimized query plan, and sends the optimized query plan to the data storage and processing sub-system for execution.

Abstract: A method for execution by a record processing and storage system includes receiving a plurality of records and generating a plurality of pages that include the plurality of records in accordance with a row-based format. The plurality of pages is stored via a page storage system. Segment generation determination data is generated based on storage utilization data of the page storage system. A plurality of segments is generated from the plurality of pages that include the plurality of records in a column-based format based on the segment generation determination data indicating segments be generated. The plurality of segments is stored via a segment storage system.

Abstract: A method for execution by a record processing and storage system includes assigning each of a plurality of key space sub-intervals of a cluster key domain to a corresponding one of a plurality of processing core resources, and generating a plurality of segments from the set of records via the plurality of processing core resources. Each processing core resource in the plurality of processing core resources generates a subset of the plurality of segments by identifying a proper subset of the set of records based on having cluster key values included in a corresponding one of the plurality of key space sub-intervals, and by generating the subset of the plurality of segments to include the proper subset of the set of records.

Abstract: A method for execution by a record processing and storage system includes receiving a plurality of records and corresponding row numbers. Pages are generated from the received records. Page metadata is generated for each page that includes row number span data based on row numbers of the records included in each page. Pairs of pages are identified in the plurality of pages based on having row number span data in their page metadata that include a row number span overlap. For each pair of pages, row number span data is updated for a first page in the pair by removing the row number span overlap with a second page in the pair. Reads of pages are performed based on their row number span data. Only records of each first page of each pair of pages having row numbers that are within the updated row number span data are read.

Abstract: A method for execution by a node in a database system includes determining to send a data block from a node to a destination node of a database system in accordance with execution of a query. An initial route to the destination node that includes a predefined number of relay nodes in a sequential relay ordering is determined. The second relay node in the sequential relay ordering is identified, and a set of intermediate node options that are each capable of receiving data from the node via a direct connection and capable of sending data to the second relay node via a direct connection is identified. One intermediate node is selected from the set of intermediate node options, and the data block is transmitted from the node to the one intermediate node to facilitate sending of the data to the destination node via exactly the predefined number of relay nodes.

Abstract: A method for execution by a query processing system includes receiving a query request that indicates a query for execution by a database system. A plurality of query execution mode options for execution of the query via the database system can be determined. A plurality of execution success conditions corresponding to the plurality of query execution mode options can be determined. A plurality of resultant correctness guarantee data corresponding to the plurality of query execution mode options based on the plurality of execution success conditions can be generated. Query execution mode selection data can be generated by selecting a query execution mode from the plurality of query execution mode options based having resultant correctness guarantee data that compares favorably to determined resultant correctness requirement data. A resultant for the query can be generated by facilitating execution of the query in accordance with the selected execution mode.

Abstract: A method includes a computing device receiving a sort request regarding data of a table. The method further includes the computing device determining probability traits of the data. The method further includes the computing device dividing the sort request into sub-sort requests based on the probability traits. The method further includes the computing device identifying processing core resources to perform the sub-sort requests based on the probability traits. The method further includes the computing device allocating the sub-sort requests to the identified processing core resources in accordance with the probability traits. The method further includes the computing device allocating data portions to the identified processing core resources in accordance with the probability traits. The method further includes the computing core resources executing allocated sub-sort requests on corresponding divided data portions to produce sorted data portions.

Abstract: A method includes receiving, by a first computing entity of a database system, data that is organized in rows and columns. The method further includes determining, by the first computing entity, one or more key columns from the columns based on a desired sort criteria for the data. The method further includes sorting, by the first computing entity, other columns of the columns based on the one or more key columns to produce sorted other columns. The method further includes sending, by the first computing entity, the one or more key columns to a second computing entity of the database system for storage in a first storage location associated with the second computing entity. The method further includes sending, by the first computing entity, the sorted other columns to the second computing entity for storage in a second storage location associated with the second computing entity.