Abstract: A store and compute sub-system of a database system includes a computing cluster that is operable to receive a plurality of tasks. The computing cluster is further operable to execute, in a concurrent manner, the plurality of tasks. For a first task, a first lead computing device of the computing cluster is operable to: generate a plurality of first partial tasks based on the first task and allocate the plurality of first partial tasks to the plurality of computing devices, wherein the plurality of computing devices executes the plurality of first partial tasks. For a second task, a second lead computing device of the computing cluster is operable to: generate a plurality of second partial tasks based on the second task; and allocate the plurality of second partial tasks to the plurality of computing devices, wherein the plurality of computing device executes the plurality of second partial tasks.

Abstract: A method for execution by a computer of a database management system includes obtaining a dataset that includes a set of data records, where the data set is associated with a set of data characteristics. The method includes executing a selected ranked analytical calculation of a ranked list of analytical calculations on the dataset to produce an analytical calculation result, where the ranked list of analytical calculations is generated by ranking a list of analytical calculations that are able to be executed on the dataset, based on a set of analytical calculation characteristics associated with the list of analytical calculations, where an analytical calculation characteristic of the set of analytical calculation characteristics indicates an estimated execution time to perform an analytical calculation of the list of analytical calculations, and the selected ranked analytical calculation is selected based on the set of data characteristics to produce the selected ranked analytical calculation.

Abstract: A parallelized data input sub-system of a database system includes a first set of loader nodes of pluralities of computing nodes of a plurality of computing device clusters. The first set of loader nodes includes a plurality of memory devices and a plurality of processing modules. The first set of loader node is operable to ingest at least a portion of a dataset (“data”), a set of the memory devices stores the data, and a set of the processing modules determines whether the data is regarding a query. When it is regarding the query, the set of processing modules provides the data to a query and response sub-system. When it is not regarding the query, the set of processing modules determines long term storage parameters, processes the data in accordance with the parameters to produce formatted data, and provides the formatted data to a store and compute sub-system.

Abstract: A data input sub-system of a parallelized database system includes processing core resources. Data blocks of a first memory device of a first processing core resource correspond to a first set of logical data block addresses. The processing core resources are operable to obtain divisions of data slabs, compress the divisions of data slabs, and store a respective division of compressed data slabs. A first data slab of a first division of data slabs is mapped to at least a portion of the first set of logical data block addresses that includes at least a portion of a first set of fixed size data fields. The first data slab is compressed to produce a first compressed data slab and the first compressed data slab is mapped to a reduced amount of fixed size data fields of the 10 at least the portion of the first set of fixed size data fields.

Abstract: A data input sub-system of a parallelized database system includes lead processing core resources of a plurality of computing nodes that are operable to receive sub-segments of segments of segment groups of dataset partitions, each partition including rows of columnar data. The lead processing core resources are operable to divide the sub-segments along columnar lines to produce divisions of data slabs, each data slab corresponding to a column of data. The lead processing core resources are further operable to store first divisions of the data slabs and transmit other divisions of the data slabs to additional processing core resources of the plurality of computing nodes.

Abstract: A data input sub-system of a parallelized database system includes processing core resources operable to obtain divisions of data slabs of a dataset and compress the divisions of data slabs using a null elimination compression scheme to produce divisions of compressed data slabs, A first data slab of a first division of data slabs of the divisions of data slabs is compressed using the null elimination compression scheme to produce a first compressed data slab. The first compressed data slab includes first compressed data and first compression information. The processing core resources are further operable to store a respective division of compressed data slabs of the divisions of compressed data slabs.

Abstract: A database system includes a parallelized data input sub-system including a first plurality of nodes, a parallelized data store, retrieve, and process sub-system including a second plurality of nodes, and a parallelized query and response sub-system including a third plurality of nodes. The third plurality of nodes is operable to receive a plurality of queries in parallel and assign a first query to a first node of the third plurality of nodes. The first node is operable to convert an instruction set of the first query into a hierarchical tree structure of code constructs, map database operations to the hierarchical tree structure of code constructs to produce a hierarchical tree structure of database operations, generate an initial query plan from the hierarchical tree structure of database operations in accordance with dataset storage information and a set of available nodes, and optimize the initial query plan to produce an optimized query plan.

Abstract: A method includes a host computing device receiving a segment group of data. The method further includes the host computing device evaluating availability status of other computing devices in the storage cluster of computing devices. When one of the other computing devices is unavailable, the method further includes the host computing device dividing the segment group of data into a plurality of lines of data blocks. For a line of the data blocks, the method further includes the host computing device generating at least one parity block. The method further includes the host computing device sending a first data segment that includes first positioned data blocks to a first available computing device. The method further includes the host computing device sending a second data segment that includes second positioned data blocks to a second available computing device. The method further includes the host computing device storing a parity segment.

Abstract: A method for execution, by a first intermediate node of a plurality of nodes in a database management system, includes processing a message that includes data that is being sent in accordance with a routing path from a source node to a destination node, is a first size, and indicates a next node of the routing path, wherein the first intermediate node is limited to communication with a subset of nodes of the plurality of nodes, and wherein the subset of nodes includes the next node. The method further includes maintaining a tracking table that indicates a total amount of data sent to each node of the subset of nodes during a first time period. The method further includes resetting the total amount of data sent to each of the subset of nodes to zero based one or more of a command and an initiation of a second time period.

Abstract: A computing system is operable to generating a set of segments of a segment group. Each segment of the set of segments includes a data and parity section that includes a corresponding set of sorted data slabs, a manifest section that includes metadata regarding the corresponding set of sorted data slabs, at least one index section that includes index data for the corresponding set of sorted data slabs, and a statistics section storing statistical information regarding the each segments. The set of segments of the segment group are stored across a plurality of computing devices of a storage cluster of the computing system.

Abstract: A networked database management system (DBMS) is disclosed. In particular, the disclosed DBMS includes a plurality of nodes, one of which is elected as a designated leader. The designated leader is elected using a consensus algorithm, such as tabulated random votes, RAFT or PAXOS. The designated leader is responsible for managing open coding lines, and determining when to close an open coding line.

Abstract: A method for execution, by a first intermediate node of a plurality of nodes in a database management system, includes receiving a message, where the first intermediate node is limited to communication with a subset of nodes of the plurality of nodes, where the message: includes data that is being sent in accordance with a routing path, is a first size, and indicates a next node of the routing path, and where the subset of nodes includes the next node. The method continues by generating a revised message, wherein the revised message includes the data and has a second size. The method continues by determining whether there is at least one additional intermediate node after the next node in the routing path. When yes, determining an optimal route for forwarding the revised message via a node of the subset of nodes, and sending the revised message to the node.

Abstract: A computing system is operable to generate a first plurality of lines of coding blocks that includes a first plurality of data blocks and a first plurality of parity blocks. A first set of segments of a first segment group that collectively include the first plurality of lines of coding blocks are generated, and the first set of segments of the first segment group are stored via a first plurality of computing devices of a first storage cluster of the computing system. A second plurality of lines of coding blocks that includes a second plurality of data blocks and a second plurality of parity blocks are generated. A second set of segments of a second segment group that collectively include the second plurality of lines of coding blocks are generated. The second set of segments of the second segment group are stored via a second plurality of computing devices of a second storage cluster of the computing system.

Abstract: A computing device includes a plurality of nodes, wherein a first node of the plurality of nodes operates in accordance with a computing device operation system (OS) and remaining nodes of the plurality of nodes operate in accordance with a database OS and process a plurality of threads of an application. The computing device further includes a divided main memory that is divided into a computing device memory section and a database memory section, and the database OS determines an allocation of the divided main memory between the computing device memory section and the database memory section, where a first database thread is assigned a buffer of a plurality of buffers of the database memory section, and a first computing device thread utilizes the computing device memory section of the divided main memory.

Abstract: A method for execution, by a first intermediate node of a plurality of nodes in a database management system, includes processing a message that includes data that is being sent in accordance with a routing path from a source node to a destination node, is a first size, and indicates a next node of the routing path, wherein the first intermediate node is limited to communication with a subset of nodes of the plurality of nodes, and wherein the subset of nodes includes the next node. The method further includes maintaining a tracking table that indicates a total amount of data sent to each node of the subset of nodes during a first time period. The method further includes resetting the total amount of data sent to each of the subset of nodes to zero based one or more of a command and an initiation of a second time period.

Abstract: A method for execution by one or more database storage nodes of a database management system includes obtaining a row of data that includes a plurality of columns. The method further includes generating a data confidence value for data within one or more columns of the plurality of columns. The method further includes storing the data confidence value in a memory of the database management system.

Abstract: A method for execution by at least one computing entity of a database system, the method includes obtaining a plurality of data partitions of a data set for storage in the database system, where the data set is organized in rows and columns, and the rows correspond to data records and the columns correspond to fields of the data records. The method further includes dividing a first partition of a plurality of data partitions to produce a first number of first raw data segments for storage in the database system, where the first number is based on a first data storage coding scheme. The method further includes dividing a second partition of the plurality of data partitions to produce a second number of second raw data segments for storage in the database system, where the second number is based on a second data storage coding scheme.

Abstract: A networked database management system (DBMS) is disclosed. In particular, the disclosed DBMS includes a plurality of nodes, one of which is elected as a designated leader. The designated leader is elected using a consensus algorithm, such as tabulated random votes, RAFT or PAXOS. The designated leader is responsible for managing open coding lines, and determining when to close an open coding line.

Abstract: A payload store within a database management system includes a first set of nodes that include a first node that includes memory, a network interface, a storage device, and at least one processing unit. The at least one processing unit is operable to receive, via the network interface, a first set of data rows that includes a plurality of subsets of data rows and a last subset of data rows. The processing module is further operable to randomly assign the plurality of subsets of data rows and the last subset of data rows between the first set of nodes, where a first number of data rows of the plurality of subsets of data rows is assigned to the first node and a second number of data rows is assigned to another node. The processing module is further operable to store the first number of data rows in the storage device.

Abstract: A method includes a host computing device receiving a segment group of data. The method further includes the host computing device evaluating availability status of other computing devices in the storage cluster of computing devices. When one of the other computing devices is unavailable, the method further includes the host computing device dividing the segment group of data into a plurality of lines of data blocks. For a line of the data blocks, the method further includes the host computing device generating at least one parity block. The method further includes the host computing device sending a first data segment that includes first positioned data blocks to a first available computing device. The method further includes the host computing device sending a second data segment that includes second positioned data blocks to a second available computing device. The method further includes the host computing device storing a parity segment.