Abstract: A large highly parallel database management system includes thousands of nodes storing huge volume of data. The database management system includes multiple query optimizers for determining low cost execution plans for queries. The database management system is adapted to receive a data query. An execution plan generator component of the database management system generates an initial execution plan for the query. The initial execution plan is fed as input to more than one query optimizers. Each optimizer starts with the initial execution plan, generates alternative execution plans, and determines a satisfactory execution plan that incurs the lowest cost. The database management system compares the selected execution plans by the optimizers and selects one with the lowest cost. The multiple query optimizers run in parallel.

Abstract: A networked database management system (DBMS) is disclosed. In particular, the disclosed DBMS includes a plurality of nodes, one of which is elected as a designated leader. The designated leader is elected using a consensus algorithm, such as tabulated random votes, RAFT or PAXOS. The designated leader is responsible for managing open coding lines, and determining when to close an open coding line.

Abstract: A computing system is operable to generate a first plurality of lines of coding blocks that includes a first plurality of data blocks and a first plurality of parity blocks. A first set of segments of a first segment group that collectively include the first plurality of lines of coding blocks are generated, and the first set of segments of the first segment group are stored via a first plurality of computing devices of a first storage cluster of the computing system. A second plurality of lines of coding blocks that includes a second plurality of data blocks and a second plurality of parity blocks are generated. A second set of segments of a second segment group that collectively include the second plurality of lines of coding blocks are generated. The second set of segments of the second segment group are stored via a second plurality of computing devices of a second storage cluster of the computing system.

Abstract: A method includes obtaining a plurality of data partitions of a data set for storage in a database system. The method further includes determining a first data storage coding scheme for a first partition of the plurality of data partitions, where the first data storage coding scheme includes first encoding parameters regarding encoding the first partition into first data segments and first parity segments. The method further includes determining a second data storage coding scheme for a second partition of the plurality of data partitions. The method further includes dividing the first partition to produce a first number of first raw data segments, where the first number is based on the first data storage coding scheme. The method further includes dividing the second partition to produce a second number of second raw data segments, where the second number is based on the second data storage coding scheme.

Abstract: A method by one or more processing units of a database management system includes determining to transition a collection of data from storage in a set of first temperature storage nodes to storage in a set of second temperature nodes based on one or more criteria associated with the collection of data, where each first temperature storage node of the set of first temperature storage nodes is paired with a corresponding second temperature storage node of the set of second temperature storage nodes to form a set of parallel node pairs. The method further includes transitioning storage of the collection of data in parallel, by each parallel node pair of the set of parallel node pairs, from storage in the set of first temperature storage nodes to storage in the set of second temperature storage nodes.

Abstract: A computing system is operable to generate a plurality of lines of coding blocks that includes a plurality of data blocks and a plurality of parity blocks. Each of the plurality of lines of coding blocks includes a corresponding subset of data blocks a corresponding subset of parity blocks. A set of segments of a segment group are generated to collectively include the plurality of lines of coding blocks. Different coding blocks of each of the plurality of lines of coding blocks are included within different ones of the set of segments, and the plurality of parity blocks are dispersed across all of the set of segments. The set of segments are stored via a plurality of nodes sets, where different segments of the set of segments are stored via memory resources of different node sets of the plurality of node sets.

Abstract: A method includes receiving, by a first computing entity of a database system, a data set that is organized in rows and columns. The method further includes determining whether to partition the data set based on a parameter associated with the data set. When determining to partition the data set, the method includes determining partitioning parameters for the data set, and partitioning the data set into a plurality of data partitions in accordance with the partitioning parameters. The method further includes determining a first coding scheme for a first data partition and determining a first number of first raw data segments for a first segment group of the first partition based on the coding scheme. The method further includes dividing the first partition to produce the first number of first raw data segments for storage in the database system.

Abstract: A networked database management system (DBMS) is disclosed. In particular, the disclosed DBMS includes a plurality of nodes, one of which is elected as a designated leader. The designated leader is elected using a consensus algorithm, such as tabulated random votes, RAFT or PAXOS. The designated leader is responsible for managing open coding lines, and determining when to close an open coding line.

Abstract: A method for execution by one or more database storage nodes of a database management system includes obtaining a row of data that includes a plurality of columns. The method further includes generating a data confidence value for data within one or more columns of the plurality of columns. The method further includes storing the data confidence value in a memory of the database management system.

Abstract: A method for managing data storage and retrieval and operated within a database management system includes retrieving a set of confidence data records from memory of the database management system, where a confidence data record of the set of confidence data records includes a set of data columns and one or more confidence columns populated with one or more data confidence values associated with data within one or more data columns of the set of data columns. The method further includes analyzing the set of confidence data records to produce a result, where the result includes a confidence level that is based on corresponding ones of the one or more data confidence values associated with the data within one or more data columns of the set of data columns.

Abstract: A payload store within a database management system includes a first set of nodes that include a first node that includes a memory, a network interface, a storage device, and a processing unit. The processing unit receives, via the network interface, a first portion of data to be stored in a plurality of storage devices of the first set of nodes, and splits the first portion into a first set of data rows that includes a plurality of subsets of data rows and a last subset of data rows. The processing unit further randomly assigns the plurality of subsets of data rows and the last subset of data rows between the first set of nodes and stores a first number of data rows in the storage device. The processing unit further sends, via the network interface, a second number of data rows to the second node for storage therein.

Abstract: A computing system is operable to generate a plurality of lines of coding blocks that includes a plurality of data blocks and a plurality of parity blocks. Each of the plurality of lines of coding blocks includes a corresponding subset of data blocks a corresponding subset of parity blocks. A set of segments of a segment group are generated to collectively include the plurality of lines of coding blocks. Different coding blocks of each of the plurality of lines of coding blocks are included within different ones of the set of segments, and the plurality of parity blocks are dispersed across all of the set of segments. The set of segments are stored via a plurality of nodes sets, where different segments of the set of segments are stored via memory resources of different node sets of the plurality of node sets.

Abstract: A networked database management system (DBMS) is disclosed. In particular, the disclosed DBMS includes a plurality of nodes, one of which is elected as a designated leader. The designated leader is elected using a consensus algorithm, such as tabulated random votes, RAFT or PAXOS. The designated leader is responsible for managing open coding lines, and determining when to close an open coding line.

Abstract: A method includes generating a plurality of parity blocks from a plurality of lines of data blocks. The plurality of lines of data blocks are stored in data sections of memory of a cluster of computing devices of the computing system by distributing storage of individual data blocks of the plurality of lines of data blocks among unique data sections of the cluster of computing devices. The plurality of parity blocks are stored in parity sections of memory of the cluster of computing devices by distributing storage of parity blocks of the plurality of parity blocks among unique parity sections of the cluster of computing devices.

Abstract: A massively parallel database management system includes an index store and a payload store including a set of storage systems of different temperatures. Both the index store and the storage system each include a list of clusters. Each cluster includes a set of nodes with storage devices forming a group of segments. Nodes and clusters are connected over high speed links. Each cluster receives data and splits the data into data rows based on a predetermined size. The data rows are randomly and evenly distributed between all nodes of the cluster.

Abstract: A method for execution by a computer of a database management system includes obtaining a dataset that includes a set of data records, where the data set is associated with a set of data characteristics. The method further includes ranking analytical calculations of a list of analytical calculations that are able to be executed on the dataset based on the set of data characteristics and a set of analytical calculation characteristics associated with the list of analytical calculations to produce a ranked list of analytical calculations. The method further includes selecting an analytical calculation from the ranked list of analytical calculations based on the ranking. The method further includes executing the selected analytical calculation on the dataset to produce an analytical calculation result.

Abstract: A large highly parallel database management system includes thousands of nodes storing huge volume of data. The database management system includes multiple query optimizers for determining low cost execution plans for queries. The database management system is adapted to receive a data query. An execution plan generator component of the database management system generates an initial execution plan for the query. The initial execution plan is fed as input to more than one query optimizers. Each optimizer starts with the initial execution plan, generates alternative execution plans, and determines a satisfactory execution plan that incurs the lowest cost. The database management system compares the selected execution plans by the optimizers and selects one with the lowest cost. The multiple query optimizers run in parallel.

Abstract: A large highly parallel database management system includes thousands of nodes storing huge volume of data. The database management system includes multiple query optimizers for determining low cost execution plans for queries. The database management system is adapted to receive a data query. An execution plan generator component of the database management system generates an initial execution plan for the query. The initial execution plan is fed as input to more than one query optimizers. Each optimizer starts with the initial execution plan, generates alternative execution plans, and determines a satisfactory execution plan that incurs the lowest cost. The database management system compares the selected execution plans by the optimizers and selects one with the lowest cost. The multiple query optimizers run in parallel.

Abstract: A method for managing data storage and retrieval and operated within a database management system includes retrieving a set of confidence data records from memory of the database management system, where a confidence data record of the set of confidence data records includes a set of data columns and one or more confidence columns populated with one or more data confidence values associated with data within one or more data columns of the set of data columns. The method further includes analyzing the set of confidence data records to produce a result, where the result includes a confidence level that is based on corresponding ones of the one or more data confidence values associated with the data within one or more data columns of the set of data columns.

Abstract: A networked database management system (DBMS) is disclosed. In particular, the disclosed DBMS includes a plurality of nodes, one of which is elected as a designated leader. The designated leader is elected using a consensus algorithm, such as tabulated random votes, RAFT or PAXOS. The designated leader is responsible for managing open coding lines, and determining when to close an open coding line.