Abstract: A computing system is operable to generating a set of segments of a segment group. Each segment of the set of segments includes a data and parity section that includes a corresponding set of sorted data slabs, a manifest section that includes metadata regarding the corresponding set of sorted data slabs, at least one index section that includes index data for the corresponding set of sorted data slabs, and a statistics section storing statistical information regarding the each segments. The set of segments of the segment group are stored across a plurality of computing devices of a storage cluster of the computing system.

Abstract: A networked database management system (DBMS) is disclosed. In particular, the disclosed DBMS includes a plurality of nodes, one of which is elected as a designated leader. The designated leader is elected using a consensus algorithm, such as tabulated random votes, RAFT or PAXOS. The designated leader is responsible for managing open coding lines, and determining when to close an open coding line.

Abstract: A computing system is operable to generate a first plurality of lines of coding blocks that includes a first plurality of data blocks and a first plurality of parity blocks. A first set of segments of a first segment group that collectively include the first plurality of lines of coding blocks are generated, and the first set of segments of the first segment group are stored via a first plurality of computing devices of a first storage cluster of the computing system. A second plurality of lines of coding blocks that includes a second plurality of data blocks and a second plurality of parity blocks are generated. A second set of segments of a second segment group that collectively include the second plurality of lines of coding blocks are generated. The second set of segments of the second segment group are stored via a second plurality of computing devices of a second storage cluster of the computing system.

Abstract: A method for execution, by a first intermediate node of a plurality of nodes in a database management system, includes receiving a message, where the first intermediate node is limited to communication with a subset of nodes of the plurality of nodes, where the message: includes data that is being sent in accordance with a routing path, is a first size, and indicates a next node of the routing path, and where the subset of nodes includes the next node. The method continues by generating a revised message, wherein the revised message includes the data and has a second size. The method continues by determining whether there is at least one additional intermediate node after the next node in the routing path. When yes, determining an optimal route for forwarding the revised message via a node of the subset of nodes, and sending the revised message to the node.

Abstract: A computing device includes a plurality of nodes, wherein a first node of the plurality of nodes operates in accordance with a computing device operation system (OS) and remaining nodes of the plurality of nodes operate in accordance with a database OS and process a plurality of threads of an application. The computing device further includes a divided main memory that is divided into a computing device memory section and a database memory section, and the database OS determines an allocation of the divided main memory between the computing device memory section and the database memory section, where a first database thread is assigned a buffer of a plurality of buffers of the database memory section, and a first computing device thread utilizes the computing device memory section of the divided main memory.

Abstract: A method for execution, by a first intermediate node of a plurality of nodes in a database management system, includes processing a message that includes data that is being sent in accordance with a routing path from a source node to a destination node, is a first size, and indicates a next node of the routing path, wherein the first intermediate node is limited to communication with a subset of nodes of the plurality of nodes, and wherein the subset of nodes includes the next node. The method further includes maintaining a tracking table that indicates a total amount of data sent to each node of the subset of nodes during a first time period. The method further includes resetting the total amount of data sent to each of the subset of nodes to zero based one or more of a command and an initiation of a second time period.

Abstract: A method for execution by one or more database storage nodes of a database management system includes obtaining a row of data that includes a plurality of columns. The method further includes generating a data confidence value for data within one or more columns of the plurality of columns. The method further includes storing the data confidence value in a memory of the database management system.

Abstract: A method for execution by at least one computing entity of a database system, the method includes obtaining a plurality of data partitions of a data set for storage in the database system, where the data set is organized in rows and columns, and the rows correspond to data records and the columns correspond to fields of the data records. The method further includes dividing a first partition of a plurality of data partitions to produce a first number of first raw data segments for storage in the database system, where the first number is based on a first data storage coding scheme. The method further includes dividing a second partition of the plurality of data partitions to produce a second number of second raw data segments for storage in the database system, where the second number is based on a second data storage coding scheme.

Abstract: A networked database management system (DBMS) is disclosed. In particular, the disclosed DBMS includes a plurality of nodes, one of which is elected as a designated leader. The designated leader is elected using a consensus algorithm, such as tabulated random votes, RAFT or PAXOS. The designated leader is responsible for managing open coding lines, and determining when to close an open coding line.

Abstract: A payload store within a database management system includes a first set of nodes that include a first node that includes memory, a network interface, a storage device, and at least one processing unit. The at least one processing unit is operable to receive, via the network interface, a first set of data rows that includes a plurality of subsets of data rows and a last subset of data rows. The processing module is further operable to randomly assign the plurality of subsets of data rows and the last subset of data rows between the first set of nodes, where a first number of data rows of the plurality of subsets of data rows is assigned to the first node and a second number of data rows is assigned to another node. The processing module is further operable to store the first number of data rows in the storage device.

Abstract: A method includes a host computing device receiving a segment group of data. The method further includes the host computing device evaluating availability status of other computing devices in the storage cluster of computing devices. When one of the other computing devices is unavailable, the method further includes the host computing device dividing the segment group of data into a plurality of lines of data blocks. For a line of the data blocks, the method further includes the host computing device generating at least one parity block. The method further includes the host computing device sending a first data segment that includes first positioned data blocks to a first available computing device. The method further includes the host computing device sending a second data segment that includes second positioned data blocks to a second available computing device. The method further includes the host computing device storing a parity segment.

Abstract: A computing device includes a plurality of nodes, where a first node operates in accordance with a computing device operation system (OS) and remaining nodes operate in accordance with a custom OS. The remaining nodes include a plurality of sets of processing core resources that process a plurality of sets of threads of an application. The computing device also includes a main memory divided into a computing device memory section and a custom memory section that includes portions logically allocated as a plurality of buffers. The computing device also includes a memory access control module operable to coordinate access to the plurality of buffers by at least some of the plurality of sets of threads in accordance with the custom OS. The computing device also includes disk memory and a disk memory access control module operable to coordinate access to the disk memory in accordance with the computing device OS.

Abstract: A method includes obtaining a plurality of data partitions of a data set for storage in a database system. The method further includes determining a first data storage coding scheme for a first partition of the plurality of data partitions, where the first data storage coding scheme includes first encoding parameters regarding encoding the first partition into first data segments and first parity segments. The method further includes determining a second data storage coding scheme for a second partition of the plurality of data partitions. The method further includes dividing the first partition to produce a first number of first raw data segments, where the first number is based on the first data storage coding scheme. The method further includes dividing the second partition to produce a second number of second raw data segments, where the second number is based on the second data storage coding scheme.

Abstract: A method includes receiving, by a first computing entity of a database system, a query request that is formatted in accordance with a generic query format. The method further includes generating, by the first computing entity, an initial query plan based on the query request and a query instruction set. The method further includes determining, by the first computing entity, storage parameters. The method further includes determining, by the first computing entity, processing resources for processing the query request based on the storage parameters. The method further includes generating, by the first computing entity, an optimized query plan from the initial query plan based on the storage parameters, the processing resources, and optimization tools. The method further includes sending, by the first computing entity, the optimized query plan to a second computing entity for distribution and execution of the optimized query plan.

Abstract: A large highly parallel database management system includes thousands of nodes storing huge volume of data. The database management system includes multiple query optimizers for determining low cost execution plans for queries. The database management system is adapted to receive a data query. An execution plan generator component of the database management system generates an initial execution plan for the query. The initial execution plan is fed as input to more than one query optimizers. Each optimizer starts with the initial execution plan, generates alternative execution plans, and determines a satisfactory execution plan that incurs the lowest cost. The database management system compares the selected execution plans by the optimizers and selects one with the lowest cost. The multiple query optimizers run in parallel.

Abstract: A method for execution by a computer of a database management system includes obtaining a dataset that includes a set of data records, where the data set is associated with a set of data characteristics. The method includes executing a selected ranked analytical calculation of a ranked list of analytical calculations on the dataset to produce an analytical calculation result, where the ranked list of analytical calculations is generated by ranking a list of analytical calculations that are able to be executed on the dataset, based on a set of analytical calculation characteristics associated with the list of analytical calculations, where an analytical calculation characteristic of the set of analytical calculation characteristics indicates an estimated execution time to perform an analytical calculation of the list of analytical calculations, and the selected ranked analytical calculation is selected based on the set of data characteristics to produce the selected ranked analytical calculation.

Abstract: A method includes a host computing device receiving a segment group of data. The method further includes the host computing device evaluating availability status of other computing devices in the storage cluster of computing devices. When one of the other computing devices is unavailable, the method further includes the host computing device dividing the segment group of data into a plurality of lines of data blocks. For a line of the data blocks, the method further includes the host computing device generating at least one parity block. The method further includes the host computing device sending a first data segment that includes first positioned data blocks to a first available computing device. The method further includes the host computing device sending a second data segment that includes second positioned data blocks to a second available computing device. The method further includes the host computing device storing a parity segment.

Abstract: A payload store within a database management system includes a first set of nodes that include a first node that includes a memory, a network interface, a storage device, and a processing unit. The processing unit receives, via the network interface, a first portion of data to be stored in a plurality of storage devices of the first set of nodes, and splits the first portion into a first set of data rows that includes a plurality of subsets of data rows and a last subset of data rows. The processing unit further randomly assigns the plurality of subsets of data rows and the last subset of data rows between the first set of nodes and stores a first number of data rows in the storage device. The processing unit further sends, via the network interface, a second number of data rows to the second node for storage therein.

Abstract: A method by one or more processing units of a database management system includes transitioning storage of a collection of data from storage in a set of first temperature storage nodes of the database management system to storage in a set of second temperature storage nodes of the database management system based on one or more criteria associated with the collection of data, where each first temperature storage node of the set of first temperature storage nodes is paired with a corresponding second temperature storage node of the set of second temperature storage nodes to form a set of parallel node pairs. The method further includes, after determining the collection of data has been stored in the set of second temperature storage nodes, deleting the collection of data in the set of first temperature storage nodes.

Abstract: A method for execution by a computer of a database management system includes obtaining a dataset that includes a set of data records, where the data set is associated with a set of data characteristics. The method further includes ranking analytical calculations of a list of analytical calculations that are able to be executed on the dataset based on the set of data characteristics and a set of analytical calculation characteristics associated with the list of analytical calculations to produce a ranked list of analytical calculations. The method further includes selecting an analytical calculation from the ranked list of analytical calculations based on the ranking. The method further includes executing the selected analytical calculation on the dataset to produce an analytical calculation result.