Abstract: A system includes a coordinator node and worker nodes in communication with the coordinator node. Each worker node stores data blocks. Each data block has data in a semi-structured format and each data block has an associated foreign table declaration specifying conversion of the data in the semi-structured format into a tabular format interpretable by a query language. A query processor executed by the coordinator node produces a distributed query plan in response to a query language query. The distributed query plan includes sub-queries. The sub-queries are executed by selected worker nodes of the worker nodes. The selected worker nodes use foreign table declarations to convert data in semi-structured formats into tabular formats of a distributed database to provide tabular data in response to the query language query.

Abstract: A database system includes a query planner with instructions executed by a processor to generate a logical plan tree. Each node of the logical plan tree is a distributed relational algebra operator. Each child node of the logical plan tree produces results processed by a parent node. The logical plan tree includes a distributed relational operator that reparations tuples of results that are at least 1 GB on a dimension and regroups the tuples on the dimension to avoid broadcasting the tuples between machines and thereby avoid consumption of network bandwidth associated with broadcasting the tuples. The logical plan tree is modified according to algebraic transformation rules. The logical plan tree is mapped to distributed query execution primitives. The distributed query execution primitives are processed on machines storing partitions of a distributed database table.

Abstract: A database system includes a query planner with instructions executed by a processor to generate a logical plan tree. Each node of the logical plan tree is a distributed relational algebra operator. Each child node of the logical plan tree produces results processed by a parent node. The logical plan tree includes a distributed relational operator that reparations tuples of results that are at least 1 GB on a dimension and regroups the tuples on the dimension to avoid broadcasting the tuples between machines and thereby avoid consumption of network bandwidth associated with broadcasting the tuples. The logical plan tree is modified according to algebraic transformation rules. The logical plan tree is mapped to distributed query execution primitives. The distributed query execution primitives are processed on machines storing partitions of a distributed database table.

Abstract: A system includes a coordinator node and worker nodes in communication with the coordinator node. Each worker node stores data blocks. Each data block has data in a semi-structured format and each data block has an associated foreign table declaration specifying conversion of the data in the semi-structured format into a tabular format interpretable by a query language. A query processor executed by the coordinator node produces a distributed query plan in response to a query language query. The distributed query plan includes sub-queries. The sub-queries are executed by selected worker nodes of the worker nodes. The selected worker nodes use foreign table declarations to convert data in semi-structured formats into tabular formats of a distributed database to provide tabular data in response to the query language query.

Abstract: A method implemented by a computer network includes receiving a database table update to a distributed database. The database table update is loaded into a database table partition on a node of a computer network executing the distributed database. The database table update is added to an append-only log file corresponding to the database table partition. The added update represents data associated with the database table update in a logical representation that is independent of the physical representation of the data at the node. This facilitates transferring the database table partition to other nodes within the computer network.

Abstract: A method implemented by a computer network includes storing a database table in a distributed database resident on the computer network. The database table is represented with append-only files, where each append-only file stores data in a logical representation that is independent of the physical representation of the data at a node of the computer network. Data in the append-only files are repartitioned across nodes in the computer network to form repartitioned files. Information about a mapping between the append-only files and the repartitioned files is maintained.

Abstract: A system includes a master node and worker nodes controlled by the master node. Each worker node stores 25 or more modular blocks of a distributed database, where each modular block has a size of 5 G bytes or less and has an associated log file.

Abstract: A method implemented by a computer includes receiving a segment of data that has a time dimension, where the time dimension of the segment of data is bounded by a start time stamp and an end time stamp. The segment of data is added to an append-only database table of a distributed database. The addition operation imposes an inherent data order based upon the start time stamp and end time stamp without the manual definition of a database table partition in the distributed database.