Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for a distributed parallel processing database system that persists table data in memory to a distributed file system. A distributed parallel processing database system persists table data in memory to a distributed file system. A parameter of creating a database table specifies that data records in the database table and history of changes to the data records can be stored in memory as well as in the distributed file system. When the database table is populated or otherwise modified, data records and the history in memory are evicted to the distributed file system as log files and removed from memory. The log files can be designated as write-only, where the data records, once written, cannot be read by structured query language (SQL) queries, or as read-write, where the data records, once written, can be read by SQL queries.

Abstract: Methods, systems, and apparatus for obtaining one or more metadata files, determining, by one or more computers and in accordance with a minor compaction setting, to perform a minor compaction of the one or more metadata files, creating one or more intermediate metadata files that each include at least compacted contacts of one or more of the metadata files, according to the determination to perform minor compaction of the one or more metadata files, determining, in accordance with a major compaction setting, to perform a major compaction of one or more of the intermediate metadata files, and creating one or more snapshot metadata files that each include at least compacted contents of one or more of the intermediate metadata files, according to the determination to perform the major compaction of one or more of the intermediate metadata files.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributed data management. One of the methods includes maintaining, by a first member in a distributed data management system having multiple computing members installed on multiple respective computers, a first garbage collection version vector that includes, for each member in the distributed data management system, a garbage collection version that represents a number of garbage collection processes performed by the member on a respective copy of a replicated data region maintained by the member in the data management system. If the first garbage collection version vector is different than a second garbage collection version vector received from a different provider member, a first replication process is performed that is different than a second replication process that is performed when the first garbage collection version vector matches the second garbage collection version vector.

Abstract: A distributed parallel processing database that processes data in a Java environment allocates memory both on a Java heap and off a Java heap. The distributed parallel processing database includes multiple servers. Each server executes a Java virtual machine (JVM) in which data allocated to the server is processed. When a JVM of a server starts, the JVM can specify an off-heap memory size, based on a JVM start parameter. The server can designate memory of the specified size that is off JVM memory heap as off-heap memory. The off-heap memory is different from heap memory in the Java environment, and is managed by a garbage collector that is outside of the Java environment. The server can process data designated as off-heap memory eligible in the off-heap memory. The off-heap memory can improve database operations that create a large number of similar-sized objects in memory by reducing Java memory management overhead.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributed data management. One of the methods includes maintaining, by a first member in a distributed data management system having multiple computing members installed on multiple respective computers, a first garbage collection version vector that includes, for each member in the distributed data management system, a garbage collection version that represents a number of garbage collection processes performed by the member on a respective copy of a replicated data region maintained by the member in the data management system. If the first garbage collection version vector is different than a second garbage collection version vector received from a different provider member, a first replication process is performed that is different than a second replication process that is performed when the first garbage collection version vector matches the second garbage collection version vector.

Abstract: Example embodiments of the present invention relate to a method, a system, and a computer program product for federated service requests of a logical data store comprising a plurality of heterogeneous data stores. The method includes providing a data abstraction services layer as a control plane for federated service requests of a logical data store, comprising a plurality of heterogeneous data stores, by an application operating on a processing device, receiving a service request at the data abstraction services layer from the application operating on the processing device directed to the logical data store, and returning results of the service request from the data abstraction services layer to the application operating on the processing device.

Abstract: Methods, systems, and apparatus for obtaining one or more metadata files, determining, by one or more computers and in accordance with a minor compaction setting, to perform a minor compaction of the one or more metadata files, creating one or more intermediate metadata files that each include at least compacted contacts of one or more of the metadata files, according to the determination to perform minor compaction of the one or more metadata files, determining, in accordance with a major compaction setting, to perform a major compaction of one or more of the intermediate metadata files, and creating one or more snapshot metadata files that each include at least compacted contents of one or more of the intermediate metadata files, according to the determination to perform the major compaction of one or more of the intermediate metadata files.

Abstract: Methods, systems, and apparatus for identifying two or more files, each of which include multiple entries, determining a respective size of each of the files, each size being an estimate of how many distinct entries exist in the respective file that are not garbage entries, determining a combined size of the files, where the combined size of the files is an arithmetic sum of the respective sizes of the files, estimating a compacted size of the files, where the estimated compacted size of the files is an estimate of how many distinct entries exist in the files that are not garbage entries, selecting the two or more files for compaction, based at least on a comparison of the combined size of the files to the estimated compacted size of the files, and compacting the two or more selected files.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for a distributed parallel processing database system that persists table data in memory to a distributed file system. A distributed parallel processing database system persists table data in memory to a distributed file system. A parameter of creating a database table specifies that data records in the database table and history of changes to the data records can be stored in memory as well as in the distributed file system. When the database table is populated or otherwise modified, data records and the history in memory are evicted to the distributed file system as log files and removed from memory. The log files can be designated as write-only, where the data records, once written, cannot be read by structured query language (SQL) queries, or as read-write, where the data records, once written, can be read by SQL queries.

Abstract: Methods, systems, and apparatus for obtaining one or more metadata files, determining, by one or more computers and in accordance with a minor compaction setting, to perform a minor compaction of the one or more metadata files, creating one or more intermediate metadata files that each include at least compacted contacts of one or more of the metadata files, according to the determination to perform minor compaction of the one or more metadata files, determining, in accordance with a major compaction setting, to perform a major compaction of one or more of the intermediate metadata files, and creating one or more snapshot metadata files that each include at least compacted contents of one or more of the intermediate metadata files, according to the determination to perform the major compaction of one or more of the intermediate metadata files.

Abstract: Methods, systems, and apparatus for identifying two or more files, each of which include multiple entries, determining a respective size of each of the files, each size being an estimate of how many distinct entries exist in the respective file that are not garbage entries, determining a combined size of the files, where the combined size of the files is an arithmetic sum of the respective sizes of the files, estimating a compacted size of the files, where the estimated compacted size of the files is an estimate of how many distinct entries exist in the files that are not garbage entries, selecting the two or more files for compaction, based at least on a comparison of the combined size of the files to the estimated compacted size of the files, and compacting the two or more selected files.

Abstract: Methods, systems, and apparatus for obtaining one or more metadata files, determining, by one or more computers and in accordance with a minor compaction setting, to perform a minor compaction of the one or more metadata files, creating one or more intermediate metadata files that each include at least compacted contacts of one or more of the metadata files, according to the determination to perform minor compaction of the one or more metadata files, determining, in accordance with a major compaction setting, to perform a major compaction of one or more of the intermediate metadata files, and creating one or more snapshot metadata files that each include at least compacted contents of one or more of the intermediate metadata files, according to the determination to perform the major compaction of one or more of the intermediate metadata files.