Abstract: A system includes determination of a plurality of secondary data structures of a database to be rebuilt, determination, for each of the plurality of secondary data structures, of a current ranking based on a pre-crash workload, a crash-time workload, the post-crash workload, and a rebuild time of the secondary data structure, determination to rebuild one of the plurality of secondary data structures based on the determined rankings, and rebuilding of the one of the plurality of secondary data structures in a dynamic random access memory based on primary data of a database stored in non-volatile random access memory.

Abstract: Some embodiments provide a non-transitory machine-readable medium that stores a program. The program receives a query for a set of records in a database system having values in a field of a table that fall within a range of values. The program also determines a number of bits used to represent the values in the field of the table. The program further determines a set of operations to perform on the values in the field of the table based on the determined number of bits. The program also performs the determined set of operations on the values in the field of the table in order to identify the set of records in the database.

Abstract: A method and system for transforming a serial schedule of transactions into a parallel schedule of transaction is disclosed. In one example, a computer system stores a list of data transactions in a transaction log. The computer system then reads a respective data transaction from the transaction log. The computer system determines whether the respective data transaction is dependent on any other currently pending data transaction. In accordance with a determination that the respective data transaction is not dependent on any other currently pending data transaction, the computer system applies the data changes to a reconstructed data set. In accordance with a determination that the respective data transaction is dependent on a currently pending second data transaction, the computer system delays commitment of the respective data transaction until the second data transaction has been applied to the reconstructed data set.

Abstract: A method and system, the system including a plurality of machines each having a processor and a main memory component; a shared distributed storage facility storing a set of data and accessible by the plurality of machines over a communication network; a controller to select, in response to a state of a query execution plan comprising a plurality of executable jobs for the set of data, which one of a set of scheduling algorithms to execute; an execution engine to execute the selected scheduling algorithm to determine, for each job in the plurality of jobs, which server to schedule to execute the respective job; and providing an indication of the scheduling of the servers determined to be schedules for the execution of the jobs.

Abstract: A plurality of reserve and commit log operations are initiated in a database system. Thereafter, at least a portion of the database operations are logged in a log such that transient data structures are kept in-memory of the database system and persistent data structures are kept in byte-addressable memory. Next, each of one or more clients concurrently accessing the log are registered to enable such clients to access the log.

Abstract: A system includes determination of a plurality of secondary data structures of a database to be rebuilt, determination, for each of the plurality of secondary data structures, of a current ranking based on a pre-crash workload, a crash-time workload, the post-crash workload, and a rebuild time of the secondary data structure, determination to rebuild one of the plurality of secondary data structures based on the determined rankings, and rebuilding of the one of the plurality of secondary data structures in a dynamic random access memory based on primary data of a database stored in non-volatile random access memory.

Abstract: Disclosed herein are system, method, and computer program product embodiments for distributed transaction processing in massively parallel processing (MPP) databases. An embodiment operates by receiving a transaction including a read timestamp corresponding to a version of data of a database associated with the query. It is determined whether a locally stored portion of the data of the database has a local timestamp that is equivalent to or after the read timestamp. A lock may be requested, and if received, the data is updated.

Abstract: Disclosed herein are system, method, and computer program product embodiments for accessing a distributed shared log that supports a total order over log writes and multiple logical log streams. An embodiment operates by selecting a range of log entries for a logical log stream from the distributed shared log and mapping the selected range of log entries to associated storage units in a plurality of storage units. The selected range of log entries are stored at the associated storage units according to metadata that indexes the selected range of log entries by logical log stream. The selected range of log entries are then requested from the associated storage units based on the mapping. The selected range of log entries for the logical log stream are then returned from the associated storage units based on a scan of the metadata. The received log entries are then reassembled.

Abstract: Disclosed herein are systems, methods, and computer readable storage media for a database system using solid state drives as a second level cache. A database system includes random access memory configured to operate as a first level cache, solid state disk drives configured to operate as a persistent second level cache, and hard disk drives configured to operate as disk storage. The database system also includes a cache manager configured to receive a request for a data page and determine whether the data page is in cache or disk storage. If the data page is on disk, or in the second level cache, it is copied to the first level cache. If copying the data page results in an eviction, the evicted data page is copied to the second level cache. At checkpoint, dirty pages stored in the second level cache are flushed in place in the second level cache.

Abstract: Disclosed herein are system, method, and computer program product embodiments for constructing an index for a database table. An index that comprises a data structure may be created. The index can then be populated with data from the database table. When a request to modify the database table is received, the method may determine that the request to modify the database table relates to a portion of the database table corresponding to a portion of the index that has yet to be populated. An entry indicating the requested modification can be inserted into the portion of the index that has yet to be populated.

Abstract: A method and system for transforming a serial schedule of transactions into a parallel schedule of transaction is disclosed. In one example, a computer system stores a list of data transactions in a transaction log. The computer system then reads a respective data transaction from the transaction log. The computer system determines whether the respective data transaction is dependent on any other currently pending data transaction. In accordance with a determination that the respective data transaction is not dependent on any other currently pending data transaction, the computer system applies the data changes to a reconstructed data set. In accordance with a determination that the respective data transaction is dependent on a currently pending second data transaction, the computer system delays commitment of the respective data transaction until the second data transaction has been applied to the reconstructed data set.

Abstract: A data recovery system and method are disclosed. Primary data is stored a database in byte-addressable NVRAM, where the database includes one or more persistent tables of data in a byte-addressable, RAM format, and a persistent memory allocator that maps persistent memory pointers of the persistent memory to virtual memory pointers of a virtual memory associated with the database. Secondary data is stored in volatile DRAM. A failure recovery includes recovering the persistent memory allocator, mapping the persistent memory to the virtual memory to recover primary data using their persistent memory pointers, translating the persistent memory pointers to virtual memory pointers, undoing changes to the primary data made by unfinished transactions of the query execution at the time of failure of one of the one or more queries, and reconstructing the secondary data from the primary data.

Abstract: Systems, methods and computer program products for compression of database indexes are described herein. A system embodiment includes a sequence determiner to scan a database index and to determine a start of a range and end of a range of consistently changing values in one or more index pages of said database index and an index updater to update said database index based on a sequence determined by said sequence determiner, while suspending writing of one or more values that lie within start of said range and end of said range of values. A method embodiment includes scanning an index, determining a pattern of changing values in one or more index pages of said index and selectively updating said index based on said determining step to minimize index insertions.

Abstract: Disclosed herein are system, method, and computer program product embodiments for constructing an index for a database table. An index that comprises a data structure may be created. The index can then be populated with data from the database table. When a request to modify the database table is received, the method may determine that the request to modify the database table relates to a portion of the database table corresponding to a portion of the index that has yet to be populated. An entry indicating the requested modification can be inserted into the portion of the index that has yet to be populated.

Abstract: Disclosed herein are systems, methods, and computer readable storage media for a database system using solid state drives as a second level cache. A database system includes random access memory configured to operate as a first level cache, solid state disk drives configured to operate as a persistent second level cache, and hard disk drives configured to operate as disk storage. The database system also includes a cache manager configured to receive a request for a data page and determine whether the data page is in cache or disk storage. If the data page is on disk, or in the second level cache, it is copied to the first level cache. If copying the data page results in an eviction, the evicted data page is copied to the second level cache. At checkpoint, dirty pages stored in the second level cache are flushed in place in the second level cache.

Abstract: A cache management system providing improved page latching methodology. A method providing access to data in a multi-threaded computing system comprises: providing a cache containing data pages and a mapping to pages in memory of the multi-threaded computing system; associating a latch with each page in cache to regulate access, the latch allowing multiple threads to share access to the page for reads and a single thread to obtain exclusive access to the page for writes; in response to a request from a first thread to read a particular page, determining whether the particular page is in cache without acquiring any synchronization object regulating access and without blocking access by other threads; if the particular page is in cache, reading the particular page unless another thread has exclusively latched the particular page; and otherwise, if the particular page is not in cache, bringing the page into cache.

Abstract: Systems, methods and computer program products for compression of database indexes are described herein. A system embodiment includes a sequence determiner to scan a database index and to determine a start of a range and end of a range of consistently changing values in one or more index pages of said database index and an index updater to update said database index based on a sequence determined by said sequence determiner, while suspending writing of one or more values that lie within start of said range and end of said range of values. A method embodiment includes scanning an index, determining a pattern of changing values in one or more index pages of said index and selectively updating said index based on said determining step to minimize index insertions.

Abstract: A cache management system providing improved page latching methodology is described. In one embodiment, for example, a method is described for providing access to data in a multi-threaded computing system, the method comprises steps of: providing a cache containing pages of data in memory of the multi-threaded computing system; associating a latch with each page in the cache to regulate access to the page, the latch allowing multiple threads to share access to the page for read operations and a single thread to obtain exclusive access to the page for write operations; in response to a request from a first thread to read a particular page, determining whether the particular page is in the cache without blocking access by other threads to pages in the cache; if the particular page is in the cache, attempting to obtain the latch for purposes of reading the particular page; and allowing the first thread to read the particular page unless a second thread has latched the particular page on an exclusive basis.

Abstract: An improved method for creating an index based on a path-compressed binary trie in a database system comprising database tables and indexes on those tables is described. For a given index to be created, a path-compressed binary trie for the given index is determined. The path-compressed binary trie comprises internal nodes and leaf nodes. Based on a traversal of the path-compressed binary trie, an index is created comprising a first array of internal nodes encountered during the traversal, and a second array of leaf nodes encountered during the traversal. The database system employs said first and second arrays for providing index-based access for a given key value.

Abstract: An improved method for creating an index based on a path-compressed binary trie in a database system comprising database tables and indexes on those tables is described. For a given index to be created, a path-compressed binary trie for the given index is determined. The path-compressed binary trie comprises internal nodes and leaf nodes. Based on a traversal of the path-compressed binary trie, an index is created comprising a first array of internal nodes encountered during the traversal, and a second array of leaf nodes encountered during the traversal. The database system employs said first and second arrays for providing index-based access for a given key value.