Abstract: A computer implemented method for updating a secondary database includes receiving transaction log information from a primary database system at a secondary database system. The transaction log information comprises a first marker identifying a beginning of an operation comprising a plurality of sub-operations and a second marker identifying an end of the operation. The transaction log information is sequentially parsed, a first marker is identified, and a lock state is created after replaying the first marker to ensure proper serialization of the plurality of sub-operations. The lock state is terminated after replaying the second marker. Related apparatus, systems, techniques and articles are also described.

Abstract: A request for an entry of a dictionary having multiple dictionary blocks may be received, where the dictionary stores string values associated with corresponding value identifiers (IDs) for dictionary compression. One of the dictionary blocks may be selectively loaded into memory, and the dictionary block that has been loaded into memory may be searched. A value ID directory may be constructed in memory, where the value ID directory includes last value IDs for the dictionary blocks, and each of the last value IDs is mapped to an index of one of the dictionary blocks that includes a string value for that last value ID. A separator directory may also be constructed in memory, where the separator directory includes separators for the dictionary blocks, and each of the separators is mapped to an index of one of the dictionary blocks that includes a string value for that separator.

Abstract: Techniques for handling large string values in a string dictionary within an in-memory database system are presented. The string dictionary can be a paged-column main dictionary. For example, a value block of a dictionary may be loaded into memory, where the dictionary stores values for dictionary compression. The value block may include a first portion of a large string value and one or more logical pointers to one or more large string pages containing a remainder of the large string value. The large string value may be materialized into a contiguous location in memory, and a pointer to the materialized large string value may be created.

Abstract: Disclosed herein are system, method, and computer program product embodiments for determining row visibility states. An embodiment operates by initializing a result bitmap for rows based on visible row state values of row state bitmaps corresponding to the rows, wherein values of the result bitmap indicate whether the row are visible or not visible; determining one or more rows whose corresponding visible row state values indicate that the one or more rows are not visible; and determining whether to update the result bitmap to indicate that the one or more rows are visible based on one or more versioned row state values, wherein the one or more versioned row state values indicate whether the corresponding rows are not visible or possibly visible.

Abstract: Disclosed herein are system, method, and computer program product embodiments for performing garbage collection on a multi-version concurrency control information in the database management system. An embodiment operates by determining, using multi-version concurrency control (MVCC) information, when a row manipulated by a write transaction is visible to a plurality of readers accessing a table that includes the row. The MVCC information for the row includes at least a creation timestamp, a destruction timestamp and a row state. Once the row is visible to the plurality of readers, garbage collecting at least the creation timestamp or the destruction timestamp in the MVCC information. After the creation timestamp or destruction timestamp have been garbage collected, the plurality of readers use the row state to determine accessibility of the row in the table.

Abstract: The subject matter disclosed herein provides methods for the dual storage of data using an in-memory array and an on-disk page structure. An in-memory array holding a column of data can be maintained. One or more pages can be maintained. Each of the one or more pages can have one or more rows for storing the column of data. Random access can be provided to a subset of the one or more rows by at least loading the subset of rows from the one or more pages to the in-memory array without loading all of the rows from the one or more pages. Related apparatus, systems, techniques, and articles are also described.

Abstract: Disclosed herein are system, method, and computer program product embodiments for bloom filter versioning. A request to add a new value to a first bitmap is received, wherein the first bitmap includes a bit sequence corresponding to a most recently added value. It is determined that the first bitmap has exceeded a threshold. A second bitmap is generated, the second bitmap including more bits than the first bitmap. The first bitmap is deleted based upon a determination that a number of readers accessing the first bitmap is zero.

Abstract: A delta store giving row-level versioning semantics to a non-row-level versioning underlying store is described. An example method includes establishing a column-based in-memory database including a main store and a delta store, where the main store allows only non-concurrent transactions on a same table and the delta store has a plurality of row-visibility bitmaps implementing a row-level versioning mechanism that allows concurrent transactions on the same table. A local RID space is established for a table fragment, that for each table in the database, the data of the table is stored in one or more main table fragment in the main store and in one or more delta table fragments in the delta store. Each table fragment has a local RID space, and the local RID space is a collection of one-based contiguous integer local RIDs (Row IDs) describing local positions of the rows of the table fragment.

Abstract: Persisting data in logical tiers of an in-memory database based on aging rules of the database system. The database system can be configured to age data by moving the data from a first logical tier supporting high performance data access to a second logical tier supporting lower performance data access, thereby improving the operational efficiency of the database system.

Abstract: Cracking page-loadable columns for in-memory data management is described herein. An embodiment operates by accessing a column according to a received query, determining that the received query requires a non-critical data structure associated with the column, and rebuilding the non-critical data structure from data associated with the column.

Abstract: The subject matter disclosed herein provides methods for materializing data from an in-memory array to one or more pages. An in-memory array holding a column of data can be maintained. One or more pages can be maintained. Each of the one or more pages can have one or more rows for storing the column of data. At least one of the one or more pages can be marked for materialization. The column of data can be materialized by copying the data from the in-memory array to the one or more rows of the one or more pages. The materializing can be based on the marking. Related apparatus, systems, techniques, and articles are also described.

Abstract: A plurality of transactions is handled in a database. Each transaction includes a plurality of operations on at least one record in the database with at least two of the transactions being handled concurrently. Thereafter, a temporary timestamp is assigned to each record. The temporary timestamp are based, at least in part, on the corresponding transaction. Further, a final timestamp is assigned to each record with a commit operation. It can later be determined, using a visibility function and based on the assigned temporary timestamps and final timestamps, which of the records are visible in a consistent view for a first transaction. Based on such determination, access can be provided to those records determined to be visible to be accessed to the first transaction.

Abstract: A plurality of transactions are handled in a database. Each transaction comprises a plurality of operations on at least one record in the database with at least two of the transactions being handled concurrently. Thereafter, a temporary timestamp is assigned to each record that is based, at least in part, on the corresponding transaction. A first transaction among the plurality of transactions is subsequently committed. Afterwards, re-stamping of at least one commit timestamp modified by the first transaction is delegated. Related apparatus, systems, techniques and articles are also described.

Abstract: Disclosed herein are system, method, and computer program product embodiments for exporting and importing database tables in a database management system using multi-version concurrency control (MVCC). An embodiment operates by first initiating an export of a database table in the database management system. During the export transaction, temporary creation and deletion timestamps are identified from MVCC information associated with each row in the database table. Each temporary timestamp is associated with a database transaction. A timestamp map is then generated that associates the identified temporary timestamps with commit timestamps of the associated transactions. The database table, associated MVCC information, timestamp map, and snapshot timestamp of the export transaction are written to a binary file.

Abstract: Log records are accessed as part of a database operation in a database. The log records log insert, update, and delete operations in the database and include, for each row, a row position, a fragment identifier (ID), and a row ID. Thereafter, as part of the database operation, rows specified by the log records are located by: using the fragment identifier and the row position within the corresponding record of the log if the fragment with the corresponding fragment identifier is still available, otherwise, using the row identifier within the corresponding record of the log to look up the row position in an index of a corresponding row identifier column. The database operation is then finalized using the located rows. Related apparatus, systems, techniques and articles are also described.

Abstract: Disclosed herein are system and method embodiments for generating a paged inverted index. An embodiment is generated by storing a first data structure and the second data structure in a plurality of pages, where the plurality of pages are stored in the one or more memories. The first data structure is stored in the plurality of pages and includes a plurality of value identifiers, where a value identifier corresponds to an offset. The second data structure stored in the plurality of pages includes a plurality of row positions, wherein a row position is at a location that corresponds to the offset in the first data structure and identifies a position of row in a table that stores data associated with the value ID.

Abstract: Disclosed herein are system, method, and computer program product embodiments for performing garbage collection in a database management system with a multi-version concurrency control. An embodiment operate by qualifying a multi-version concurrency control (MVCC) block for garbage collection, where the MVCC block includes multiple cells, each cell corresponding to a row of a table that was acted on by a transaction. Determining that the MVCC block can be garbage collected based on MVCC information in the MVCC block, where the MVCC information includes information that determines whether changes made to rows in the multiple cells are visible in a database management system. Based on the determining, garbage collecting the MVCC block.

Abstract: First page data from one or more pages stored on a persistent storage can be loaded into a page buffer in a main system memory of one or more computing systems, and second page data that includes first dictionary values of a first dictionary for a first database column can be loaded and pinned into the page buffer in the main system memory. First contents of a first dictionary block containing first re-direction references to the first dictionary can be copied from the loaded first page data into a first in-memory array, and the first page data can be deallocated from the page buffer. The first re-direction references can direct to the first dictionary values of the first dictionary in the pinned second page data.

Abstract: A delta store giving row-level versioning semantics to a non-row-level versioning underlying store is described. An example method includes establishing a column-based in-memory database including a main store and a delta store, where the main store does not allow concurrent transactions on a same table and the delta store has a plurality of row-visibility bitmaps implementing a row-level versioning mechanism that allows concurrent transactions on the same table. A transaction associated with the column-based in-memory database is received. For each table read by the transaction, a version of the table in the delta store that represents a transaction-consistent snapshot of the database visible to the transaction is determined. Each table is represented in the main store and the delta store; and each version of the table is represented by one or more bitmaps.

Abstract: The subject matter disclosed herein provides methods for materializing data from an in-memory array to one or more pages. An in-memory array holding a column of data can be maintained. One or more pages can be maintained. Each of the one or more pages can have one or more rows for storing the column of data. At least one of the one or more pages can be marked for materialization. The column of data can be materialized by copying the data from the in-memory array to the one or more rows of the one or more pages. The materializing can be based on the marking. Related apparatus, systems, techniques, and articles are also described.