Abstract: A compression manager may store, within an index vector, a plurality of value identifiers (IDs), each value ID representing a value within a database. A page generator may designate a number of the value IDs as defining a page within the index vector, so that the index vector includes a plurality of pages, each page including the number of value IDs. The page generator may store the index vector in a secondary memory of a main memory database. An iterator may access a requested value ID, and a page loader may load a corresponding page of the index vector that contains the requested value ID into the main memory database.

Abstract: Objects across multiple database manipulation language (DML) redo log records are cached, during continuous replay of redo log records on a secondary system, for the same table partition to enable reuse of such cached objects. Later, these cached objects can be reused as they are accessed during the sequential processing of DML redo records for a specific table partition.

Abstract: A computer implemented method for manipulating data comprises receiving log transaction information from a primary database system at a secondary database system and parsing the log transaction information to identify data objects that have been modified on the primary database system. The secondary database system determines whether the data objects are present in main memory on the secondary database system and whether the log transaction corresponds to a data manipulation language (DML) query or a data definition language (DDL) query. If the data objects are not in the main memory and the query is a DML query, they are copied from persistence data volumes to the main memory. The data objects in main memory are modified by replaying the log transactions on the data objects in the main memory. Then, the data objects in the persistence data volumes are modified. Related apparatus, systems, techniques and articles are also described.

Abstract: A computer implemented method for initializing a secondary database system includes receiving table state information from a primary database system at a secondary database system. The table state information includes information identifying which tables had an open operation during a savepoint event. Metadata associated with the tables is parsed to identify table state information from the metadata and a lock is created for each table identified as having an open operation during the savepoint event. Afterwards, log transaction information is sequentially parsed. Related apparatus, systems, techniques and articles are also described.

Abstract: A request for an entry of a dictionary having multiple dictionary blocks may be received, where the dictionary stores string values associated with corresponding value identifiers (IDs) for dictionary compression. One of the dictionary blocks may be selectively loaded into memory, and the dictionary block that has been loaded into memory may be searched. A value ID directory may be constructed in memory, where the value ID directory includes last value IDs for the dictionary blocks, and each of the last value IDs is mapped to an index of one of the dictionary blocks that includes a string value for that last value ID. A separator directory may also be constructed in memory, where the separator directory includes separators for the dictionary blocks, and each of the separators is mapped to an index of one of the dictionary blocks that includes a string value for that separator.

Abstract: A computer implemented method for updating a secondary database includes receiving transaction log information from a primary database system at a secondary database system. The transaction log information comprises a first marker identifying a beginning of an operation comprising a plurality of sub-operations and a second marker identifying an end of the operation. The transaction log information is sequentially parsed, a first marker is identified, and a lock state is created after replaying the first marker to ensure proper serialization of the plurality of sub-operations. The lock state is terminated after replaying the second marker. Related apparatus, systems, techniques and articles are also described.

Abstract: Techniques for handling large string values in a string dictionary within an in-memory database system are presented. The string dictionary can be a paged-column main dictionary. For example, a value block of a dictionary may be loaded into memory, where the dictionary stores values for dictionary compression. The value block may include a first portion of a large string value and one or more logical pointers to one or more large string pages containing a remainder of the large string value. The large string value may be materialized into a contiguous location in memory, and a pointer to the materialized large string value may be created.

Abstract: A request for an entry of a dictionary having multiple dictionary blocks may be received, where the dictionary stores string values associated with corresponding value identifiers (IDs) for dictionary compression. One of the dictionary blocks may be selectively loaded into memory, and the dictionary block that has been loaded into memory may be searched. A value ID directory may be constructed in memory, where the value ID directory includes last value IDs for the dictionary blocks, and each of the last value IDs is mapped to an index of one of the dictionary blocks that includes a string value for that last value ID. A separator directory may also be constructed in memory, where the separator directory includes separators for the dictionary blocks, and each of the separators is mapped to an index of one of the dictionary blocks that includes a string value for that separator.

Abstract: Disclosed herein are system, method, and computer program product embodiments for determining row visibility states. An embodiment operates by initializing a result bitmap for rows based on visible row state values of row state bitmaps corresponding to the rows, wherein values of the result bitmap indicate whether the row are visible or not visible; determining one or more rows whose corresponding visible row state values indicate that the one or more rows are not visible; and determining whether to update the result bitmap to indicate that the one or more rows are visible based on one or more versioned row state values, wherein the one or more versioned row state values indicate whether the corresponding rows are not visible or possibly visible.

Abstract: A delta store giving row-level versioning semantics to a non-row-level versioning underlying store is described. An example method includes establishing a column-based in-memory database including a main store and a delta store, where the main store allows only non-concurrent transactions on a same table and the delta store has a plurality of row-visibility bitmaps implementing a row-level versioning mechanism that allows concurrent transactions on the same table. A local RID space is established for a table fragment, that for each table in the database, the data of the table is stored in one or more main table fragment in the main store and in one or more delta table fragments in the delta store. Each table fragment has a local RID space, and the local RID space is a collection of one-based contiguous integer local RIDs (Row IDs) describing local positions of the rows of the table fragment.

Abstract: Data records of a data set can be stored in multiple main part fragments retained in on-disk storage. Each fragment can include a number of data records that is equal to or less than a defined maximum fragment size. Using a compression that is optimized for each fragment, each fragment can be compressed. After reading at least one of the fragments into main system memory from the on-disk storage, an operation can be performed on the fragment or fragments while the in the main system memory.

Abstract: A plurality of data records that comprise a data set can be stored in a plurality of main part fragments such that each main part fragment includes a subset of the set of data records. Each fragment of the plurality of main part fragments can be assigned a relative data temperature. A newly arrived data record for storage in the data set can be placed in a delta part, and a merge can be performed to add the newly arrived data record to a corresponding main part fragment. The performing of the merge can occur more quickly if the corresponding main part fragment has a higher relative data temperature than if the corresponding main part fragment has a lower relative data temperature.

Abstract: Data records of a data set can be stored in multiple main part fragments retained in on-disk storage. Each fragment can include a number of data records that is equal to or less than a defined maximum fragment size. Using a compression that is optimized for each fragment, each fragment can be compressed. After reading at least one of the fragments into main system memory from the on-disk storage, an operation can be performed on the fragment or fragments while the in the main system memory.

Abstract: Disclosed herein are system and method embodiments for generating a paged inverted index. An embodiment is generated by storing a first data structure and the second data structure in a plurality of pages, where the plurality of pages are stored in the one or more memories. The first data structure is stored in the plurality of pages and includes a plurality of value identifiers, where a value identifier corresponds to an offset. The second data structure stored in the plurality of pages includes a plurality of row positions, wherein a row position is at a location that corresponds to the offset in the first data structure and identifies a position of row in a table that stores data associated with the value ID.

Abstract: A delta store giving row-level versioning semantics to a non-row-level versioning underlying store is described. An example method includes establishing a column-based in-memory database including a main store and a delta store, where the main store does not allow concurrent transactions on a same table and the delta store has a plurality of row-visibility bitmaps implementing a row-level versioning mechanism that allows concurrent transactions on the same table. A transaction associated with the column-based in-memory database is received. For each table read by the transaction, a version of the table in the delta store that represents a transaction-consistent snapshot of the database visible to the transaction is determined. Each table is represented in the main store and the delta store; and each version of the table is represented by one or more bitmaps.

Abstract: The subject matter disclosed herein relates to out of memory error handling in a database system. A database operation can be received by an in-memory database. The database operation can be for a database object stored in a database table that can be represented as a plurality of pages that can be persisted in a page chain. The in-memory database can reserve out of memory space sufficient to load a predetermined number of pages in the page chain into memory. The in-memory database can iteratively process each page in the page chain until completion of the database operation. The iterative process can include loading the page from persistence into memory of the in-memory database, performing at least a portion of the database operation using the loaded page, and unloading the page from the memory of the in-memory database. Related apparatus, systems, techniques, and articles are also described.

Abstract: The present disclosure involves systems, software, and computer implemented methods for moving a table from a source node to a destination node. In one example, the method includes receiving metadata associated with an in-memory database table stored at a source node. A table container is created responsive to receiving the metadata. The destination node sequentially requests, from the source node, portions of the table, wherein the table is serialized at the source node to provide a serialized sequence of table portions. Sequentially requesting comprises sending a request for a next portion of the table after processing a received portion, which includes receiving a portion of the serialized table, deserializing the received portion, adding the deserialized portion to the created table container, and in response to an end of file indication associated with the received portion, ending the requests and finalizing the table.

Abstract: Data records of a data set can be stored in multiple main part fragments retained in on-disk storage. Each fragment can include a number of data records that is equal to or less than a defined maximum fragment size. Using a compression that is optimized for each fragment, each fragment can be compressed. After reading at least one of the fragments into main system memory from the on-disk storage, an operation can be performed on the fragment or fragments while the in the main system memory.

Abstract: The present disclosure involves systems, software, and computer implemented methods for moving a table from a source node to a destination node. In one example, the method includes receiving metadata associated with an in-memory database table stored at a source node. A table container is created responsive to receiving the metadata. The destination node sequentially requests, from the source node, portions of the table, wherein the table is serialized at the source node to provide a serialized sequence of table portions. Sequentially requesting comprises sending a request for a next portion of the table after processing a received portion, which includes receiving a portion of the serialized table, deserializing the received portion, adding the deserialized portion to the created table container, and in response to an end of file indication associated with the received portion, ending the requests and finalizing the table.

Abstract: Disclosed herein are system, method, and computer program product embodiments for altering the data type of a column in a database. An embodiment operates by converting an original dictionary associated with a column into a new dictionary. The new dictionary stores the values of the original dictionary using a different data type. An index vector containing the keys of the original dictionary is then updated to contain the associated keys of the new dictionary. Because the size of the original dictionary is often substantially smaller than the number of rows in the associated column, this dictionary conversion decreases the computation cost to the database system of altering the data type of the column and reduces or even minimizes database downtime for users.