Abstract: Disclosed herein are system, method, and computer program product embodiments for determining row visibility states. An embodiment operates by initializing a result bitmap for rows based on visible row state values of row state bitmaps corresponding to the rows, wherein values of the result bitmap indicate whether the row are visible or not visible; determining one or more rows whose corresponding visible row state values indicate that the one or more rows are not visible; and determining whether to update the result bitmap to indicate that the one or more rows are visible based on one or more versioned row state values, wherein the one or more versioned row state values indicate whether the corresponding rows are not visible or possibly visible.

Abstract: A plurality of data records of a data set can be stored in a plurality of main part fragments, at least one of which is an old fragment stored on-disk. A number of one or more data records in the old fragment that have been marked for deletion can be determined to be greater than a threshold number, and the old fragment can be loaded into main system memory. A merge of the old fragment can be performed to remove the one or more data records marked for deletion.

Abstract: Data records of a data set can be stored in multiple main part fragments retained in on-disk storage. Each fragment can include a number of data records that is equal to or less than a defined maximum fragment size. Using a compression that is optimized for each fragment, each fragment can be compressed. After reading at least one of the fragments into main system memory from the on-disk storage, an operation can be performed on the fragment or fragments while the in the main system memory.

Abstract: A locking mechanism in a delta-store-based database to support long running transactions across multiple RID spaces is described. An example method includes establishing a column-based in-memory database including a main store and a delta store. A delete or an update statement is executed with a transaction on a table having plurality of table versions. The table versions are represented by bitmaps in the delta store and the bitmaps and table fragments corresponding to the table versions implement RID spaces for the table. A lock on a row of the table manipulated by the delete or the update statement is requested to preclude other transaction from deleting or updating an obsolete version of data. Upon a successful validation that the row to be locked is not the obsolete version in the RID spaces of the table, a lock is granted to the transaction.

Abstract: A compression manager may store, within an index vector, a plurality of value identifiers (IDs), each value ID representing a value within a database. A page generator may designate a number of the value IDs as defining a page within the index vector, so that the index vector includes a plurality of pages, each page including the number of value IDs. The page generator may store the index vector in a secondary memory of a main memory database. An iterator may access a requested value ID, and a page loader may load a corresponding page of the index vector that contains the requested value ID into the main memory database.

Abstract: Data records of a data set can be stored in multiple main part fragments, each of which includes a subset of the set of data records. A relative age can be assigned to each main part fragment, and a fragment-specific index segment can be created for a newest of the main part fragments. The fragment-specific index segment can provide a lookup ability for logical identifiers of data records in just the newest of the main part fragments. A multi-fragment index segment can span two or more older main fragments. The multi-fragment index segment can provide a lookup ability for logical identifiers of data records in the two or more older main part fragments.

Abstract: A locking mechanism in a delta-store-based database to support long running transactions across multiple RID spaces is described. An example method includes establishing a column-based in-memory database including a main store and a delta store. A delete or an update statement is executed with a transaction on a table having plurality of table versions. The table versions are represented by bitmaps in the delta store and the bitmaps and table fragments corresponding to the table versions implement RID spaces for the table. A lock on a row of the table manipulated by the delete or the update statement is requested to preclude other transaction from deleting or updating an obsolete version of data. Upon a successful validation that the row to be locked is not the obsolete version in the RID spaces of the table, a lock is granted to the transaction.

Abstract: A delta store giving row-level versioning semantics to a non-row-level versioning underlying store is described. An example method includes establishing a column-based in-memory database including a main store and a delta store, where the main store allows only non-concurrent transactions on a same table and the delta store has a plurality of row-visibility bitmaps implementing a row-level versioning mechanism that allows concurrent transactions on the same table. A local RID space is established for a table fragment, that for each table in the database, the data of the table is stored in one or more main table fragment in the main store and in one or more delta table fragments in the delta store. Each table fragment has a local RID space, and the local RID space is a collection of one-based contiguous integer local RIDs (Row IDs) describing local positions of the rows of the table fragment.

Abstract: Techniques for handling large string values in a string dictionary within an in-memory database system are presented. The string dictionary can be a paged-column main dictionary. For example, a value block of a dictionary may be loaded into memory, where the dictionary stores values for dictionary compression. The value block may include a first portion of a large string value and one or more logical pointers to one or more large string pages containing a remainder of the large string value. The large string value may be materialized into a contiguous location in memory, and a pointer to the materialized large string value may be created.

Abstract: A request for an entry of a dictionary having multiple dictionary blocks may be received, where the dictionary stores string values associated with corresponding value identifiers (IDs) for dictionary compression. One of the dictionary blocks may be selectively loaded into memory, and the dictionary block that has been loaded into memory may be searched. A value ID directory may be constructed in memory, where the value ID directory includes last value IDs for the dictionary blocks, and each of the last value IDs is mapped to an index of one of the dictionary blocks that includes a string value for that last value ID. A separator directory may also be constructed in memory, where the separator directory includes separators for the dictionary blocks, and each of the separators is mapped to an index of one of the dictionary blocks that includes a string value for that separator.

Abstract: Data records of a data set can be stored in a plurality of main part fragments retained in on-disk storage. A size of the data set can be compared to an available size of main system memory. All of the plurality of main part fragments can be fully loaded into the main system memory when the available size of the main system memory is larger than the size of the data set. Alternatively, one or more of the of main part fragments can be paged into the main system memory on demand in response to a data access request when the available size of the main system memory is smaller than the size of the data set and the data access request can be satisfied by providing access to a subset of the main part fragments, or access can be provided directly to the on-disk main part fragments when the data access request involves random access for projection in the data set and the available size of the main system memory is smaller than the size of the data set.

Abstract: A delta store giving row-level versioning semantics to a non-row-level versioning underlying store is described. An example method includes establishing a column-based in-memory database including a main store and a delta store, where the main store does not allow concurrent transactions on a same table and the delta store has a plurality of row-visibility bitmaps implementing a row-level versioning mechanism that allows concurrent transactions on the same table. A transaction associated with the column-based in-memory database is received. For each table read by the transaction, a version of the table in the delta store that represents a transaction-consistent snapshot of the database visible to the transaction is determined. Each table is represented in the main store and the delta store; and each version of the table is represented by one or more bitmaps.

Abstract: Embodiments described herein generally relate to providing durability of in memory columnar datastores using multiple logs. A computer-implemented method is described. The method includes receiving a request to perform a transaction involving multiple tables in a column-oriented database system. The method further includes performing the transaction on each of the two or more database tables. Performance of the transaction generates two or more commit fragments corresponding to each of the two or more database tables. The method further includes writing each commit fragment to a transaction log for each of the two or more database tables. The method also includes writing a commit record for the transaction to a commit log after all commit fragments have been written.

Abstract: A plurality of data records of a data set can be stored in a plurality of main part fragments, at least one of which is an old fragment stored on-disk. A number of one or more data records in the old fragment that have been marked for deletion can be determined to be greater than a threshold number, and the old fragment can be loaded into main system memory. A merge of the old fragment can be performed to remove the one or more data records marked for deletion.

Abstract: Data records of a data set can be stored in multiple main part fragments, each of which includes a subset of the set of data records. A relative age can be assigned to each main part fragment, and a fragment-specific index segment can be created for a newest of the main part fragments. The fragment-specific index segment can provide a lookup ability for logical identifiers of data records in just the newest of the main part fragments. A multi-fragment index segment can span two or more older main fragments. The multi-fragment index segment can provide a lookup ability for logical identifiers of data records in the two or more older main part fragments.

Abstract: A plurality of data records that comprise a data set can be stored in a plurality of main part fragments such that each main part fragment includes a subset of the set of data records. Each fragment of the plurality of main part fragments can be assigned a relative data temperature. A newly arrived data record for storage in the data set can be placed in a delta part, and a merge can be performed to add the newly arrived data record to a corresponding main part fragment. The performing of the merge can occur more quickly if the corresponding main part fragment has a higher relative data temperature than if the corresponding main part fragment has a lower relative data temperature.

Abstract: Data records of a data set can be stored in multiple main part fragments retained in on-disk storage. Each fragment can include a number of data records that is equal to or less than a defined maximum fragment size. Using a compression that is optimized for each fragment, each fragment can be compressed. After reading at least one of the fragments into main system memory from the on-disk storage, an operation can be performed on the fragment or fragments while the in the main system memory.

Abstract: Data records of a data set can be stored in a plurality of main part fragments retained in on-disk storage. A size of the data set can be compared to an available size of main system memory. All of the plurality of main part fragments can be fully loaded into the main system memory when the available size of the main system memory is larger than the size of the data set. Alternatively, one or more of the of main part fragments can be paged into the main system memory on demand in response to a data access request when the available size of the main system memory is smaller than the size of the data set and the data access request can be satisfied by providing access to a subset of the main part fragments, or access can be provided directly to the on-disk main part fragments when the data access request involves random access for projection in the data set and the available size of the main system memory is smaller than the size of the data set.

Abstract: System, method, computer program product embodiments and combinations and sub-combinations thereof for snapshot isolation support for distributed query processing in a shared disk database cluster are provided. The embodiments include the forming of object identification data in a leader node of query execution in the shared disk database cluster. At least one fragment of the query execution is distributed to at least one worker node in the shared disk database cluster, and the object identification data is utilized for multiversion concurrency control during query execution by the leader node and at least one worker node.

Abstract: System, method, computer program product embodiments and combinations and sub-combinations thereof for snapshot isolation support for distributed query processing in a shared disk database cluster are provided. The embodiments include the forming of object identification data in a leader node of query execution in the shared disk database cluster. At least one fragment of the query execution is distributed to at least one worker node in the shared disk database cluster, and the object identification data is utilized for multiversion concurrency control during query execution by the leader node and at least one worker node.