Abstract: As part of a columnar in-memory database, value identifiers are inserted into a backing array in-memory until such time that it is determined that such backing array does not have sufficient capacity. A new backing array is then generated that includes the value identifiers in the old backing array and which has sufficient capacity. The old backing array can be flushed from memory when there are no active operations using such backing array. Such an arrangement allows for readers and non-structural writers to operate concurrently.

Abstract: A computer implemented method for manipulating data comprises receiving log transaction information from a primary database system at a secondary database system and parsing the log transaction information to identify data objects that have been modified on the primary database system. The secondary database system determines whether the data objects are present in main memory on the secondary database system. If the data objects are not in the main memory, they are copied from persistence data volumes to the main memory. The data objects are modified by replaying the log transactions on the data objects in the main memory. Then, the data objects in the persistence data volumes are modified. Related apparatus, systems, techniques and articles are also described.

Abstract: Disclosed herein are system, method, and computer program product embodiments for bloom filter versioning. A request to add a new value to a first bitmap is received, wherein the first bitmap includes a bit sequence corresponding to a most recently added value. It is determined that the first bitmap has exceeded a threshold. A second bitmap is generated, the second bitmap including more bits than the first bitmap. The first bitmap is deleted based upon a determination that a number of readers accessing the first bitmap is zero.

Abstract: A delta store giving row-level versioning semantics to a non-row-level versioning underlying store is described. An example method includes establishing a column-based in-memory database including a main store and a delta store, where the main store allows only non-concurrent transactions on a same table and the delta store has a plurality of row-visibility bitmaps implementing a row-level versioning mechanism that allows concurrent transactions on the same table. A local RID space is established for a table fragment, that for each table in the database, the data of the table is stored in one or more main table fragment in the main store and in one or more delta table fragments in the delta store. Each table fragment has a local RID space, and the local RID space is a collection of one-based contiguous integer local RIDs (Row IDs) describing local positions of the rows of the table fragment.

Abstract: Persisting data in logical tiers of an in-memory database based on aging rules of the database system. The database system can be configured to age data by moving the data from a first logical tier supporting high performance data access to a second logical tier supporting lower performance data access, thereby improving the operational efficiency of the database system.

Abstract: Disclosed herein are system, method, and computer program product embodiments for providing a spatio-temporal index for high-update workloads and query processing. An embodiment operates by a first thread retrieving an update record from a first queue, the update record comprising a location component and a temporal component indicating a location of one of a plurality of mobile devices at a specified time, and updating a columnar-store database with the update record. The embodiment further operates by a second thread identifying a spatial grid of a spatial temporal index within a memory corresponding to the location component of the update record, and updating a temporal index of the spatial grid based on the temporal component of the update record.

Abstract: The subject matter disclosed herein provides methods for reformatting a page due to a rollover. An in-memory array holding a column of data can be maintained. One or more pages can be maintained. Each page can have one or more rows for storing the column of data. The column of data in the in-memory array can be monitored for a change. A rollover can be performed on at least one of the pages based on the change. The rollover can reformat the at least one page by rewriting metadata associated with the at least one page. Related apparatus, systems, techniques, and articles are also described.

Abstract: Disclosed herein are system, method, and computer program product embodiments for bloom filter versioning. A request to add a new value to a first bitmap is received, wherein the first bitmap includes a bit sequence corresponding to a most recently added value. It is determined that the first bitmap has exceeded a threshold. A second bitmap is generated, the second bitmap including more bits than the first bitmap. The first bitmap is deleted based upon a determination that a number of readers accessing the first bitmap is zero.

Abstract: At least one read operation is concurrently performed with at least one write operation that each insert a key/value pair into a backing array of a backing hash table of a hash table forming part of a columnar in-memory database. The backing array maps a plurality of pointers each to a respective bucket. Each bucket includes at least one state bit and a hashed value of a corresponding key. Thereafter, for each write operation, a first available position in the backing array at which a pointer to a new bucket containing the key/value pair can be inserted is iteratively determined (such that each first available position has no corresponding pre-existing pointer). Subsequently, for each write operation, the pointer to the new bucket containing the key/value pair is inserted at the corresponding first determined position in the backing array. Related apparatus, systems, techniques and articles are also described.

Abstract: As part of a columnar in-memory database, value identifiers are inserted into a backing array in-memory until such time that it is determined that such backing array does not have sufficient capacity. A new backing array is then generated that includes the value identifiers in the old backing array and which has sufficient capacity. The old backing array can be flushed from memory when there are no active operations using such backing array. Such an arrangement allows for readers and non-structural writers to operate concurrently. Related apparatus, systems, techniques and articles are also described.

Abstract: A set-oriented locking based on in-memory bitmap for a column store database is described. An example method includes establishing a column-based in-memory database including a main store and a delta store, where the delta store has a plurality of row-visibility lock bitmaps visible to transactions at various points in time. The lock bitmaps represent a bit encoding to indicate whether there are granted row locks tables in the database. A delete or an update statement is executed with a transaction on a table. A set of row locks on rows of the table manipulated by the delete or the update statement are requested to preclude other transactions from currently deleting or updating the same rows. Accordingly, set operations are performed on the lock bitmap to manage the set of row locks associated with the transaction.

Abstract: Disclosed herein are system, method, and computer program product embodiments for implementing a bitmap for a column store database. An embodiment operates by creating, by at least one processor, a bitmap identifying rows in a column store database. The bitmap may include a list of bit chunks, a bit chunk including an offset being a natural number indicating a chunk size, and a bit specification including one of an ordered row id list, a contiguous row id sequence, and a bit vector. In addition, the embodiment includes performing database operations using the bitmap.

Abstract: A locking mechanism in a delta-store-based database to support long running transactions across multiple RID spaces is described. An example method includes establishing a column-based in-memory database including a main store and a delta store. A delete or an update statement is executed with a transaction on a table having plurality of table versions. The table versions are represented by bitmaps in the delta store and the bitmaps and table fragments corresponding to the table versions implement RID spaces for the table. A lock on a row of the table manipulated by the delete or the update statement is requested to preclude other transaction from deleting or updating an obsolete version of data. Upon a successful validation that the row to be locked is not the obsolete version in the RID spaces of the table, a lock is granted to the transaction.

Abstract: Disclosed herein are system, method, and computer program product embodiments for populating a hash index and returning a handle to the hash index. An embodiment operates by determining, by at least one processor, during query optimization that a first database query has a query execution plan comprising a sub-query which executes N times a correlated predicate having an operator being one of equal and not equal to a base column. A cost of creating and probing the hash index N times and a cost of fully scanning the base column N times are compared based on the correlated predicate. Based on the comparing, it is determined whether to create on-demand a hash index.

Abstract: A set-oriented locking based on in-memory bitmap for a column store database is described. An example method includes establishing a column-based in-memory database including a main store and a delta store, where the delta store has a plurality of row-visibility lock bitmaps visible to transactions at various points in time. The lock bitmaps represent a bit encoding to indicate whether there are granted row locks tables in the database. A delete or an update statement is executed with a transaction on a table. A set of row locks on rows of the table manipulated by the delete or the update statement are requested to preclude other transactions from currently deleting or updating the same rows. Accordingly, set operations are performed on the lock bitmap to manage the set of row locks associated with the transaction.

Abstract: A locking mechanism in a delta-store-based database to support long running transactions across multiple RID spaces is described. An example method includes establishing a column-based in-memory database including a main store and a delta store. A delete or an update statement is executed with a transaction on a table having plurality of table versions. The table versions are represented by bitmaps in the delta store and the bitmaps and table fragments corresponding to the table versions implement RID spaces for the table. A lock on a row of the table manipulated by the delete or the update statement is requested to preclude other transaction from deleting or updating an obsolete version of data. Upon a successful validation that the row to be locked is not the obsolete version in the RID spaces of the table, a lock is granted to the transaction.

Abstract: A delta store giving row-level versioning semantics to a non-row-level versioning underlying store is described. An example method includes establishing a column-based in-memory database including a main store and a delta store, where the main store allows only non-concurrent transactions on a same table and the delta store has a plurality of row-visibility bitmaps implementing a row-level versioning mechanism that allows concurrent transactions on the same table. A local RID space is established for a table fragment, that for each table in the database, the data of the table is stored in one or more main table fragment in the main store and in one or more delta table fragments in the delta store. Each table fragment has a local RID space, and the local RID space is a collection of one-based contiguous integer local RIDs (Row IDs) describing local positions of the rows of the table fragment.

Abstract: Disclosed herein are system, method, and computer program product embodiments for providing a spatio-temporal index for high-update workloads and query processing. An embodiment operates by a first thread retrieving an update record from a first queue, the update record comprising a location component and a temporal component indicating a location of one of a plurality of mobile devices at a specified time, and updating a columnar-store database with the update record. The embodiment further operates by a second thread identifying a spatial grid of a spatial temporal index within a memory corresponding to the location component of the update record, and updating a temporal index of the spatial grid based on the temporal component of the update record.

Abstract: Disclosed herein are system, method, and computer program product embodiments for populating a hash index and returning a handle to the hash index. An embodiment operates by determining, by at least one processor, during query optimization that a first database query has a query execution plan comprising a sub-query which executes N times a correlated predicate having an operator being one of equal and not equal to a base column. A cost of creating and probing the hash index N times and a cost of fully scanning the base column N times are compared based on the correlated predicate. Based on the comparing, it is determined whether to create on-demand a hash index.

Abstract: Disclosed herein are system, method, and computer program product embodiments for implementing a bitmap for a column store database. An embodiment operates by creating, by at least one processor, a bitmap identifying rows in a column store database. The bitmap may include a list of bit chunks, a bit chunk including an offset being a natural number indicating a chunk size, and a bit specification including one of an ordered row id list, a contiguous row id sequence, and a bit vector. In addition, the embodiment includes performing database operations using the bitmap.