Abstract: Data records of a data set can be stored in multiple main part fragments retained in on-disk storage. Each fragment can include a number of data records that is equal to or less than a defined maximum fragment size. Using a compression that is optimized for each fragment, each fragment can be compressed. After reading at least one of the fragments into main system memory from the on-disk storage, an operation can be performed on the fragment or fragments while the in the main system memory.

Abstract: Each of a plurality of database objects can be assigned to a specific message-passing worker of a plurality of message-passing workers, each executing on a first logical core that shares with at least a second logical core one or more resources of a physical processor core of a plurality of physical processor cores. The second logical core can execute a job worker of a plurality of job workers that implement procedural code. Exclusive write access can be provided to a database object of the plurality of database objects via a message-passing worker of the plurality of message-passing workers while read-only access is provided to any database object of the plurality of database objects via any of the plurality of job workers. Operations can be executed by the message-passing worker in an order in which request messages are received in a message queue of the message-passing worker.

Abstract: Methods and systems for partitioning and dynamically merging a database index are described. A database index includes a single first-level index partition stored in a data cache. As the first-level index partition in the data cache reaches a predetermined size, it is copied to secondary storage and a new index partition is generated in the data cache. When the number of index partitions in secondary storage reaches some predetermined number, the index partitions are merged to create a single index partition of a higher level in a hierarchy of index partitions having an exponentially increasing size with each increase in level within the hierarchy.

Abstract: A system may include an application programming interface (API) layer, a cache layer, and an object storage/access layer. The API layer may expose an interface to store a business object and an interface to retrieve the business object, and may transmit a request to store the business object and a request to retrieve the business object, and the cache layer may cache the business object and transmit the request to store the business object and the request to retrieve the business object. The object storage/access layer may receive the request to store the business object and, in response to the request to store the business object, to invoke a serialization method exposed by the business object to store the data associated with the object in a data structure.

Abstract: First logical cores supported on physical processor cores in a computing system can be designated for execution of message-passing workers of a plurality of message workers while at least second logical cores supported on the physical processor cores can be designated for execution of procedural code such that resources of a physical processor core supporting the first logical core and the second logical core are shared between a first logical core and a second logical core. A database object in a repository can be assigned to one message-passing worker, which can execute operations on the database object while procedurally coded operations are processed using the second logical core on one or more of the plurality of physical processor cores while the first logical core executes the message-passing worker.

Abstract: Disclosed herein are system and method embodiments for generating a paged inverted index. An embodiment is generated by storing a first data structure and the second data structure in a plurality of pages, where the plurality of pages are stored in the one or more memories. The first data structure is stored in the plurality of pages and includes a plurality of value identifiers, where a value identifier corresponds to an offset. The second data structure stored in the plurality of pages includes a plurality of row positions, wherein a row position is at a location that corresponds to the offset in the first data structure and identifies a position of row in a table that stores data associated with the value ID.

Abstract: Disclosed herein are system, method, and computer program product embodiments for performing garbage collection in a database management system with a multi-version concurrency control. An embodiment operate by qualifying a multi-version concurrency control (MVCC) block for garbage collection, where the MVCC block includes multiple cells, each cell corresponding to a row of a table that was acted on by a transaction. Determining that the MVCC block can be garbage collected based on MVCC information in the MVCC block, where the MVCC information includes information that determines whether changes made to rows in the multiple cells are visible in a database management system. Based on the determining, garbage collecting the MVCC block.

Abstract: First page data from one or more pages stored on a persistent storage can be loaded into a page buffer in a main system memory of one or more computing systems, and second page data that includes first dictionary values of a first dictionary for a first database column can be loaded and pinned into the page buffer in the main system memory. First contents of a first dictionary block containing first re-direction references to the first dictionary can be copied from the loaded first page data into a first in-memory array, and the first page data can be deallocated from the page buffer. The first re-direction references can direct to the first dictionary values of the first dictionary in the pinned second page data.

Abstract: A first block in a page chain can be assigned to a first dictionary for a first column of a database. The page chain can include a plurality of dictionary blocks allocated to a plurality of dictionaries of a plurality of columns including the first column such that the plurality of dictionary blocks is interleaved in the page chain. A new unique value to be added to the first column can be appended to an end of the first dictionary. The plurality of dictionaries can be loaded from the page chain into memory of one or more computers performing operations of a database management system. The loading can include scanning the page chain a single time and assigning the plurality of dictionary blocks to the respective dictionaries of the plurality of dictionaries.

Abstract: A new unique value to be added to a column of a database table can be written to an in-memory array representing a dictionary of the column, and space can be allocated in at least one dictionary block on at least one page maintained on a persistent storage without writing the new unique value to the at least one dictionary block. A pending materialization bit for the at least one page can be set to identify the at least one page for deferred materialization.

Abstract: The subject matter disclosed herein relates to out of memory error handling in a database system. A database operation can be received by an in-memory database. The database operation can be for a database object stored in a database table that can be represented as a plurality of pages that can be persisted in a page chain. The in-memory database can reserve out of memory space sufficient to load a predetermined number of pages in the page chain into memory. The in-memory database can iteratively process each page in the page chain until completion of the database operation. The iterative process can include loading the page from persistence into memory of the in-memory database, performing at least a portion of the database operation using the loaded page, and unloading the page from the memory of the in-memory database. Related apparatus, systems, techniques, and articles are also described.

Abstract: At least one read operation of at least one object of a data container is initiated. The data container includes an anchor object, a first internal data object and a first garbage collection object, the anchor object comprising a pointer to a versioned structure tree. Thereafter, in response to the at least one incompatible write operation, a second internal data object and a second garbage collection object are created for the data container. The second garbage collection object has a reference to the second internal data object. Subsequently, the second internal data object is installed in the anchor object and the first garbage collection object is passed to a garbage collection process so that space used by the first garbage collection object in a database can be reused. Related apparatus, systems, techniques and articles are also described.

Abstract: The subject matter disclosed herein provides methods for materializing data from an in-memory array to one or more pages. An in-memory array holding a column of data can be maintained. One or more pages can be maintained. Each of the one or more pages can have one or more rows for storing the column of data. At least one of the one or more pages can be marked for materialization. The column of data can be materialized by copying the data from the in-memory array to the one or more rows of the one or more pages. The materializing can be based on the marking. Related apparatus, systems, techniques, and articles are also described.

Abstract: First logical cores supported on physical processor cores in a computing system can be designated for execution of message-passing workers of a plurality of message workers while at least second logical cores supported on the physical processor cores can be designated for execution of procedural code such that resources of a physical processor core supporting the first logical core and the second logical core are shared between a first logical core and a second logical core. A database object in a repository can be assigned to one message-passing worker, which can execute operations on the database object while procedurally coded operations are processed using the second logical core on one or more of the plurality of physical processor cores while the first logical core executes the message-passing worker.

Abstract: A database operation is initiated in an in-memory database that stores persistent data in a secondary storage and has at least one associated log. The database operation involves data having a size above a pre-defined threshold. Thereafter, the data is split into a plurality of chunks and the database operation is executed by parallelizing each chunk over a plurality of hierarchically arranged threads. Related systems, apparatus, methods and computer program products are also described.

Abstract: A system and method of logless atomic data movement. An internal transaction is started within a multi-level storage architecture, the internal transaction to merge data from the first level storage structure to the second level storage structure. Committed data is read from a first level storage structure of the multi-level storage architecture as specified by the internal transaction. The committed data from the first level storage structure is inserted into a second level storage structure in a bulk insertion process, and the committed data is marked as being deleted from the first level storage. The internal transaction is then committed to the multi-level storage architecture when the committed data has been inserted into the second level storage structure.

Abstract: A database memory manager determines a size class for each of a plurality of memory allocation requests. The memory manager then, based on the determined size classes, assigns which of a plurality of sub-allocators forming part of a plurality of memory pools should handle each memory allocation request. The sub-allocators assignments are based on predefined size ranges for each size class. The corresponding assigned sub-allocators then identify locations within the memory for each memory allocation request. The corresponding assigned sub-allocators next handle the memory allocation requests to allocate memory at the identified locations such that one of the sub-allocators utilizes both thread local storage and core-striped memory management.

Abstract: A record within a destination virtual file is generated on a destination node of a distributed data storage system. The record comprises (i) a link directed to a partition of a source virtual file stored on a source node and (ii) partition criteria characterizing the partition. The source virtual file is mapped to a chain of linked pages stored in a page buffer of the distributed data storage system and the partitioning criteria is used by at least one of the source node and the destination node to identify data associated with the partition. A request is later received at the destination node to access data defined by the destination virtual file. Data is provided, in response to the request, from the partition of the source virtual file stored on the source node using the link and the partitioning criteria. Related apparatus, systems, techniques and articles are also described.

Abstract: In an executing database instance including a plurality of database nodes, creation of a backup of the executing database instance includes creation of a current savepoint in one of the plurality of database nodes by storing first modified pages of a cache of the database node in a datastore of the database node, transmitting a confirmation after storing the first modified pages, repeatedly identifying second modified pages of the cache and storing the identified second modified pages in the datastore, receiving an instruction to enter a critical phase and stopping the repeated identifying and storing in response to the instruction, blocking updates to the database node and transmitting a second confirmation, and receiving a second instruction and, in response to receiving the second instruction, identifying third modified pages of the cache and storing the third modified pages of the cache in the datastore.

Abstract: Disclosed herein are system, method, and computer program product embodiments for managing non-uniform memory access (NUMA) in a database management system. An embodiment operates by receiving a request to load data from a disk into an in-memory database. An optimal one of the plurality of nodes onto which to load the data from the disk based on a system state is determined. It is determined whether the optimal node has sufficient free local memory in a free list to store the data. If the optimal node does not have sufficient free list memory, a memory allocation is requested from an operating system. An address of the memory storing the data is provided.