Abstract: Techniques and solutions are described for implementing virtual table schemas. A virtual table schema includes a logical pointer that is used to target a table that includes data and is defined according to the virtual table schema. Values assigned to the logical pointer can be used to target tables at different locations, such as in a federated database system or in a cache of a database management system of a central computing system. When data associated with the virtual table is requested, or prior to receiving the request, the data can be stored in a table in the cache. The logical pointer can be updated to reference the cache. If the cache is full, the table can be removed from the cache, and the logical pointer can be updated to reference a table at the federated database system.

Abstract: Techniques and solutions are described for implementing virtual table schemas. A virtual table schema includes a logical pointer that is used to target a table that includes data and is defined according to the virtual table schema. Values assigned to the logical pointer can be used to target tables at different locations, such as in-memory storage or in dynamic storage of a database management system. The dynamic storage includes an in-memory buffer cache and persistent storage. The logical pointer can be updated to point to the in-memory storage or the dynamic storage, including based on use statistics. Similarly, a table having data for the virtual table can be transitioned between the in-memory buffer cache and the persistent storage, including based on use statistics for the virtual table. A value for the logical pointer can be updated when the table is transferred between the in-memory buffer cache and the dynamic storage.

Abstract: Techniques and solutions are described for implementing virtual table schemas. A virtual table schema includes a logical pointer that is used to target a table that includes data and is defined according to the virtual table schema. Values assigned to the logical pointer can be used to target tables at different locations, such as tables located in a remote database system, replica tables containing data replicated from a table in a remote database system, or cached table data. Replica tables targeted by a logical pointer of a virtual table schema can have various locations, such as being located in main memory of a database system, being located in a dynamic storage system, or being located in a data lake. Replica tables can be transferred between locations based on various criteria.

Abstract: Disclosed herein are system, method, and computer program product embodiments for providing a memory optimization system for inverted indexes. An embodiment operates by determining a value identifier corresponding to a value to be searched for in a database based on a query. From a lookup table, a prefix of an offset corresponding to a location in a compressed posting list that identifies row locations where the value is found in the database is retrieved. From a directory page, a suffix of the offset corresponding to the location is retrieved. The offset is generated based on the prefix and the suffix. From the posting list, row identifiers corresponding to the generated offset are retrieved. The retrieved row identifiers are returned responsive to the query.

Abstract: Persisting data in logical tiers of an in-memory database based on aging rules of the database system. The database system can be configured to age data by moving the data from a first logical tier supporting high performance data access to a second logical tier supporting lower performance data access, thereby improving the operational efficiency of the database system.

Abstract: Provided is a system and method for improving memory management in a database. In one example, the method may include receiving a request to store a data object within a database, determining a category type associated with the data object from among a plurality of category types based on an attribute of the data object, and storing the data object via a memory pool corresponding to the determined category from among a plurality of memory pools corresponding to the plurality of respective categories, where the storing comprises allocating a first category type of data object to a first memory pool locked to main memory and allocating a second category type of data object to a second memory pool that is swapped out to disk over time. The locked memory pool can ensure that more important data items remain available even when they are the least recently used.

Abstract: A method, a system and a computer program product for implicitly partitioning data. An index in a plurality of indexes is selected. The index corresponds to a plurality of ranges of data values stored in a plurality of database slices of a database. A partitioning structure for the selected index is generated. The partitioning structure includes a plurality of hierarchically arranged nodes. Each node corresponds to a range of data values in the plurality of ranges of data values stored and at least one database slice storing corresponding range of data values. Another range of data values is received for storage in the database. Using the partitioning structure, a node in the partitioning structure corresponding to the received range of data values and a corresponding database slice for storing the received range of data values are identified. The received range of data values is stored in the identified database slice.

Abstract: Disclosed herein are system, method, and computer-program product embodiments for generating a paged and in-memory representation of a database object. An embodiment operates by maintaining in-memory and paged form primitives unique to the database object or a substructure thereof in a database such that the in-memory and paged form primitives are capable of providing the in-memory and paged representations of the database objects, respectively. Thereafter, a load configuration for the database object is determined. Based on the load configuration, the in-memory and/or paged representations of the database object are generated using the in-memory form primitive or the paged form primitive unique to the database object, respectively. Subsequently, the in-memory and/or paged representations of the database object are stored in the database.

Abstract: Systems, apparatus, techniques, articles, and methods include a unified table container for storing data and metadata in pages organized in one or more page chains. A determination is made as to whether the query includes a data definition statement. In response to determining that the query includes a data definition statement, in-memory runtime data is persisted to a persistent volume that is configured to be accessed by the database system. The in-memory runtime data includes one or more first data fields and one or more second data fields that are different from the one or more first data fields. Furthermore, the unified table container includes a persistent runtime data descriptor that is associated with the one or more first data fields and persists the first one or more first data fields when persisting the in-memory runtime data.

Abstract: Implementations of the present disclosure include providing, at each node in a set of nodes of a database system, a table partition of a plurality of table partitions, the plurality of table partitions being provided by partitioning a table using a primary key, providing, at each node in the set of nodes of the database system, a secondary index partition of a plurality of secondary index partitions, each secondary index partition including a replicate table of at least a portion of the table, the plurality of secondary index partitions being provided by partitioning the table using one or more secondary keys, and for at least one operation executed on a table partition, executing a replication protocol to replicate the at least one operation on a secondary index partition that corresponds to the table partition.

Abstract: A computer implemented method for updating a secondary database includes receiving transaction log information from a primary database system at a secondary database system. The transaction log information comprises a first marker identifying a beginning of an operation comprising a plurality of sub-operations and a second marker identifying an end of the operation. The transaction log information is sequentially parsed, a first marker is identified, and a lock state is created after replaying the first marker to ensure proper serialization of the plurality of sub-operations. The lock state is terminated after replaying the second marker. Related apparatus, systems, techniques and articles are also described.

Abstract: Provided is a system and method for improving memory management in a database. In one example, the method may include receiving a request to store a data object within a database, determining a category type associated with the data object from among a plurality of category types based on an attribute of the data object, and storing the data object via a memory pool corresponding to the determined category from among a plurality of memory pools corresponding to the plurality of respective categories, where the storing comprises allocating a first category type of data object to a first memory pool locked to main memory and allocating a second category type of data object to a second memory pool that is swapped out to disk over time. The locked memory pool can ensure that more important data items remain available even when they are the least recently used.

Abstract: The subject matter disclosed herein provides methods for the forced ordering of a dictionary based on row identifier values. A plurality of concurrent transactions can be handled in an in-memory database. At least one of the transactions can include at least one write operation to a dictionary. Each write operation can be assigned a row identifier (ID). At least one of the write operations can be written to the dictionary out of sequence. The sequence can be based on the row ID. Each row ID in the dictionary can be mapped to a corresponding value identifier in the dictionary. The dictionary can position the value identifiers so that the corresponding row IDs are in a sorted sequential order based on the row ID. Related apparatus, systems, techniques, and articles are also described.

Abstract: Computer-implemented systems and methods for categorizing tables of a database into groups are provided. A set of queries used to access a database are processed to generate a log that identifies for each of the queries (i) a database operation associated with the query, (ii) a table of the database accessed via the query, and (iii) a column or columns of the table used by the query to access the table. Database operation criteria for categorizing database tables into groups are accessed. The criteria are based on a total number of times that a table is accessed by the set of queries and a number of times that the table is accessed using a column or group of columns. An algorithm that evaluates data of the log against the criteria is applied. The algorithm is configured to automatically categorize tables identified in the log into groups based on the evaluation.

Abstract: Cracking page-loadable columns for in-memory data management is described herein. An embodiment operates by accessing a column according to a received query, determining that the received query requires a non-critical data structure associated with the column, and rebuilding the non-critical data structure from data associated with the column.

Abstract: A system and method, including determining, by a processor, a data type for each column of a database table; determining, by the processor and based on the determined data type for each column of the database table, an indication of a size of the database table; calculating, by the processor and based on the determined indication of the size of the database table, a start nbit size for a nbit compression process to be used on the database table; specifying, by the processor, the calculated start nbit size for the nbit compression process; and compressing the database table by executing the nbit data compression process using the specified start nbit size.

Abstract: A computer implemented method for manipulating data comprises receiving log transaction information from a primary database system at a secondary database system and parsing the log transaction information to identify data objects that have been modified on the primary database system. The secondary database system determines whether the data objects are present in main memory on the secondary database system. If the data objects are not in the main memory, they are copied from persistence data volumes to the main memory. The data objects are modified by replaying the log transactions on the data objects in the main memory. Then, the data objects in the persistence data volumes are modified. Related apparatus, systems, techniques and articles are also described.

Abstract: In one aspect, there is provided a method. The method may include accessing a multi-version concurrency control block providing row state for a block of rows in a table of a database, the multi-version concurrency control block including a header portion and a data portion, the header portion including a type indicator indicating whether all of the rows of the block are visible to a plurality of threads at a database management system or invisible to the plurality of threads at the database management system. Related systems, methods, and articles of manufacture are also disclosed.

Abstract: A system includes a volatile random access memory storing first header data and a first data block, the first header data including descriptive information and a first pointer to the first data block, and a non-volatile random access memory. The system includes determination of a memory size associated with the first header data and the first data block, allocation of a first memory block of the non-volatile random access memory based on the determined memory size, determination of an address of the non-volatile random access memory associated with the allocated first memory block, and writing of the descriptive information and a binary copy of the first data block at the address of the non-volatile random access memory.

Abstract: The present disclosure involves systems, software, and computer implemented methods for using declarative rules for optimized access to data. One example method includes receiving a structured query language (SQL) query, the SQL query associated with at least one database table and at least one predicate. A determination is made as to whether the SQL query is associated with at least one declarative rule based on the at least one database table and the at least one predicate. In response to determining that the SQL query is associated with at least one declarative rule, the at least one associated declarative rule is applied, during optimization of the SQL query. The at least one declarative rule defines a transformation to the SQL query to be performed during query optimization. An updated SQL query is generated based on the applied at least one associated declarative rule and the updated SQL query is provided.