Abstract: Disclosed herein are system, method, and computer-program product embodiments for generating a paged and in-memory representation of a database object. An embodiment operates by maintaining in-memory and paged form primitives unique to the database object or a substructure thereof in a database such that the in-memory and paged form primitives are capable of providing the in-memory and paged representations of the database objects, respectively. Thereafter, a load configuration for the database object is determined. Based on the load configuration, the in-memory and/or paged representations of the database object are generated using the in-memory form primitive or the paged form primitive unique to the database object, respectively. Subsequently, the in-memory and/or paged representations of the database object are stored in the database.

Abstract: System, method, and various embodiments for providing a data access and recommendation system are described herein. An embodiment operates by identifying a column access of one or more data values of a first column of a plurality of columns of a table of a database during a sampling period. A count of how many of the one or more data values are accessed during the column access are recorded. A first counter, corresponding to the first column and stored in a distributed hash table, is incremented by the count. The sampling period is determined to have expired. A load recommendation on how to load data values into the first column based on the first counter is computed. The load recommendation for implementation into the database for one or more subsequent column accesses is provided.

Abstract: The subject matter disclosed herein relates to out of memory error handling in a database system. A database operation can be received by an in-memory database. The database operation can be for a database object stored in a database table that can be represented as a plurality of pages that can be persisted in a page chain. The in-memory database can reserve out of memory space sufficient to load a predetermined number of pages in the page chain into memory. The in-memory database can iteratively process each page in the page chain until completion of the database operation. The iterative process can include loading the page from persistence into memory of the in-memory database, performing at least a portion of the database operation using the loaded page, and unloading the page from the memory of the in-memory database. Related apparatus, systems, techniques, and articles are also described.

Abstract: Disclosed herein are system, method, and computer program product embodiments for providing a memory optimization system for inverted indexes. An embodiment operates by determining a value identifier corresponding to a value to be searched for in a database based on a query. From a lookup table, a prefix of an offset corresponding to a location in a compressed posting list that identifies row locations where the value is found in the database is retrieved. From a directory page, a suffix of the offset corresponding to the location is retrieved. The offset is generated based on the prefix and the suffix. From the posting list, row identifiers corresponding to the generated offset are retrieved. The retrieved row identifiers are returned responsive to the query.

Abstract: Disclosed herein are system, method, and computer program product embodiments for storing an object onto a first or second page. An embodiment operates by receiving the object and determining that the first page has sufficient unused space for storing at least one byte of the object. Thereafter, a data block of the object is created to comprise at least one byte of the object. The data block is then stored on the first page or the second page, and a location of the object's first data block is recorded. Thereafter, a pointer corresponding to the location of the object's first data block for loading the object is provided.

Abstract: System, method, and various embodiments for providing a hybrid data storage and load system are described herein. An embodiment operates by receiving a lookup request for a row identifier (ROWID) of a row of a table of a database, wherein the lookup request identifies a row position of the ROWID. A page corresponding to the row position is identified from an address vector stored in memory storage. The identified page is loaded from the disk storage to the memory storage. The row position is identified within the page from the memory storage. The ROWID stored in the row position is identified within the page. The ROWID is returned responsive to the lookup request.

Abstract: Disclosed herein are system, method, and computer program product embodiments for converting column loadable data in a database into a page loadable format. When a column oriented in-memory database needs to perform an operation on a table, the affected columns must be loaded fully into memory. In some cases, a database may partition tables, allowing for column fragments corresponding to those partitions to be loaded into memory. These column fragments may exceed available memory, such that it is beneficial to convert the column fragment into a page loadable format. Approaches are discussed herein for performing the conversion without the need to update global metadata during the process, allowing for multiple parallel column fragments to be updated, allowing data manipulation language (DML) operations to continue on the column during the conversion process, and reducing the impact of blocked parallel savepoint operations during the conversion process by chunking the conversion process.

Abstract: Disclosed herein are system, method, and computer program product embodiments for maintaining of a geometric object in a database. An embodiment operates by a database maintaining a first page storing a data block in the database's on-disk store such that the data block stores at least one byte of the geometric object. After receiving the request for the geometric object, the database loads the page storing the geometric object in the in-memory store and determines the size of the geometric object. Based on the size of the geometric object, the database stores the geometric object in the in-memory store directly or in a heap of the in-memory store.

Abstract: Disclosed herein are system, method, and computer program product embodiments for identifying and loading a relevant page of a dictionary into temporary memory. An embodiment operates by receiving a query to be executed. The query includes a value for executing the query. The server queries a dictionary to retrieve a value ID. The server executes a binary search on a helper vector of the dictionary based on the value. The helper vector includes a last value for each page of a dictionary. The server identifies a page of the dictionary including the value. The server loads the page into temporary memory and retrieves the value ID of the value from the page. The server executes the query on a column using the value ID.

Abstract: Disclosed herein are system, method, and computer-program product embodiments for generating a paged and in-memory representation of a database object. An embodiment operates by maintaining in-memory and paged form primitives unique to the database object or a substructure thereof in a database such that the in-memory and paged form primitives are capable of providing the in-memory and paged representations of the database objects, respectively. Thereafter, a load configuration for the database object is determined. Based on the load configuration, the in-memory and/or paged representations of the database object are generated using the in-memory form primitive or the paged form primitive unique to the database object, respectively. Subsequently, the in-memory and/or paged representations of the database object are stored in the database.

Abstract: System, method, and various embodiments for providing a hybrid data storage and load system are described herein. An embodiment operates by receiving a lookup request comprising a row identifier (ROWID) corresponding to a row of a table of a database. Which of a first set of pages corresponds to a sorted position of the ROWID is identified. The identified page of the first set of pages is loaded from the disk storage to the memory storage. A sorted row position of the ROWID is identified from a memory block of the loaded page from the first set of pages. One of a second set of pages that includes the sorted row position is identified. The identified one of the second set of pages is loaded from the disk storage into memory. The loaded identified one of the second set of pages is searched for the unsorted position of the ROWID.

Abstract: Disclosed herein are system, method, and computer program product embodiments for providing paged and compressed storage for column data, while maintaining existing access mechanisms for the data. In order to reduce an in-memory footprint for column data, columns may be stored in pageable format using page chains, and only those pages of the column data needed to resolve a query will be placed in memory, and evicted from memory when no longer needed. In order to further reduce the footprint for these columns, compression can be applied, and the compressed column data stored in the same pageable format using page chains. The compressed data includes a plurality of vectors, each of which is converted into pages and stored on the page chain with the others so that they can be efficiently retrieved during database retrieval operations.

Abstract: Disclosed herein are system, method, and computer program product embodiments for storing a large object (LOB) in a database. An embodiment operates by identifying the LOB to be stored in an on-disk store of the database and determining the size of the LOB. Thereafter, an index vector for the on-disk store to include an identifier corresponding to the LOB is created. Additionally, a dictionary for the on-disk store to include a copy of the identifier and corresponding LOB data is also created. The LOB data corresponds to the LOB itself or a LOB location inside of the on-disk store based on the LOB's size.

Abstract: Objects across multiple database manipulation language (DML) redo log records are cached, during continuous replay of redo log records on a secondary system, for the same table partition to enable reuse of such cached objects. Later, these cached objects can be reused as they are accessed during the sequential processing of DML redo records for a specific table partition.

Abstract: A computer implemented method for updating a secondary database includes receiving transaction log information from a primary database system at a secondary database system. The transaction log information comprises a first marker identifying a beginning of an operation comprising a plurality of sub-operations and a second marker identifying an end of the operation. The transaction log information is sequentially parsed, a first marker is identified, and a lock state is created after replaying the first marker to ensure proper serialization of the plurality of sub-operations. The lock state is terminated after replaying the second marker. Related apparatus, systems, techniques and articles are also described.

Abstract: Disclosed herein are system, method, and computer program product embodiments for altering the data type of a column in a database. An embodiment operates by converting an original dictionary associated with a column into a new dictionary. The new dictionary stores the values of the original dictionary using a different data type. An index vector containing the keys of the original dictionary is then updated to contain the associated keys of the new dictionary. Because the size of the original dictionary is often substantially smaller than the number of rows in the associated column, this dictionary conversion decreases the computation cost to the database system of altering the data type of the column and reduces or even minimizes database downtime for users.

Abstract: Data records of a data set can be stored in multiple main part fragments retained in on-disk storage. Each fragment can include a number of data records that is equal to or less than a defined maximum fragment size. Using a compression that is optimized for each fragment, each fragment can be compressed. After reading at least one of the fragments into main system memory from the on-disk storage, an operation can be performed on the fragment or fragments while the in the main system memory.

Abstract: A computer implemented method for initializing a secondary database system includes receiving table state information from a primary database system at a secondary database system. The table state information includes information identifying which tables had an open operation during a savepoint event. Metadata associated with the tables is parsed to identify table state information from the metadata and a lock is created for each table identified as having an open operation during the savepoint event. Afterwards, log transaction information is sequentially parsed. Related apparatus, systems, techniques and articles are also described.

Abstract: Disclosed herein are system, method, and computer program product embodiments for providing a memory optimization system for inverted indexes. An embodiment operates by determining a value identifier corresponding to a value to be searched for in a database based on a query. From a lookup table, a prefix of an offset corresponding to a location in a compressed posting list that identifies row locations where the value is found in the database is retrieved. From a directory page, a suffix of the offset corresponding to the location is retrieved. The offset is generated based on the prefix and the suffix. From the posting list, row identifiers corresponding to the generated offset are retrieved. The retrieved row identifiers are returned responsive to the query.

Abstract: Disclosed herein are system and method embodiments for generating a paged inverted index. An embodiment is generated by storing a first data structure and the second data structure in a plurality of pages, where the plurality of pages are stored in the one or more memories. The first data structure is stored in the plurality of pages and includes a plurality of value identifiers, where a value identifier corresponds to an offset. The second data structure stored in the plurality of pages includes a plurality of row positions, wherein a row position is at a location that corresponds to the offset in the first data structure and identifies a position of row in a table that stores data associated with the value ID.