Abstract: The present disclosure involves systems, software, and computer implemented methods for compressed columnar data search using fingerprints. One example method includes compressing columnar data that includes dividing the columnar data into multiple data blocks and generating a fingerprint for each data block, storing the compressed columnar data and the generated fingerprints in an in-memory database, receiving a query for the columnar data, for each in-memory data block stored in the in-memory database, determining whether the in-memory data block satisfies the query and in response to a determination that the in-memory data block does not satisfy the query, pruning the in-memory data block from the multiple data blocks to generate an unpruned set of data blocks, decompressing the unpruned set of data blocks, and performing a query search on the decompressed unpruned set of data blocks for the received query.

Abstract: Disclosed herein are system, method, and computer-program product embodiments for generating a paged and in-memory representation of a database object. An embodiment operates by maintaining in-memory and paged form primitives unique to the database object or a substructure thereof in a database such that the in-memory and paged form primitives are capable of providing the in-memory and paged representations of the database objects, respectively. Thereafter, a load configuration for the database object is determined. Based on the load configuration, the in-memory and/or paged representations of the database object are generated using the in-memory form primitive or the paged form primitive unique to the database object, respectively. Subsequently, the in-memory and/or paged representations of the database object are stored in the database.

Abstract: Disclosed herein are system, method, and computer program product embodiments for partition pruning via non-homogenous synopsis information. An embodiment operates by maintaining synopsis information for a data partition in accordance with a first synopsis strategy, monitoring performance of the synopsis information within a partition pruning system, determining that the performance of the synopsis information is insufficient, and updating the synopsis information in accordance with a second synopsis strategy better suited for the attributes of the data partition. In some embodiments, a first data partition of a partitioned data table may employ a first synopsis strategy and a second data partition of the partitioned data table may employ a second synopsis strategy.

Abstract: Techniques and solutions are described that can be used to determine data subsets of a data set that may be responsive to a query. In one aspect, imprints are created for the data subsets, where an imprint indicates whether, for a plurality of bins, a data subset contains values within respective bins. The bins are defined at least in part using data for a set of queries. In another aspect, at least one bin of a plurality of bins is designated to be associated with values of the data set satisfying a threshold frequency. In a further aspect, imprints are created for a plurality of bins, where an imprint indicates whether the plurality of data subsets comprise a value for a given bin. A query imprint can be compared with the data subset imprints or bin imprints to determine data subsets that may be responsive to an associated query.

Abstract: Techniques and solutions are described for compressing data and facilitating access to compressed data. Compression can be applied to proper data subsets of a data set, such as to columns of a table. Using various methods, the proper data subsets can be evaluated to be included in a group of proper data subsets to be compressed using a first compression technique, where unselected proper data subsets are not compressed using the first compression technique. Data in the data set can be reordered based on a reordering sequence for the proper data subsets. Reordering data in the data set can improve compression when at least a portion of the proper data subsets are compressed. A data structure is provided that facilitates accessing specified data stored in a compressed format.

Abstract: Disclosed herein are system, method, and computer program product embodiments for partition pruning using globally encoded range partition information. An embodiment operates by partitioning a data table into a plurality of data partitions, determining a plurality of sub-partitions within the data partitions, and determining a global range table based at least in part on value ranges corresponding to the sub-partitions. Further, the global range table may be used to determine bit vectors for the data partitions and query predicates. In some examples, the bit vectors may be used to determine whether to prune data partitions during execution of a query over a large data volume.

Abstract: A computer implemented system and method of memory management for an in-memory database. The system implements a paged data vector using non-uniform compression of its chunks. In this manner, the system achieves greater compression than systems that use uniform compression.

Abstract: Disclosed herein are system, method, and computer-program product embodiments for generating a paged and in-memory representation of a database object. An embodiment operates by maintaining in-memory and paged form primitives unique to the database object or a substructure thereof in a database such that the in-memory and paged form primitives are capable of providing the in-memory and paged representations of the database objects, respectively. Thereafter, a load configuration for the database object is determined. Based on the load configuration, the in-memory and/or paged representations of the database object are generated using the in-memory form primitive or the paged form primitive unique to the database object, respectively. Subsequently, the in-memory and/or paged representations of the database object are stored in the database.

Abstract: A computer implemented system and method of memory management for an in-memory database. The system implements a paged data vector using non-uniform compression of its chunks. In this manner, the system achieves greater compression than systems that use uniform compression.

Abstract: Persisting data in logical tiers of an in-memory database based on aging rules of the database system. The database system can be configured to age data by moving the data from a first logical tier supporting high performance data access to a second logical tier supporting lower performance data access, thereby improving the operational efficiency of the database system.

Abstract: Disclosed herein are system, method, and computer program product embodiments for providing a memory optimization system for inverted indexes. An embodiment operates by determining a value identifier corresponding to a value to be searched for in a database based on a query. From a lookup table, a prefix of an offset corresponding to a location in a compressed posting list that identifies row locations where the value is found in the database is retrieved. From a directory page, a suffix of the offset corresponding to the location is retrieved. The offset is generated based on the prefix and the suffix. From the posting list, row identifiers corresponding to the generated offset are retrieved. The retrieved row identifiers are returned responsive to the query.

Abstract: Disclosed herein are system, method, and computer program product embodiments for identifying and loading a relevant page of a dictionary into temporary memory. An embodiment operates by receiving a query to be executed. The query includes a value for executing the query. The server queries a dictionary to retrieve a value ID. The server executes a binary search on a helper vector of the dictionary based on the value. The helper vector includes a last value for each page of a dictionary. The server identifies a page of the dictionary including the value. The server loads the page into temporary memory and retrieves the value ID of the value from the page. The server executes the query on a column using the value ID.

Abstract: Disclosed herein are system, method, and computer-program product embodiments for generating a paged and in-memory representation of a database object. An embodiment operates by maintaining in-memory and paged form primitives unique to the database object or a substructure thereof in a database such that the in-memory and paged form primitives are capable of providing the in-memory and paged representations of the database objects, respectively. Thereafter, a load configuration for the database object is determined. Based on the load configuration, the in-memory and/or paged representations of the database object are generated using the in-memory form primitive or the paged form primitive unique to the database object, respectively. Subsequently, the in-memory and/or paged representations of the database object are stored in the database.

Abstract: Disclosed herein are system, method, and computer program product embodiments for storing a large object (LOB) in a database. An embodiment operates by identifying the LOB to be stored in an on-disk store of the database and determining the size of the LOB. Thereafter, an index vector for the on-disk store to include an identifier corresponding to the LOB is created. Additionally, a dictionary for the on-disk store to include a copy of the identifier and corresponding LOB data is also created. The LOB data corresponds to the LOB itself or a LOB location inside of the on-disk store based on the LOB's size.

Abstract: Disclosed herein are system, method, and computer program product embodiments for providing paged and compressed storage for column data, while maintaining existing access mechanisms for the data. In order to reduce an in-memory footprint for column data, columns may be stored in pageable format using page chains, and only those pages of the column data needed to resolve a query will be placed in memory, and evicted from memory when no longer needed. In order to further reduce the footprint for these columns, compression can be applied, and the compressed column data stored in the same pageable format using page chains. The compressed data includes a plurality of vectors, each of which is converted into pages and stored on the page chain with the others so that they can be efficiently retrieved during database retrieval operations.

Abstract: The present disclosure involves systems, software, and computer implemented methods for compressed columnar data search using fingerprints. One example method includes compressing columnar data that includes dividing the columnar data into multiple data blocks and generating a fingerprint for each data block, storing the compressed columnar data and the generated fingerprints in an in-memory database, receiving a query for the columnar data, for each in-memory data block stored in the in-memory database, determining whether the in-memory data block satisfies the query and in response to a determination that the in-memory data block does not satisfy the query, pruning the in-memory data block from the multiple data blocks to generate an unpruned set of data blocks, decompressing the unpruned set of data blocks, and performing a query search on the decompressed unpruned set of data blocks for the received query.

Abstract: A computer implemented system and method of memory management for an in-memory database. The system implements a paged data vector using non-uniform compression of its chunks. In this manner, the system achieves greater compression than systems that use uniform compression.

Abstract: A computer implemented system and method of memory management for an in-memory database. The system implements a paged data vector using non-uniform compression of its chunks. In this manner, the system achieves greater compression than systems that use uniform compression.

Abstract: Cracking page-loadable columns for in-memory data management is described herein. An embodiment operates by accessing a column according to a received query, determining that the received query requires a non-critical data structure associated with the column, and rebuilding the non-critical data structure from data associated with the column.

Abstract: A computer implemented system and method of memory management for an in-memory database. The system implements a paged data vector using non-uniform compression of its chunks. In this manner, the system achieves greater compression than systems that use uniform compression.