Abstract: Disclosed herein are system, method, and computer program product embodiments for partition pruning via non-homogenous synopsis information. An embodiment operates by maintaining synopsis information for a data partition in accordance with a first synopsis strategy, monitoring performance of the synopsis information within a partition pruning system, determining that the performance of the synopsis information is insufficient, and updating the synopsis information in accordance with a second synopsis strategy better suited for the attributes of the data partition. In some embodiments, a first data partition of a partitioned data table may employ a first synopsis strategy and a second data partition of the partitioned data table may employ a second synopsis strategy.

Abstract: Disclosed herein are system, method, and computer program product embodiments for partition pruning using globally encoded range partition information. An embodiment operates by partitioning a data table into a plurality of data partitions, determining a plurality of sub-partitions within the data partitions, and determining a global range table based at least in part on value ranges corresponding to the sub-partitions. Further, the global range table may be used to determine bit vectors for the data partitions and query predicates. In some examples, the bit vectors may be used to determine whether to prune data partitions during execution of a query over a large data volume.

Abstract: Disclosed herein are system and method embodiments for generating a paged inverted index. An embodiment is generated by storing a first data structure and the second data structure in a plurality of pages, where the plurality of pages are stored in the one or more memories. The first data structure is stored in the plurality of pages and includes a plurality of value identifiers, where a value identifier corresponds to an offset. The second data structure stored in the plurality of pages includes a plurality of row positions, wherein a row position is at a location that corresponds to the offset in the first data structure and identifies a position of row in a table that stores data associated with the value ID.

Abstract: Disclosed herein are system, method, and computer program product embodiments for determining a constraint data statistics object is consistent. An embodiment operates by receiving a query for a database table. The embodiment determines a constraint data statistic object associated with a partition of the database table. The embodiment determines the constraint data statistics object is consistent with a data state of the partition of the database table based on consistency metadata of the constraint data statistics object. The embodiment processes the query for the database table based on the determining the constraint data statistics object is consistent with the data state of the partition of the database table.

Abstract: Disclosed herein are system, method, and computer program product embodiments for using a data statistic as a dynamic data integrity constraint. An embodiment operates by defining a data statistic for a column or a set of columns of a partition of a plurality of partitions of a database table. The embodiment creates a constraint data statistics object based on the data statistic. The embodiment receives a query for the database table. The embodiment determines the constraint data statistics object is consistent with a data state of the partition. The embodiment derives an implied constraint based on the constraint data statistics object. The embodiment processes the query for the partition based on the implied constraint.

Abstract: Disclosed herein are system, method, and computer program product embodiments for performing dynamic partition pruning using data statistic objects as data integrity constraints. An embodiment operates by partitioning a database table into a plurality of partitions based on a partition criterion. The embodiment creates a data statistics object for a partition in the plurality of partitions. The embodiment receives a query for the database table. The embodiment determines the data statistics object is consistent with data in the partition. The embodiment processes the query for the partition based on the data statistics object.

Abstract: A compression manager may store, within an index vector, a plurality of value identifiers (IDs), each value ID representing a value within a database. A page generator may designate a number of the value IDs as defining a page within the index vector, so that the index vector includes a plurality of pages, each page including the number of value IDs. The page generator may store the index vector in a secondary memory of a main memory database. An iterator may access a requested value ID, and a page loader may load a corresponding page of the index vector that contains the requested value ID into the main memory database.

Abstract: As part of a columnar in-memory database, value identifiers are inserted into a backing array in-memory until such time that it is determined that such backing array does not have sufficient capacity. A new backing array is then generated that includes the value identifiers in the old backing array and which has sufficient capacity. The old backing array can be flushed from memory when there are no active operations using such backing array. Such an arrangement allows for readers and non-structural writers to operate concurrently.

Abstract: Persisting data in logical tiers of an in-memory database based on aging rules of the database system. The database system can be configured to age data by moving the data from a first logical tier supporting high performance data access to a second logical tier supporting lower performance data access, thereby improving the operational efficiency of the database system.

Abstract: Cracking page-loadable columns for in-memory data management is described herein. An embodiment operates by accessing a column according to a received query, determining that the received query requires a non-critical data structure associated with the column, and rebuilding the non-critical data structure from data associated with the column.

Abstract: A method, computer program product and system are provided. The method, computer program product and system execute a process for storing an object in an object container that is stored in a persistency of a disk storage. The object container has size criteria whereby objects meeting the size criteria of the object container can be assigned to the object container. The object container can facilitate storing multiple objects to optimize disk storage usage by facilitating the assigning of multiple objects to the same disk storage page.

Abstract: Disclosed herein are system and method embodiments for generating a paged inverted index. An embodiment is generated by storing a first data structure and the second data structure in a plurality of pages, where the plurality of pages are stored in the one or more memories. The first data structure is stored in the plurality of pages and includes a plurality of value identifiers, where a value identifier corresponds to an offset. The second data structure stored in the plurality of pages includes a plurality of row positions, wherein a row position is at a location that corresponds to the offset in the first data structure and identifies a position of row in a table that stores data associated with the value ID.

Abstract: Disclosed herein are systems, methods, and computer readable storage media for a database system using solid state drives as a second level cache. A database system includes random access memory configured to operate as a first level cache, solid state disk drives configured to operate as a persistent second level cache, and hard disk drives configured to operate as disk storage. The database system also includes a cache manager configured to receive a request for a data page and determine whether the data page is in cache or disk storage. If the data page is on disk, or in the second level cache, it is copied to the first level cache. If copying the data page results in an eviction, the evicted data page is copied to the second level cache. At checkpoint, dirty pages stored in the second level cache are flushed in place in the second level cache.

Abstract: As part of a columnar in-memory database, value identifiers are inserted into a backing array in-memory until such time that it is determined that such backing array does not have sufficient capacity. A new backing array is then generated that includes the value identifiers in the old backing array and which has sufficient capacity. The old backing array can be flushed from memory when there are no active operations using such backing array. Such an arrangement allows for readers and non-structural writers to operate concurrently. Related apparatus, systems, techniques and articles are also described.

Abstract: A compression manager may store, within an index vector, a plurality of value identifiers (IDs), each value ID representing a value within a database. A page generator may designate a number of the value IDs as defining a page within the index vector, so that the index vector includes a plurality of pages, each page including the number of value IDs. The page generator may store the index vector in a secondary memory of a main memory database. An iterator may access a requested value ID, and a page loader may load a corresponding page of the index vector that contains the requested value ID into the main memory database.

Abstract: Disclosed herein are systems, methods, and computer readable storage media for a database system using solid state drives as a second level cache. A database system includes random access memory configured to operate as a first level cache, solid state disk drives configured to operate as a persistent second level cache, and hard disk drives configured to operate as disk storage. The database system also includes a cache manager configured to receive a request for a data page and determine whether the data page is in cache or disk storage. If the data page is on disk, or in the second level cache, it is copied to the first level cache. If copying the data page results in an eviction, the evicted data page is copied to the second level cache. At checkpoint, dirty pages stored in the second level cache are flushed in place in the second level cache.

Abstract: A method, information processing system, and computer program storage product to compress sorted values. At least a first prefix and a second prefix in a plurality of prefixes are compared. Each prefix comprises at least a portion of a plurality of sorted values. A respective prefix comprises a set of consecutive characters including at least a first character of a respective sorted value. The respective sorted value further comprising a respective suffix comprising consecutive characters of the respective sorted value that are after the respective prefix. At least a respective first character of the first prefix and a respective first character of the second prefix are determined to be substantially identical. The first prefix is merged with the second prefix into a single prefix comprising the first character. A set of suffixes associated with the first prefix is updated to reflect an association with the second prefix.

Abstract: A method, information processing system, and computer program storage product for compressing sorted values is disclosed. At least a first prefix and a second prefix in a plurality of prefixes are compared. Each prefix comprises at least a portion of a plurality of sorted values. A respective prefix comprises a set of consecutive characters including at least a first character of a respective sorted value. The respective sorted value further comprising a respective suffix comprising consecutive characters of the respective sorted value that are after the respective prefix. At least a respective first character of the first prefix and a respective first character of the second prefix are determined to be substantially identical. The first prefix is merged with the second prefix into a single prefix comprising the first character. A set of suffixes associated with the first prefix is updated to reflect an association with the second prefix.

Abstract: A micro-programmable filter (MFE) engine includes multiple programmable filter elements and a microcode controller. The filter elements can be configured, controlled, and combined in different ways to implement different types of filters. The MFE preferably supports multiple-execution instructions that allow a single instruction to perform multiple moves into accumulators for efficient data movement inside MFE.