Abstract: A method, apparatus, and system for tracking row and object database activity into block level heatmaps is provided. Database activity including reads, writes, and creates can be tracked by a database management system at the finest possible level of granularity, or the row and object level. To efficiently record the tracked database activity, a two-part structure is described for writing the activity into heatmaps. A hierarchical in-memory component may use a dynamically allocated sparse pool of bitmap blocks. Periodically, the in-memory component is persisted to a stored representation component, sharable with multiple database instances, which may include consolidated last access times and/or a history of heatmap snapshots to reflect access over time. The heatmaps may then be externalized to database users and applications to provide and support a variety of features.

Abstract: A method and apparatus for compression units in an index block is provided. A particular leaf block of a database index for a table is generated. The particular leaf block corresponds to a particular range of key values. A compression scheme is determined for a plurality of index entries with key values within the particular range. A compression unit that includes a plurality of compressed index entries is generated by applying the compression scheme to the plurality of index entries. Compression metadata describing the compression scheme is generated. The compression unit and the compression metadata is stored in the particular leaf block. In response to a request, at least a portion of the compression unit is decompressed to access one or more index entries.

Abstract: A method, apparatus, and system for tracking row and object database activity into block level heatmaps is provided. Database activity including reads, writes, and creates can be tracked by a database management system at the finest possible level of granularity, or the row and object level. To efficiently record the tracked database activity, a two-part structure is described for writing the activity into heatmaps. A hierarchical in-memory component may use a dynamically allocated sparse pool of bitmap blocks. Periodically, the in-memory component is persisted to a stored representation component, sharable with multiple database instances, which may include consolidated last access times and/or a history of heatmap snapshots to reflect access over time. The heatmaps may then be externalized to database users and applications to provide and support a variety of features.

Abstract: Techniques are described herein for executing queries on distinct portions of a database object that has been separate into chunks and distributed across the volatile memories of a plurality of nodes in a clustered database system. The techniques involve redistributing the in-memory database object portions on changes to the clustered database system. Each node may maintain a mapping indicating which nodes in the clustered database system store which chunks, and timestamps indicating when each mapping entry was created or updated. A query coordinator may use the timestamps to select a database server instance with local in memory access to data required by a portion of a query to process that portion of the query.

Abstract: A method, apparatus, and system for policy driven data placement and information lifecycle management in a database management system are provided. A user or database application can specify declarative policies that define the movement and transformation of stored database objects. The policies are associated with a database object and may also be inherited. A policy defines, for a database object, an archiving action to be taken, a scope, and a condition before the archiving action is triggered. Archiving actions may include compression, data movement, table clustering, and other actions to place the database object into an appropriate storage tier for a lifecycle phase of the database object. Conditions based on access statistics can be specified at the row level and may use segment or block level heatmaps. Policy evaluation occurs periodically in the background, with actions queued as tasks for a task scheduler.

Abstract: A database server stores compressed units in data blocks of a database. A table (or data from a plurality of rows thereof) is first compressed into a “compression unit” using any of a wide variety of compression techniques. The compression unit is then stored in one or more data block rows across one or more data blocks. As a result, a single data block row may comprise compressed data for a plurality of table rows, as encoded within the compression unit. Storage of compression units in data blocks maintains compatibility with existing data block-based databases, thus allowing the use of compression units in preexisting databases without modification to the underlying format of the database. The compression units may, for example, co-exist with uncompressed tables. Various techniques allow a database server to optimize access to data in the compression unit, so that the compression is virtually transparent to the user.

Abstract: Row locking is performed at the row level of granularity for database data stored in columnar form. Row level locking entails use of a lock vector that is stored in a compression unit in a data block, the compression unit storing rows in columnar-major format. On an as needed basis, the lock vector is expanded to identify more transactions affecting the rows in the compression unit.

Abstract: A method, apparatus, and system for automated information lifecycle management using low access patterns in a database management system are provided. A user or the database can store policy data that defines an archiving action when meeting an activity-level condition on one or more database objects. The archiving actions may include compression, data movement, and other actions to place the database object in an appropriate storage tier for a lifecycle phase of the database object. The activity-level condition may specify the database object meeting a low access pattern, optionally for a minimum time period. Various criteria including access statistics for the database object and cost characteristics of current and target compression levels or storage tiers may be considered to determine the meeting of the activity-level condition. The policies may be evaluated on an adjustable periodic basis and may utilize a task scheduler for minimal performance impact.

Abstract: Techniques are provided for automatic verification of query results obtained from data formatted different from row-major format. Techniques are also provided for automatic triage of query results obtained from data formatted different from row-major format. In an embodiment, the query results may be obtained from querying data formatted in column-major format. In an embodiment, when the query results are determined to be incorrect, a cause of the error in the query results can be automatically identified.

Abstract: A method, apparatus, and system for policy driven data placement and information lifecycle management in a database management system are provided. A user or database application can specify declarative policies that define the movement and transformation of stored database objects. The policies are associated with a database object and may also be inherited. A policy defines, for a database object, an archiving action to be taken, a scope, and a condition before the archiving action is triggered. Archiving actions may include compression, data movement, table clustering, and other actions to place the database object into an appropriate storage tier for a lifecycle phase of the database object. Conditions based on access statistics can be specified at the row level and may use segment or block level heatmaps. Policy evaluation occurs periodically in the background, with actions queued as tasks for a task scheduler.

Abstract: A method, apparatus, and system for policy driven data placement and information lifecycle management in a database management system are provided. A user or database application can specify declarative policies that define the movement and transformation of stored database objects. The policies are associated with a database object and may also be inherited. A policy defines, for a database object, an archiving action to be taken, a scope, and a condition before the archiving action is triggered. Archiving actions may include compression, data movement, table clustering, and other actions to place the database object into an appropriate storage tier for a lifecycle phase of the database object. Conditions based on access statistics can be specified at the row level and may use segment or block level heatmaps. Policy evaluation occurs periodically in the background, with actions queued as tasks for a task scheduler.

Abstract: A method, apparatus, and system for periodic performance optimization through heatmap based management of an in-memory area are provided. A heatmap is maintained to track database accesses, and a sliding most recent time window of the heatmap is externalized to a desired granularity level to provide access statistics regarding candidate elements to be possibly placed in the in-memory area. Initially and on a periodic basis, an appropriate knapsack algorithm is chosen based on an analysis on the computational costs versus the benefits of applying various knapsack algorithms for the candidate elements. Using the chosen algorithm in conjunction with a selected performance model, an optimized configuration of the in-memory area is determined. The optimized configuration indicates a set of elements chosen from the candidate elements, optionally specified with compression levels.

Abstract: A method, apparatus, and system for tracking row and object database activity into block level heatmaps is provided. Database activity including reads, writes, and creates can be tracked by a database management system at the finest possible level of granularity, or the row and object level. To efficiently record the tracked database activity, a two-part structure is described for writing the activity into heatmaps. A hierarchical in-memory component may use a dynamically allocated sparse pool of bitmap blocks. Periodically, the in-memory component is persisted to a stored representation component, sharable with multiple database instances, which may include consolidated last access times and/or a history of heatmap snapshots to reflect access over time. The heatmaps may then be externalized to database users and applications to provide and support a variety of features.

Abstract: A method, apparatus, and system for tracking row and object database activity into block level heatmaps is provided. Database activity including reads, writes, and creates can be tracked by a database management system at the finest possible level of granularity, or the row and object level. To efficiently record the tracked database activity, a two-part structure is described for writing the activity into heatmaps. A hierarchical in-memory component may use a dynamically allocated sparse pool of bitmap blocks. Periodically, the in-memory component is persisted to a stored representation component, sharable with multiple database instances, which may include consolidated last access times and/or a history of heatmap snapshots to reflect access over time. The heatmaps may then be externalized to database users and applications to provide and support a variety of features.

Abstract: Techniques are described herein for executing queries on distinct portions of a database object that has been separate into chunks and distributed across the volatile memories of a plurality of nodes in a clustered database system. The techniques involve redistributing the in-memory database object portions on changes to the clustered database system. Each node may maintain a mapping indicating which nodes in the clustered database system store which chunks, and timestamps indicating when each mapping entry was created or updated. A query coordinator may use the timestamps to select a database server instance with local in memory access to data required by a portion of a query to process that portion of the query.

Abstract: Techniques are described herein for executing queries on distinct portions of a database object that has been separate into chunks and distributed across the volatile memories of a plurality of nodes in a clustered database system. The techniques involve redistributing the in-memory database object portions on changes to the clustered database system. Each node may maintain a mapping indicating which nodes in the clustered database system store which chunks, and timestamps indicating when each mapping entry was created or updated. A query coordinator may use the timestamps to select a database server instance with local in memory access to data required by a portion of a query to process that portion of the query.

Abstract: In an approach, the database server generates a request for data representing rows of a table, the request identifying one or more data blocks stored on a storage system. The database server then generates metadata describing one or more filtering conditions to be applied to the rows and sends the metadata along with the request to the storage system. The storage system, when applying filtering conditions to a column containing a LOB, determines whether the LOB is stored in-line or out-of-line. If the column contains an out-of-line LOB, the storage system skips the filtering conditions on the column. If the column contains an in-line LOB, the storage system applies the filtering conditions to the column. Upon obtaining the filtered data from the storage system, the database server retrieves the data blocks for out-of-line LOBs and applies the skipped filtering conditions to create a final result set.

Abstract: A method, apparatus, and system for OZIP, a data compression and decompression codec, is provided. OZIP utilizes a fixed size static dictionary, which may be generated from a random sampling of input data to be compressed. Compression by direct token encoding to the static dictionary streamlines the encoding and avoids expensive conditional branching, facilitating hardware implementation and high parallelism. By bounding token definition sizes and static dictionary sizes to hardware architecture constraints such as word size or processor cache size, hardware implementation can be made fast and cost effective. For example, decompression may be accelerated by using SIMD instruction processor extensions. A highly granular block mapping in optional stored metadata allows compressed data to be accessed quickly at random, bypassing the processing overhead of dynamic dictionaries. Thus, OZIP can support low latency random data access for highly random workloads, such as for OLTP systems.

Abstract: In an approach, the database server generates a request for data representing rows of a table, the request identifying one or more data blocks stored on a storage system. The database server then generates metadata describing one or more filtering conditions to be applied to the rows and sends the metadata along with the request to the storage system. The storage system, when applying filtering conditions to a column containing a LOB, determines whether the LOB is stored in-line or out-of-line. If the column contains an out-of-line LOB, the storage system skips the filtering conditions on the column. If the column contains an in-line LOB, the storage system applies the filtering conditions to the column. Upon obtaining the filtered data from the storage system, the database server retrieves the data blocks for out-of-line LOBs and applies the skipped filtering conditions to create a final result set.

Abstract: Techniques for compressing leaf blocks of an index are provided. The decision whether (and/or how much) to compress leaf blocks of an index is performed on a per block basis. Thus, for example, some leaf blocks of the index may be compressed while other leaf blocks of the index may be uncompressed. As another example, different leaf blocks of an index may be compressed a different amount. In one technique, a set of index entries is identified for storage in a leaf block. Multiple estimates of potential space savings are calculated for compressing the set of index entries, one estimate for each different variation of a compression technique. The compression technique variation that is associated with the most space savings is selected and applied to the set of index entries. Thus, some techniques prevent negative compression altogether. Additionally, users require no knowledge of the underlying data to perform optimal compression.