Abstract: A method, apparatus, and system for policy driven data placement and information lifecycle management in a database management system are provided. A user or database application can specify declarative policies that define the movement and transformation of stored database objects. The policies are associated with a database object and may also be inherited. A policy defines, for a database object, an archiving action to be taken, a scope, and a condition before the archiving action is triggered. Archiving actions may include compression, data movement, table clustering, and other actions to place the database object into an appropriate storage tier for a lifecycle phase of the database object. Conditions based on access statistics can be specified at the row level and may use segment or block level heatmaps. Policy evaluation occurs periodically in the background, with actions queued as tasks for a task scheduler.

Abstract: A method, apparatus, and system for policy driven data placement and information lifecycle management in a database management system are provided. A user or database application can specify declarative policies that define the movement and transformation of stored database objects. The policies are associated with a database object and may also be inherited. A policy defines, for a database object, an archiving action to be taken, a scope, and a condition before the archiving action is triggered. Archiving actions may include compression, data movement, table clustering, and other actions to place the database object into an appropriate storage tier for a lifecycle phase of the database object. Conditions based on access statistics can be specified at the row level and may use segment or block level heatmaps. Policy evaluation occurs periodically in the background, with actions queued as tasks for a task scheduler.

Abstract: Techniques are described herein for sharing a dictionary across multiple in-memory compression units (IMCUs). After a dictionary is used to encode a first column vector in a first IMCU, the same dictionary is used to encode a second column vector in a second IMCU. The entries in the dictionary are in sort order to facilitate binary searching when performing value-to-code look-ups. If, during the encoding of the second column vector, values are encountered for which the dictionary does not already have codes, then a “sort-order-boundary” is established after the last entry in the dictionary, and entries for the newly encountered values are added to the dictionary, after the sort-order-boundary. To facilitate value-to-code look-ups, the new entries are also sorted relative to each other, creating a second “sort order set”. A new version of the dictionary may be created when the number of sort order sets in the first version of the dictionary reaches a configurable threshold.

Abstract: Techniques are described for encoding join columns that belong to the same domain with a common dictionary. The tables are encoded with dictionary indexes that make the comparison operation of a join query a quick equality check of two integers and there is no need to compute any hashes during execution. Additionally, the techniques described herein minimize the bloom filter creation and evaluation cost as well because the dictionary indexes serve as hash values into the bloom filter. If the bloom filter is as large as the range of dictionary indexes, then the filter is no longer a probabilistic structure and can be used to filter rows in the probe phase with full certainty without any significant overhead.

Abstract: A method, apparatus, and system for custom policy driven data placement and information lifecycle management in a database management system are provided. A user or database application can specify declarative custom policies that define the movement and transformation of stored database objects. A custom policy defines, for a database object, a custom function to evaluate on an associated database object to determine whether an archiving action is triggered. Archiving actions may include compression, data movement, table clustering, and other actions to place the database object into an appropriate storage tier for a lifecycle phase of the database object. The custom function is defined by the database user, and can flexibly include any customized business logic using data sources internal and external to the database, including database access statistics such as segment level or block level heatmaps.

Abstract: A server receives a query and checks how columns in rows responsive to the query are to be sent to a client. Based on one result of checking, the server prepares and transmits a redacted version of one or more rows or portions thereof. The redacted version may be prepared by discarding from the retrieved rows or portions thereof, any columns that are to not be sent, by applying a specific redaction technique, while maintaining columns that are to be sent in a database storage format in which the retrieved rows are stored. Based on another result of checking, the server prepares and transmits a raw version of the retrieved rows or portions thereof, in the database storage format. Based on yet another result of checking, the server extracts columns selected by the query to memory and sends processed versions of the rows or portions thereof.

Abstract: A method, apparatus, and system for automated information lifecycle management using low access patterns in a database management system are provided. A user or the database can store policy data that defines an archiving action when meeting an activity-level condition on one or more database objects. The archiving actions may include compression, data movement, and other actions to place the database object in an appropriate storage tier for a lifecycle phase of the database object. The activity-level condition may specify the database object meeting a low access pattern, optionally for a minimum time period. Various criteria including access statistics for the database object and cost characteristics of current and target compression levels or storage tiers may be considered to determine the meeting of the activity-level condition. The policies may be evaluated on an adjustable periodic basis and may utilize a task scheduler for minimal performance impact.

Abstract: A method, apparatus, and system for automated information lifecycle management using low access patterns in a database management system are provided. A user or the database can store policy data that defines an archiving action when meeting an activity-level condition on one or more database objects. The archiving actions may include compression, data movement, and other actions to place the database object in an appropriate storage tier for a lifecycle phase of the database object. The activity-level condition may specify the database object meeting a low access pattern, optionally for a minimum time period. Various criteria including access statistics for the database object and cost characteristics of current and target compression levels or storage tiers may be considered to determine the meeting of the activity-level condition. The policies may be evaluated on an adjustable periodic basis and may utilize a task scheduler for minimal performance impact.

Abstract: Techniques are described for encoding join columns that belong to the same domain with a common dictionary. The tables are encoded with dictionary indexes that make the comparison operation of a join query a quick equality check of two integers and there is no need to compute any hashes during execution. Additionally, the techniques described herein minimize the bloom filter creation and evaluation cost as well because the dictionary indexes serve as hash values into the bloom filter. If the bloom filter is as large as the range of dictionary indexes, then the filter is no longer a probabilistic structure and can be used to filter rows in the probe phase with full certainty without any significant overhead.

Abstract: A data block stores one or more rows of a database table or relation. An entire row may not fit in a data block. Part of the row is stored in one data block, and another part is stored in another data block. Each row part is referred to herein as a row segment and the data blocks are referred to as row-segmented data blocks. Data block dictionary compression is used to compress row-segmented data blocks. Each data block contains a dictionary that is used to compress rows in the data block, including row segments. The dictionary in a data block is used to compress row segments in the data block. Hence, multiple dictionaries may be used to decompress a row comprised of row segments.

Abstract: Techniques are described herein for reducing the number of redundant evaluations that occur when an expression is evaluated against an encoded column vector by caching results of expression evaluations. When executing a query that includes an expression that references columns for which dictionary-encoded column vectors exist, the database server performs a cost-based analysis to determine which expressions (or sub-expressions) would benefit from caching the expression's evaluation result. For each such expression, the database server performs the necessary computations and caches the results for each of the possible distinct input values. When evaluating an expression for a row with a particular set of input codes, a look-up is performed based on the input code combination to retrieve the pre-computed results of that evaluation from the cache.

Abstract: Techniques are described herein for sharing a dictionary across multiple in-memory compression units (IMCUs). After a dictionary is used to encode a first column vector in a first IMCU, the same dictionary is used to encode a second column vector in a second IMCU. The entries in the dictionary are in sort order to facilitate binary searching when performing value-to-code look-ups. If, during the encoding of the second column vector, values are encountered for which the dictionary does not already have codes, then a “sort-order-boundary” is established after the last entry in the dictionary, and entries for the newly encountered values are added to the dictionary, after the sort-order-boundary. To facilitate value-to-code look-ups, the new entries are also sorted relative to each other, creating a second “sort order set”. A new version of the dictionary may be created when the number of sort order sets in the first version of the dictionary reaches a configurable threshold.

Abstract: A data block stores one or more rows of a database table or relation. An entire row may not fit in a data block. Part of the row is stored in one data block, and another part is stored in another data block. Each row part is referred to herein as a row segment and the data blocks are referred to as row-segmented data blocks. Data block dictionary compression is used to compress row-segmented data blocks. Each data block contains a dictionary that is used to compress rows in the data block, including row segments. The dictionary in a data block is used to compress row segments in the data block. Hence, multiple dictionaries may be used to decompress a row comprised of row segments.

Abstract: A method, apparatus, and system for policy driven data placement and information lifecycle management in a database management system are provided. A user or database application can specify declarative policies that define the movement and transformation of stored database objects. The policies are associated with a database object and may also be inherited. A policy defines, for a database object, an archiving action to be taken, a scope, and a condition before the archiving action is triggered. Archiving actions may include compression, data movement, table clustering, and other actions to place the database object into an appropriate storage tier for a lifecycle phase of the database object. Conditions based on access statistics can be specified at the row level and may use segment or block level heatmaps. Policy evaluation occurs periodically in the background, with actions queued as tasks for a task scheduler.

Abstract: A method, apparatus, and system for custom policy driven data placement and information lifecycle management in a database management system are provided. A user or database application can specify declarative custom policies that define the movement and transformation of stored database objects. A custom policy defines, for a database object, a custom function to evaluate on an associated database object to determine whether an archiving action is triggered. Archiving actions may include compression, data movement, table clustering, and other actions to place the database object into an appropriate storage tier for a lifecycle phase of the database object. The custom function is defined by the database user, and can flexibly include any customized business logic using data sources internal and external to the database, including database access statistics such as segment level or block level heatmaps.

Abstract: A method, apparatus, and system for automated information lifecycle management using low access patterns in a database management system are provided. A user or the database can store policy data that defines an archiving action when meeting an activity-level condition on one or more database objects. The archiving actions may include compression, data movement, and other actions to place the database object in an appropriate storage tier for a lifecycle phase of the database object. The activity-level condition may specify the database object meeting a low access pattern, optionally for a minimum time period. Various criteria including access statistics for the database object and cost characteristics of current and target compression levels or storage tiers may be considered to determine the meeting of the activity-level condition. The policies may be evaluated on an adjustable periodic basis and may utilize a task scheduler for minimal performance impact.