Abstract: A data block stores one or more rows of a database table or relation. An entire row may not fit in a data block. Part of the row is stored in one data block, and another part is stored in another data block. Each row part is referred to herein as a row segment and the data blocks are referred to as row-segmented data blocks. Data block dictionary compression is used to compress row-segmented data blocks. Each data block contains a dictionary that is used to compress rows in the data block, including row segments. The dictionary in a data block is used to compress row segments in the data block. Hence, multiple dictionaries may be used to decompress a row comprised of row segments.

Abstract: Row locking is performed at the row level of granularity for database data stored in columnar form. Row level locking entails use of a lock vector that is stored in a compression unit in a data block, the compression unit storing rows in columnar-major format. On an as needed basis, the lock vector is expanded to identify more transactions affecting the rows in the compression unit.

Abstract: A method, apparatus, and system for hierarchical organization of tablespace free space metadata in a database are provided. The hierarchy is divided into at least two levels: level 1 or L1 bitmap blocks are stored as a shared data structure and map free space in a tablespace, whereas level 2 or L2 bitmap blocks are stored as separate local copies at each database instance and map to the L1 bitmap blocks. This hierarchical organization provides a mechanism for finer grained concurrency control, enabling highly parallel tablespace metadata processing to accommodate the performance requirements of large tablespaces with big data sets. By integrating the hierarchical organization as part of the tablespace stack layer in a database management system (DBMS), implementations can be provided transparently to database end users without demanding any additional administrative, maintenance, or development burdens.

Abstract: A method and apparatus is provided for optimizing queries received by a database system that relies on an intelligent data storage server to manage storage for the database system. Storing compression units in hybrid columnar format, the storage manager evaluates simple predicates and only returns data blocks containing rows that satisfy those predicates. The returned data blocks are not necessarily stored persistently on disk. That is, the storage manager is not limited to returning disc block images. The hybrid columnar format enables optimizations that provide better performance when processing typical database workloads including both fetching rows by identifier and performing table scans.

Abstract: A method and apparatus for updating databases are disclosed. In response to a flashback request, a database object is restored to a state as of a specified flashback time. The changes introduced into the database object are reversed in a sequence starting with a change introduced into the database object immediately before issuance of the flashback request and ending with a change introduced into the database object immediately after the specified flashback time.

Abstract: In token compression, a column value is represented by a encoded token. During early predicate evaluation, the result that a decoded token in a column does not satisfy a predicate condition in a predicate conjunction is recorded in a cache as a “FALSE”. Such a result not only means that when the column contains that token that the predicate condition is not satisfied, but also that a predicate conjunction containing the predicate condition is not satisfied. When performing early predicate evaluation on a subsequent row on a token in a column, the conjunction result cache is referred to. If the cache records a FALSE for that token in that column, then it is known the predicate conjunction cannot be satisfied by the decoded token.

Abstract: A method and apparatus for data recovery are disclosed. Undo tablespace size is calculated for user-specified undo retention time based on system statistics collected over a period of time specified by a history time parameter.

Abstract: Techniques for lifecycle state management and in-database archiving are described. Activity tracking refers to techniques that collect statistics related to user access patterns, such as the frequency or recency with which users access particular database elements. The statistics gathered through activity tracking can be supplied to data classification techniques to automatically classify the database elements or to assist users with manually classifying the database elements. Then, once the database elements have been classified, in-database archiving techniques can be employed to move database elements to different storage tiers based on the classifications. However, although the techniques related to activity tracking, data classification, and in-database archiving may be used together as described above; each technique may also be practiced separately.

Abstract: A method, apparatus, and system for custom policy driven data placement and information lifecycle management in a database management system are provided. A user or database application can specify declarative custom policies that define the movement and transformation of stored database objects. A custom policy defines, for a database object, a custom function to evaluate on an associated database object to determine whether an archiving action is triggered. Archiving actions may include compression, data movement, table clustering, and other actions to place the database object into an appropriate storage tier for a lifecycle phase of the database object. The custom function is defined by the database user, and can flexibly include any customized business logic using data sources internal and external to the database, including database access statistics such as segment level or block level heatmaps.

Abstract: A method, apparatus, and system for tracking row and object database activity into block level heatmaps is provided. Database activity including reads, writes, and creates can be tracked by a database management system at the finest possible level of granularity, or the row and object level. To efficiently record the tracked database activity, a two-part structure is described for writing the activity into heatmaps. A hierarchical in-memory component may use a dynamically allocated sparse pool of bitmap blocks. Periodically, the in-memory component is persisted to a stored representation component, sharable with multiple database instances, which may include consolidated last access times and/or a history of heatmap snapshots to reflect access over time. The heatmaps may then be externalized to database users and applications to provide and support a variety of features.

Abstract: In token compression, a column value is represented by a encoded token. During early predicate evaluation, the result that a decoded token in a column does not satisfy a predicate condition in a predicate conjunction is recorded in a cache as a “FALSE”. Such a result not only means that when the column contains that token that the predicate condition is not satisfied, but also that a predicate conjunction containing the predicate condition is not satisfied. When performing early predicate evaluation on a subsequent row on a token in a column, the conjunction result cache is referred to. If the cache records a FALSE for that token in that column, then it is known the predicate conjunction cannot be satisfied by the decoded token.

Abstract: Techniques for activity tracking, data classification, and in-database archiving are described. Activity tracking refers to techniques that collect statistics related to user access patterns, such as the frequency or recency with which users access particular database elements. The statistics gathered through activity tracking can be supplied to data classification techniques to automatically classify the database elements or to assist users with manually classifying the database elements. Then, once the database elements have been classified, in-database archiving techniques can be employed to move database elements to different storage tiers based on the classifications. However, although the techniques related to activity tracking, data classification, and in-database archiving may be used together as described above; each technique may also be practiced separately.

Abstract: A method, apparatus, and system for policy driven data placement and information lifecycle management in a database management system are provided. A user or database application can specify declarative policies that define the movement and transformation of stored database objects. The policies are associated with a database object and may also be inherited. A policy defines, for a database object, an archiving action to be taken, a scope, and a condition before the archiving action is triggered. Archiving actions may include compression, data movement, table clustering, and other actions to place the database object into an appropriate storage tier for a lifecycle phase of the database object. Conditions based on access statistics can be specified at the row level and may use segment or block level heatmaps. Policy evaluation occurs periodically in the background, with actions queued as tasks for a task scheduler.

Abstract: A method, apparatus, and system for automated information lifecycle management using low access patterns in a database management system are provided. A user or the database can store policy data that defines an archiving action when meeting an activity-level condition on one or more database objects. The archiving actions may include compression, data movement, and other actions to place the database object in an appropriate storage tier for a lifecycle phase of the database object. The activity-level condition may specify the database object meeting a low access pattern, optionally for a minimum time period. Various criteria including access statistics for the database object and cost characteristics of current and target compression levels or storage tiers may be considered to determine the meeting of the activity-level condition. The policies may be evaluated on an adjustable periodic basis and may utilize a task scheduler for minimal performance impact.

Abstract: A database server stores compressed units in data blocks of a database. A table (or data from a plurality of rows thereof) is first compressed into a “compression unit” using any of a wide variety of compression techniques. The compression unit is then stored in one or more data block rows across one or more data blocks. As a result, a single data block row may comprise compressed data for a plurality of table rows, as encoded within the compression unit. Storage of compression units in data blocks maintains compatibility with existing data block-based databases, thus allowing the use of compression units in preexisting databases without modification to the underlying format of the database. The compression units may, for example, co-exist with uncompressed tables. Various techniques allow a database server to optimize access to data in the compression unit, so that the compression is virtually transparent to the user.

Abstract: Techniques for maintaining a cascading index are provided. In one approach, one or more branch node compression techniques are applied to the main index of a cascading index. In an approach, a Bloom filter is generated and associated with, e.g., a branch node in the main index. The Bloom filter is used to determine whether, without accessing any leaf blocks, a particular key value exists, e.g., in leaf blocks associated with the branch node. In an approach, a new redo record is generated in response to a merge operation between two levels of the cascading index. The new redo record comprises (a) one or more addresses of blocks that are affected by the merge operation, (b) data is that being “pushed down” to a lower level of the cascading index, and (c) one or more addresses of blocks that are written to storage as a result of the merge operation.

Abstract: A method and mechanism for identifying one or more transactions that have modified or created an object in a database system. Also disclosed is a method and mechanism to compensate for the effects of a transaction in a database system is disclosed. The disclosed approach can be applied to perform auditing for changes and/or transactions in a database system and for identifying the scope of changes and data accesses made by a transaction in the system.

Abstract: A database server stores compressed units in data blocks of a database. A table (or data from a plurality of rows thereof) is first compressed into a “compression unit” using any of a wide variety of compression techniques. The compression unit is then stored in one or more data block rows across one or more data blocks. As a result, a single data block row may comprise compressed data for a plurality of table rows, as encoded within the compression unit. Storage of compression units in data blocks maintains compatibility with existing data block-based databases, thus allowing the use of compression units in preexisting databases without modification to the underlying format of the database. The compression units may, for example, co-exist with uncompressed tables. Various techniques allow a database server to optimize access to data in the compression unit, so that the compression is virtually transparent to the user.

Abstract: A method of processing data from a file includes obtaining a first portion of the file, creating a first compression unit by compressing at least the first portion of the file, obtaining a second portion of the file, creating a second compression unit by compressing at least the second portion of the file, and storing the first and second compression unit such that each of the first and the second compression units can be individually accessed. A method of processing data from a file includes receiving a request to access a portion of the file, determining one or more sub-units that include compressed data associated with the portion of the file, de-compressing the compressed data in the one or more sub-units to obtain de-compressed data, and transmitting the de-compressed data in response to the request.

Abstract: A method and apparatus for efficiently processing data in various formats in a single instruction multiple data (“SIMD”) architecture is presented. Specifically, a method to unpack a fixed-width bit values in a bit stream to a fixed width byte stream in a SIMD architecture is presented. A method to unpack variable-length byte packed values in a byte stream in a SIMD architecture is presented. A method to decompress a run length encoded compressed bit-vector in a SIMD architecture is presented. A method to return the offset of each bit set to one in a bit-vector in a SIMD architecture is presented. A method to fetch bits from a bit-vector at specified offsets relative to a base in a SIMD architecture is presented. A method to compare values stored in two SIMD registers is presented.