Abstract: Techniques are disclosed for creating and using a filter index in order to identify registered queries whose result sets are likely to have been changed by changes made to tables. The filter index entries are based on filter conditions. The filter conditions are created based on predicates contained in the registered queries. The filter conditions may include exclusive predicates and join predicates. Join predicates that join a table T1 with a table T2 may be instantiated by replacing references to table T2 with values extracted from table T2. Various techniques are described for recognizing situations in which a query can be included in, or excluded from, the set of likely-changed queries, without fully evaluating the filter conditions that correspond to the queries.

Abstract: A highly flexible and extensible structure is provided for physically storing tabular data. The structure, referred to as a compression unit, may be used to physically store tabular data that logically resides in any type of table-like structure. Techniques are employed to avoid changing tabular data within existing compression units. Deleting tabular data within compression units is avoided by merely tracking deletion requests, without actually deleting the data. Inserting new tabular data into existing compression units is avoided by storing the new data external to the compression units. If the number of deletions exceeds a threshold, and/or the number of new inserts exceeds a threshold, new compression units may be generated. When new compression units are generated, the previously-existing compression units may be discarded to reclaim storage, or retained to allow reconstruction of prior states of the tabular data.

Abstract: A method and apparatus is provided for optimizing queries received by a database system that relies on an intelligent data storage server to manage storage for the database system. Storing compression units in hybrid columnar format, the storage manager evaluates simple predicates and only returns data blocks containing rows that satisfy those predicates. The returned data blocks are not necessarily stored persistently on disk. That is, the storage manager is not limited to returning disc block images. The hybrid columnar format enables optimizations that provide better performance when processing typical database workloads including both fetching rows by identifier and performing table scans.

Abstract: Disclosed is a mechanism, system, interface, and process to determine one or more life intervals when given a key. In addition, one or more life intervals can be determined for each key within a range of keys. This type of process is useful for auditing purposes or if it is desire to implement a query upon past versions when a key or range of keys is specified for the data for which past versions is desired. This facilitates algorithms, mechanisms, and processes to implement a query upon past versions.

Abstract: Techniques are described to allow a query to be registered as a persistent stored entity within the database, and to generate notifications as and when the query result changes continuously as long as the query continues to be registered with the database. According to one aspect, for a table referenced in a query, a filter condition is generated based, at least in part, on a predicate of the query. Then, the database server determines whether the filter condition is satisfied by either a before image of a row, or an after image of the row, that was modified by a transaction. If the filter condition is satisfied by either the before image or the after image, then the query is added to a first set of queries whose result sets may have been affected by the transaction. From among the first set of queries, a second set of queries that have result sets that were actually affected by the transaction is determined. Notifications are then sent based on the second set of queries.

Abstract: Disclosed are algorithms, mechanisms, processes, mechanisms, and systems to implement a query upon past versions. In one approach, the version query can be defined as either “rowid”-based access or non-rowid based access. With rowid based access, a given rowid is used to identify the data for which past versions are desired. Also disclosed is a mechanism, system, interface, and process to allow specification of a new table decoration clause to obtain one or more past version of one or more data items. The version query provides for a direct interface to get multiple versions of the row(s). In addition, information about the past versions can be retrieved and displayed along with the past versions.

Abstract: Disclosed is a mechanism, system, interface, and process to determine one or more life intervals when given a key. In addition, one or more life intervals can be determined for each key within a range of keys. This type of process is useful for auditing purposes or if it is desire to implement a query upon past versions when a key or range of keys is specified for the data for which past versions is desired. This facilitates algorithms, mechanisms, and processes to implement a query upon past versions.

Abstract: A query result set is an example of a type of body of data whose content is defined by a query. Modifications to a database can change the result of a query and therefore what the content of such a body of data, such as a result set, should be. Active Queries is an approach that enables a database server to detect database changes that alter the result of a query registered for a client and to notify a client of those changes. The notifications may be sent to, for example, a cache manager responsible for maintaining coherency of a result set cache generated by the query. The notification notifies the cache manager of when and/or what changes are needed to maintain the coherency of the cache.

Abstract: Processing a database statement received from a database client, which requests information about the timing of a last committed change to a row in a database table, comprises retrieving a logical time value that represents a commit time associated with the row and providing the value to the client. In one embodiment, providing the logical time value to the client is enabled by database infrastructure in which a virtual column is used to temporarily “store” the value so that the value can be manipulated as if the value resided in a column of the row. Performance of operations that involve the virtual column are processed similar to real columns, as if data was actually logically stored in the virtual column. For example, the virtual column can be used in condition clauses of database statements for concurrency control.

Abstract: Disclosed is a mechanism, system, interface, and process to allow specification of a new table decoration clause to obtain one or more past version of one or more data items. The version query provides for a direct interface to get multiple versions of the row(s). In addition, information about the past versions can be retrieved and displayed along with the past versions. Also disclosed are algorithms, mechanisms, and processes that are employed to implement a query upon past versions. In one approach, the version query can be defined as either “rowid”-based access or non-rowid based access. With rowid based access, a given rowid is used to identify the data for which past versions are desired. Also disclosed is a process for determining one or more life intervals when given a key.

Abstract: A method and mechanism for identifying one or more transactions that have modified or created an object in a database system. Also disclosed is a method and mechanism to compensate for the effects of a transaction in a database system is disclosed. The disclosed approach can be applied to perform auditing for changes and/or transactions in a database system and for identifying the scope of changes and data accesses made by a transaction in the system.

Abstract: A method of processing data from a file includes obtaining a first portion of the file, encrypting the first portion of the file to create a first encrypted portion, obtaining a second portion of the file, encrypting the second portion of the file to create a second encrypted portion, and storing the first and second encrypted portions such that each of the first and the second encrypted portions can be individually accessed. A method of processing data from a file includes receiving a request to access a first portion of the file, wherein data in the first portion of the file is encrypted, and data in a second portion of the file is encrypted, and decrypting the data in the first portion, and not the data in the second portion.

Abstract: Disclosed is a system and method for implementing space management for large objects stored in the computing system. According to some approaches, storage of large objects are managed by dynamically creating contiguous chunks of storage space of varying lengths. The length of each chunk may vary depending upon object size being stored, fragmentation of the storage space, available free space, and/or expected length of the object.

Abstract: Techniques for storing and manipulating tabular data are provided. According to one embodiment, a user may control whether tabular data is stored in row-level or column-major format. Furthermore, the user may control the level of data compression to achieve an optimal balance between query performance and compression ratios. Tabular data from within the same table may be stored in both column-major and row-major format and compressed at different levels. In addition, tabular data can migrate between column-major format and row-major format in response to various events. For example, in response to a request to update or lock a row stored in column-major format, the row may be migrated and subsequently stored into row-major format. In one embodiment, table partitions are used to enhance data compression techniques. For example, compression tests are performed on a representative table partition, and a compression map is generated and applied to other table partitions.

Abstract: A method and apparatus for selective removal of user data changes is provided. In one embodiment, the methodology is implemented by a logical recovery mechanism. Upon receiving a request to selectively remove user data changes stored in a database, the mechanism creates an empty set of transactions to be populated for selective removal, identifies one or more transactions and places them in the causal set of transactions. The mechanism further identifies other transactions that logically depend on any transaction already in the causal set of transactions. Finally, either while the causal set of transactions is being identified or after the identification is completed, the mechanism also determines user data changes made by the causal set of transactions. One or more compensating transactions may be generated by the logical recovery mechanism, which when executed, reverse the user data changes.

Abstract: One embodiment of the present invention provides a system for automatically classifying data in a database. During operation, the system receives and executes a database operation. Next, the system automatically determines if any data was modified as a result of executing the database operation. If so, for each data item that was modified, the system automatically determines if the data item is associated with a classification-rule. If so, the system automatically reclassifies the data item according to the classification-rule. If not, the system leaves a classification of the data item unchanged.

Abstract: A database server stores compressed units in data blocks of a database. A table (or data from a plurality of rows thereof) is first compressed into a “compression unit” using any of a wide variety of compression techniques. The compression unit is then stored in one or more data block rows across one or more data blocks. As a result, a single data block row may comprise compressed data for a plurality of table rows, as encoded within the compression unit. Storage of compression units in data blocks maintains compatibility with existing data block-based databases, thus allowing the use of compression units in preexisting databases without modification to the underlying format of the database. The compression units may, for example, co-exist with uncompressed tables. Various techniques allow a database server to optimize access to data in the compression unit, so that the compression is virtually transparent to the user.

Abstract: Techniques are described herein for automatically selecting the compression techniques to be used on tabular data. A compression analyzer gives users high-level control over the selection process without requiring the user to know details about the specific compression techniques that are available to the compression analyzer. Users are able to specify, for a given set of data, a “balance point” along the spectrum between “maximum performance” and “maximum compression”. The point thus selected is used by the compression analyzer in a variety of ways. For example, in one embodiment, the compression analyzer uses the user-specified balance point to determine which of the available compression techniques qualify as “candidate techniques” for the given set of data. The compression analyzer selects the compression technique to use on a set of data by actually testing the candidate compression techniques against samples from the set of data.

Abstract: A highly flexible and extensible structure is provided for physically storing tabular data. The structure, is referred to as a compression unit, and may be used to physically store tabular data that logically resides in any type of table-like structure. According to one embodiment, compression units are recursive. Thus, a compression unit may have a “parent” compression unit to which it belongs, and may have one or more “child” compression units that belong to it. In one embodiment, compression units include metadata that indicates how the tabular data is stored within them. The metadata for a compression unit may indicate, for example, whether the data within the compression unit is stored in row-major or column major-format (or some combination thereof), the order of the columns within the compression unit (which may differ from the logical order of the columns dictated by the definition of their logical container), a compression technique for the compression unit, the child compression units (if any), etc.

Abstract: Performing an operation directly on data stored in a database table provides for avoiding generation and processing of SQL statements. A Data Layer interface, comprising a set of APIs, allows internal database server components to call directly to the Data Layer for making fast row changes to a table. According to an embodiment, before making any row change to a table through this Data Layer interface, a shared state data structure is created to hold the metadata about the table. A single shared state can be used for all DML changes to a particular table regardless of what columns are updated or inserted during each DML operation. When a process needs to change a row, the process uses a mutable state based on the shared state of the table.