Abstract: Techniques are provided for more efficiently using the bandwidth of the I/O path between a CPU and volatile memory during the performance of database operation. Relational data from a relational table is stored in volatile memory as column vectors, where each column vector contains values for a particular column of the table. A binary-comparable format may be used to represent each value within a column vector, regardless of the data type associated with the column. The column vectors may be compressed and/or encoded while in volatile memory, and decompressed/decoded on-the-fly within the CPU. Alternatively, the CPU may be designed to perform operations directly on the compressed and/or encoded column vector data. In addition, techniques are described that enable the CPU to perform vector processing operations on the column vector values.

Abstract: Techniques for undropping objects (e.g., tables) and dependent objects in a database systems are provided. When an object is dropped, the object is moved to a recycle bin where it resides until the user undrops the objects or the object is purged. Dependent objects are also moved into the recycle bin with the object to which they depend. The object can be purged from the recycle bin explicitly by a user or when more storage space is needed. Purging of dependent objects and partitions can be deferred if not required to obtain more storage space.

Abstract: Techniques are provided for more efficiently using the bandwidth of the I/O path between a CPU and volatile memory during the performance of database operation. Relational data from a relational table is stored in volatile memory as column vectors, where each column vector contains values for a particular column of the table. A binary-comparable format may be used to represent each value within a column vector, regardless of the data type associated with the column. The column vectors may be compressed and/or encoded while in volatile memory, and decompressed/decoded on-the-fly within the CPU. Alternatively, the CPU may be designed to perform operations directly on the compressed and/or encoded column vector data. In addition, techniques are described that enable the CPU to perform vector processing operations on the column vector values.

Abstract: Techniques for storing and manipulating tabular data are provided. According to one embodiment, a user may control whether tabular data is stored in row-level or column-major format. Furthermore, the user may control the level of data compression to achieve an optimal balance between query performance and compression ratios. Tabular data from within the same table may be stored in both column-major and row-major format and compressed at different levels. In addition, tabular data can migrate between column-major format and row-major format in response to various events. For example, in response to a request to update or lock a row stored in column-major format, the row may be migrated and subsequently stored into row-major format. In one embodiment, table partitions are used to enhance data compression techniques. For example, compression tests are performed on a representative table partition, and a compression map is generated and applied to other table partitions.

Abstract: A computer is programmed to compress data of a database in response to database modification language (DML) statements generated by on-line transaction processing (OLTP) systems. In several embodiments, data that is initially added to a database block is left uncompressed until a predetermined condition is satisfied, which happens infrequently (relative to OLTP transactions on the block). When satisfied, the computer automatically compresses all uncompressed data in the block, which increases the amount of unused space in the block. New data is thereafter added uncompressed to the partially compressed block, until satisfaction of a predetermined condition whereby the partially compressed block is again compressed, i.e. re-compressed. Adding of new data to a partially compressed block and its compression are repeated unless another predetermined condition is met, in response to which the block is not further re-compressed, thereby to recognize a limit on the benefit from compression.

Abstract: An approach is disclosed for implementing failover and resume when using ordered sequences in a multi-instance database environment. The approach commences by instantiating a first database instance initially to serve as an active instance, then instantiating a second database instance to serve as an instance of one or more passive instances. The active database establishes mastership over a sequence and then processes requests for the ‘next’ symbol by accessing a shared sequence cache only after accessing a first instance semaphore. The active instance and the passive instance perform a protocol such that upon passive database detection of a failure of the active database, one of the passive database instances takes over mastership of the sequence cache, and then proceeds to satisfy sequence value requests. The particular order is observed in spite of the failure.

Abstract: In a database system having a plurality of concurrently executing session processes, the method commences by establishing a master list of sequences, the master list comprising a plurality of sequence objects which in turn define a sequence of values used for numbering and other identification within the database system. To reduce sequence cache latch access contention, multiple tiers of latches are provided. Methods of the system provide a first tier having a first tier “global” latch to serialize access to the master list such that at any point in time, only one of the concurrently executing session processes is granted access to the master list, from which master list are allocated sequences on demand. A second tier of latches is provided, the second tier having multiple second tier latches to serialize access to corresponding allocated sequences of values such that at any point in time, only one of the concurrently executing session processes is granted access to the allocated sequence.

Abstract: A computer is programmed to compress data of a database in response to database modification language (DML) statements generated by on-line transaction processing (OLTP) systems. In several embodiments, data that is initially added to a database block is left uncompressed until a predetermined condition is satisfied, which happens infrequently (relative to OLTP transactions on the block). When satisfied, the computer automatically compresses all uncompressed data in the block, which increases the amount of unused space in the block. New data is thereafter added uncompressed to the partially compressed block, until satisfaction of a predetermined condition whereby the partially compressed block is again compressed, i.e. re-compressed. Adding of new data to a partially compressed block and its compression are repeated unless another predetermined condition is met, in response to which the block is not further re-compressed, thereby to recognize a limit on the benefit from compression.

Abstract: Techniques are described herein for automatically selecting the compression techniques to be used on tabular data. A compression analyzer gives users high-level control over the selection process without requiring the user to know details about the specific compression techniques that are available to the compression analyzer. Users are able to specify, for a given set of data, a “balance point” along the spectrum between “maximum performance” and “maximum compression”. The point thus selected is used by the compression analyzer in a variety of ways. For example, in one embodiment, the compression analyzer uses the user-specified balance point to determine which of the available compression techniques qualify as “candidate techniques” for the given set of data. The compression analyzer selects the compression technique to use on a set of data by actually testing the candidate compression techniques against samples from the set of data.

Abstract: For automatic data placement of database data, a plurality of access-tracking data is maintained. The plurality of access-tracking data respectively corresponds to a plurality of data rows that are managed by a database server. While the database server is executing normally, it is automatically determined whether a data row, which is stored in first one or more data blocks, has been recently accessed based on the access-tracking data that corresponds to that data row. After determining that the data row has been recently accessed, the data row is automatically moved from the first one or more data blocks to one or more hot data blocks that are designated for storing those data rows, from the plurality of data rows, that have been recently accessed.

Abstract: Described herein are compression and processing optimizations by using data transformation techniques. In example embodiments, a byte-wise differential transformation is applied to columnar data represented as a list of length-value pairs to determine a list of delta pairs that is subsequently compressed and stored on persistent storage. A length separation transformation is applied to separate a list of length-value pairs into a length array and a corresponding data value array, where these two arrays are subsequently compressed and stored separately on persistent storage. A native number transformation is applied to a set of number values to remove the lengths stored in the number values, where the transformed set is stored on persistent storage instead of the original set of number values.

Abstract: A highly flexible and extensible structure is provided for physically storing tabular data. The structure, referred to as a compression unit, may be used to store tabular data that logically resides in any type of table-like structure. According to one embodiment, compression units are recursive. Thus, a compression unit may have a “parent” compression unit to which it belongs, and may have one or more “child” compression units that belong to it. In one embodiment, compression units include metadata that indicates how the tabular data is stored within them. The metadata for a compression unit may indicate, for example, whether the data is stored in row-major or column major-format the order of the columns within the compression unit (which may differ from the logical order of the columns dictated by the definition of their logical container), a compression technique for the compression unit, the child compression units (if any), etc.

Abstract: A highly flexible and extensible structure is provided for physically storing tabular data. The structure, referred to as a compression unit, may be used to physically store tabular data that logically resides in any type of table-like structure. Techniques are employed to avoid changing tabular data within existing compression units. Deleting tabular data within compression units is avoided by merely tracking deletion requests, without actually deleting the data. Inserting new tabular data into existing compression units is avoided by storing the new data external to the compression units. If the number of deletions exceeds a threshold, and/or the number of new inserts exceeds a threshold, new compression units may be generated. When new compression units are generated, the previously-existing compression units may be discarded to reclaim storage, or retained to allow reconstruction of prior states of the tabular data.

Abstract: A method and apparatus is provided for optimizing queries received by a database system that relies on an intelligent data storage server to manage storage for the database system. Storing compression units in hybrid columnar format, the storage manager evaluates simple predicates and only returns data blocks containing rows that satisfy those predicates. The returned data blocks are not necessarily stored persistently on disk. That is, the storage manager is not limited to returning disc block images. The hybrid columnar format enables optimizations that provide better performance when processing typical database workloads including both fetching rows by identifier and performing table scans.

Abstract: Techniques for storing and manipulating tabular data are provided. According to one embodiment, a user may control whether tabular data is stored in row-level or column-major format. Furthermore, the user may control the level of data compression to achieve an optimal balance between query performance and compression ratios. Tabular data from within the same table may be stored in both column-major and row-major format and compressed at different levels. In addition, tabular data can migrate between column-major format and row-major format in response to various events. For example, in response to a request to update or lock a row stored in column-major format, the row may be migrated and subsequently stored into row-major format. In one embodiment, table partitions are used to enhance data compression techniques. For example, compression tests are performed on a representative table partition, and a compression map is generated and applied to other table partitions.

Abstract: Techniques are described herein for automatically selecting the compression techniques to be used on tabular data. A compression analyzer gives users high-level control over the selection process without requiring the user to know details about the specific compression techniques that are available to the compression analyzer. Users are able to specify, for a given set of data, a “balance point” along the spectrum between “maximum performance” and “maximum compression”. The point thus selected is used by the compression analyzer in a variety of ways. For example, in one embodiment, the compression analyzer uses the user-specified balance point to determine which of the available compression techniques qualify as “candidate techniques” for the given set of data. The compression analyzer selects the compression technique to use on a set of data by actually testing the candidate compression techniques against samples from the set of data.

Abstract: A database server stores compressed units in data blocks of a database. A table (or data from a plurality of rows thereof) is first compressed into a “compression unit” using any of a wide variety of compression techniques. The compression unit is then stored in one or more data block rows across one or more data blocks. As a result, a single data block row may comprise compressed data for a plurality of table rows, as encoded within the compression unit. Storage of compression units in data blocks maintains compatibility with existing data block-based databases, thus allowing the use of compression units in preexisting databases without modification to the underlying format of the database. The compression units may, for example, co-exist with uncompressed tables. Various techniques allow a database server to optimize access to data in the compression unit, so that the compression is virtually transparent to the user.

Abstract: A highly flexible and extensible structure is provided for physically storing tabular data. The structure, is referred to as a compression unit, and may be used to physically store tabular data that logically resides in any type of table-like structure. According to one embodiment, compression units are recursive. Thus, a compression unit may have a “parent” compression unit to which it belongs, and may have one or more “child” compression units that belong to it. In one embodiment, compression units include metadata that indicates how the tabular data is stored within them. The metadata for a compression unit may indicate, for example, whether the data within the compression unit is stored in row-major or column major-format (or some combination thereof), the order of the columns within the compression unit (which may differ from the logical order of the columns dictated by the definition of their logical container), a compression technique for the compression unit, the child compression units (if any), etc.

Abstract: A system for managing dropped objects of a database. The system comprises a finite amount of disk space for temporarily storing the dropped objects. The system further includes an indexer for assigning time stamps to each of the dropped objects and a reclaimer for reclaiming the dropped objects based on their time stamps. These time stamps can also be used to guarantee the amount of time the object remains in the recycle bin.

Abstract: A computer is programmed to compress data of a database in response to database modification language (DML) statements generated by on-line transaction processing (OLTP) systems. In several embodiments, data that is initially added to a database block is left uncompressed until a predetermined condition is satisfied, which happens infrequently (relative to OLTP transactions on the block). When satisfied, the computer automatically compresses all uncompressed data in the block, which increases the amount of unused space in the block. New data is thereafter added uncompressed to the partially compressed block, until satisfaction of a predetermined condition whereby the partially compressed block is again compressed, i.e. re-compressed. Adding of new data to a partially compressed block and its compression are repeated unless another predetermined condition is met, in response to which the block is not further re-compressed, thereby to recognize a limit on the benefit from compression.