Abstract: Techniques for maintaining a cascading index are provided. In one approach, one or more branch node compression techniques are applied to the main index of a cascading index. In an approach, a Bloom filter is generated and associated with, e.g., a branch node in the main index. The Bloom filter is used to determine whether, without accessing any leaf blocks, a particular key value exists, e.g., in leaf blocks associated with the branch node. In an approach, a new redo record is generated in response to a merge operation between two levels of the cascading index. The new redo record comprises (a) one or more addresses of blocks that are affected by the merge operation, (b) data is that being “pushed down” to a lower level of the cascading index, and (c) one or more addresses of blocks that are written to storage as a result of the merge operation.

Abstract: In a consumer-producer distributed model of computing, consumers request data from producers and perform computing work on the data received from the producers, thereby transforming the data. At least some of the computing work performed by a consumer on data provided by the producer is commutable between the consumer and producer, that is, either the producer as well as the consumer can perform the computing work on the data. A producer redistributes commutable based on the work load of the producer and consumer as well as other factors.

Abstract: Techniques are provided for using an intermediate cache between the shared cache of an application and the non-volatile storage of a storage system. The application may be any type of application that uses a storage system to persistently store data. The intermediate cache may be local to the machine upon which the application is executing, or may be implemented within the storage system. In one embodiment where the application is a database server, the database system includes both a DB server-side intermediate cache, and a storage-side intermediate cache. The caching policies used to populate the intermediate cache are intelligent, taking into account factors that may include which object an item belongs to, the item type of the item, a characteristic of the item, or the type of operation in which the item is involved.

Abstract: Methods, computer-readable media, and computer systems are provided for initiating storage of data on multiple storage devices and confirming storage of the data after the data has been stored on one but not necessarily all of the devices. A storage server receives, from a client, a request to store data. In response to the request, the storage server initiates, in parallel, storage of the data on multiple storage systems. The storage server detects that the data has been stored on any one of the storage systems, such as an auxiliary system, and, in response, indicates, to the client, that the data has been stored. The storage server may flush or discard data on the auxiliary storage system upon detecting that the data has been successfully stored on a target storage system, where the data persists.

Abstract: One embodiment of the present invention provides a system for performing selective encryption/decryption in a data storage system. During operation, the system receives a data block from a storage medium at an input/output layer, wherein the input/output layer serves as an interface between the storage medium and a buffer cache. Next, the system determines whether the data block is an encrypted data block. If not, the system stores the data block in the buffer cache. Otherwise, if the data block is an encrypted data block, the system retrieves a storage-key, wherein the storage-key is associated with a subset of storage, which is associated with the encrypted data block. Using the storage-key, the system then decrypts the encrypted data block to produce a decrypted data block. Finally, the system stores the decrypted data block in the buffer cache, wherein the data block remains encrypted in the storage medium.

Abstract: Techniques are provided for managing, within a storage system, the sequence in which I/O requests are processed by the storage system based, at least in part, on one or more logical characteristics of the I/O requests. The logical characteristics may include, for example, the identity of the user for whom the I/O request was submitted, the service that submitted the I/O request, the database targeted by the I/O request, an indication of a consumer group to which the I/O request maps, the reason why the I/O request was issued, a priority category of the I/O request, etc. Techniques are also provided for automatically establishing a scheduling policy within a storage system, and for dynamically changing the scheduling policy in response to changes in workload.

Abstract: Techniques are provided for using an intermediate cache between the shared cache of a database server and the non-volatile storage of a storage system. The intermediate cache may be local to the machine upon which the database server is executing, or may be implemented within the storage system. In one embodiment, the database system includes both a DB server-side intermediate cache, and a storage-side intermediate cache. The caching policies used to populate the intermediate cache are intelligent, taking into account factors that may include which database object an item belongs to, the item type of the item, a characteristic of the item; or the database operation in which the item is involved.

Abstract: No-loss rapid recovery performs resynchronization efficiently while concurrently allowing availability to mirrored data on the storage device. No-loss rapid recovery has two stages and involves storage devices that have both a non-volatile cache and primary storage and that operate as mirror buddies. The first stage is referred to herein as the buddy-retention stage. During the buddy-retention stage, writes to mirrored data are not performed on the offline mirror buddy but are performed on the online mirror buddy. The mirrored data changed in the online mirrored buddy is retained in the non-volatile cache of the retention buddy. The next stage is referred to herein as the rapid resynchronization stage. In this stage, the changed mirrored data retained by the retention buddy for no-loss rapid recovery is used to resynchronize the offline buddy. The storage device is resynchronized using the changed mirrored data retained in the cache of the mirror buddy.

Abstract: Techniques for maintaining a cascading index are provided. In one approach, one or more branch node compression techniques are applied to the main index of a cascading index. In an approach, a Bloom filter is generated and associated with, e.g., a branch node in the main index. The Bloom filter is used to determine whether, without accessing any leaf blocks, a particular key value exists, e.g., in leaf blocks associated with the branch node. In an approach, a new redo record is generated in response to a merge operation between two levels of the cascading index. The new redo record comprises (a) one or more addresses of blocks that are affected by the merge operation, (b) data is that being “pushed down” to a lower level of the cascading index, and (c) one or more addresses of blocks that are written to storage as a result of the merge operation.

Abstract: A method and system for replicating database data is provided. One or more standby database replicas can be used for servicing read-only queries, and the amount of storage required is scalable in the size of the primary database storage. One technique is described for combining physical database replication to multiple physical databases residing within a common storage system that performs de-duplication. Having multiple physical databases allows for many read-only queries to be processed, and the de-duplicating storage system provides scalability in the size of the primary database storage. Another technique uses one or more diskless standby database systems that share a read-only copy of physical standby database files. Notification messages provide consistency between each diskless system's in-memory cache and the state of the shared database files.

Abstract: Detecting an operating condition in a multi-server data storage system. One or more of the data storage servers are designated with a status within the cluster. The designating may be performed by at least one of the data storage servers. The status is encoded into log data having records that report changes to the primary data storage performed by the cluster of data storage servers. The log data is transmitted from the primary data storage to the standby data storage system. The standby data storage system determines an operating condition of the cluster of data storage servers based, at least in part, on the status.

Abstract: Techniques are provided for managing, within a storage system, the sequence in which I/O requests are processed by the storage system based, at least in part, on one or more logical characteristics of the I/O requests. The logical characteristics may include, for example, the identity of the user for whom the I/O request was submitted, the service that submitted the I/O request, the database targeted by the I/O request, an indication of a consumer group to which the I/O request maps, the reason why the I/O request was issued, a priority category of the I/O request, etc. Techniques are also provided for automatically establishing a scheduling policy within a storage system, and for dynamically changing the scheduling policy in response to changes in workload.

Abstract: Dirty data in a storage device is made current through rapid re-silvering, which uses a mirrored and up-to-date version of the dirty data from another storage device to recover the data. Because under rapid re-silvering cache metadata in volatile memory survives the failure of the cache, the cache metadata is used to determine which subset of data from the other storage device needs to be copied to the storage device being re-silvered. During re-silvering, cache metadata is used to determine which I/O requests from clients are requests for data that is not stale.

Abstract: Techniques for maintaining a cascading index are provided. In one approach, one or more branch node compression techniques are applied to the main index of a cascading index. In an approach, a Bloom filter is generated and associated with, e.g., a branch node in the main index. The Bloom filter is used to determine whether, without accessing any leaf blocks, a particular key value exists, e.g., in leaf blocks associated with the branch node. In an approach, a new redo record is generated in response to a merge operation between two levels of the cascading index. The new redo record comprises (a) one or more addresses of blocks that are affected by the merge operation, (b) data is that being “pushed down” to a lower level of the cascading index, and (c) one or more addresses of blocks that are written to disk storage as a result of the merge operation.

Abstract: A method and system for replicating database data is provided. One or more standby database replicas can be used for servicing read-only queries, and the amount of storage required is scalable in the size of the primary database storage. One technique is described for combining physical database replication to multiple physical databases residing within a common storage system that performs de-duplication. Having multiple physical databases allows for many read-only queries to be processed, and the de-duplicating storage system provides scalability in the size of the primary database storage. Another technique uses one or more diskless standby database systems that share a read-only copy of physical standby database files. Notification messages provide consistency between each diskless system's in-memory cache and the state of the shared database files.

Abstract: In a consumer-producer distributed model of computing, consumers request data from producers and perform computing work on the data received from the producers, thereby transforming the data. At least some of the computing work performed by a consumer on data provided by the producer is commutable between the consumer and producer, that is, either the producer as well as the consumer can perform the computing work on the data. A producer redistributes commutable based on the work load of the producer and consumer as well as other factors.

Abstract: Techniques are provided for managing, within a storage system, the sequence in which I/O requests are processed by the storage system based, at least in part, on one or more logical characteristics of the I/O requests. The logical characteristics may include, for example, the identity of the user for whom the I/O request was submitted, the service that submitted the I/O request, the database targeted by the I/O request, an indication of a consumer group to which the I/O request maps, the reason why the I/O request was issued, a priority category of the I/O request, etc. Techniques are also provided for automatically establishing a scheduling policy within a storage system, and for dynamically changing the scheduling policy in response to changes in workload.

Abstract: A partial reverse key index is described, which allows distributed contention as resources vie to insert data into an index as well as allows range scans to be performed on the index. To do so, before an index entry for a key value is inserted into an index, the key value is transformed using a transformation operation that affects a subset of the order of the key value. The index entry is then inserted based on the transformed key value. Because the transformation operation affects the order of the key value, the transformed values associated with two consecutive key values will not necessarily be consecutive. Therefore, the index entries associated with the consecutive key values may be inserted into unrelated portions of the index.

Abstract: A storage device uses non-volatile memory devices for caching. The storage device operates in a mode referred to herein as write-back mode. In write-back mode, a storage device responds to a request to write data by persistently writing the data to a cache in a non-volatile memory device and acknowledges to the requestor that the data is written persistently in the storage device. The acknowledgement is sent without necessarily having written the data that was requested to be written to primary storage. Instead, the data is written to primary storage later.

Abstract: Methods, computer-readable media, and computer systems are provided for initiating storage of data on multiple storage devices and confirming storage of the data after the data has been stored on one but not necessarily all of the devices. A storage server receives, from a client, a request to store data. In response to the request, the storage server initiates, in parallel, storage of the data on multiple storage systems. The storage server detects that the data has been stored on any one of the storage systems, such as an auxiliary system, and, in response, indicates, to the client, that the data has been stored. The storage server may flush or discard data on the auxiliary storage system upon detecting that the data has been successfully stored on a target storage system, where the data persists.