Abstract: Techniques are provided for managing, within a storage system, the sequence in which I/O requests are processed by the storage system based, at least in part, on one or more logical characteristics of the I/O requests. The logical characteristics may include, for example, the identity of the user for whom the I/O request was submitted, the service that submitted the I/O request, the database targeted by the I/O request, an indication of a consumer group to which the I/O request maps, the reason why the I/O request was issued, a priority category of the I/O request, etc. Techniques are also provided for automatically establishing a scheduling policy within a storage system, and for dynamically changing the scheduling policy in response to changes in workload.

Abstract: Dirty data in a storage device is made current through rapid re-silvering, which uses a mirrored and up-to-date version of the dirty data from another storage device to recover the data. Because under rapid re-silvering cache metadata in volatile memory survives the failure of the cache, the cache metadata is used to determine which subset of data from the other storage device needs to be copied to the storage device being re-silvered. During re-silvering, cache metadata is used to determine which I/O requests from clients are requests for data that is not stale.

Abstract: Techniques for maintaining a cascading index are provided. In one approach, one or more branch node compression techniques are applied to the main index of a cascading index. In an approach, a Bloom filter is generated and associated with, e.g., a branch node in the main index. The Bloom filter is used to determine whether, without accessing any leaf blocks, a particular key value exists, e.g., in leaf blocks associated with the branch node. In an approach, a new redo record is generated in response to a merge operation between two levels of the cascading index. The new redo record comprises (a) one or more addresses of blocks that are affected by the merge operation, (b) data is that being “pushed down” to a lower level of the cascading index, and (c) one or more addresses of blocks that are written to disk storage as a result of the merge operation.

Abstract: A method and system for replicating database data is provided. One or more standby database replicas can be used for servicing read-only queries, and the amount of storage required is scalable in the size of the primary database storage. One technique is described for combining physical database replication to multiple physical databases residing within a common storage system that performs de-duplication. Having multiple physical databases allows for many read-only queries to be processed, and the de-duplicating storage system provides scalability in the size of the primary database storage. Another technique uses one or more diskless standby database systems that share a read-only copy of physical standby database files. Notification messages provide consistency between each diskless system's in-memory cache and the state of the shared database files.

Abstract: In a consumer-producer distributed model of computing, consumers request data from producers and perform computing work on the data received from the producers, thereby transforming the data. At least some of the computing work performed by a consumer on data provided by the producer is commutable between the consumer and producer, that is, either the producer as well as the consumer can perform the computing work on the data. A producer redistributes commutable based on the work load of the producer and consumer as well as other factors.

Abstract: Techniques are provided for managing, within a storage system, the sequence in which I/O requests are processed by the storage system based, at least in part, on one or more logical characteristics of the I/O requests. The logical characteristics may include, for example, the identity of the user for whom the I/O request was submitted, the service that submitted the I/O request, the database targeted by the I/O request, an indication of a consumer group to which the I/O request maps, the reason why the I/O request was issued, a priority category of the I/O request, etc. Techniques are also provided for automatically establishing a scheduling policy within a storage system, and for dynamically changing the scheduling policy in response to changes in workload.

Abstract: A partial reverse key index is described, which allows distributed contention as resources vie to insert data into an index as well as allows range scans to be performed on the index. To do so, before an index entry for a key value is inserted into an index, the key value is transformed using a transformation operation that affects a subset of the order of the key value. The index entry is then inserted based on the transformed key value. Because the transformation operation affects the order of the key value, the transformed values associated with two consecutive key values will not necessarily be consecutive. Therefore, the index entries associated with the consecutive key values may be inserted into unrelated portions of the index.

Abstract: A storage device uses non-volatile memory devices for caching. The storage device operates in a mode referred to herein as write-back mode. In write-back mode, a storage device responds to a request to write data by persistently writing the data to a cache in a non-volatile memory device and acknowledges to the requestor that the data is written persistently in the storage device. The acknowledgement is sent without necessarily having written the data that was requested to be written to primary storage. Instead, the data is written to primary storage later.

Abstract: Methods, computer-readable media, and computer systems are provided for initiating storage of data on multiple storage devices and confirming storage of the data after the data has been stored on one but not necessarily all of the devices. A storage server receives, from a client, a request to store data. In response to the request, the storage server initiates, in parallel, storage of the data on multiple storage systems. The storage server detects that the data has been stored on any one of the storage systems, such as an auxiliary system, and, in response, indicates, to the client, that the data has been stored. The storage server may flush or discard data on the auxiliary storage system upon detecting that the data has been successfully stored on a target storage system, where the data persists.

Abstract: Techniques for compressing branch nodes in an index are provided. The branch nodes may be part of a main index of a multi-level index that also includes one or more journal indexes. A Bloom filter may be generated and associated with, e.g., a branch node in the main index. The Bloom filter is used to determine whether, without accessing any leaf blocks, a particular key value exists, e.g., in leaf blocks associated with the branch node.

Abstract: Techniques are provided for managing, within a storage system, the sequence in which I/O requests are processed by the storage system based, at least in part, on one or more logical characteristics of the I/O requests. The logical characteristics may include, for example, the identity of the user for whom the I/O request was submitted, the service that submitted the I/O request, the database targeted by the I/O request, an indication of a consumer group to which the I/O request maps, the reason why the I/O request was issued, a priority category of the I/O request, etc. Techniques are also provided for automatically establishing a scheduling policy within a storage system, and for dynamically changing the scheduling policy in response to changes in workload.

Abstract: A system and method for enabling a second database instance to more quickly process a request to execute a database statement that has previously been executed by a first database instance is described. In one embodiment, the method involves sending the database statement from the first database instance to the second database instance, and generating by the second database instance one or more structures needed to prepare the statement for execution, such as a parse tree and an execution plan for the statement. If at some point in the future, the second database instance receives a request to execute the same statement, the above structures can be used for execution, thereby eliminating the need for one or more potentially time-consuming operations, such as generation of a parse tree or execution plan for the statement.

Abstract: Techniques are provided for managing, within a storage system, the sequence in which I/O requests are processed by the storage system based, at least in part, on a one or more logical characteristics of the I/O requests. The logical characteristics may include, for example, the identity of the user for whom the I/O request was submitted, the service that submitted the I/O request, the database targeted by the I/O request, an indication of a consumer group to which the I/O request maps, the reason why the I/O request was issued, a priority category of the I/O request, etc. Techniques are also provided for automatically establishing a scheduling policy within a storage system, and for dynamically changing the scheduling policy in response to changes in workload.

Abstract: A method and system for replicating database data is provided. One or more standby database replicas can be used for servicing read-only queries, and the amount of storage required is scalable in the size of the primary database storage. One technique is described for combining physical database replication to multiple physical databases residing within a common storage system that performs de-duplication. Having multiple physical databases allows for many read-only queries to be processed, and the de-duplicating storage system provides scalability in the size of the primary database storage. Another technique uses one or more diskless standby database systems that share a read-only copy of physical standby database files. Notification messages provide consistency between each diskless system's in-memory cache and the state of the shared database files.

Abstract: A system and method for enabling a second database instance to more quickly process a request to execute a database statement that has previously been executed by a first database instance is described. In one embodiment, the method involves sending the database statement from the first database instance to the second database instance, and generating by the second database instance one or more structures needed to prepare the statement for execution, such as a parse tree and an execution plan for the statement. If at some point in the future, the second database instance receives a request to execute the same statement, the above structures can be used for execution, thereby eliminating the need for one or more potentially time-consuming operations, such as generation of a parse tree or execution plan for the statement.

Abstract: A system and method for enabling a second database instance to more quickly process a request to execute a database statement that has previously been executed by a first database instance is described. In one embodiment, the method involves sending the database statement from the first database instance to the second database instance, and generating by the second database instance one or more structures needed to prepare the statement for execution, such as a parse tree and an execution plan for the statement. If at some point in the future, the second database instance receives a request to execute the same statement, the above structures can be used for execution, thereby eliminating the need for one or more potentially time-consuming operations, such as generation of a parse tree or execution plan for the statement.

Abstract: Techniques are provided for using an intermediate cache between the shared cache of a database server and the non-volatile storage of a storage system. The intermediate cache may be local to the machine upon which the database server is executing, or may be implemented within the storage system. In one embodiment, the database system includes both a DB server-side intermediate cache, and a storage-side intermediate cache. The caching policies used to populate the intermediate cache are intelligent, taking into account factors that may include which database object an item belongs to, the item type of the item, a characteristic of the item; or the database operation in which the item is involved.

Abstract: Space on disk, designated for storage of recovery files is automatically managed so as to not exceed a predetermined limit. Specifically, after creation of information to be stored in a new recovery file, an automatic check is made to see if creation of the new recovery file will cause the space to exceed the limit. If so, at least one recovery file currently in the space is deleted. The new recovery file is automatically created, when the space occupied by existing recovery files becomes sufficiently small to accommodate the new file without exceeding the limit. The recovery file(s) deleted to make room for the new recovery file may be automatically identified in accordance with a user-specified policy. Such a policy may be used to automatically identify a subset of recovery files available for deletion, well ahead of time, i.e. prior to a need for deletion arises.

Abstract: Allocation of memory is optimized across multiple pools of memory, based on minimizing the time it takes to successfully retrieve a given data item from each of the multiple pools. First data is generated that indicates a hit rate per pool size for each of multiple memory pools. In an embodiment, the generating step includes continuously monitoring attempts to access, or retrieve a data item from, each of the memory pools. The first data is converted to second data that accounts for a cost of a miss with respect to each of the memory pools. In an embodiment, the second data accounts for the cost of a miss in terms of time. How much of the memory to allocate to each of the memory pools is determined, based on the second data. In an embodiment, the steps of converting and determining are automatically performed, on a periodic basis.

Abstract: Techniques are provided for using an intermediate cache to provide some of the items involved in a scan operation, while other items involved in the scan operation are provided from primary storage. Techniques are also provided for determining whether to service an I/O request for an item with a copy of the item that resides in the intermediate cache based on factors such as a) an identity of the user for whom the I/O request was submitted, b) an identity of a service that submitted the I/O request, c) an indication of a consumer group to which the I/O request maps, d) whether the I/O request is associated with an offloaded filter provided by the database server to the storage system, or e) whether the intermediate cache is overloaded. Techniques are also provided for determining whether to store items in an intermediate cache in response to the items being retrieved, based on logical characteristics associated with the requests that retrieve the items.