Abstract: A slave storage is provisioned using metadata of a master B-tree and updates to references (e.g., offsets) pertaining to data operations of the master B-tree. Master-slave pairs can be used to provide data redundancy, and a master copy can include the master B-tree with references to corresponding data. When provisioning a slave copy, the master sends a B-tree copy to the slave, which stores the slave B-tree copy, allocates the necessary space on local storage, and updates respective offsets of the slave B-tree copy to point to the local storage. Data from the master can then be transferred to the slave and stored according to a note and commit process that ensures operational sequence of the data. Operations received to the master during the process can be committed to the slave copy until the slave is consistent with the master and able to take over as master in the event of a failure.

Abstract: A block-based storage system may implement efficient replication for restoring a data volume from a reduced durability state. A storage node that is not replicating write requests for a data volume may determine that replication for the data volume is to be enabled. A peer storage node may be identified that maintains a stale replica of the data volume. One or more replication operations may be performed to update stale data chunks in the stale replica of the data volume with current data chunks without updating data chunks in the stale replica of the data volume that are current. Stale replicas that are no longer needed may be deleted according timeouts or the amount of stale data in the replica.

Abstract: A block-based storage system may implement reducing durability state for a data volume. A determination may be made that storage node replicating write requests for a data volume is unavailable. In response, subsequent write requests may be processed according to a reduced durability state for the data volume such that replication for the data volume may be disabled for the storage node. Write requests may then be completed at a fewer number of storage nodes prior to acknowledging the write request as complete. Durability state for the data volume may be increase in various embodiments. A storage node may be identified and replication operations may be performed to synchronize the current data volume at the storage node with a replica of the data volume maintained at the identified storage node.

Abstract: A block-based storage system may implement reducing durability state for a data volume. A determination may be made that storage node replicating write requests for a data volume is unavailable. In response, subsequent write requests may be processed according to a reduced durability state for the data volume such that replication for the data volume may be disabled for the storage node. Write requests may then be completed at a fewer number of storage nodes prior to acknowledging the write request as complete. Durability state for the data volume may be increase in various embodiments. A storage node may be identified and replication operations may be performed to synchronize the current data volume at the storage node with a replica of the data volume maintained at the identified storage node.

Abstract: Systems and methods are described for a storage processing service that processes multiple storage commands. The storage processing service uses tags from test storage commands to determine whether a test storage service is to be instantiated that reflects a corresponding production service. Test storage commands with the same tag are tested on that test service. Additionally, the storage processing service determines a strategy for testing processes on production services when the storage system is overloaded. In one embodiment, the test service manager can determine to stop testing processes for a period of time, and issue a shed command that queues or sheds test storage commands. Advantageously, a shed command, while active at a storage processing service, may alleviate the overload on production services. The test service manager can continue to monitor the storage system to determine whether the overload continues to exist.

Abstract: Write optimization for block-based storage performing snapshot operations may be implemented. Write requests for a particular data volume may be received for which a snapshot operation is in progress. A determination may be made as to whether a data chunk of the data volume modified as part of the write request has not yet been stored to a remote snapshot data store as part of the snapshot operation. For a data chunk that is to be modified and that has not yet been stored, the data chunk may be stored in a local in-memory volume snapshot buffer. Once the data chunk is stored in the in-memory volume snapshot buffer, the write request may be performed and acknowledged as complete. The data chunk may be sent to the remote snapshot data store asynchronously with regard to the acknowledgment of the write request.

Abstract: Write optimization for block-based storage performing snapshot operations may be implemented. Write requests for a particular data volume may be received for which a snapshot operation is in progress. A determination may be made as to whether a data chunk of the data volume modified as part of the write request has not yet been stored to a remote snapshot data store as part of the snapshot operation. For a data chunk that is to be modified and that has not yet been stored, the data chunk may be stored in a local in-memory volume snapshot buffer. Once the data chunk is stored in the in-memory volume snapshot buffer, the write request may be performed and acknowledged as complete. The data chunk may be sent to the remote snapshot data store asynchronously with regard to the acknowledgment of the write request.

Abstract: A block-based storage system may implement dynamic durability adjustment for page cache write logging. A rate of incoming write requests for data volumes maintained at a storage node may be monitored. Based, at least in part, on the rate of incoming write requests, a dynamic modification to a durability property for a data volume may be made, such as enabling page cache write logging the data volume or disabling write logging for the data volume. When incoming write requests are received, a determination may be made as to whether page cache write logging for a particular data volume is enabled. For write requests with disabled page cache write logging, the page cache may be updated and the write request may be acknowledged without storing a log record describing the update in a page cache write log.

Abstract: A block-based storage system may implement page cache write logging. Write requests for a data volume maintained at a storage node may be received at a storage node. A page cache for may be updated in accordance with the request. A log record describing the page cache update may be stored in a page cache write log maintained in a persistent storage device. Once the write request is performed in the page cache and recorded in a log record in the page cache write log, the write request may be acknowledged. Upon recovery from a system failure where data in the page cache is lost, log records in the page cache write log may be replayed to restore to the page cache a state of the page cache prior to the system failure.

Abstract: A block-based storage system may implement reducing durability state for a data volume. A determination may be made that storage node replicating write requests for a data volume is unavailable. In response, subsequent write requests may be processed according to a reduced durability state for the data volume such that replication for the data volume may be disabled for the storage node. Write requests may then be completed at a fewer number of storage nodes prior to acknowledging the write request as complete. Durability state for the data volume may be increase in various embodiments. A storage node may be identified and replication operations may be performed to synchronize the current data volume at the storage node with a replica of the data volume maintained at the identified storage node.

Abstract: A block-based storage system may implement dynamic durability adjustment for page cache write logging. A rate of incoming write requests for data volumes maintained at a storage node may be monitored. Based, at least in part, on the rate of incoming write requests, a dynamic modification to a durability property for a data volume may be made, such as enabling page cache write logging the data volume or disabling write logging for the data volume. When incoming write requests are received, a determination may be made as to whether page cache write logging for a particular data volume is enabled. For write requests with disabled page cache write logging, the page cache may be updated and the write request may be acknowledged without storing a log record describing the update in a page cache write log.