Abstract: A computing system recovers volumes in a distributed computing environment while reducing downtime of storage servers. In an embodiment, a storage server contacts a control plane after a storage failure has occurred. If the storage server hosts an authoritative copy of an offline volume, the storage server is requested to restore the volume. Non-authoritative volumes are removed from the storage server and the storage server provides read access to the restored volume while resuming storage services.

Abstract: A failure of a storage device used to provide a mirrored storage volume can be managed without a full re-mirroring of the volume. The volume can be provided using a set of similar storage devices on each of a master server and a slave server, and a technique such as data striping can be used to store the data for the volume across the various devices. When a storage device becomes unavailable, the data from the corresponding storage device on the other mirrored server can be written to the remaining storage devices on the server experiencing the device failure. The data interface can be virtualized such that the user can continue to send input and output (I/O) requests using the same address information. A translation layer can map the virtualized addresses to the physical addresses where the data is stored.

Abstract: A block-level data storage system receives a request to delete a data storage volume. As a result, the data storage volume is deleted and the areas comprising the volume are released and reaped. The areas may contain non-zero data within a plurality of data storage chunks that comprise the areas. An area cleaner is configured to zero out the areas for allocation to a newly created data storage volume.

Abstract: Commitments against various resources can be dynamically adjusted for customers in a shared-resource environment. A customer can provision a data volume with a committed rate of Input/Output Operations Per Second (IOPS) and pay only for that commitment (plus any overage), for example, as well as the amount of storage requested. The customer can subsequently adjust the committed rate of IOPS by submitting an appropriate request, or the rate can be adjusted automatically based on any of a number of criteria. Data volumes for the customer can be migrated, split, or combined in order to provide the adjusted rate. The interaction of the customer with the data volume does not need to change, independent of adjustments in rate or changes in the data volume, other than the rate at which requests are processed.

Abstract: The present disclosure provides persistent storage for a master copy using operation numbers. A master copy can include a B-tree with references to corresponding data. When provisioning a slave copy, the master copy sends a point-in-time copy of the B-tree to the slave copy, which stores a copy of the B-tree, allocates the necessary space, and updates the references of the B-tree to point to a local storage before the data is transferred. When writing the data to persistent storage, a snapshot created on the master copy is an operation that is replicated to the slave copy. The snapshot is generated using a volume view that includes changes to chunks of data of the master copy since a previous snapshot, as determined using the operation number for the previous snapshot. Data (and metadata) for the snapshot is written to persistent storage while new I/O operations are processed.

Abstract: A request to create a volume to store data is received. A block within the storage node is selected, dependent at least in part on metadata indicating regions of available storage space in a storage node, to associate with a volume. Information is generated that includes an address to the block. A second computer system is determined to lacks the address to the block. The second computer system is enabled, by providing at least a portion of the information to the second computer system, to perform an operation to the block.

Abstract: The relative health of data storage drives may be determined based, at least in some aspects, on data access information and/or other drive operation information. In some examples, upon receiving the operation information from a computing device, a health level of a drive may be determined. The health level determination may be based at least in part on operating information received from a client entity. Additionally, a storage space allocation instruction or operation may be determined for execution. The allocation instruction or operation determined to be performed may be based at least in part on the determined health level.

Abstract: Persistent storage for a master copy is provided using operation numbers. A master copy can include a persistent key-value store such as a B-tree with references to corresponding data. When provisioning a slave copy, the master copy sends a point-in-time copy of the B-tree to the slave copy, which stores a copy of the B-tree, allocates the necessary space, and updates the references of the B-tree to point to a local storage before the data is transferred. When writing the data to persistent storage, a snapshot created on the master copy is an operation that is replicated to the slave copy. The snapshot is generated using a volume view that includes changes to chunks of data of the master copy since a previous snapshot, as determined using the operation number for the previous snapshot. Data (and metadata) for the snapshot is written to persistent storage while new input/output operations are processed.

Abstract: A first location in one or more storage nodes is determined, with the first location being associated with a first block of a plurality of blocks associated with a storage volume. First information that maps the first block to the first location is generated. At least a portion of data is obtained from the first block at the first location. A second location in the one or more storage nodes is determined, with the second location being associated with a second block. Second information that maps the second block to the second location is included in the first information. A second computer system, different from the first computer system, is enabled, by providing at least a portion of the first information, to perform an operation to the storage volume.

Abstract: A block-based storage system may implement page cache write logging. Write requests for a data volume maintained at a storage node may be received at a storage node. A page cache for may be updated in accordance with the request. A log record describing the page cache update may be stored in a page cache write log maintained in a persistent storage device. Once the write request is performed in the page cache and recorded in a log record in the page cache write log, the write request may be acknowledged. Upon recovery from a system failure where data in the page cache is lost, log records in the page cache write log may be replayed to restore to the page cache a state of the page cache prior to the system failure.

Abstract: Commitments against various resources can be dynamically adjusted for customers in a shared-resource environment. A customer can provision a data volume with a committed rate of Input/Output Operations Per Second (IOPS) and pay only for that commitment (plus any overage), for example, as well as the amount of storage requested. The customer can subsequently adjust the committed rate of IOPS by submitting an appropriate request, or the rate can be adjusted automatically based on any of a number of criteria. Data volumes for the customer can be migrated, split, or combined in order to provide the adjusted rate. The interaction of the customer with the data volume does not need to change, independent of adjustments in rate or changes in the data volume, other than the rate at which requests are processed.

Abstract: A block-based storage system may implement page cache write logging. Write requests for a data volume maintained at a storage node may be received at a storage node. A page cache for may be updated in accordance with the request. A log record describing the page cache update may be stored in a page cache write log maintained in a persistent storage device. Once the write request is performed in the page cache and recorded in a log record in the page cache write log, the write request may be acknowledged. Upon recovery from a system failure where data in the page cache is lost, log records in the page cache write log may be replayed to restore to the page cache a state of the page cache prior to the system failure.

Abstract: A distributed system may implement dynamic resource creation for connecting resources. Resource creation requests may be received for resources which are accessed by other resources hosted in a distributed system. An intention for the creation request is stored to perform the creation request. At a subsequent time, a request to connect another resource to the resource of the creation request is received. An infrastructure locality for the other resource may then be identified and utilized to determine a placement location for the resource. The resource may then be created at the placement location according to the stored intention. The resource may then be connected with the other resource for access.

Abstract: A data storage system includes a head node and mass storage devices. The head node is configured to flush data stored in a storage of the head node to a set of the mass storage devices of the data storage system. A head node may flush both current version data and point-in-time version data to the set of mass storage devices. Also, the data storage system maintains an index that indicates storage locations of data for particular portions of a volume before and after the data is flushed to the set of mass storage devices. In some embodiments an index includes a current version reference for a volume or portion of a volume and one or more point-in-time snapshot references.

Abstract: A data storage system includes a head node and mass storage devices. The head node is configured to flush data stored in a storage of the head node, based at least in part on one or more triggers being met, from the storage of the head node to a set of the mass storage devices of the data storage system. The flushed data is written to a segment of free storage space across the set of the mass storage devices allocated for the given data flush operation. In some embodiments, a head node may flush both current version data and point-in-time version data to the set of mass storage devices. Also, the data storage system maintains an index that indicates storage locations of data for particular portions of a volume before and after the data is flushed to the set of mass storage devices.

Abstract: A user can set or modify operational parameters of a data volume stored on a network-accessible storage device in a data center. For example, the user may be provided access to a data volume and may request a modification to the operational parameters of the data volume. Instead of modifying the existing data volume, the data center can provision a new data volume and migrate data stored on the existing data volume to the new data volume. While the data migration takes place, the existing data volume may block input/output (I/O) requests and the new data volume may handle such requests instead. If a request is received for data not yet migrated to the new data volume, then the new data volume prioritizes a migration of the requested data.

Abstract: A data storage system includes multiple data storage units and a zonal control plane. The zonal control plane assigns volumes to respective ones of the data storage units. The data storage units include multiple head nodes and data storage sleds. At least one of the head nodes implements a local control plane for the data storage unit. Also, the head nodes of each data storage unit are configured to service read and write requests directed to one or more volumes serviced by the data storage unit independent of the zonal control plane.

Abstract: A data storage system includes a rack, multiple head nodes, multiple data storage sleds, and at least two networking devices. The at least two network devices are configured to implement at least two redundant networks within the data storage system. Also, each of the head nodes is assigned at least two network addresses for communication with the data storage sleds of the data storage system via the at least two networking devices. The data storage sleds each include multiple mass storage devices and a sled controller that is configured to couple with the at least two network switches. In some embodiments, the data storage system further includes redundant power systems within a rack in which the head nodes, the data storage sleds, and the at least two networking devices are mounted.

Abstract: A data storage system includes multiple head nodes and data storage sleds. The data storage sleds include multiple mass storage devices and a sled controller. Respective ones of the head nodes are configured to obtain credentials for accessing particular portions of the mass storage devices of the data storage sleds. A sled controller of a data storage sled determines whether a head node attempting to perform a write on a mass storage device of a data storage sled that includes the sled controller is presenting with the write request a valid credential for accessing the mass storage devices of the data storage sled. If the credentials are valid, the sled controller causes the write to be performed and if the credentials are invalid, the sled controller returns a message to the head node indicating that it has been fenced off from the mass storage device.

Abstract: A data storage system includes multiple head nodes and multiple data storage sleds mounted in a rack. For a particular volume or volume partition one of the head nodes is designated as a primary head node for the volume or volume partition. The primary head node is configured to store data for the volume in a data storage of the primary head node and cause the data to be replicated to a secondary head node. The primary head node is also configured to cause the data for the volume to be stored in a plurality of respective mass storage devices each in different ones of the plurality of data storage sleds of the data storage system.