Abstract: The present disclosure generally relates to creating virtualized block storage devices whose data is replicated across isolated computing systems to lower risk of data loss even in wide-scale events, such as natural disasters. The virtualized device can include at least two volumes, each of which is implemented in a distinct computing system. In the case of a failed volume, a new volume can be created and populated with data from the surviving volume. During population, new writes can continue to be replicated to the new volume. The population process can write data from the surviving volume to the new volume “under” new writes, such that the population process does not overwrite data included in the new writes.

Abstract: A data storage system includes multiple data storage units and a zonal control plane. The zonal control plane assigns volumes to respective ones of the data storage units. The data storage units include multiple head nodes and data storage sleds. At least one of the head nodes implements a local control plane for the data storage unit. Also, the head nodes of each data storage unit are configured to service read and write requests directed to one or more volumes serviced by the data storage unit independent of the zonal control plane.

Abstract: The present disclosure generally relates to creating virtualized block storage devices whose data is replicated across isolated computing systems to lower risk of data loss even in wide-scale events, such as natural disasters. The virtualized device can include at least two volumes, each of which is implemented in a distinct computing system. Each volume can be implemented by at least two computing devices, a first of which is configured as a primary device to which reads from and writes to the volume are directed. To ensure consistency in the distributed device, a multi-tier authority service is implemented, in which a cross-computing system authority service designates a volume as having authority to accept writes to the virtualized device, and in which a second tier authority service designates a computing device as having authority to accept writes to the volume.

Abstract: A block-based storage system may implement page cache write logging. Write requests for a data volume maintained at a storage node may be received at a storage node. A page cache for may be updated in accordance with the request. A log record describing the page cache update may be stored in a page cache write log maintained in a persistent storage device. Once the write request is performed in the page cache and recorded in a log record in the page cache write log, the write request may be acknowledged. Upon recovery from a system failure where data in the page cache is lost, log records in the page cache write log may be replayed to restore to the page cache a state of the page cache prior to the system failure.

Abstract: Generally described, one or more aspects of the present application correspond to a highly distributed replica of a volume stored in a networked elastic computing environment. First and second replicas of the volume can be synchronously replicated, and some implementations of the tertiary replica can be asynchronously replicated. The highly distributed nature of the tertiary replica supports parallel data transfer of the data of the volume, resulting in faster creation of backups and new copies of the volume.

Abstract: A data storage system includes multiple head nodes and data storage sleds. Volume data is replicated between a primary and one or more secondary head nodes for a volume partition and is further flushed to a set of mass storage devices of the data storage sleds. Volume metadata is maintained in a primary and one or more secondary head nodes for a volume partition and is updated in response to volume data being flushed to the data storage sleds. Also, the primary and secondary head nodes store check-points of volume metadata to the data storage sleds, wherein in response to a failure of a primary or secondary head node for a volume partition, a replacement secondary head node for the volume partition recreates a secondary replica for the volume partition based, at least in part, on a stored volume metadata checkpoint.

Abstract: A failure of a storage device used to provide a mirrored storage volume can be managed without a full re-mirroring of the volume. The volume can be provided using a set of similar storage devices on each of a master server and a slave server, and a technique such as data striping can be used to store the data for the volume across the various devices. When a storage device becomes unavailable, the data from the corresponding storage device on the other mirrored server can be written to the remaining storage devices on the server experiencing the device failure. The data interface can be virtualized such that the user can continue to send input and output (I/O) requests using the same address information. A translation layer can map the virtualized addresses to the physical addresses where the data is stored.

Abstract: A data storage system includes a head node and mass storage devices. The head node is configured to flush data stored in a storage of the head node to a set of the mass storage devices of the data storage system. A head node may flush both current version data and point-in-time version data to the set of mass storage devices. Also, the data storage system maintains an index that indicates storage locations of data for particular portions of a volume before and after the data is flushed to the set of mass storage devices. In some embodiments an index includes a current version reference for a volume or portion of a volume and one or more point-in-time snapshot references.

Abstract: A snapshot object or other type of object may be stored in a first storage system and may be accelerated in another storage system, wherein an accelerated snapshot or other type of object can be used to populate volumes in the other storage system with data more rapidly than a non-accelerated snapshot or other type of object. The accelerated snapshot or other object may be implemented using an intermediate volume implemented in the other storage system that is populated with data from the snapshot object or the other object stored in the first storage system.

Abstract: A block-based storage system hosts logical volumes that are implemented via multiple replicas of volume data stored on multiple resource hosts in different failure domains. Also, the block-based storage service allows multiple client computing devices to attach to a same given logical volume at the same time. In order to prevent unnecessary failovers, a primary node storing a primary replica is configured with a health check application programmatic interface (API) and a secondary node storing a secondary replica determines whether or not to initiate a failover based on the health of the primary replica.

Abstract: One or more configuration parameters for an object gateway instance are received at an interface to a provider network, the parameters including an identifier of a first object store of the provider network for which to cache objects in a first object cache of the object gateway instance and an indication of a data transfer mode that controls when objects written to the first object cache are written to the first object store. The one or more configuration parameters are stored in a data store of the provider network and sent to the object gateway instance. A read request that includes the identifier of the first object store and a first object identifier is received from the object gateway instance, and a first object associated with the first object identifier and stored in the first object store is sent to the object gateway instance.

Abstract: Generally described, aspects of the present application correspond to enabling rapid duplication of data within a data volume hosted on a network storage system. The network storage system can maintain a highly distributed replica of the data volume, designated for duplication of data within the volume and separate from one or more other replicas designated for handling modifications to the data volume. By providing increased parallelization, the highly distributed replica can facilitate rapid duplication of the volume. When a sufficiently large request to duplicate the data volume is received, the system can create additional duplicate portions of the volume to further increase parallelization. For example, a partition of the highly distributed replica may be repeatedly duplicated to create a large number of intermediary duplicate partitions. The intermediary duplicate partitions can then be used to service the duplication request rapidly, due to increased parallelism.

Abstract: A data storage system includes a head node and mass storage devices. The head node is configured to flush data stored in a storage of the head node to a set of the mass storage devices of the data storage system. A head node may flush both current version data and point-in-time version data to the set of mass storage devices. Also, the data storage system maintains an index that indicates storage locations of data for particular portions of a volume before and after the data is flushed to the set of mass storage devices. In some embodiments an index includes a current version reference for a volume or portion of a volume and one or more point-in-time snapshot references.

Abstract: Generally described, one or more aspects of the present application correspond to techniques for automatic recovery from dual isolation in which both the primary and secondary replicas of a volume are stored on isolating servers. The disclosed techniques use handshakes between the client and the replicas to determine which has a better health score. The replica with the better health score becomes the primary replica, and confirms that it and the secondary replica are both in an isolating state. In response, the primary replica seeks a solo blessing, undoes the isolating state at the volume level (the server host will still be in isolating state), and continues handling I/O and peer replication until its healthy peer is complete. These techniques can avoid availability drops when the servers hosting the primary and secondary replicas of a volume enter the isolating state at around the same time.

Abstract: A block-based storage system hosts logical volumes that are implemented via multiple replicas of volume data stored on multiple resource hosts in different failure domains. Also, the block-based storage service allows multiple client computing devices to attach to a same given logical volume at the same time. In order to prevent unnecessary failovers, a primary node storing a primary replica is configured with a health check application programmatic interface (API) and a secondary node storing a secondary replica determines whether or not to initiate a failover based on the health of the primary replica.

Abstract: The present disclosure generally relates to creating virtualized block storage devices whose data is replicated across isolated computing systems to lower risk of data loss even in wide-scale events, such as natural disasters. The virtualized device can include at least two volumes, each of which is implemented in a distinct computing system. In the case of a failed volume, a new volume can be created and populated with data from the surviving volume. During population, new writes can continue to be replicated to the new volume. The population process can write data from the surviving volume to the new volume “under” new writes, such that the population process does not overwrite data included in the new writes.

Abstract: The present disclosure generally relates to creating virtualized block storage devices whose data is replicated across isolated computing systems to lower risk of data loss even in wide-scale events, such as natural disasters. The virtualized device can include at least two volumes, each of which is implemented in a distinct computing system. Each volume can be implemented by at least two computing devices, a first of which is configured as a primary device to which reads from and writes to the volume are directed. To ensure consistency in the distributed device, a multi-tier authority service is implemented, in which a cross-computing system authority service designates a volume as having authority to accept writes to the virtualized device, and in which a second tier authority service designates a computing device as having authority to accept writes to the volume.

Abstract: Commitments against various resources can be dynamically adjusted for customers in a shared-resource environment. A customer can provision a data volume with a committed rate of Input/Output Operations Per Second (IOPS) and pay only for that commitment (plus any overage), for example, as well as the amount of storage requested. The customer can subsequently adjust the committed rate of IOPS by submitting an appropriate request, or the rate can be adjusted automatically based on any of a number of criteria. Data volumes for the customer can be migrated, split, or combined in order to provide the adjusted rate. The interaction of the customer with the data volume does not need to change, independent of adjustments in rate or changes in the data volume, other than the rate at which requests are processed.

Abstract: A block-based storage system hosts logical volumes that are implemented via multiple replicas of volume data stored on multiple resource hosts in different failure domains. Also, the block-based storage service allows multiple client computing devices to attach to a same given logical volume at the same time. A membership group authority authorizes sequence numbers for a given logical volume and an associated membership group. The members of the membership group ensure that the members are in agreement on the latest sequence number for the given logical volume before responding to read or write requests directed to the given logical volume.

Abstract: Generally described, one or more aspects of the present application correspond to techniques for creating multiple encrypted block store volumes of data from an unencrypted source. These encryption techniques can use a transform fleet as an intermediary use between the unencrypted source and the encrypted volumes. The transform fleet can obtain data of the volume from one or both of two sources—an object storage “snapshot” a block storage “source volume”—and can then apply the appropriate encryption key for performing the encryption of a particular volume.