Abstract: A data storage system includes multiple head nodes and data storage sleds. The data storage sleds include multiple mass storage devices and a sled controller. Respective ones of the head nodes are configured to obtain credentials for accessing particular portions of the mass storage devices of the data storage sleds. A sled controller of a data storage sled determines whether a head node attempting to perform a write on a mass storage device of a data storage sled that includes the sled controller is presenting with the write request a valid credential for accessing the mass storage devices of the data storage sled. If the credentials are valid, the sled controller causes the write to be performed and if the credentials are invalid, the sled controller returns a message to the head node indicating that it has been fenced off from the mass storage device.

Abstract: Systems and methods for an incast mitigation approach that first modifies network responses for content requests above a threshold size to be delayed according a response latency selected from an established latency range. Additionally, as incast characteristics are analyzed from network traffic, the volumes can selectively modify the individual established latency range to increase the latency range during periods of higher incast characteristics and to decrease the latency range when incast characteristics appear to be decreasing.

Abstract: A user can set or modify operational parameters of a data volume stored on a network-accessible storage device in a data center. For example, the user may be provided access to a data volume and may request a modification to the operational parameters of the data volume. Instead of modifying the existing data volume, the data center can provision a new data volume and migrate data stored on the existing data volume to the new data volume. While the data migration takes place, the existing data volume may block input/output (I/O) requests and the new data volume may handle such requests instead. Once the data migration is complete, the data center may deallocate the data blocks of the existing data volume such that the data blocks can be reused by other data volumes.

Abstract: Customers of a shared-resource environment can provision resources in a fine-grained manner that meets specific performance requirements. A customer can provision a data volume with a committed rate of Input/Output Operations Per Second (IOPS) and pay only for that commitment (plus any overage), and the amount of storage requested. The customer will then at any time be able to complete at least the committed rate of IOPS. If the customer generates submissions at a rate that exceeds the committed rate, the resource can still process at the higher rate when the system is not under pressure. Even under pressure, the system will deliver at least the committed rate. Multiple customers can be provisioned on the same resource, and more than one customer can have a committed rate on that resource. Customers without committed or guaranteed rates can utilize the uncommitted portion, or committed portions that are not being used.

Abstract: A data storage system includes a head node and mass storage devices. The head node is configured to flush data stored in a storage of the head node, based at least in part on one or more triggers being met, from the storage of the head node to a set of the mass storage devices of the data storage system. The flushed data is written to a segment of free storage space across the set of the mass storage devices allocated for the given data flush operation. In some embodiments, a head node may flush both current version data and point-in-time version data to the set of mass storage devices. Also, the data storage system maintains an index that indicates storage locations of data for particular portions of a volume before and after the data is flushed to the set of mass storage devices.

Abstract: A data storage system includes multiple head nodes and multiple data storage sleds mounted in a rack. For a particular volume or volume partition one of the head nodes is designated as a primary head node for the volume or volume partition. The primary head node is configured to store data for the volume in a data storage of the primary head node and cause the data to be replicated to a secondary head node. The primary head node is also configured to cause the data for the volume to be stored in a plurality of respective mass storage devices each in different ones of the plurality of data storage sleds of the data storage system.

Abstract: A data storage system includes multiple data storage units and a zonal control plane. The zonal control plane assigns volumes to respective ones of the data storage units. The data storage units include multiple head nodes and data storage sleds. At least one of the head nodes implements a local control plane for the data storage unit. Also, the head nodes of each data storage unit are configured to service read and write requests directed to one or more volumes serviced by the data storage unit independent of the zonal control plane.

Abstract: A block-based storage system may implement page cache write logging. Write requests for a data volume maintained at a storage node may be received at a storage node. A page cache for may be updated in accordance with the request. A log record describing the page cache update may be stored in a page cache write log maintained in a persistent storage device. Once the write request is performed in the page cache and recorded in a log record in the page cache write log, the write request may be acknowledged. Upon recovery from a system failure where data in the page cache is lost, log records in the page cache write log may be replayed to restore to the page cache a state of the page cache prior to the system failure.

Abstract: A block-based storage system may implement reducing durability state for a data volume. A determination may be made that storage node replicating write requests for a data volume is unavailable. In response, subsequent write requests may be processed according to a reduced durability state for the data volume such that replication for the data volume may be disabled for the storage node. Write requests may then be completed at a fewer number of storage nodes prior to acknowledging the write request as complete. Durability state for the data volume may be increase in various embodiments. A storage node may be identified and replication operations may be performed to synchronize the current data volume at the storage node with a replica of the data volume maintained at the identified storage node.

Abstract: A data storage system includes multiple head nodes and data storage sleds. The data storage sleds include multiple mass storage devices and a sled controller. Respective ones of the head nodes are configured to obtain credentials for accessing particular portions of the mass storage devices of the data storage sleds. A sled controller of a data storage sled determines whether a head node attempting to perform a write on a mass storage device of a data storage sled that includes the sled controller is presenting with the write request a valid credential for accessing the mass storage devices of the data storage sled. If the credentials are valid, the sled controller causes the write to be performed and if the credentials are invalid, the sled controller returns a message to the head node indicating that it has been fenced off from the mass storage device.

Abstract: A user can set or modify operational parameters of a data volume stored on a network-accessible storage device in a data center. For example, the user may be provided access to a data volume and may request a modification to the operational parameters of the data volume. Instead of modifying the existing data volume, the data center can provision a new data volume and migrate data stored on the existing data volume to the new data volume. While the data migration takes place, the existing data volume may block input/output (I/O) requests and the new data volume may handle such requests instead. Once the data migration is complete, the data center may deallocate the data blocks of the existing data volume such that the data blocks can be reused by other data volumes.

Abstract: Generally described, one or more aspects of the present application correspond to a highly distributed replica of a volume stored in a networked computing environment. First and second replicas of the volume can be synchronously replicated, and some implementations of the tertiary replica can be asynchronously replicated. The highly distributed nature of the tertiary replica supports parallel data transfer of the data of the volume, resulting in faster creation of backups and new copies of the volume.

Abstract: Generally described, aspects of the present application correspond to enabling rapid duplication of data within a data volume hosted on a network storage system. The network storage system can maintain a highly distributed replica of the data volume, designated for duplication of data within the volume and separate from one or more other replicas designated for handling modifications to the data volume. By providing increased parallelization, the highly distributed replica can facilitate rapid duplication of the volume. When a sufficiently large request to duplicate the data volume is received, the system can create additional duplicate portions of the volume to further increase parallelization. For example, a partition of the highly distributed replica may be repeatedly duplicated to create a large number of intermediary duplicate partitions. The intermediary duplicate partitions can then be used to service the duplication request rapidly, due to increased parallelism.

Abstract: Generally described, one or more aspects of the present application correspond to a highly distributed replica of a volume stored in a networked elastic computing environment. First and second replicas of the volume can be synchronously replicated, and some implementations of the tertiary replica can be asynchronously replicated. The highly distributed nature of the tertiary replica supports parallel data transfer of the data of the volume, resulting in faster creation of backups and new copies of the volume.

Abstract: Generally described, one or more aspects of the present application correspond to a highly distributed replica of a volume stored in a networked elastic computing environment. First and second replicas of the volume can be synchronously replicated, and some implementations of the tertiary replica can be asynchronously replicated. The highly distributed nature of the tertiary replica supports parallel data transfer of the data of the volume, resulting in faster creation of backups and new copies of the volume.

Abstract: A data storage system includes multiple head nodes and data storage sleds. A control plane of the data storage system designates, for a volume partition, one of the head nodes to function as a primary head node storing a primary replica of the volume partition and designates two or more other head nodes to function as reserve head nodes storing reserve replicas of the volume partition. Additionally, the primary head node causes volume data for the volume partition to be erasure encoded and stored on multiple mass storage devices in different ones of the data storage sleds.

Abstract: A snapshot object or other type of object may be stored in a first storage system and may be accelerated in another storage system, wherein an accelerated snapshot or other type of object can be used to populate volumes in the other storage system with data more rapidly than a non-accelerated snapshot or other type of object. The accelerated snapshot or other object may be implemented using an intermediate volume implemented in the other storage system that is populated with data from the snapshot object or the other object stored in the first storage system.

Abstract: Master-slave pairs can be used to provide data redundancy in an electronic data environment. A master peer can include a B-tree with references to the corresponding data. When provisioning a slave, the master can send a point-in-time copy of the B-tree to the slave, which can allocate the necessary space on local storage and update the references of the B-tree to point to the local storage for the slave. If the master and slave become disconnected, one of the peers can function as a solo master until the peers are again connected, at which point the old peer can be brought current or a new slave provisioned. A log peer can also be provisioned by a solo master, which can store data for operations received during the disconnect for use in catching up a slave peer, which could be the old slave, the log peer, or a new peer.

Abstract: Generally described, one or more aspects of the present application correspond to techniques for creating encrypted block store volumes of data from unencrypted object storage snapshots of the volumes. These encryption techniques use a special pool of servers for performing the encryption. These encryption servers are not accessible to users, and they perform encryption and pass encrypted volumes to other block store servers for user access. The encryption context for the volumes can be persisted on the encryption severs for as long as needed for encryption and not shared with the user-facing servers in order to prevent user access to encryption context.

Abstract: A failure of a storage device used to provide a mirrored storage volume can be managed without a full re-mirroring of the volume. The volume can be provided using a set of similar storage devices on each of a master server and a slave server, and a technique such as data striping can be used to store the data for the volume across the various devices. When a storage device becomes unavailable, the data from the corresponding storage device on the other mirrored server can be written to the remaining storage devices on the server experiencing the device failure. The data interface can be virtualized such that the user can continue to send input and output (I/O) requests using the same address information. A translation layer can map the virtualized addresses to the physical addresses where the data is stored.