Abstract: A block-based storage system may implement page cache write logging. Write requests for a data volume maintained at a storage node may be received at a storage node. A page cache for may be updated in accordance with the request. A log record describing the page cache update may be stored in a page cache write log maintained in a persistent storage device. Once the write request is performed in the page cache and recorded in a log record in the page cache write log, the write request may be acknowledged. Upon recovery from a system failure where data in the page cache is lost, log records in the page cache write log may be replayed to restore to the page cache a state of the page cache prior to the system failure.

Abstract: A block-based storage system may implement reducing durability state for a data volume. A determination may be made that storage node replicating write requests for a data volume is unavailable. In response, subsequent write requests may be processed according to a reduced durability state for the data volume such that replication for the data volume may be disabled for the storage node. Write requests may then be completed at a fewer number of storage nodes prior to acknowledging the write request as complete. Durability state for the data volume may be increase in various embodiments. A storage node may be identified and replication operations may be performed to synchronize the current data volume at the storage node with a replica of the data volume maintained at the identified storage node.

Abstract: A method for selecting a service endpoint from a plurality of service endpoints in a distributed system of a service provider may include storing processing data for each of the plurality of endpoints. A success rate may be calculated for each of the plurality of service endpoints and based on a number of processed requests from a plurality of received requests. An average latency may be calculated based on latency associated with each of the processed requests. A latency score may be calculated based on a minimum average latency and the average latency. A raw score may be calculated based on the latency score and the success rate. A selection weight may be calculated based on the raw score and a balancing parameter. One of the plurality of endpoints may be selected based on the selection weight.

Abstract: Embodiments presented herein describe an I/O client that switches between generating high-resolution and low-resolution performance data in response to a defined event indicating there is a performance issue with a storage volume. In one embodiment, the I/O client uses one or more event rules that define a condition or threshold for determining whether to generate high-resolution or low-resolution performance data. In one embodiment, the I/O client transmits the performance data to a central monitoring system which can generate a chart plotting the performance data overtime. Because the I/O client may generate the high resolution performance data only when an event rule indicates there is a performance issue, the amount of performance data transmitted to the central monitoring system can be reduced.

Abstract: A request to create a volume to store data is received. A block within the storage node is selected, dependent at least in part on metadata indicating regions of available storage space in a storage node, to associate with a volume. Information is generated that includes an address to the block. A second computer system is determined to lacks the address to the block. The second computer system is enabled, by providing at least a portion of the information to the second computer system, to perform an operation to the block.

Abstract: A distributed system may implement identifying correlated events in a distributed system according to operational metrics. A distributed system may collect large numbers of operational metrics from multiple different sources. Some operational metrics may be monitored, analyzing the operational metrics for correlation with other operational metrics. The monitored operational metrics may be manually selected, or identified according to anomalous events detected for the operational metrics. Based on the monitoring, a correlated event may be detected. A response for the correlated event may be determined and performed. In some embodiments, a notification of the correlated event may be sent. Corrective actions may be performed at the distributed system, in some embodiments.

Abstract: At a first client-side component of a storage service, a metric of a storage workload directed from the first client-side component towards server components of the storage service is determined. The metric is transmitted to a particular server component, which is configured to propagate the metric to a different client-side component of the storage service. The first client-side component receives, from a server component, a different metric of a storage workload of a second client-side component. The first client-side component reschedules at least one storage request based at least in part on the different metric.

Abstract: A first location in one or more storage nodes is determined, with the first location being associated with a first block of a plurality of blocks associated with a storage volume. First information that maps the first block to the first location is generated. At least a portion of data is obtained from the first block at the first location. A second location in the one or more storage nodes is determined, with the second location being associated with a second block. Second information that maps the second block to the second location is included in the first information. A second computer system, different from the first computer system, is enabled, by providing at least a portion of the first information, to perform an operation to the storage volume.

Abstract: A block-based storage system may implement page cache write logging. Write requests for a data volume maintained at a storage node may be received at a storage node. A page cache for may be updated in accordance with the request. A log record describing the page cache update may be stored in a page cache write log maintained in a persistent storage device. Once the write request is performed in the page cache and recorded in a log record in the page cache write log, the write request may be acknowledged. Upon recovery from a system failure where data in the page cache is lost, log records in the page cache write log may be replayed to restore to the page cache a state of the page cache prior to the system failure.

Abstract: Data storage services may provide customers with access to computing resources of data storage server and throttling methods may be used manage the computing resources of the data storage service in order to ensure that the customers do not overload the data storage servers. Tokens may represent I/O operations executed by the customer of the data storage service. When I/O request are received the data storage service may determine if the I/O request is a member of a sequence and remove a reduced number for tokens from the work token bucket as a result. The data storage server may determine the I/O request is a member of a sequence based at least in part on a logical chunk of a storage volume the I/O request maps to.

Abstract: The estimated rate of work requests expected during a time period at a first block storage device, implemented at a particular server of a storage service, exceeds a provisioned rate of the first device. At a client-side component of the storage service, a different storage server is identified, at which the rate of work requests directed during the time period to a second block storage device is anticipated to be less than the provisioned rate of the second device. At least one admission control parameter of the first device is modified to enable the first storage server to accept work requests at a rate that exceeds the provisioned rate of the first device.

Abstract: A block-based storage system may implement page cache write logging. Write requests for a data volume maintained at a storage node may be received at a storage node. A page cache for may be updated in accordance with the request. A log record describing the page cache update may be stored in a page cache write log maintained in a persistent storage device. Once the write request is performed in the page cache and recorded in a log record in the page cache write log, the write request may be acknowledged. Upon recovery from a system failure where data in the page cache is lost, log records in the page cache write log may be replayed to restore to the page cache a state of the page cache prior to the system failure.

Abstract: A block-based storage system may implement efficient replication for restoring a data volume from a reduced durability state. A storage node that is not replicating write requests for a data volume may determine that replication for the data volume is to be enabled. A peer storage node may be identified that maintains a stale replica of the data volume. One or more replication operations may be performed to update stale data chunks in the stale replica of the data volume with current data chunks without updating data chunks in the stale replica of the data volume that are current. Stale replicas that are no longer needed may be deleted according timeouts or the amount of stale data in the replica.

Abstract: A storage system may implement dynamic configuration of data volumes. Client utilization of a data volume in a storage system may be tracked or monitored. Based on the utilization of the data volume, configuration recommendations to reconfigure the data volume according to data volume offerings may be determined. The data volume may be configured according to an authorized configuration recommendation. In some embodiments, these recommendations may be provided to a client and selection of the configuration recommendation to perform may be received. In some embodiments, a configuration recommendation may be automatically performed based on previously provided authorization to configure the data volume.

Abstract: Systems and methods are described providing detailed input/output (I/O) metric information that is collected and gathered by an agent of the storage volume. An I/O request is received by a storage volume, and the agent associates primary and secondary identifiers with that I/O request. For example, a trace may be associated with that I/O request. The agent may store this I/O metric information in a ring buffer. Further, after collection and aggregation, statistics may be published by an I/O metric service that further processes the data provided by the agent. Advantageously, interdependent relationships associated with the I/O request or I/O operations of that request may be included in those statistics. This may allow an operator to evaluate the performance of I/O requests for a network.

Abstract: Techniques are described for formally expressing whether sequences of operations performed on block storage devices are sequential or random. In embodiments, determinations of whether these sequences of operations are sequential or random may be used to predict latencies involved with running particular workloads, and to predict representative workloads for particular latencies.

Abstract: A block-based storage system may implement efficient replication for restoring a data volume from a reduced durability state. A storage node that is not replicating write requests for a data volume may determine that replication for the data volume is to be enabled. A peer storage node may be identified that maintains a stale replica of the data volume. One or more replication operations may be performed to update stale data chunks in the stale replica of the data volume with current data chunks without updating data chunks in the stale replica of the data volume that are current. Stale replicas that are no longer needed may be deleted according timeouts or the amount of stale data in the replica.

Abstract: Systems and methods are described for dynamically detecting outliers in a set of input/output (I/O) metrics collected and aggregated by a storage volume network. An I/O request is received by a storage volume network, and an agent of the storage volume network associates primary and secondary identifiers with that I/O request. For example, a trace may be associated with a request to write data to a storage volume network, and spans may be associated with the individual operations required to fulfill that request. Once gathered, I/O metrics may be aggregated based on the associated identifiers. I/O metric information regarding outliers may be received from the storage volume network, processed, and published by an I/O metrics service to identify the outliers among the primary and secondary identifiers. These outliers may then be stored for further analysis, and may be utilized to determine improvements to the performance of a storage volume network.

Abstract: Write optimization for block-based storage performing snapshot operations may be implemented. Write requests for a particular data volume may be received for which a snapshot operation is in progress. A determination may be made as to whether a data chunk of the data volume modified as part of the write request has not yet been stored to a remote snapshot data store as part of the snapshot operation. For a data chunk that is to be modified and that has not yet been stored, the data chunk may be stored in a local in-memory volume snapshot buffer. Once the data chunk is stored in the in-memory volume snapshot buffer, the write request may be performed and acknowledged as complete. The data chunk may be sent to the remote snapshot data store asynchronously with regard to the acknowledgment of the write request.

Abstract: A storage system may implement dynamic configuration of data volumes. Client utilization of a data volume in a storage system may be tracked or monitored. Based on the utilization of the data volume, configuration recommendations to reconfigure the data volume according to data volume offerings may be determined. The data volume may be configured according to an authorized configuration recommendation. In some embodiments, these recommendations may be provided to a client and selection of the configuration recommendation to perform may be received. In some embodiments, a configuration recommendation may be automatically performed based on previously provided authorization to configure the data volume.