Abstract: Techniques to satisfy quality of service (QoS) requirements on a per virtual machine basis natively in a storage system are disclosed. In various embodiments, for each of a plurality of virtual machines a corresponding input/output (I/O) request queue is stored on the storage system. Requests are scheduled to be pulled from the respective request queues and added to a corresponding storage operation pipeline at the storage system in an order determined based at least in part on a per virtual machine quality of service parameter.

Abstract: Detecting and pinpointing data corruption is disclosed, including: storing an object-level metadata structure corresponding to a stored object, wherein the stored object comprises a plurality of blocks; and determining for a block included in the plurality of blocks, based at least in part on a piece of identifying information of the block, two or more locations in the object-level metadata structure at which to store a value computed based at least in part on data comprising the block.

Abstract: Techniques to automatically allocate resources among storage system resource consumers are disclosed. In various embodiments, for each of a plurality of heterogeneous categories of resource consumer of the storage system a corresponding allocated portion of storage system resources to be available for use to perform operations associated with that category of resource consumer is determined. A storage system is configured automatically to provide access to resources of the storage system based at least in part on the respective portions of storage system resources allocated to each category of resource consumer, including by ensuring availability to workloads associated with each category at any given time at least a minimum amount of storage system resources corresponding to the portion of storage system resources allocated to that category.

Abstract: Techniques to satisfy quality of service (QoS) requirements on a per virtual machine basis natively in a storage system are disclosed. In various embodiments, for each of a plurality of virtual machines a corresponding input/output (I/O) request queue is stored on the storage system. Requests are scheduled to be pulled from the respective request queues and added to a corresponding storage operation pipeline at the storage system in an order determined based at least in part on a per virtual machine quality of service parameter.

Abstract: Techniques to automatically allocate resources among storage system resource consumers are disclosed. In various embodiments, for each of a plurality of heterogeneous categories of resource consumer of the storage system a corresponding allocated portion of storage system resources to be available for use to perform operations associated with that category of resource consumer is determined. A storage system is configured automatically to provide access to resources of the storage system based at least in part on the respective portions of storage system resources allocated to each category of resource consumer, including by ensuring availability to workloads associated with each category at any given time at least a minimum amount of storage system resources corresponding to the portion of storage system resources allocated to that category.

Abstract: Cluster storage comprises an interface and a processor. The interface is to send a tag to a selected node and receive tags from the selected node. The tags received from the selected node comprise tags for likely similar segments stored on the selected node. The processor is to break a segment into subsegments, calculate subsegment tags for each subsegment, identify one or more references to one or more previously stored subsegments and/or one or more segment data using the tags from the selected node and the subsegment tags, and send the one or more references to the one or more previously stored subsegments and/or segment data and associated tags to the selected node.

Abstract: A computer-implemented method and system for deduplicating sub-chunks in a data storage system selects a data chunk to deduplicate and generates a sketch for the selected data chunk. A similar data chunk is searched for using the sketch. A set of fingerprints corresponding to sub-chunks of the similar data chunk is loaded. The set of fingerprints for the similar data chunk is compared to a set of fingerprints of the selected data chunk and the selected chunk is encoded as a set of references to identical sub-chunks of the similar data chunk and at least one unmatched sub-chunk.

Abstract: Efficient use of spare device(s) associated with a group of devices is disclosed, including: receiving a set of data; storing the set of data across a group of devices according to a protocol; determining a set of additional data units associated with the set of data; and storing the set of additional data units in a spare device associated with the group of devices, wherein the spare device is available to be used in the event of a failure of a device included in the group of devices, including by overwriting the set of additional data units as needed.

Abstract: Efficient use of spare device(s) associated with a group of devices is disclosed, including: receiving a set of data; storing the set of data across a group of devices according to a protocol; determining a set of additional data units associated with the set of data; and storing the set of additional data units in a spare device associated with the group of devices, wherein the spare device is available to be used in the event of a failure of a device included in the group of devices, including by overwriting the set of additional data units as needed.

Abstract: Avoiding long access latencies in redundant storage systems is disclosed, including: determining a first device associated with a request is in a slow access period based at least in part on a scheduled slow access period, wherein the first device is included in a plurality of devices associated with a redundant storage system; and reconstructing data associated with the request from one or more devices included in the plurality of devices other than the first device.

Abstract: Performing data synchronization is disclosed, including: receiving an indication to synchronize a container to a snapshot, wherein the container has a first data state and an identity, wherein the snapshot corresponds to a second data state; causing the container to have the second data state corresponding to the snapshot; and maintaining the identity of the container.

Abstract: Performing data synchronization is disclosed, including: receiving an indication to synchronize a container to a snapshot, wherein the container has a first data state and an identity, wherein the snapshot corresponds to a second data state; causing the container to have the second data state corresponding to the snapshot; and maintaining the identity of the container.

Abstract: Synchronizing data is disclosed, including: generating a snapshot index associated with a first set of data; generating a set of difference data between a snapshot index associated with a second set of data and the snapshot index associated with the first set of data; and applying the set of difference data to a set of metadata associated with the first set of data, wherein the set of metadata reflects a current state of the first set of data.

Abstract: Stream locality delta compression is disclosed. A previous stream indicated locale of data segments is selected. A first data segment is then determined to be similar to a data segment in the stream indicated locale.

Abstract: Synchronizing data is disclosed, including: generating a snapshot index associated with a first set of data; generating a set of difference data between a snapshot index associated with a second set of data and the snapshot index associated with the first set of data; and applying the set of difference data to a set of metadata associated with the first set of data, wherein the set of metadata reflects a current state of the first set of data.

Abstract: Avoiding long access latencies in redundant storage systems is disclosed, including: determining a first device associated with a request is in a slow access period based at least in part on a scheduled slow access period, wherein the first device is included in a plurality of devices associated with a redundant storage system; and reconstructing data associated with the request from one or more devices included in the plurality of devices other than the first device.

Abstract: Storing data is disclosed, including: creating a clone of a snapshot comprising a first set of metadata associated with a source data, at least in part by generating for the clone a second set of metadata that includes an active index that is associated with the snapshot and is configured to store metadata associated with data values that are written to the clone; receiving a write operation to write a requested data value to the clone; and in response to the write operation, updating the active index based at least in part on the requested data value.

Abstract: Avoiding long access latencies in redundant storage systems is disclosed, including: determining a first device associated with a request is in a slow access period; and reconstructing data associated with the request from one or more other devices comprising a redundant storage system in which the first device and the one or more other devices are included.

Abstract: Techniques to automatically allocate resources among storage system resource consumers are disclosed. In various embodiments, for each of a plurality of heterogeneous categories of resource consumer of the storage system a corresponding allocated portion of storage system resources to be available for use to perform operations associated with that category of resource consumer is determined. A storage system is configured automatically to provide access to resources of the storage system based at least in part on the respective portions of storage system resources allocated to each category of resource consumer, including by ensuring availability to workloads associated with each category at any given time at least a minimum amount of storage system resources corresponding to the portion of storage system resources allocated to that category.

Abstract: Storing data that includes receiving an indication to perform an operation with respect to data associated with a virtual machine storage abstraction; and using a meta information identifying which data stored on a storage system is associated with the virtual machine storage abstraction to perform the requested operation.