Abstract: Efficient snapshot restore is disclosed. An indication to restore a snapshot from a replication destination to a storage system is received. At least a subset of data associated with the snapshot that is already present at the storage system is determined. A restore operation of the snapshot is performed by obtaining the at least subset of the data associated with the snapshot that is already present at the storage system locally from the storage system and by obtaining the remaining data associated with the snapshot from the replication destination.

Abstract: Efficient deduplication for storage systems is disclosed, including: using the current data structure to track a set of deduplication data entries associated with a collection group; determining a flush criterion has been met; and storing the set of deduplication data entries of the current data structure as one or more persistent objects associated with the collection group in a persistent storage, wherein the one or more persistent objects are usable to update a set of persistently stored metadata associated with the collection group that tracks deduplication data entries generated before a previous merge operation.

Abstract: A system for policy-driven RAID rebuild includes an interface to a group of devices each having stripe units. At least one of the devices is a spare device available to be used in the event of a failure of a device in the group of devices. The system further includes a processor coupled to the interface and configured to determine, based at least in part on an ordering criteria, an order in which to reconstruct stripe units to rebuild a failed device in the group of devices. The processor is further configured to rebuild the failed device including by reconstructing stripe units in the determined order using the spare device to overwrite stripe units as needed.

Abstract: Performing replication of snapshots between storage systems is disclosed. A request to store at a destination system a snapshot data to represent at the destination system a state of a set of data at a first point-in-time is received. A first source system snapshot data that represents at a source system the state of the set of data at the first point-in-time depends on a second source system snapshot data that represents at the source system a state of the set of data at a second point-in-time. The snapshot data to represent at the destination system the state of the set of data at the first point-in-time is determined based on data comprising the first source system snapshot data and a destination system snapshot data that represents at the destination system a state of the set of data at a third point-in-time.

Abstract: Efficient recovery of deduplication data for high capacity systems is disclosed, including: reading from the data storage device a data structure that tracks a plurality of segments to which a plurality of persistent objects have been recently written, wherein segments are written to in a monotonically increasing numerical order; selecting a checkpoint segment from among the plurality of segments based at least in part on a plurality of segment numbers corresponding to respective ones of the plurality of segments; using the checkpoint segment and a segment associated with a latest available segment number to determine a set of candidate segments; reading at least a portion of the set of candidate segments to identify a data storage block for which a corresponding deduplication data entry is not already stored in persistently stored deduplication data entries; and storing a new deduplication data entry to insert a fingerprint associated with the data storage block in a current data structure stored in a memory

Abstract: Dynamic block optimization for space and performance is disclosed, including: determining that a data pattern associated with a data block included in a write request matches a promoted data pattern; and performing the write request by associating the data block with a previously stored copy of the data block without updating an associated reference count. Dynamic block optimization for space and performance further includes determining that a data pattern associated with a data block included in a write request matches a predetermined data pattern; and performing the write request by storing a static representation associated with the data block and not storing the data block, wherein the static representation is usable to generate the data block.

Abstract: Detecting and pinpointing data corruption is disclosed, including: storing an object-level metadata structure corresponding to a stored object, wherein the stored object comprises a plurality of blocks; and determining for a block included in the plurality of blocks, based at least in part on a piece of identifying information of the block, two or more locations in the object-level metadata structure at which to store a value computed based at least in part on data comprising the block.

Abstract: Dynamic RAID expansion is disclosed, including: incrementally expanding a redundant storage system comprising a plurality of storage drives by adding one or more storage drives to the plurality of storage drives; and storing, for each stripe included in the redundant storage system, metadata indicating which storage drives in the redundant storage system store valid data for the stripe.

Abstract: Efficient deletion of a shared snapshot and other workflows are described herein, including: determining to delete a shared snapshot associated with a first container, wherein the shared snapshot shares with a second container at least a subset of data values that are stored by the shared snapshot, the determination is based at least in part on the shared snapshot sharing data values with not more than a prescribed number of containers other than the second container and an occurrence of a deletion triggering event; and causing zero or more data values that are stored by the shared snapshot and shared with the second container to be associated with the second container prior to deleting the shared snapshot.

Abstract: Efficient use of spare device(s) associated with a group of devices is disclosed, including: receiving a set of data; storing the set of data across a group of devices according to a protocol; determining a set of additional data units associated with the set of data; and storing the set of additional data units in a spare device associated with the group of devices, wherein the spare device is available to be used in the event of a failure of a device included in the group of devices, including by overwriting the set of additional data units as needed.

Abstract: Efficient deduplication for storage systems is disclosed, including: using the current data structure to track a set of deduplication data entries associated with a collection group; determining a flush criterion has been met; and storing the set of deduplication data entries of the current data structure as one or more persistent objects associated with the collection group in a persistent storage, wherein the one or more persistent objects are usable to update a set of persistently stored metadata associated with the collection group that tracks deduplication data entries generated before a previous merge operation.

Abstract: Efficient recovery of deduplication data for high capacity systems is disclosed, including: reading from the data storage device a data structure that tracks a plurality of segments to which a plurality of persistent objects have been recently written, wherein segments are written to in a monotonically increasing numerical order; selecting a checkpoint segment from among the plurality of segments based at least in part on a plurality of segment numbers corresponding to respective ones of the plurality of segments; using the checkpoint segment and a segment associated with a latest available segment number to determine a set of candidate segments; reading at least a portion of the set of candidate segments to identify a data storage block for which a corresponding deduplication data entry is not already stored in persistently stored deduplication data entries; and storing a new deduplication data entry to insert a fingerprint associated with the data storage block in a current data structure stored in a memory

Abstract: Efficient use of spare device(s) associated with a group of devices is disclosed, including: receiving a set of data; storing the set of data across a group of devices according to a protocol; determining a set of additional data units associated with the set of data; and storing the set of additional data units in a spare device associated with the group of devices, wherein the spare device is available to be used in the event of a failure of a device included in the group of devices, including by overwriting the set of additional data units as needed.

Abstract: Avoiding long access latencies in redundant storage systems is disclosed, including: determining a first device associated with a request is in a slow access period based at least in part on a scheduled slow access period, wherein the first device is included in a plurality of devices associated with a redundant storage system; and reconstructing data associated with the request from one or more devices included in the plurality of devices other than the first device.

Abstract: Avoiding long access latencies in redundant storage systems is disclosed, including: determining a first device associated with a request is in a slow access period based at least in part on a scheduled slow access period, wherein the first device is included in a plurality of devices associated with a redundant storage system; and reconstructing data associated with the request from one or more devices included in the plurality of devices other than the first device.

Abstract: Avoiding long access latencies in redundant storage systems is disclosed, including: determining a first device associated with a request is in a slow access period; and reconstructing data associated with the request from one or more other devices comprising a redundant storage system in which the first device and the one or more other devices are included.

Abstract: Avoiding long access latencies in redundant storage systems is disclosed, including: determining a first device associated with a request is in a slow access period; and reconstructing data associated with the request from one or more other devices comprising a redundant storage system in which the first device and the one or more other devices are included.

Abstract: Avoiding long access latencies in redundant storage systems is disclosed, including: determining a first device associated with a request is in a slow access period; and reconstructing data associated with the request from one or more other devices comprising a redundant storage system in which the first device and the one or more other devices are included.

Abstract: Performing replication of snapshots between storage systems is disclosed, including: receiving a request to store at a destination system a snapshot data to represent at the destination system a state of a set of data at a first point-in-time, wherein a first source system snapshot data that represents at a source system the state of the set of data at the first point-in-time depends on a second source system snapshot data that represents at the source system a state of the set of data at a second point-in-time; and determining the snapshot data to represent at the destination system the state of the set of data at the first point-in-time, wherein the snapshot data is determined based on data comprising the first source system snapshot data and a destination system snapshot data that represents at the destination system a state of the set of data at a third point-in-time.