Abstract: The disclosure relates data protection management (e.g. data re-protection) for distributed storage systems. Specifically, the systems (and methods) of the disclosure implement erasure coding to protect replicated data efficiently while reducing the storage capacity overhead. Traditional approaches for data re-protection that implement erasure coding often require performing a complete re-encoding to reflect changes in data (e.g. a removal of data). The disclosure provides an improved mechanism for data re-protection by implementing an efficient re-encoding (or un-encoding) process that reduces the requirements for data reads and operations, and thus, reduces the resource requirements for data re-protection.

Abstract: A method comprising: receiving an I/O request for object data; determining one or more data fragments wherein the object data is stored; determining that one or more of the data fragments are unavailable; determining, from within the one or more unavailable data fragments, a set of slices storing the object data, each slice comprising k small data fragments and m coded fragments; for each slice, retrieving at least k small data and coded fragments within the slice from storage; and recovering a segment of the object data using the retrieved small data and coded fragments. A related system and computer program product are also described.

Abstract: A method for generating coded fragments comprises receiving data to be encoded, splitting the data into a plurality of data fragments, identifying a first group of data fragments from among the plurality of data fragments using a coding matrix, summing the data fragments within the first group of data fragments to generate a first group sum, and using the first group sum to calculate at least a portion of two or more coded fragments.

Abstract: Method for tree management of trees under multi-version concurrency control is described herein. Method starts by detecting change in a size of a cluster. The number of nodes in the cluster is counted to obtain the changed size of the cluster. The number of trees needed for the number of nodes in the cluster is determined. The number of trees may be based on the number of nodes in the cluster and predetermined system balancing coefficient. When the number of trees needed is greater than existing number of trees, existing number of trees is doubled as a single-step operation which includes modifying and scaling hash function used to derive each tree to generate new hash function and using new hash function to partition each tree instantly into two new trees. Scaling happens on demand without service disruption. Hash function scales automatically when number of trees increases. Other embodiments are described.

Abstract: One embodiment is related to a method for avoiding recovery and storage of useless data, comprising: determining whether all of one or more unavailable data fragments of a data chunk correspond only to useless data; and in response to determining that all of the one or more unavailable data fragments of the data chunk correspond only to useless data, generating redundancy data for protecting available data fragments of the data chunk without recovering content of the one or more unavailable data fragments.

Abstract: One embodiment is related to a method for creating a redundancy data chunk for data protection with a chain topology, comprising: transmitting a data chunk of a first frontend zone of a data storage system to a second frontend zone of the data storage system; creating a redundancy data chunk at the second frontend zone of the data storage system based on the data chunk of the first frontend zone and a data chunk of the second frontend zone; passing the redundancy data chunk onto one or more subsequent frontend zones of the data storage system from the second frontend zone, wherein at each subsequent frontend zone the redundancy data chunk is updated based on the received redundancy data chunk and a data chunk of the respective subsequent frontend zone, and wherein the redundancy data chunk is passed through each subsequent frontend zone exactly once; and at a last subsequent frontend zone of the data storage system, forwarding the updated redundancy data chunk to a backend zone of the data storage system for

Abstract: Data protection in a distributed storage system is provided using a combination of a data protection operation and an exclusive or XOR operation. Chunks of data subject to replication are encoded with a protection operation and an XOR operation commutative with the protection operation to generate a combined protected data chunk. The protected data chunks from which the combined protected data chunk was generated can be safely deleted to reduce data protection overhead. Portions of the protected data that later become unavailable due to failure in the distributed storage system can be recovered from other portions that are available and the combined protect data chunk using the XOR operation. The protection operation includes a matrix-based erasure coding operation commutative with the XOR operation.

Abstract: One embodiment is related to a method for remote replication recovery, comprising: determining that a damaged data chunk at a first zone of a cloud storage system is not recoverable locally; determining one or more data fragments of the damaged data chunk that are to be recovered with remote replication recovery based on data stored at a second zone and at a third zone of the cloud storage system, wherein the damaged data chunk comprises a plurality of data fragments; recovering the one or more data fragments of the damaged data chunk with remote replication recovery based on the data stored at the second zone and at the third zone of the cloud storage system; and repairing the damaged data chunk with the recovered data fragments at the first zone.

Abstract: Method of implementing generational garbage collection for trees under MVCC starts by detecting live objects in trees. Trees include normal trees and frozen trees. Poorly-filled young chunks and poorly-filled old chunks of hard-drive memory are identified. Hard-drive memory includes young chunks storing young elements, old chunks storing old elements, and immortal chunks storing immortal elements. One or more old chunks are opened for writes and elements from poorly-filled young chunks and old chunks are copied to one or more opened old chunks. Elements above elements from poorly-filled young chunks and old chunks in the normal trees are updated and stored in the young chunks. One or more immortal chunks are opened for writes and tree leaves of frozen trees from young chunks and from old chunks are copied to one or more opened immortal chunks. All nodes of frozen trees are updated and stored in immortal chunks.

Abstract: Described is a system that allows for the efficient management of reallocating data between tiers of an automated storage tiering system. In certain configurations, protected data that is stored within the storage system may include a user data portion and a redundant data portion. Accordingly, to conserve space on higher storage tiers, the system may separate user data from the redundant data when reallocating data between tiers. For example, the system may only allocate the user data portion to higher storage tiers thereby conserving the space that would otherwise be taken by the redundant data, which remains, or is demoted to a lower tier. Moreover, the reallocation may occur during scheduled reallocation cycles, and accordingly, the reallocation of the separated protected data may occur without any additional tiering overhead.

Abstract: A computer program product, system, and method for receiving I/Os to write a plurality of objects; allocating one or more storage chunks for the plurality of objects; storing the objects as segments within the allocated storage chunks; receiving an I/O to delete an object from the plurality of objects; detecting one or more dedicated storage chunks from one or more storage chunks in which the object to delete is stored; determining one or more unused chunks from the one or more of the dedicated chunks; and deleting the unused chunks and reclaiming storage capacity for the unused chunks.

Abstract: A method comprising: receiving an I/O request for object data; determining one or more data fragments wherein the object data is stored; determining that one or more of the data fragments are unavailable; determining, from within the one or more unavailable data fragments, a set of slices storing the object data, each slice comprising k small data fragments and m coded fragments; for each slice, retrieving at least k small data and coded fragments within the slice from storage; and recovering a segment of the object data using the retrieved small data and coded fragments. A related system and computer program product are also described.

Abstract: A method comprising: generating a storage chunk having a plurality of data fragments, the storage chunk stored in one or more storage devices; allocating space in a primary memory to store a plurality of coded fragments; receiving a plurality of I/O requests to write data; allocating space in the primary memory to store a next unfilled data fragment; processing a plurality of I/O requests to write data; and copying the coded fragments from the primary memory to the one or more storage devices. For each I/O request, writing the data to the next unfilled data fragment in the one or more storage devices, writing the data to the next unfilled data fragment in the primary memory, and if the data fragment in the primary memory is full, updating the coded fragments in the primary memory using the filled data fragment in the primary memory.

Abstract: A method, computer program product, and computer system for receiving a change to a configuration parameter associated with a first cluster node in a cluster configuration. The configuration parameter may be stored in a shared cluster memory. The configuration parameter may be distributed from the shared cluster memory to the cluster configuration.

Abstract: A method, computer program product, and computer system for determining, by a computing device, that an object of a plurality of objects is an orphan. It may be determined that the object is older than a threshold age. A capacity occupied by the object may be reclaimed based upon, at least in part, determining that the object is an orphan and determining that the object is older than the threshold age.

Abstract: A method for generating coded fragments comprises receiving data to be encoded, splitting the data into a plurality of data fragments, identifying a first group of data fragments from among the plurality of data fragments using a coding matrix, summing the data fragments within the first group of data fragments to generate a first group sum, and using the first group sum to calculate at least a portion of two or more coded fragments.