Abstract: Method for tree management of trees under multi-version concurrency control is described herein. Method starts by detecting change in a size of a cluster. The number of nodes in the cluster is counted to obtain the changed size of the cluster. The number of trees needed for the number of nodes in the cluster is determined. The number of trees may be based on the number of nodes in the cluster and predetermined system balancing coefficient. When the number of trees needed is greater than existing number of trees, existing number of trees is doubled as a single-step operation which includes modifying and scaling hash function used to derive each tree to generate new hash function and using new hash function to partition each tree instantly into two new trees. Scaling happens on demand without service disruption. Hash function scales automatically when number of trees increases. Other embodiments are described.

Abstract: Inter-zone data replication in a distributed storage system can be organized. In one aspect, a replication destination zone can send a new data portion to a replication source zone to reduce inter-zone traffic. The new data portion can be stored within one or more chunks within the replication source zone, which can then provide the replication destination zone with location information (e.g., chunk identifier (ID), offset within chunk, segment size, etc.) indicative of a location of the new data portion within one or more chunks. The location information can be utilized by the replication destination zone to store a backup copy of the new data portion. Since the backup copy is stored without data replication from the replication source zone to the replication destination zone, inter-zone network traffic is significantly reduced.

Abstract: The described technology is generally directed towards storing an object in different ways based upon evaluating the object's size information. An object classified as small with respect to a threshold size value is an embedded object, and has its object metadata and object data stored in an object table that generally references object data. A non-embedded object can be further classified based on its object metadata size, as a normal object (metadata size below a threshold size) or an inflated object (metadata size not below the threshold size). A normal object has its object data stored in a repository (in a chunk in a chunk store) and its object metadata and pointer to the chunk stored in the object table. An inflated object has its object metadata and object data stored in the repository (in a chunk) with a pointer to the chunk stored in the object table.

Abstract: One embodiment is related to a method for remote replication recovery, comprising: determining that a damaged data chunk at a first zone of a cloud storage system is not recoverable locally; determining one or more data fragments of the damaged data chunk that are to be recovered with remote replication recovery based on data stored at a second zone and at a third zone of the cloud storage system, wherein the damaged data chunk comprises a plurality of data fragments; recovering the one or more data fragments of the damaged data chunk with remote replication recovery based on the data stored at the second zone and at the third zone of the cloud storage system; and repairing the damaged data chunk with the recovered data fragments at the first zone.

Abstract: Synchronization of metadata-based system snapshots with a state of user data is presented herein. A snapshot component can generate, at respective times, snapshots of roots of respective trees of a system—the respective trees comprising metadata representing respective states of the system corresponding, via the snapshots, to the respective times. Further, a garbage collection component can select a snapshot of the snapshots of the roots that is older than remaining snapshots of the snapshots of the roots, and determine, using an object table of the snapshot referencing data chunks comprising respective objects of the system, whether a data chunk of the data chunks comprises an inactive object of the respective objects to facilitate a selection of the data chunk as a garbage collection (GC) candidate for deletion via a GC procedure.

Abstract: The described technology is generally directed towards generational garbage collection, in which objects copied from earlier generation chunks during garbage collection are copied into a chunk of a later generation. Chunks are associated with generation numbers, starting with an original generation number (e.g., zero). When a garbage collection cycle occurs, objects of one generation of chunks are copied into chunks with a next generation number. As a result, over garbage collection cycles, longer-lived objects get grouped together into later generation chunks. Because of the objects' longer lifetimes, such later generation chunks are not copied often during subsequent garbage collection cycles, thereby avoiding the expense of copying for many objects.

Abstract: Opportunistic combining of data chunks is disclosed. Data chunks stored in storage devices of different zones of a zone storage system can be convolved to conserve memory. The zone storage system can be a geographically diverse storage system. A convolved chunk can be stored at a zone that does not contribute a local data chunk to the data represented in the convolved chunk. A zone storage component can be androgynous, rather than being explicitly configured to act as a front/back end storage device. This androgyny can enable the zone storage system to store a complete chunk at a zone based on real time use. In an aspect, an androgynous zone storage component can take on, or transition between, a de facto front-end storage device character or de facto back-end storage device character in response to deployment of the androgynous zone storage component in the storage system.

Abstract: Facilitating multi-level data deduplication in an elastic cloud storage environment is provided herein. A system can comprise a processor and a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations. The operations can comprise performing a first deduplication on a group of data objects at a data block level of a storage device. The operations can also comprise performing a second deduplication of the group of data objects at an object level of the storage device.

Abstract: Facilitating intra-cluster migration of data in an elastic cloud storage environment is provided herein. A system can comprise a processor and a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations. The operations can comprise scheduling a migration of a data chunk from a source storage node to a target storage node. Further, the operations can comprise facilitating the migration of the data chunk from a first location in the source storage node to a second location in the target storage node. Data indicative of an identity of the data chunk can be migrated with the data chunk. The operations can also comprise replacing the first location with the second location in a storage mapping table.

Abstract: A method, computer program product, and computer system for determining, by a computing device, that an object of a plurality of objects is an orphan. It may be determined that the object is older than a threshold age. A capacity occupied by the object may be reclaimed based upon, at least in part, determining that the object is an orphan and determining that the object is older than the threshold age.

Abstract: The described technology is generally directed towards storing an object in different ways based upon evaluating the object's size information. An object classified as small with respect to a threshold size value is an embedded object, and has its object metadata and object data stored in an object table that generally references object data. A non-embedded object can be further classified based on its object metadata size, as a normal object (metadata size below a threshold size) or an inflated object (metadata size not below the threshold size). A normal object has its object data stored in a repository (in a chunk in a chunk store) and its object metadata and pointer to the chunk stored in the object table. An inflated object has its object metadata and object data stored in the repository (in a chunk) with a pointer to the chunk stored in the object table.

Abstract: Erasure coding is utilized to facilitate data protection in several high-end storage systems. After a failure, the system commences the process of data recovery. Typically, the system can detect data portions impacted by the failure. In one aspect, the system can trigger recovery of the impacted data portions in a sequence that reduces a probability of data losses and a probability of temporary read failures.

Abstract: One embodiment is related to a method for promoting or demoting data objects in a multi-tier storage system, comprising: creating one or more chunks to store data objects therein at a first tier of the multi-tier storage system, wherein data objects stored in each chunk are associated with a same lifecycle policy and an approximately same creation time; and moving one of the one or more chunks to a second tier in its entirety based on the lifecycle policy associated with the data objects stored in the moved chunk.

Abstract: Data availability in geographically-distributed object storage systems that utilize erasure coding is increased without any run-time overheads. In one aspect, non-intersecting sub matrices of a great encoding matrix can be utilized to erasure code data fragments of a chunk at the zone level and generate coding fragments. Accordingly, the data fragments that are stored within different zones are identical, while the coding fragments stored within the different zones are disparate. Subsequent to a multi-zone data failure, wherein it is determined that a decoding operation cannot be performed at the zone level, available fragments associated with the chunk can be collected from the zones and collectively decoded to recover the chunk. In one aspect, the fragments can be decoded by utilizing a great decoding matrix that corresponds to the great encoding matrix.

Abstract: Inter-zone network traffic generated during deletion of a data chunk that has been replicated by employing geographically distributed (GEO) erasure coding is reduced. In one aspect, if a data chunk is to be deleted, partial coding chunks are generated by a source zone and provided to destination zones that store complete coding chunks for updating the complete coding chunks based on combining them with the received partial coding chunks. In another aspect, if a first data chunk is to be deleted and a second data chunk is to be replicated, partial coding chunks are generated by the source zone for each data chunk. Further, the partial coding chunks created for different data chunks can be combined to generate transforming chunks, which can then be transferred to the destination zones. The destination zones can then update the complete coding chunks based on combining them with the received transforming chunks.

Abstract: Method of implementing generational garbage collection for trees under MVCC starts by detecting live objects in trees. Trees include normal trees and frozen trees. Poorly-filled young chunks and poorly-filled old chunks of hard-drive memory are identified. Hard-drive memory includes young chunks storing young elements, old chunks storing old elements, and immortal chunks storing immortal elements. One or more old chunks are opened for writes and elements from poorly-filled young chunks and old chunks are copied to one or more opened old chunks. Elements above elements from poorly-filled young chunks and old chunks in the normal trees are updated and stored in the young chunks. One or more immortal chunks are opened for writes and tree leaves of frozen trees from young chunks and from old chunks are copied to one or more opened immortal chunks. All nodes of frozen trees are updated and stored in immortal chunks.

Abstract: A method for creating distributed erasure coding chunks in a distributed storage system with unbalanced load is disclosed. The operations comprise configuring the distributed storage system into at least k+m zones, wherein each zone accumulates at least l primary backup chunks of original data chunks replicated from different remote zones, and wherein l<k, and preparing the distributed storage system for recovery from a failure of 1 to m zones of the at least k+m zones including, in each zone, encoding the at least l primary backup chunks to create m coding chunks using an erasure coding having parameters k+m, wherein in each zone in which fewer than k primary backup chunks have been accumulated, at most k?l predetermined virtual chunks are used to create partial coding chunks.

Abstract: A method for avoiding recovery and storage of useless data is disclosed. The operations comprise: determining that a subset of data fragments out of a plurality of data fragments have become unavailable, the plurality of data fragments forming a data chunk, wherein the plurality of data fragments are protected by additional redundancy data; determining whether all of data that were stored in the unavailable subset of data fragments when the unavailable subset of data fragments were available would be useless if recovered using remaining available data fragments of the data chunk and the redundancy data; and in response to determining that all of the data that were stored in the unavailable subset of data fragments would be useless if recovered, generating new redundancy data for protecting the remaining available data fragments of the data chunk without recovering the data that were stored in the unavailable subset of data fragments.

Abstract: The disclosure relates data protection management (e.g. data recovery) for distributed storage systems. Specifically, the systems (and methods) of the disclosure provide an advanced mechanism for data recovery based on a notion of a data fragment's peer group, which may be used for “peer” recovery. Peer recovery allows a data fragment to be recovered when all the data fragments from its peer group are available. Accordingly, the described mechanism leverages the power of erasure coding and XOR operations to support recovery of data in situations where such data would previously be considered unrecoverable.

Abstract: A method comprising: generating a storage chunk having a plurality of data fragments, the storage chunk stored in one or more storage devices; allocating space in a primary memory to store a plurality of coded fragments; receiving a plurality of I/O requests to write data; allocating space in the primary memory to store a next unfilled data fragment; processing a plurality of I/O requests to write data; and copying the coded fragments from the primary memory to the one or more storage devices. For each I/O request, writing the data to the next unfilled data fragment in the one or more storage devices, writing the data to the next unfilled data fragment in the primary memory, and if the data fragment in the primary memory is full, updating the coded fragments in the primary memory using the filled data fragment in the primary memory.