Abstract: A size of a metadata-based system snapshot is determined based on performing incremental calculations in background mode to assure accurate size reporting without producing considerable workload. Snapshot sizes are relative to each other. For example, a size of a first snapshot is set when a second snapshot is generated. Moreover, a size of the latest snapshot is relative to a current tree version and a size of an older snapshot is relative to a snapshot that follows it. An aggregated size of all system snapshots can be determined by adding the individual snapshot sizes. This provides simplicity during snapshot management.

Abstract: Data protection with meta chunks increases capacity use efficiency without verification and data copying. In one aspect, a meta chunk is a data protection unit, which combines two or more source chunks that are determined to have a reduced sets of data fragments. The meta chunk can be encoded to generate a set of coding fragments, which can be stored and utilized to recover data fragments of any of the two or more source chunks. Further, the source chunks can be linked to the meta chunk. Furthermore, the sets of coding fragments, that were previously generated by individually encoding each source chunk, can be deleted.

Abstract: Facilitating data deduplication in an elastic cloud storage environment is provided herein. A system can comprise a processor and a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations. The operations can comprise facilitating a first deduplication of first data at a first storage device based on a determination that the first storage device comprises duplicated data. The operations can also comprise sending, by the system, a request for a second deduplication at a second storage device after completion of the first deduplication at the first storage device. In addition, the operations can comprise facilitating, by the system, the second deduplication of second data at the second storage device, wherein the second data comprises a copy of the duplicated data.

Abstract: Resource-efficient data protection is performed by generating meta chunks in storage systems that utilize erasure coding. During erasure coding with a k+m configuration, a data chunk can be divided into k data fragments, having indices 1 to k, that can be encoded by combining them with corresponding coefficients of a coding matrix, to generate coding fragments. Source portions that have a reduced set (e.g., less than k data fragments) of data fragments and that are complementary (e.g., that do not have common indices) can be determined and combined to generate a meta chunk. The coding fragments of the source portions can be added to generate coding fragments for the meta chunk, which can then be utilized to recover data fragments of any of the source portions. Further, the coding fragments, that were previously generated by individually encoding each source portion, can be deleted.

Abstract: A size of a metadata-based system snapshot is determined based on performing incremental calculations in background mode to assure accurate size reporting without producing considerable workload. Snapshot sizes are relative to each other. For example, a size of a first snapshot is set when a second snapshot is generated. Moreover, a size of the latest snapshot is relative to a current tree version and a size of an older snapshot is relative to a snapshot that follows it. An aggregated size of all system snapshots can be determined by adding the individual snapshot sizes. This provides simplicity during snapshot management.

Abstract: A transformation job associated with a storage container is created. The transformation job creation includes turning specified comparison predicates into a composite filter. A transformation service is performed for the created transformation job. The transformation service performance includes enumerating content of the storage container and applying the composite filter to each found object. One or more objects that are determined to pass the composite filter are added to an object index of a primary storage.

Abstract: Resource-efficient data protection is performed by generating meta chunks in storage systems that utilize erasure coding. During erasure coding with a k+m configuration, a data chunk can be divided into k data fragments, having indices 1 to k, that can be encoded by combining them with corresponding coefficients of a coding matrix, to generate coding fragments. Source portions that have a reduced set (e.g., less than k data fragments) of data fragments and that are complementary (e.g., that do not have common indices) can be determined and combined to generate a meta chunk. The coding fragments of the source portions can be added to generate coding fragments for the meta chunk, which can then be utilized to recover data fragments of any of the source portions. Further, the coding fragments, that were previously generated by individually encoding each source portion, can be deleted.

Abstract: One embodiment is related to a method for promoting or demoting data objects in a multi-tier storage system, comprising: creating one or more chunks to store data objects therein at a first tier of the multi-tier storage system, wherein data objects stored in each chunk are associated with a same lifecycle policy and an approximately same creation time; and moving one of the one or more chunks to a second tier in its entirety based on the lifecycle policy associated with the data objects stored in the moved chunk.

Abstract: Data protection with meta chunks increases capacity use efficiency without verification and data copying. In one aspect, a meta chunk is a data protection unit, which combines two or more source chunks that are determined to have a reduced sets of data fragments. The meta chunk can be encoded to generate a set of coding fragments, which can be stored and utilized to recover data fragments of any of the two or more source chunks. Further, the source chunks can be linked to the meta chunk. Furthermore, the sets of coding fragments, that were previously generated by individually encoding each source chunk, can be deleted.

Abstract: One embodiment is related to a method for implementing a demoted chunk, comprising: replicating a first chunk from a first zone of a cloud storage system to a third zone of the cloud storage system; replicating a second chunk from a second zone of the cloud storage system to the third zone; creating an exclusive or (XOR) chunk at the third zone based at least in part on the replicated first chunk and the replicated second chunk; and marking the replicated first chunk and the replicated second chunk as demoted first chunk and demoted second chunk, respectively, at the third zone.

Abstract: A method for creating distributed erasure coding chunks in a distributed storage system with unbalanced load is disclosed. The operations comprise configuring the distributed storage system into at least k+m zones, wherein each zone accumulates at least l primary backup chunks of original data chunks replicated from different remote zones, and wherein l<k, and preparing the distributed storage system for recovery from a failure of 1 to m zones of the at least k+m zones including, in each zone, encoding the at least l primary backup chunks to create m coding chunks using an erasure coding having parameters k+m, wherein in each zone in which fewer than k primary backup chunks have been accumulated, at most k?l predetermined virtual chunks are used to create partial coding chunks.

Abstract: Method of implementing generational garbage collection for trees under MVCC starts by detecting live objects in trees. Trees include normal trees and frozen trees. Poorly-filled young chunks and poorly-filled old chunks of hard-drive memory are identified. Hard-drive memory includes young chunks storing young elements, old chunks storing old elements, and immortal chunks storing immortal elements. One or more old chunks are opened for writes and elements from poorly-filled young chunks and old chunks are copied to one or more opened old chunks. Elements above elements from poorly-filled young chunks and old chunks in the normal trees are updated and stored in the young chunks. One or more immortal chunks are opened for writes and tree leaves of frozen trees from young chunks and from old chunks are copied to one or more opened immortal chunks. All nodes of frozen trees are updated and stored in immortal chunks.

Abstract: A method for avoiding recovery and storage of useless data is disclosed. The operations comprise: determining that a subset of data fragments out of a plurality of data fragments have become unavailable, the plurality of data fragments forming a data chunk, wherein the plurality of data fragments are protected by additional redundancy data; determining whether all of data that were stored in the unavailable subset of data fragments when the unavailable subset of data fragments were available would be useless if recovered using remaining available data fragments of the data chunk and the redundancy data; and in response to determining that all of the data that were stored in the unavailable subset of data fragments would be useless if recovered, generating new redundancy data for protecting the remaining available data fragments of the data chunk without recovering the data that were stored in the unavailable subset of data fragments.

Abstract: Data object content verification systems and processes provide perfect reliability and low storage overhead. Object data is generated in a reproducible manner based upon object locally stored object metadata. The object data is stored to an object storage system. The stored object data is subsequently verified by retrieving the object metadata, reproducing the original object data, and comparing the stored and original object data. The data object content verification systems and processes support both mutable and immutable data object content verification.

Abstract: One embodiment is related to a method for avoiding recovery and storage of useless data, comprising: determining whether all of one or more unavailable data fragments of a data chunk correspond only to useless data; and in response to determining that all of the one or more unavailable data fragments of the data chunk correspond only to useless data, generating redundancy data for protecting available data fragments of the data chunk without recovering content of the one or more unavailable data fragments.

Abstract: Method of implementing generational garbage collection for trees under MVCC starts by detecting live objects in trees. Trees include normal trees and frozen trees. Poorly-filled young chunks and poorly-filled old chunks of hard-drive memory are identified. Hard-drive memory includes young chunks storing young elements, old chunks storing old elements, and immortal chunks storing immortal elements. One or more old chunks are opened for writes and elements from poorly-filled young chunks and old chunks are copied to one or more opened old chunks. Elements above elements from poorly-filled young chunks and old chunks in the normal trees are updated and stored in the young chunks. One or more immortal chunks are opened for writes and tree leaves of frozen trees from young chunks and from old chunks are copied to one or more opened immortal chunks. All nodes of frozen trees are updated and stored in immortal chunks.

Abstract: A computer program product, system, and method for receiving I/Os to write a plurality of objects; allocating one or more storage chunks for the plurality of objects; storing the objects as segments within the allocated storage chunks; receiving an I/O to delete an object from the plurality of objects; detecting one or more dedicated storage chunks from one or more storage chunks in which the object to delete is stored; determining one or more unused chunks from the one or more of the dedicated chunks; and deleting the unused chunks and reclaiming storage capacity for the unused chunks.

Abstract: Data object content verification systems and processes provide perfect reliability and low storage overhead. Object data is generated in a reproducible manner based upon object locally stored object metadata. The object data is stored to an object storage system. The stored object data is subsequently verified by retrieving the object metadata, reproducing the original object data, and comparing the stored and original object data. The data object content verification systems and processes support both mutable and immutable data object content verification.

Abstract: A data object content verification technique provides perfect reliability and low storage overhead. Object data is generated in a reproducible manner based upon object locally stored object metadata. The object data is stored to an object storage system. The stored object data is subsequently verified by retrieving the object metadata, regenerating the original object data, and comparing the stored and original object data.