Abstract: Overhead associated with data re-protection during scaling out and/or scaling up of a distributed cloud storage system can be reduced. A coding matrix that is to be utilized for erasure coding in a potential final configuration of the distributed cloud storage can be determined. During initial data protection, a portion of the coding matrix can be utilized to determine coding chunks for protecting data chunks stored within different geographical zones of the distributed cloud storage system. When additional zones are added to the distributed cloud storage system, a larger portion of the coding matrix can be utilized to erasure code the new configuration and accordingly, the existing coding chunks are considered as partially complete. Further, the partially complete coding chunks can be combined with data chunks stored within the newly added zones and coefficients of the larger portion of the coding matrix to generate complete coding chunks.

Abstract: Synchronization of metadata-based system snapshots with a state of user data is presented herein. A snapshot component can generate, at respective times, snapshots of roots of respective trees of a system—the respective trees comprising metadata representing respective states of the system corresponding, via the snapshots, to the respective times. Further, a garbage collection component can select a snapshot of the snapshots of the roots that is older than remaining snapshots of the snapshots of the roots, and determine, using an object table of the snapshot referencing data chunks comprising respective objects of the system, whether a data chunk of the data chunks comprises an inactive object of the respective objects to facilitate a selection of the data chunk as a garbage collection (GC) candidate for deletion via a GC procedure.

Abstract: The technology describes shallow memory tables, comprising data maintained at a backup node of a data storage system that contain digest information related to a main node memory table that represents a metadata tree. If the main node fails, the shallow memory table's digest information contains sufficient information to recover the failed main node's memory table data. In response to receiving an update operation at a main node, the main node updates a memory table, journals an update record in a tree-related journal, and sends a digest representing the update to a backup node, which maintains the digest in a shallow memory table. If the main node fails, the backup node transforms the shallow memory table into a memory table by using the digest information to locate the corresponding update journal records. The backup node is able to handle create, read, update and delete requests during the transformation.

Abstract: The disclosed technology generally describes separating types of data chunks in a copy-on-write/MVCC B+ tree, chunk-based data storage system, and also allocating the sizes of leaf chunks to be smaller than that of other (e.g., internal and root node) chunks. By having leaf chunks separate from node chunks, the probability of having a fully reclaimable (without copying) chunk is increased. Similarly, by having smaller sized leaf chunks relative to node chunks, the probability of having a fully reclaimable (without copying) leaf chunks is increased. The technology thus facilitates more efficient garbage collection.

Abstract: The described technology is generally directed towards recovery of an impacted (damaged) tree in an impacted zone in a geographically distributed data storage environment, using a peer tree in a remote zone. A peer zone is selected for recovery, and updated to have current data. Logic at the impacted zone requests recovery of an impacted tree, identifying one or more recovery ranges for which recovery data is needed. Logic at the peer zone locates missing objects via the peer tree, and provides recovery information to the impacted zone, by which recovery of the impacted tree is able to be accomplished. For example, a replication journal may be returned as a tree recovery journal comprising add leaf (object) instructions, whereby the impacted zone processes the journal with respect to a reduced representation of the impacted tree to obtain a recovered tree.

Abstract: The described technology is generally directed towards quasi-compacting data storage chunks that obtains free capacity in used data chunks without moving data from those storage chunks. A composite data chunk is created from the unused block(s) within a data storage chunk. For example, blocks can be based on which fragments of a used data chunk are not in use (e.g., where a fragment is a one-twelfth, contiguous part of a chunk). A composite chunk thus uses the unused storage space of an existing “parent” data chunk, with mapping maintained to map from references to the composite chunks to actual addresses of their respective parent chunks. Quasi-compaction, such as used in conjunction with garbage collection, can be used to efficiently obtain more free storage capacity, without the inefficient copying of data from used chunks.

Abstract: Affinity sensitive storage of data corresponding to a mapped redundant array of independent nodes, e.g., mapped cluster, in a real storage system, e.g., a real cluster, is disclosed. Different mappings of mapped cluster data to real cluster storage locations can result in different levels of affinity between real nodes of the real cluster. A data storage scheme can be selected based on affinity scores, for example drawn from an affinity matrix, to provide access to stored data that can be more resilient against a real node becoming less available. Further, data recovery from a real node that has become less accessible can be improved where data is stored based on the affinity scores. Generally, data storage that provides greater diversity of data storage locations can be related to more desirable affinity scores. Further, data storage that provides less divergence of affinity scores across an affinity matrix can also be desirable.

Abstract: Capacity management is provided for a plurality of search trees under multi-version concurrency control. A non-volatile memory includes a plurality of chunks that are fixed-sized blocks of the non-volatile memory, each chunk including at least one page. The non-volatile memory stores the plurality of search trees, each search tree having elements including a tree root, a tree node and a tree leaf. Each element of the tree is at a different level of the search tree: a first level including the tree root, a second level including the tree node, and a third level including the tree leaf. The plurality of chunks includes a number of chunk types, each chunk type for storing the element from a different level of the search tree, such that elements from different levels are stored in separate chunks.

Abstract: Erasure coding is utilized to facilitate data protection in several high-end storage systems. After a failure, the system commences the process of data recovery. Typically, the system can detect data portions impacted by the failure. In one aspect, the system can trigger recovery of the impacted data portions in a sequence that reduces a probability of data losses and a probability of temporary read failures.

Abstract: One embodiment is related to a method for creating a redundancy data chunk for data protection with a chain topology, comprising: transmitting a data chunk of a first frontend zone of a data storage system to a second frontend zone of the data storage system; creating a redundancy data chunk at the second frontend zone of the data storage system based on the data chunk of the first frontend zone and a data chunk of the second frontend zone; passing the redundancy data chunk onto one or more subsequent frontend zones of the data storage system from the second frontend zone, wherein at each subsequent frontend zone the redundancy data chunk is updated based on the received redundancy data chunk and a data chunk of the respective subsequent frontend zone, and wherein the redundancy data chunk is passed through each subsequent frontend zone exactly once; and at a last subsequent frontend zone of the data storage system, forwarding the updated redundancy data chunk to a backend zone of the data storage system for

Abstract: The described technology is generally directed towards proactive disk recovery that operates when a failing disk is detected in a data-protected cloud data storage system. A proactive recovery process evaluates the chunks of a failing disk one-by-one. If a system process is scheduled to handle that chunk, the chunk is skipped, with recovery delegated to the system process. For non-delegated chunks protected by mirroring, a chunk copy is read by the proactive disk recovery process from a good disk copy, and copied to a new location. For non-delegated chunks protected by erasure coding, the chunk fragment is read and validated. If a portion is consistent, the proactive recovery process stores the portion to a new location on a good disk. If a portion is inconsistent, the process initiates recovery of the portion, e.g., via a fragment recovery task, for copying to a new location on a good disk.

Abstract: Facilitating intra-cluster migration of data in an elastic cloud storage environment is provided herein. A system can comprise a processor and a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations. The operations can comprise scheduling a migration of a data chunk from a source storage node to a target storage node. Further, the operations can comprise facilitating the migration of the data chunk from a first location in the source storage node to a second location in the target storage node. Data indicative of an identity of the data chunk can be migrated with the data chunk. The operations can also comprise replacing the first location with the second location in a storage mapping table.

Abstract: Resource-efficient data protection is performed by generating meta chunks in storage systems that utilize erasure coding. During erasure coding with a k+m configuration, a data chunk can be divided into k data fragments, having indices 1 to k, that can be encoded by combining them with corresponding coefficients of a coding matrix, to generate coding fragments. Source portions that have a reduced set (e.g., less than k data fragments) of data fragments can be modified such that they are made complementary (e.g., that do not have common indices) without complete data re-protection. The complementary portions can then be combined to generate a meta chunk. The coding fragments of the complementary portions can be added to generate coding fragments for the meta chunk, which can then be utilized to recover data fragments of any of the source portions.

Abstract: The described technology is generally directed towards data storage using a node cluster, and garbage collecting unused chunks (data storage units) in the cluster based on which node owns the particular unused chunks. A node determines which chunks are in use, and exchanges datasets identifying those chunks with other nodes such that the other nodes know which of the chunks that they own are in use. When a node obtains the dataset identifying the chunks in use, the node determines the chunks not in use by a difference of those owned and those in use. This difference dataset is used to garbage collect owned, unused chunks. Garbage collection via this technology is able to be performed in a single cycle.

Abstract: The technology describes shallow memory tables, comprising data maintained at a backup node of a data storage system that contain digest information related to a main node memory table that represents a metadata tree. If the main node fails, the shallow memory table's digest information contains sufficient information to recover the failed main node's memory table data. In response to receiving an update operation at a main node, the main node updates a memory table, journals an update record in a tree-related journal, and sends a digest representing the update to a backup node, which maintains the digest in a shallow memory table. If the main node fails, the backup node transforms the shallow memory table into a memory table by using the digest information to locate the corresponding update journal records. The backup node is able to handle create, read, update and delete requests during the transformation.

Abstract: Facilitating hybrid intra-cluster migration of data in an ECS storage environment is provided herein. A system can comprise a processor and a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations. The operations can comprise, based on a first determination that a first use efficiency of a first data chunk satisfies a defined use efficiency threshold, moving the first data chunk from a first storage device to a second storage device during a data migration. The operations can also comprise based on a second determination that a second use efficiency of a second data chunk fails to satisfy the defined use efficiency threshold, moving a first data segment from the second data chunk in the first storage device to a third data chunk in the second storage device during the data migration.

Abstract: In geographically-distributed object storage systems that utilize erasure coding, non-intersecting sub matrices of a great encoding matrix can be utilized to erasure code data fragments of a chunk at the zone level and generate coding fragments. Accordingly, the data fragments that are stored within different zones are identical, while the coding fragments stored within the different zones are disparate. Subsequent to a multi-zone data failure, wherein it is determined that a decoding operation cannot be performed at the zone level, available fragments associated with the chunk can be collected from the zones and collectively decoded to recover the chunk. In one aspect, the fragments can be decoded by utilizing a great decoding matrix that corresponds to the great encoding matrix.

Abstract: Geological allocation of storage space for a zone of a geographically diverse storage system is disclosed. A first allocation of storage space of the first zone can be adapted. In some embodiments, the adaptation can result in changes to a size of a storage area for a type of data stored in the first zone. In some embodiments, the adaptation can result in changes to a location of a storage area for a type of data stored in the first zone. In an aspect, the adaptation can result in improved inter-zone network utilization. In another aspect, the adaptation can result in efficient use of storage space in view of the amount and type of data to be stored in the zone. In an embodiment, the types of data stored can comprise a buffer space, local data, remote data, and combined or convolved data.

Abstract: Described is a system that allows for the efficient management of reallocating data between tiers of an automated storage tiering system. In certain configurations, protected data that is stored within the storage system may include a user data portion and a redundant data portion. Accordingly, to conserve space on higher storage tiers, the system may separate user data from the redundant data when reallocating data between tiers. For example, the system may only allocate the user data portion to higher storage tiers thereby conserving the space that would otherwise be taken by the redundant data, which remains, or is demoted to a lower tier. Moreover, the reallocation may occur during scheduled reallocation cycles, and accordingly, the reallocation of the separated protected data may occur without any additional tiering overhead.

Abstract: A method, computer program product, and computer system for determining, by a computing device, that an object of a plurality of objects is an orphan. It may be determined that the object is older than a threshold age. A capacity occupied by the object may be reclaimed based upon, at least in part, determining that the object is an orphan and determining that the object is older than the threshold age.