Abstract: Scaling out of a geographically diverse storage system is disclosed. A first chunk can be selected to be moved in response to scaling out of the geographically diverse storage system. In some embodiments, the first chunk can be convolved, combined, etc., with a second chunk prior to moving a representation of information comprised in the first chunk, e.g., via copy and delete operations. The moving can be in accord with a geographically diverse storage system schema and can be based on a criteria associated with the geographically diverse storage system. In an embodiment the convolved chunk can be copied to a new zone and, subsequently, the first chunk can be deleted from the old zone. In another embodiment the combined chunk can be retained at the old zone and the first chunk can be copied to the new zone prior to deleting the first chunk from the old zone.

Abstract: A method, computer program product, and computer system for receiving, at a first computing device, a first data chunk sent from a second computing device. It may be determined that the first data chunk includes a first type of data. The first data chunk may be stored to a cache operatively coupled to the first computing device based upon, at least in part, determining that the first data chunk includes the first type of data, wherein the cache may include a first storage device type. An acknowledgement of a successful write of the first data chunk to the second computing device may be sent based upon, at least in part, a successful storing of the first data chunk to the cache operatively coupled to the first computing device.

Abstract: A computer program product, system, and method for generating a test data set using object count and object size parameters, storing the generated data set in storage, subsequently regenerating the data set, and comparing the regenerated data set to the stored data set to validate the storage.

Abstract: Elastic Cloud Storage (ECS) uses a specific method for disk capacity management, where disk space is partitioned into a set of blocks of fixed size called chunks. Chunk data can comprise user data and all kinds of metadata. The replication mechanism implemented in ECS works at the chunk level. However, object level data can be used to provide visibility for client applications. Thus, object arrival can be determined to track replication progress at the object level for systems with a replication mechanism that works at the chunk level.

Abstract: Capacity management is provided for a plurality of search trees under multi-version concurrency control. A non-volatile memory includes a plurality of chunks that are fixed-sized blocks of the non-volatile memory, each chunk including at least one page. The non-volatile memory stores the plurality of search trees, each search tree having elements including a tree root, a tree node and a tree leaf. Each element of the tree is at a different level of the search tree: a first level including the tree root, a second level including the tree node, and a third level including the tree leaf. The plurality of chunks includes a number of chunk types, each chunk type for storing the element from a different level of the search tree, such that elements from different levels are stored in separate chunks.

Abstract: Although a distributed storage device can recover data from multiple failures this process produces excessive inter-zone network traffic when a chunk with user data is deleted. This disclosure employs an un-encoding erasure coding and partial coding chunks to facilitate data deletes while reducing inter-zone network traffic. Therefore a data chunk representative of partitioned disk space associated with a first zone of a data store can be determined to be marked for deletion. Consequently, the data chunk can be copied, resulting in a copied data chunk, to a second zone of the data store associated with a coding chunk comprising the data chunk. Based on the copied data chunk and the coding chunk, a partial coding chunk can be generated via un-encoding, wherein the partial coding chunk is a subset of the coding chunk.

Abstract: Time sensitive data convolution and de-convolution is disclosed. Data stored in chunks of memory can be convolved, based on temporal aspects of the data stored on the chunks, to conserve used memory. Convolved chunks can be de-convolved according to several schema, wherein the schema are selected based temporal aspects of the original data blocks being determined to have satisfied one or more rule(s). In an aspect, the schema can reduce an amount of data transfer between data storage devices of different zones in regard to convolving or de-convolving data blocks. In an embodiment the convolution can comprise an ‘exclusive or’ operation of a first chunk and second chunk to form a third chunk, wherein the several chunks are stored in different zones.

Abstract: Scaling out of a geographically diverse storage system is disclosed. A first chunk can be selected to be moved in response to scaling out of the geographically diverse storage system. In some embodiments, the first chunk can be convolved, combined, etc., with a second chunk prior to moving a representation of information comprised in the first chunk, e.g., via copy and delete operations. The moving can be in accord with a geographically diverse storage system schema and can be based on a criteria associated with the geographically diverse storage system. In an embodiment the convolved chunk can be copied to a new zone and, subsequently, the first chunk can be deleted from the old zone. In another embodiment the combined chunk can be retained at the old zone and the first chunk can be copied to the new zone prior to deleting the first chunk from the old zone.

Abstract: One embodiment is related to a method for promoting or demoting data objects in a multi-tier storage system, comprising: creating one or more chunks to store data objects therein at a first tier of the multi-tier storage system, wherein data objects stored in each chunk are associated with a same lifecycle policy and an approximately same creation time; and moving one of the one or more chunks to a second tier in its entirety based on the lifecycle policy associated with the data objects stored in the moved chunk.

Abstract: A method, computer program product, and computer system for monitoring health of at least one storage device of a cache in a clustered system. A recovery journal may be maintained, wherein the recovery journal may identify whether one or more chunks of data stored in the cache have been dumped from the at least one storage device to persistent storage in the clustered system. A state of the at least one storage device may be determined based upon, at least in part, the health of the at least one storage device. A recovery action may be performed on the one or more chunks of data stored in the at least one storage device based upon, at least in part, the state of the at least one storage device.

Abstract: A method for creating distributed erasure coding chunks in a distributed storage system with unbalanced load is disclosed. The operations comprise configuring the distributed storage system into at least k+m zones, wherein each zone accumulates at least l primary backup chunks of original data chunks replicated from different remote zones, and wherein l<k, and preparing the distributed storage system for recovery from a failure of 1 to m zones of the at least k+m zones including, in each zone, encoding the at least l primary backup chunks to create m coding chunks using an erasure coding having parameters k+m, wherein in each zone in which fewer than k primary backup chunks have been accumulated, at most k?l predetermined virtual chunks are used to create partial coding chunks.

Abstract: Although a distributed storage device can recover data from multiple failures this process produces excessive inter-zone network traffic when a chunk with user data is deleted. This disclosure employs an un-encoding erasure coding and partial coding chunks to facilitate data deletes while reducing inter-zone network traffic. Therefore a data chunk representative of partitioned disk space associated with a first zone of a data store can be determined to be marked for deletion. Consequently, the data chunk can be copied, resulting in a copied data chunk, to a second zone of the data store associated with a coding chunk comprising the data chunk. Based on the copied data chunk and the coding chunk, a partial coding chunk can be generated via un-encoding, wherein the partial coding chunk is a subset of the coding chunk.

Abstract: Elastic Cloud Storage (ECS) uses a specific method for disk capacity management, where disk space is partitioned into a set of blocks of fixed size called chunks. Chunk data can comprise user data and all kinds of metadata. The replication mechanism implemented in ECS works at the chunk level. However, object level data can be used to provide visibility for client applications. Thus, object arrival can be determined to track replication progress at the object level for systems with a replication mechanism that works at the chunk level.

Abstract: The disclosure relates data protection management (e.g. data recovery) for distributed storage systems. Specifically, the systems (and methods) of the disclosure provide an advanced mechanism for data recovery based on a notion of a data fragment's peer group, which may be used for “peer” recovery. Peer recovery allows a data fragment to be recovered when all the data fragments from its peer group are available. Accordingly, the described mechanism leverages the power of erasure coding and XOR operations to support recovery of data in situations where such data would previously be considered unrecoverable.

Abstract: Although a distributed storage device can recover data from multiple failures this process produces excessive inter-zone network traffic when a chunk with user data is deleted. This disclosure employs an un-encoding erasure coding and partial coding chunks to facilitate data deletes while reducing inter-zone network traffic. Therefore a data chunk representative of partitioned disk space associated with a first zone of a data store can be determined to be marked for deletion. Consequently, the data chunk can be copied, resulting in a copied data chunk, to a second zone of the data store associated with a coding chunk comprising the data chunk. Based on the copied data chunk and the coding chunk, a partial coding chunk can be generated via un-encoding, wherein the partial coding chunk is a subset of the coding chunk.

Abstract: A method, computer program product, and computer system for receiving, at a first computing device, a first data chunk sent from a second computing device. It may be determined that the first data chunk includes a first type of data. The first data chunk may be stored to a cache operatively coupled to the first computing device based upon, at least in part, determining that the first data chunk includes the first type of data, wherein the cache may include a first storage device type. An acknowledgement of a successful write of the first data chunk to the second computing device may be sent based upon, at least in part, a successful storing of the first data chunk to the cache operatively coupled to the first computing device.

Abstract: Elastic Cloud Storage (ECS) uses a specific method for disk capacity management, where disk space is partitioned into a set of blocks of fixed size called chunks. Chunk data can comprise user data and all kinds of metadata. The replication mechanism implemented in ECS works at the chunk level. However, object level data can be used to provide visibility for client applications. Thus, object arrival can be determined to track replication progress at the object level for systems with a replication mechanism that works at the chunk level.

Abstract: A method, computer program product, and computer system for receiving, at a first computing device, a first data chunk sent from a second computing device. It may be determined that the first data chunk includes a first type of data. The first data chunk may be stored to a cache operatively coupled to the first computing device based upon, at least in part, determining that the first data chunk includes the first type of data, wherein the cache may include a first storage device type. An acknowledgement of a successful write of the first data chunk to the second computing device may be sent based upon, at least in part, a successful storing of the first data chunk to the cache operatively coupled to the first computing device.

Abstract: Data object content verification systems and processes provide perfect reliability and low storage overhead. Object data is generated in a reproducible manner based upon object locally stored object metadata. The object data is stored to an object storage system. The stored object data is subsequently verified by retrieving the object metadata, reproducing the original object data, and comparing the stored and original object data. The data object content verification systems and processes support both mutable and immutable data object content verification.

Abstract: The disclosure relates to data protection management for geographically distributed storage systems. Specifically, the present disclosure provides a mechanism for load balancing by combining the techniques of XOR Star and XOR Chain to control the protection of data internally. The load balancing is done by each zone without the need for an external load balancer. In addition, the load balancing mechanism provides the ability for native (e.g. without additional configuration) support of disaster recovery.