Abstract: Disclosed is a data encoding, decoding and recovering method of a distributed storage system for data protection of the distributed storage system. The methods include using local recoverable coding, and calling Reed-Solomon coding on data blocks obtained from divisions of file segments based on coding parameters to generate global coding blocks, locally coding to data blocks and global coding blocks respectively to generate local coding blocks. The methods can also include computing decoded block indices and recovered block indices according to current node state, reading block data of assistant node, and implementing decoding of file segments and recovery of failed blocks. The coding method of the present disclosure can reduce the amount of data that needs to be transmitted when recovering a failed node by increasing local coding blocks and speed up the node recovery speed.

Abstract: A method of data recovery for a distributed storage system is a method of recovering multiple failed nodes concurrently with the minimum feasible bandwidth when failed nodes exist in a distributed storage system. By means of selecting assistant nodes, obtaining helper data sub-blocks through computing the selected assistant nodes, then computing a repair matrix and finally multiple the repair matrix and the helper data sub-blocks, the missing data blocks are reconstructed; or the missing data blocks are reconstructed by decoding. The method is applicable to data recovery in the case of any number of failed nodes and any reasonable combinations of coding parameters. The data recovery herein can reach the theoretical lower limit of the minimum recovery bandwidth.

Abstract: Disclosed is a systems and methods of distributed data storage using multi-layers consistent hashing comprising: a plurality of storage nodes providing data storage and redundancy protection; a plurality of management nodes maintaining the properties of the storage nodes and mapping information from virtual groups to storage nodes; a plurality of monitor nodes maintaining the state of storage nodes and handling the changes of states of storage nodes including joining, decommissioning and failure; and one or more clients providing entries for applications or users to access the storage system. The storage nodes is in a hierarchical tree arrangement, and each storage node in each layer of the tree is allocated with a plurality of identities and configured for remaining hash space with consistency. Instead of sharing one hash space among all storage nodes, there are a plurality of hash spaces kept consistent in each layer of the storage hierarchical tree.

Abstract: Disclosed is a method of distributed data redundancy storage using consistent hashing, which is used for selection of pre-stored data in the distributed position in storage cluster. The method is implemented by establishing a logical topology for each storage node in a distributed storage system, determining the position of each node in a storage node sequence corresponding to each hash subspace by using consistent hashing, and then determining a number of data copies and each piece of stored data according to the redundant storage policy of pre-stored data; and storing each piece of stored data in a different storage node according to storage rules.

Abstract: Disclosed is a method of data concurrent recovery for a distributed storage system, that is, a method for synchronous repair of multiple failed nodes with a minimum recovery bandwidth when a node in a distributed storage system fails. First an assistant node is selected to get helper data sub-block, then the repair matrix related to the data block stored in the node to be repaired is constructed, and finally the lost data block is reconstructed by multiplying the repair matrix and the helper data helper data sub-block; the missing data block is reconstructed by decoding, wherein the node to be recovered includes all failed systematical nodes, or all or partly failed parity nodes. The method is applicable to concurrently recover multiple failed nodes at minimal recovery bandwidth, and the nodes to be recovered are selected according to the demand to reduce the recovery bandwidth as much as possible.

Abstract: Disclosed is a method of distributed data redundancy storage using consistent hashing, which is used for selection of pre-stored data in the distributed position in storage cluster. The method is implemented by establishing a logical topology for each storage node in a distributed storage system, determining the position of each node in a storage node sequence corresponding to each hash subspace by using consistent hashing, and then determining a number of data copies and each piece of stored data according to the redundant storage policy of pre-stored data; and storing each piece of stored data in a different storage node according to storage rules.