Abstract: The present invention provides techniques for assignment and layout of redundant data in data storage system. In one aspect, the data storage system stores a number M of replicas of the data. Nodes that have sufficient resources available to accommodate a requirement of data to be assigned to the system are identified. When the number of nodes is greater than M, the data is assigned to M randomly selected nodes from among those identified. The data to be assigned may include a group of data segments and when the number of nodes is less than M, the group is divided to form a group of data segments having a reduced requirement. Nodes are then identified that have sufficient resources available to accommodate the reduced requirement. In other aspects, techniques are providing for adding a new storage device node to a data storage system having a plurality of existing storage device nodes and for removing data from a storage device node in such a data storage system.

Abstract: Read, write and recovery operations for replicated data are provided. In one aspect, a system for redundant storage of data included a plurality of storage devices and a communication medium for interconnecting the storage devices. At least two of the storage devices are designated devices for storing a block of data. Each designated device has a version of the data and a first timestamp that is indicative of when the version of data was last updated and a second timestamp that is indicative of any pending update to the block of data. The read, write and recovery operations are performed to the data using the first and second timestamps to coordinate the operations among the designated devices.

Abstract: The present invention provides techniques for assignment and layout of redundant data in data storage system. In one aspect, the data storage system stores a number M of replicas of the data. Nodes that have sufficient resources available to accommodate a requirement of data to be assigned to the system are identified. When the number of nodes is greater than M, the data is assigned to M randomly selected nodes from among those identified. The data to be assigned may include a group of data segments and when the number of nodes is less than M, the group is divided to form a group of data segments having a reduced requirement. Nodes are then identified that have sufficient resources available to accommodate the reduced requirement. In other aspects, techniques are providing for adding a new storage device node to a data storage system having a plurality of existing storage device nodes and for removing data from a storage device node in such a data storage system.

Abstract: A method of reading data comprises receiving a request for a stripe of erasure coded data, sending read messages to at least a quorum of storage devices, and receiving at least the quorum of reply messages from the devices. The quorum of the reply messages includes at least a minimum number of stripe blocks needed to decode the data. The quorum meets a quorum condition of a number such that any two selections of the number of the stripe blocks intersect in the minimum number of the stripe blocks. A method of writing data comprises generating a timestamp, encoding the data, sending query messages including the timestamp to the storage devices, receiving query reply messages from each of at least the quorum of the devices, sending write messages to the devices, and receiving a write reply message from each of at least the quorum of the devices.

Abstract: A method of reading data comprises sending read messages to storage devices holding the stripe and receiving at least a quorum of reply messages. The reply message from the storage device holding the data block includes the data block. The quorum meets a quorum condition of a number such that any two selections of the number of stripe blocks intersect in the minimum number of the stripe blocks needed to decode the stripe. A method of writing data comprises sending query messages to storage devices holding the stripe, receiving a query reply message from each of at least a first quorum of the storage devices, sending modify messages to the storage devices, and receiving a write reply message from each of at least a second quorum of the storage devices. The first and second quorums each meet the quorum condition.

Abstract: A method of recovering a stripe of erasure coded data begins with sending query messages to storage devices. The method continues with receiving query reply messages from at least a first quorum of the storage devices. The query reply messages include a minimum number of the stripe blocks needed to decode the stripe. Following this, the stripe of erasure coded data is encoded. Next, a write message is sent to each of the storage devices, which include a timestamp and the stripe block destined for the storage device. The method concludes with receiving a write reply message from at least a second quorum of the storage devices indicating that the stripe block was successfully stored. The first and second quorums each meet a quorum condition of a number such that any two selections of the number of the stripe blocks intersect in the minimum number of the stripe blocks.

Abstract: Data structure and timestamp management techniques for redundant storage. A plurality of storage devices are interconnected by a communication medium. At least two of the storage devices are designated devices for storing a block of data. Each designated device stores a version of the data and a first timestamp that is indicative of when the version of data was last updated. A second timestamp is indicative of a pending update to the block of data. When the update to the block of data is completed at one of the designated devices, the device discards the second timestamp. A storage device acting as coordinator instructs the device to discard the second timestamp. The designated storage devices store a plurality of blocks of data and corresponding timestamps according to a data structure. At least some of the entries in the data structure correspond to a range of data blocks that share a common timestamp. Entries in the data structure are arranged such that the ranges do not overlap.

Abstract: A method of reading data comprises receiving a request for a stripe of erasure coded data, sending read messages to at least a quorum of storage devices, and receiving at least the quorum of reply messages from the devices. The quorum of the reply messages includes at least a minimum number of stripe blocks needed to decode the data. The quorum meets a quorum condition of a number such that any two selections of the number of the stripe blocks intersect in the minimum number of the stripe blocks. A method of writing data comprises generating a timestamp, encoding the data, sending query messages including the timestamp to the storage devices, receiving query reply messages from each of at least the quorum of the devices, sending write messages to the devices, and receiving a write reply message from each of at least the quorum of the devices.

Abstract: A method of reading data comprises sending read messages to storage devices holding the stripe and receiving at least a quorum of reply messages. The reply message from the storage device holding the data block includes the data block. The quorum meets a quorum condition of a number such that any two selections of the number of stripe blocks intersect in the minimum number of the stripe blocks needed to decode the stripe. A method of writing data comprises sending query messages to storage devices holding the stripe, receiving a query reply message from each of at least a first quorum of the storage devices, sending modify messages to the storage devices, and receiving a write reply message from each of at least a second quorum of the storage devices. The first and second quorums each meet the quorum condition.

Abstract: A method of recovering a stripe of erasure coded data begins with sending query messages to storage devices. The method continues with receiving query reply messages from at least a first quorum of the storage devices. The query reply messages include a minimum number of the stripe blocks needed to decode the stripe. Following this, the stripe of erasure coded data is encoded. Next, a write message is sent to each of the storage devices, which include a timestamp and the stripe block destined for the storage device. The method concludes with receiving a write reply message from at least a second quorum of the storage devices indicating that the stripe block was successfully stored. The first and second quorums each meet a quorum condition of a number such that any two selections of the number of the stripe blocks intersect in the minimum number of the stripe blocks.

Abstract: Data structure and timestamp management techniques for redundant storage. A plurality of storage devices are interconnected by a communication medium. At least two of the storage devices are designated devices for storing a block of data. Each designated device stores a version of the data and a first timestamp that is indicative of when the version of data was last updated. A second timestamp is indicative of a pending update to the block of data. When the update to the block of data is completed at one of the designated devices, the device discards the second timestamp. A storage device acting as coordinator instructs the device to discard the second timestamp. The designated storage devices store a plurality of blocks of data and corresponding timestamps according to a data structure. At least some of the entries in the data structure correspond to a range of data blocks that share a common timestamp. Entries in the data structure are arranged such that the ranges do not overlap.

Abstract: The present invention provides techniques for assignment and layout of redundant data in data storage system. In one aspect, the data storage system stores a number M of replicas of the data. Nodes that have sufficient resources available to accommodate a requirement of data to be assigned to the system are identified. When the number of nodes is greater than M, the data is assigned to M randomly selected nodes from among those identified. The data to be assigned may include a group of data segments and when the number of nodes is less than M, the group is divided to form a group of data segments having a reduced requirement. Nodes are then identified that have sufficient resources available to accommodate the reduced requirement. In other aspects, techniques are providing for adding a new storage device node to a data storage system having a plurality of existing storage device nodes and for removing data from a storage device node in such a data storage system.

Abstract: Read, write and recovery operations for replicated data are provided. In one aspect, a system for redundant storage of data included a plurality of storage devices and a communication medium for interconnecting the storage devices. At least two of the storage devices are designated devices for storing a block of data. Each designated device has a version of the data and a first timestamp that is indicative of when the version of data was last updated and a second timestamp that is indicative of any pending update to the block of data. The read, write and recovery operations are performed to the data using the first and second timestamps to coordinate the operations among the designated devices.