Abstract: A distributed storage integrity system in a dispersed storage network includes a scanning agent and a control unit. The scanning agent identifies an encoded data slice that requires rebuilding, wherein the encoded data slice is one of a plurality of encoded data slices generated from a data segment using an error encoding dispersal function. The control unit retrieves at least a number T of encoded data slices needed to reconstruct the data segment based on the error encoding dispersal function. The control unit is operable to reconstruct the data segment from at least the number T of the encoded data slices and generate a rebuilt encoded data slice from the reconstructed data segment. The scanning agent is located in a storage unit and the control unit is located in the storage unit or in a storage integrity processing unit, a dispersed storage processing unit or a dispersed storage managing unit.

Abstract: A method for a storage network begins by receiving, from a requestor, an access request for metadata associated with a data object, where the data object is dispersed error encoded in accordance with dispersed storage error encoding parameters to produce a set of encoded data slices and distributedly stored among a plurality of storage units, where a decode threshold number of encoded data slices are needed to recover the data object. The method continues, by generating a metadata lookup for the data object and then receiving metadata from one or more storage units associated with the storage network. The method then continues, by mapping, based on the metadata, a file path and a storage location for accessing each encoded data slice of the set of encoded data slices.

Abstract: Methods and apparatus for efficient storage of a data object in a storage network. A method for execution by one or more processing modules of a storage network begins by receiving a store data request including a data object and a data identifier, and generating a source name for the data object. The method continues by determining whether the data object is already stored in memory of the storage network. In response to determining that the data object is already stored in the memory of the storage network, the method further includes identifying location information for the data object, storing the location information using the source name, and updating metadata for the data object to indicate that an additional copy of the data object is stored in the memory of the storage network.

Abstract: A method for a computing device begins by generating a set of write requests regarding a set of encoded data slices and sending the set of write requests to a set of storage units, where each write request includes an encoded data slice of the set of encoded data slices and a version attribute, where a data object is dispersed error encoded in accordance with dispersed error encoding parameters to produce the set of encoded data slices. The method continues with the computing device receiving a write response indicating a version number conflict and in response to the version number conflict, retrieving a latest version of the data object, generating an updated set of write requests and sending the updated set of write requests to a set of storage units.

Abstract: A data segment is encrypted to produce an encrypted data segment, and a data tag associated with the data segment is generated. The encrypted data segment is encoded to generate a set of encoded data slices. At least a read-threshold number of encoded data slices are required to reconstruct the encrypted data segment. A set of write slice requests, which includes the set of encoded data slices and the data tag, is transmitted to a DSN memory. A determination is made, based on the data tag, whether a first encoded data slice of the set of encoded data slices is a duplicate of a second encoded data slice already stored within the DSN memory. If it is a duplicate, rather of storing the first encoded data slice, a reference to a location of the second encoded data slice is stored.

Abstract: A method includes receiving, by a storage unit of a storage network, a check request message from a computing device of the storage network, where the check request message includes a group of slice names of a plurality of sets of slice names associated with a plurality of sets of encoded data slices, where a plurality of data segments are dispersed storage error encoded into the plurality of sets of encoded data slices. The method further includes determining, by the storage unit, whether an error condition exists for an encoded data slice associated with the group of slice names. When no, the method includes sending, by the storage unit, a check response message to the computing device, where the check response message includes a group of slice information regarding the group of slice names. When yes, the method includes disregarding sending the check response message.

Abstract: A storage network operates by: identifying a set of data objects having a common trait from a plurality of data objects for storage via the storage network; combining the set of data objects to produce a concatenated data object; encoding the concatenated data object in accordance with a dispersed encoding function to produce a set of encoded data blocks; and generating a mapping of the set of data objects to the set of encoded data blocks, wherein the mapping facilitates retrieval from the storage network of individual data objects of the set of data objects.

Abstract: A method includes identifying an independent data object of a plurality of independent data objects for retrieval from dispersed storage network (DSN) memory. The method further includes determining a mapping of the plurality of independent data objects into a data matrix, wherein the mapping is in accordance with the dispersed storage error encoding function. The method further includes identifying, based on the mapping, an encoded data slice of the set of encoded data slices corresponding to the independent data object. The method further includes sending a retrieval request to a storage unit of the DSN memory regarding the encoded data slice. When the encoded data slice is received, the method further includes decoding the encoding data slice in accordance with the dispersed storage error encoding function and the mapping to reproduce the independent data object.

Abstract: A method includes receiving, by a storage unit of a storage network, a check request message from a computing device of the storage network, where the check request message includes a group of slice names of a plurality of sets of slice names associated with a plurality of sets of encoded data slices, where a plurality of data segments are dispersed storage error encoded into the plurality of sets of encoded data slices. The method further includes determining, by the storage unit, whether an error condition exists for an encoded data slice associated with the group of slice names. When no, the method includes sending, by the storage unit, a check response message to the computing device, where the check response message includes a group of slice information regarding the group of slice names. When yes, the method includes disregarding sending the check response message.

Abstract: A method includes identifying an independent data object of a plurality of independent data objects for retrieval from dispersed storage network (DSN) memory. The method further includes determining a mapping of the plurality of independent data objects into a data matrix, wherein the mapping is in accordance with the dispersed storage error encoding function. The method further includes identifying, based on the mapping, an encoded data slice of the set of encoded data slices corresponding to the independent data object. The method further includes sending a retrieval request to a storage unit of the DSN memory regarding the encoded data slice. When the encoded data slice is received, the method further includes decoding the encoding data slice in accordance with the dispersed storage error encoding function and the mapping to reproduce the independent data object.

Abstract: A distributed storage integrity system in a dispersed storage network includes a scanning agent and a control unit. The scanning agent identifies an encoded data slice that requires rebuilding, wherein the encoded data slice is one of a plurality of encoded data slices generated from a data segment using an error encoding dispersal function. The control unit retrieves at least a number T of encoded data slices needed to reconstruct the data segment based on the error encoding dispersal function. The control unit is operable to reconstruct the data segment from at least the number T of the encoded data slices and generate a rebuilt encoded data slice from the reconstructed data segment. The scanning agent is located in a storage unit and the control unit is located in the storage unit or in a storage integrity processing unit, a dispersed storage processing unit or a dispersed storage managing unit.

Abstract: A method includes identifying an independent data object of a plurality of independent data objects for retrieval from dispersed storage network (DSN) memory. The method further includes determining a mapping of the plurality of independent data objects into a data matrix, wherein the mapping is in accordance with the dispersed storage error encoding function. The method further includes identifying, based on the mapping, an encoded data slice of the set of encoded data slices corresponding to the independent data object. The method further includes sending a retrieval request to a storage unit of the DSN memory regarding the encoded data slice. When the encoded data slice is received, the method further includes decoding the encoding data slice in accordance with the dispersed storage error encoding function and the mapping to reproduce the independent data object.

Abstract: A distributed storage integrity system in a dispersed storage network includes a scanning agent and a control unit. The scanning agent identifies an encoded data slice that requires rebuilding, wherein the encoded data slice is one of a plurality of encoded data slices generated from a data segment using an error encoding dispersal function. The control unit retrieves at least a number T of encoded data slices needed to reconstruct the data segment based on the error encoding dispersal function. The control unit is operable to reconstruct the data segment from at least the number T of the encoded data slices and generate a rebuilt encoded data slice from the reconstructed data segment. The scanning agent is located in a storage unit and the control unit is located in the storage unit or in a storage integrity processing unit, a dispersed storage processing unit or a dispersed storage managing unit.

Abstract: A distributed storage integrity system in a dispersed storage network includes a scanning agent and a control unit. The scanning agent identifies an encoded data slice that requires rebuilding, wherein the encoded data slice is one of a plurality of encoded data slices generated from a data segment using an error encoding dispersal function. The control unit retrieves at least a number T of encoded data slices needed to reconstruct the data segment based on the error encoding dispersal function. The control unit is operable to reconstruct the data segment from at least the number T of the encoded data slices and generate a rebuilt encoded data slice from the reconstructed data segment. The scanning agent is located in a storage unit and the control unit is located in the storage unit or in a storage integrity processing unit, a dispersed storage processing unit or a dispersed storage managing unit.

Abstract: A distributed storage integrity system in a dispersed storage network includes a scanning agent and a control unit. The scanning agent identifies an encoded data slice that requires rebuilding, wherein the encoded data slice is one of a plurality of encoded data slices generated from a data segment using an error encoding dispersal function. The control unit retrieves at least a number T of encoded data slices needed to reconstruct the data segment based on the error encoding dispersal function. The control unit is operable to reconstruct the data segment from at least the number T of the encoded data slices and generate a rebuilt encoded data slice from the reconstructed data segment. The scanning agent is located in a storage unit and the control unit is located in the storage unit or in a storage integrity processing unit, a dispersed storage processing unit or a dispersed storage managing unit.

Abstract: A data segment is encrypted to produce an encrypted data segment, and a data tag associated with the data segment is generated. The encrypted data segment is encoded to generate a set of encoded data slices. At least a read-threshold number of encoded data slices are required to reconstruct the encrypted data segment. A set of write slice requests, which includes the set of encoded data slices and the data tag, is transmitted to a DSN memory. A determination is made, based on the data tag, whether a first encoded data slice of the set of encoded data slices is a duplicate of a second encoded data slice already stored within the DSN memory. If it is a duplicate, rather of storing the first encoded data slice, a reference to a location of the second encoded data slice is stored.

Abstract: A method includes identifying an independent data object of a plurality of independent data objects for retrieval from dispersed storage network (DSN) memory. The method further includes determining a mapping of the plurality of independent data objects into a data matrix, wherein the mapping is in accordance with the dispersed storage error encoding function. The method further includes identifying, based on the mapping, an encoded data slice of the set of encoded data slices corresponding to the independent data object. The method further includes sending a retrieval request to a storage unit of the DSN memory regarding the encoded data slice. When the encoded data slice is received, the method further includes decoding the encoding data slice in accordance with the dispersed storage error encoding function and the mapping to reproduce the independent data object.

Abstract: A method for execution by a computing device includes, receiving, from a requesting device, a request for a data segment of a data object that is or is to be stored in storage units of a content delivery network. The method further includes determining whether the data segment is stored in a cache memory of the content delivery network or in the storage units. When stored in the cache memory, the method includes retrieving the cached data segment, and sending it to the requesting device. When stored in the storage units, the method includes, sending read requests regarding the data segment to the storage units, receiving, in response to the read requests, at least a decode threshold number of encoded data slices, decoding the at least the decode threshold number of encoded data slices to reproduce the data segment, and sending the data segment to the requesting device.

Abstract: A method includes identifying an independent data object of a plurality of independent data objects for retrieval from dispersed storage network (DSN) memory. The method further includes determining a mapping of the plurality of independent data objects into a data matrix, wherein the mapping is in accordance with the dispersed storage error encoding function. The method further includes identifying, based on the mapping, an encoded data slice of the set of encoded data slices corresponding to the independent data object. The method further includes sending a retrieval request to a storage unit of the DSN memory regarding the encoded data slice. When the encoded data slice is received, the method further includes decoding the encoding data slice in accordance with the dispersed storage error encoding function and the mapping to reproduce the independent data object.

Abstract: A method includes, for a current phase of a multiple phase write operation to write a set of encoded data slices to a set of storage units, determining whether to send a write performance threshold number of write requests to a subset of the set of storage units regarding a write performance threshold number of encoded data slices of the set of encoded data slices, where the set of encoded data slices includes a total number of encoded data slices, and wherein the write performance threshold number is less than the total number and greater than or equal to the write threshold number. When the write performance threshold number of write requests is to be sent to the subset of the set of storage units, the method continues by sending the write performance threshold number of write requests to the subset of the set of storage units.