Abstract: A method includes storing a plurality of data in a storage system. A plurality of identifiers corresponding to the plurality of data is determined and the plurality of identifiers are stored in the storage system. A first set of integrity information corresponding to a first system storage level is generated for the plurality of data by performing a first set of cyclic redundancy checks and the first set of integrity information is stored in the storage system. A second set of integrity information corresponding to a second system storage level is generated for the plurality of data and the second set of integrity information is stored in the storage system.

Abstract: A method includes receiving, by a storage unit of a storage network, a check request message from a computing device of the storage network, where the check request message includes a group of slice names of a plurality of sets of slice names associated with a plurality of sets of encoded data slices, where a plurality of data segments are dispersed storage error encoded into the plurality of sets of encoded data slices. The method further includes determining, by the storage unit, whether an error condition exists for an encoded data slice associated with the group of slice names. When no, the method includes sending, by the storage unit, a check response message to the computing device, where the check response message includes a group of slice information regarding the group of slice names. When yes, the method includes disregarding sending the check response message.

Abstract: A method includes determining a plurality of identifiers based on a data retrieval request. Integrity information is generated based on determining the plurality of identifiers. Stored integrity information corresponding to the data retrieval request is compared with the integrity information. When the stored integrity information compares unfavorably with the integrity information, corruption associated with the plurality of identifiers is determined.

Abstract: A distributed storage integrity system in a dispersed storage network includes a scanning agent and a control unit. The scanning agent identifies an encoded data slice that requires rebuilding, wherein the encoded data slice is one of a plurality of encoded data slices generated from a data segment using an error encoding dispersal function. The control unit retrieves at least a number T of encoded data slices needed to reconstruct the data segment based on the error encoding dispersal function. The control unit is operable to reconstruct the data segment from at least the number T of the encoded data slices and generate a rebuilt encoded data slice from the reconstructed data segment. The scanning agent is located in a storage unit and the control unit is located in the storage unit or in a storage integrity processing unit, a dispersed storage processing unit or a dispersed storage managing unit.

Abstract: A storage unit operates by: receiving a write slice request, wherein the write slice request includes a plurality of encoded data slices and wherein the write slice request corresponds to a range; determining whether a write lock conflict exists based on the range; issuing an unfavorable write slice response when the write lock conflict is determined to exist; and when the write lock conflict is determined to not exist: initiating local storage of the plurality of encoded data slices; and issuing a favorable write slice response.

Abstract: A distributed storage integrity system in a dispersed storage network includes a scanning agent and a control unit. The scanning agent identifies an encoded data slice that requires rebuilding, wherein the encoded data slice is one of a plurality of encoded data slices generated from a data segment using an error encoding dispersal function. The control unit retrieves at least a number T of encoded data slices needed to reconstruct the data segment based on the error encoding dispersal function. The control unit is operable to reconstruct the data segment from at least the number T of the encoded data slices and generate a rebuilt encoded data slice from the reconstructed data segment. The scanning agent is located in a storage unit and the control unit is located in the storage unit or in a storage integrity processing unit, a dispersed storage processing unit or a dispersed storage managing unit.

Abstract: A method includes determining a plurality of identifiers based on a data retrieval request. Integrity information is generated based on determining the plurality of identifiers. Stored integrity information corresponding to the data retrieval request is compared with the integrity information. When the stored integrity information compares unfavorably with the integrity information, corruption associated with the plurality of identifiers is determined.

Abstract: A method includes receiving a data retrieval request. A plurality of identifiers are determined in accordance with the data retrieval request. Integrity information is generated based on determining the plurality of identifiers by performing a cyclic redundancy check. Stored integrity information corresponding to the data retrieval request is compared with the integrity information, where the stored integrity information was previously generated by performing the cyclic redundancy check. When the stored integrity information compares unfavorably with the integrity information, corruption associated with the plurality of identifiers is determined.

Abstract: A system includes a plurality of storage units each including a network port operably coupled to the network, where one or more storage vaults is associated with the plurality of storage units and each storage vault of the one or more storage vaults represents a software-constructed grouping of storage units of the plurality of storage units, where the software-constructed grouping of storage units stores encoded data slices, where a data segment is encoded using an information dispersal algorithm to produce the encoded data slices, and where a storage unit: receives, via the network port, a request regarding the data segment stored in the software-constructed grouping of storage units, obtains, from a data structure pertaining to the software-constructed grouping of storage units, information regarding the request, determines whether the request is valid based on the information regarding the request, and when the request is valid, the storage unit executes the request.

Abstract: A system includes a plurality of storage units, where one or more storage vaults is associated with the plurality of storage units and each storage vault of the one or more storage vaults represents a software-constructed grouping of storage units of the plurality of storage units. The software-constructed grouping of storage units stores encoded data slices. A data segment is encoded using an information dispersal algorithm to produce the encoded data slices. The system further includes a grid access manager that generates a data structure pertaining to the software-constructed grouping of storage units. A storage unit of the software-constructed grouping of storage units receives, from a client computer of the system, a request regarding the data segment, obtains, from the data structure, information regarding the request, determines whether the request is valid based on the information regarding the request, and when the request is valid, executes the request.

Abstract: A storage unit operates by: receiving a write slice request, wherein the write slice request includes a plurality of encoded data slices and wherein the write slice request corresponds to a range; determining whether a write lock conflict exists based on the range; issuing an unfavorable write slice response when the write lock conflict is determined to exist; and when the write lock conflict is determined to not exist: initiating local storage of the plurality of encoded data slices; and issuing a favorable write slice response.

Abstract: A method includes encoding data via erasure coding to produce a plurality of data slices. The method further includes determining a plurality of identifiers corresponding to the data. The method further includes generating integrity information based on the plurality of identifiers by performing a cyclic redundancy check. The method further includes storing the plurality of data slices, the plurality of identifiers, and the integrity information in a storage system.

Abstract: A method includes encoding data via erasure coding to produce a plurality of data slices. The method further includes determining a plurality of identifiers corresponding to the data. The method further includes generating integrity information based on the plurality of identifiers by performing a cyclic redundancy check. The method further includes storing the plurality of data slices, the plurality of identifiers, and the integrity information in a storage system.

Abstract: A method for execution by a dispersed storage and task (DST) execution unit includes receiving a write slice request from a requesting entity, where the write slice request includes a plurality of encoded data slices and a corresponding plurality of slice names. The plurality of slice names are interpreted to produce a slice name range. Whether a write lock conflict exists is determined based on the slice name range. An unfavorable write slice response is issued to the requesting entity when the write lock conflict is determined to exist. When the write lock conflict is determined to not exist, local storage of the plurality of encoded data slices is initiated, a favorable write slice response is issued to the requesting entity, and the processing system indicates that the slice name range is not locked when completing the local storage of the plurality of encoded data slices.

Abstract: A distributed storage integrity system in a dispersed storage network includes a scanning agent and a control unit. The scanning agent identifies an encoded data slice that requires rebuilding, wherein the encoded data slice is one of a plurality of encoded data slices generated from a data segment using an error encoding dispersal function. The control unit retrieves at least a number T of encoded data slices needed to reconstruct the data segment based on the error encoding dispersal function. The control unit is operable to reconstruct the data segment from at least the number T of the encoded data slices and generate a rebuilt encoded data slice from the reconstructed data segment. The scanning agent is located in a storage unit and the control unit is located in the storage unit or in a storage integrity processing unit, a dispersed storage processing unit or a dispersed storage managing unit.

Abstract: A distributed storage integrity system in a dispersed storage network includes a scanning agent and a control unit. The scanning agent identifies an encoded data slice that requires rebuilding, wherein the encoded data slice is one of a plurality of encoded data slices generated from a data segment using an error encoding dispersal function. The control unit retrieves at least a number T of encoded data slices needed to reconstruct the data segment based on the error encoding dispersal function. The control unit is operable to reconstruct the data segment from at least the number T of the encoded data slices and generate a rebuilt encoded data slice from the reconstructed data segment. The scanning agent is located in a storage unit and the control unit is located in the storage unit or in a storage integrity processing unit, a dispersed storage processing unit or a dispersed storage managing unit.

Abstract: To identify slice errors, a processing module of a computing device in a dispersed storage network (DSN) sends first list digest requests to at least first and second dispersed storage (DS) units. The requests indicates a first range of slice names to include in a first list digest. The processing module receives digest responses from the DS units, and compares the digest responses to determine whether they identify the same slices. If they do not identify the same slices, the processing module sends second list digest requests indicating a sub-range of the first range of slice names to include in second list digests. The sub-range continues to be narrowed until the processing module identifies at least one sub-range of slice names where a slice error exists.

Abstract: A method includes generating integrity data of one or more slice names of one or more error encoded data slices and evaluating the integrity data, without using any of the error encoded data slices, for an inconsistency between the plurality of received integrity data. The method further includes indicating an error based on the evaluating the integrity data indicating an inconsistency, retrieving one or more slice names for the one or more error encoded data slices, and when detecting that at least one data error has occurred at a plurality of storage units at a DSN site/pillar, determine that a DSN site/pillar failure has occurred and after a predetermined time period, if the DSN site/pillar failure has not been resolved, rebuild the error encoded data slices of the DSN site/pillar and store them at another DSN site/storage unit.

Abstract: A method for execution by one or more processing modules of a storage network (SN) begins with the one or more processing modules scanning a storage node of the SN at a predetermined interval to determine whether an erasure encoded data slice (EEDS) of a set of erasure encoded data slices (EEDSs) is corrupt, where the set of erasure encoded data slices (EEDSs) is generated from object data using an erasure error encoding dispersal function. The method continues with the one or more processing modules determining that an EEDS is corrupt and in response, rebuilding the EEDS in place in the storage node.

Abstract: A method for execution by a dispersed storage and task (DST) execution unit includes receiving a write slice request from a requesting entity, where the write slice request includes a plurality of encoded data slices and a corresponding plurality of slice names. The plurality of slice names are interpreted to produce a slice name range. Whether a write lock conflict exists is determined based on the slice name range. An unfavorable write slice response is issued to the requesting entity when the write lock conflict is determined to exist. When the write lock conflict is determined to not exist, local storage of the plurality of encoded data slices is initiated, a favorable write slice response is issued to the requesting entity, and the processing system indicates that the slice name range is not locked when completing the local storage of the plurality of encoded data slices.