Abstract: A method includes determining, by a computing device of a storage network, a storage inconsistency exists for a set of encoded data slices based on an indicator of a difference list, where the difference list is regarding storage of the set of encoded data slices within a set of storage units of the storage network and where a data segment is error encoded into the set of encoded data slices. The method further includes determining a storage resolution of a plurality of storage resolutions for the storage inconsistency based on the one or more indicators. The method further includes facilitating the storage resolution to resolve the storage inconsistency for the set of encoded data slices.

Abstract: A processing system is operable to encode data to produce a first set of data slices based on a value of a width parameter. The data is stored based on maintaining storage of the first set of data slices across a set of storage units of a storage pool. Storage of the first set of data slices is maintained in the set of storage units of the storage pool after addition of an additional set of storage units added to the storage pool. The value of the width parameter is increased to an increased value to produce an updated width parameter. The data is re-encoded in accordance with the updated width parameter to produce a second set of data slices. The data is re-stored based on maintaining storage of the second set of data slices across the expanded set of storage units of the storage pool.

Abstract: A method for processing data access requests in a storage network. The method includes receiving, by one or more processing modules, a data access request for a data object. A plurality of encoded data slices corresponding to the data object are stored in a distributed manner among a plurality of storage units associated with a plurality of storage sites of the storage network. The method continues with the one or more processing modules selecting respective numbers of storage units at each of the plurality of storage sites to support the data access request. A first computing device that is associated with a first storage site of the plurality of storage sites is also selected to further process the data access request. The method continues with the one or more processing modules transmitting the data access request to the first computing device to be further processed.

Abstract: A method includes generating a plurality of pages from a plurality of records received from a plurality of data sources. Deduplication of the plurality of pages is facilitated based on a plurality of page metadata of the plurality of pages based on, for the each page of the plurality of pages. A filtered set of potentially-intersecting pages is identified for each given page as a proper subset of the plurality of pages stored in the page storage system based on first comparison parameters, and an intersecting set of pages that include a row number intersection with the given page is identified as a proper subset of the filtered set of potentially-intersecting pages based on second comparison parameters. Records with records with row numbers included in row number intersections with other pages in the intersecting set of pages are removed from the each page.

Abstract: Methods and apparatus for rebuilding and storing data in a storage network that includes a plurality of storage units. In an embodiment, a processing module(s) of storage network identifies a storage error associated with a data object stored in a first storage unit of a set of storage units. In response, the processing module obtains storage network configuration information associated with the data object, and rebuilds the data object to produce a rebuilt data object. The processing module further identifies, based on the storage network configuration information, a candidate storage unit(s) for storage of the rebuilt data object and determines an available storage capacity level of the candidate storage unit. In response to determining that the available storage capacity level of the candidate storage unit is sufficient to store the rebuilt data object, the processing module facilitates storage of the rebuilt data object in the candidate storage unit.

Abstract: A method includes encoding data in accordance with a width parameter to produce a first set of data slices. Storage of the first set of data slices is maintained in a set of storage units of a storage pool. An expanded set of storage units of the storage pool is determined based on an additional subset of storage units added to the storage pool. Storage of the first set of data slices is maintained in the set of storage units after addition of the additional subset of storage units added to the storage pool. An updated width parameter is determined by increasing the width parameter based on the expanded set of storage units. The data is encoded in accordance with the updated width parameter to produce a second set of data slices. Storage of the second set of data slices is maintained in the expanded set of storage units.

Abstract: A record processing and storage system is operable to receive a plurality of row data that each indicate a batch number. Each of the plurality of row data is added to a pending row data pool. A plurality of pages is generated from the plurality of row data via a plurality of processing core resources. Each processing core resource in the plurality of processing core resources processing a corresponding subset of the plurality of row data by retrieving, in each time slice of a plurality of time slices, one row data from the pending row data pool with a most favorably ordered batch number of row data in the pending row data pool. Each processing core resource further processes the one row data in the in each time slice to participate in generation of at least one of the plurality of pages.

Abstract: A decentralized computer system operates by: storing encoded data blocks in an existing set of decentralized storage units of the decentralized computer system, wherein the decentralized computer system operates in accordance with a decentralized agreement protocol; assigning one or more additional decentralized storage units to the decentralized computing system, wherein the one or more additional decentralized storage units do not currently store the encoded data blocks; allocating the encoded data blocks to the one or more additional decentralized storage units, in accordance with the decentralized agreement protocol; and facilitating storage of the encoded data blocks from the existing set of decentralized storage units of the decentralized computer system to the at least one of the one or more additional decentralized storage units.

Abstract: Methods and apparatus for asynchronous replication of data in a storage network. In an embodiment, a method begins by a processing module(s) of a computing device identifying at least a first storage set and a second storage set for replicated storage of data. The processing module maintains a synchronization schedule for the first storage set and the second storage set. After initiating storage of a data object in the first storage set, the processing module determines, based at least in part on the synchronization schedule, to synchronize the first storage set and the second storage set. In response to determining to synchronize the first and second storage sets, the processing module determines that the second storage set requires the data object to maintain synchronization with the first storage set, and facilitates storage of the data object in the second storage set.

Abstract: Methods and apparatus for use in a storage network operate by: storing, in a first storage unit of a first set of storage units of the storage network, a first encoded data slice corresponding to at least one data object; assigning one or more additional storage units to the storage network to form a second set of storage units, the second set of storage units including the one or more additional storage units; migrating the first encoded data slice from the first storage unit to at least one of the one or more additional storage units of the second set of storage units; and reallocating a mapping of the first encoded data slice from the first storage unit to the at least one of the one or more additional storage units of the second set of storage units.

Abstract: A record processing and storage system is operable to receive a plurality of labeled row data from a data source. Each labeled row data of the plurality of labeled row data includes at least one record and a corresponding row number of a plurality of row numbers. A plurality of pages are generated from records included in the labeled row data. The plurality of pages are stored via a page storage system. A plurality of page metadata corresponding to the plurality of pages is generated, where each of the plurality of page metadata is generated based on at least corresponding one row number of at least one labeled row data with records included in a corresponding one of the plurality of pages. Deduplication of duplicated records included the plurality of pages is facilitated based on the plurality of page metadata.

Abstract: A method includes determining an information dispersal algorithm width number and determining a number of primary storage slots from a number of storage slots associated with a set of storage units deployed across multiple sites, where the number of primary storage slots is equal to or greater than the information dispersal algorithm width number. The method further includes determining a mapping of primary storage slots to storage units. The method further includes sending configuration information to the set of storage units that includes the mapping. The method further includes storing a set of encoded data slices in the primary storage slots in accordance with the configuration information, where a data segment is error encoded into the set of encoded data slices in accordance with the information dispersal algorithm width number and a decode threshold number, which is a number of encoded data slices are needed to reconstruct the data segment.

Abstract: A record processing and storage system operates by: generating a set of pages from a plurality of row data via a plurality of processing core resources, wherein each processing core resource in the plurality of processing core resources generate a corresponding subset of the set of pages, independently from and in parallel with processing of other subsets of the set of pages via other ones of the plurality of processing core resources; facilitating performance of a single storage transaction to store the set of pages; identifying a page set interval based on a plurality of row number intervals of the set of pages; generating, based on completing the single storage transaction, row durability data indicating a least favorably ordered row number of a plurality of row numbers corresponding to the plurality of row data; and transmitting the row durability data to a computing device associated with the plurality of row data.

Abstract: A method for execution by a distributed storage network begins by receiving a request to transfer a copy of a set of encoded data slices from at least some associated virtual storage vaults to a destination virtual storage vault and continues by determining whether the destination storage unit supports a source virtual storage vault of the at least some source virtual storage vaults. When the destination storage unit supports the source virtual storage vault the method continues by determining a sub-set of encoded data slices of the set of encoded data slices for transfer and finally, by facilitating sending the sub-set of encoded data slices to the destination storage unit.

Abstract: A processing system of a storage network operates by: sending, to at least one storage unit of the storage network, at least one read request corresponding to at least a read threshold number of a set of encoded data slices to be retrieved, wherein the set of encoded data slices correspond to data, wherein the data is coded in accordance with dispersed error coding parameters that include a write threshold number and the read threshold number, wherein the write threshold number is a number of encoded data slices in the set of encoded data slices and wherein the read threshold number is a number of the set of encoded data slices that is required to decode the data; receiving, via the at least one processing circuit and from the at least one storage unit, a first subset of the set of encoded data slices, wherein the first subset is missing at least one missing encoded data slice that was not received from the at least one storage unit in response to the at least one read request and wherein a number of encoded

Abstract: A method for execution by one or more computing devices of a storage network includes obtaining performance impact information regarding a data reconstruction operation associated with a computing device of the one or more computing devices, where the data reconstruction operation is regarding reconstructing data at a first reconstruction rate of a plurality of reconstruction rates, and where the performance impact information includes performance metrics of the storage network affected by the data reconstruction operation. The method further includes determining a second reconstruction rate of the plurality of reconstruction rates to utilize for the data reconstruction operation based on the performance impact information to achieve a first performance metric of the performance metrics. The method further includes executing the data reconstruction operation in accordance with the second reconstruction rate.

Abstract: A database system operates by: receiving a plurality of row data associated with a first data source; identifying a subset of row data from the plurality of row data that includes only ones of the plurality of row data that compare favorably to maintained row durability data; generating at least one page from ones of the plurality of row data included in the subset of row data; storing the at least one page in long term storage; generating updated row durability data indicating a least favorably ordered row number of a plurality of row numbers corresponding to the subset of row data based on storing the at least one page in long term storage; and updating the maintained row durability data to indicate the least favorably ordered row number of the updated row durability data.

Abstract: A method for execution by one or more processing modules of a storage network begins by receiving an access request for a set of encoded data slices, where the data object is segmented into a plurality of data segments, a data segment of which is dispersed error encoded in accordance with dispersed error encoding parameters to produce the set of encoded data slices. The method continues by determining whether a revision for one or more encoded data slices in a first storage module is a most recent revision level for the one or more encoded data slices and when the one or more encoded data slices is a most recent revision level, determining whether the revision level for the one or more encoded data slices in a second storage module is the most recent revision level.

Abstract: A method includes generating a predictive model mapping batch size to estimated memory utilization based on a plurality of prior batch sizes applied for performing a corresponding plurality of prior data format conversions, and further based on a plurality of prior actual memory utilization occurring in performing the corresponding plurality of prior data format conversions. A next batch size is automatically selected for performance of a subsequent data format conversion based on the predictive model and a current memory availability. A batch of data for performance of the subsequent data format conversion is identified based on the batch of data having the next batch size. The subsequent data format conversion is performed the batch of data, and an actual amount of memory utilized in performing the subsequent data format conversion is determined. The predictive model is updated based on the next batch size and the actual amount of memory utilized.

Abstract: method and apparatus for synchronizing replicated data in a storage network. In an embodiment, a method begins by a processing module of a computing device identifying a first storage set and a second storage set for replicated storage of a data object. The processing module initiates storage of the data object in both the first and second storage sets, and further maintains a synchronization status for the data object. The processing module determines, based at least in part on the synchronization status, to resynchronize the first storage set and the second storage set. In response to determining to resynchronize the first storage set and the second storage set, the processing module identifies a latest available revision of the data object, determines that the second storage set requires the latest available revision of the data object to maintain synchronization, and facilitates storage of the identified latest available revision of the data object in the second storage set.