Abstract: A method for execution by a computing device of a storage network begins or continues by generating a plurality of estimated efficiency models associated with a plurality of processing units of the storage network, where an estimated efficiency model of the plurality of estimated efficiency models includes estimated efficiency probabilities, and where the estimated efficiency probabilities correspond to data access request types for a processing unit of the plurality of processing units. The method further includes storing the plurality of estimated efficiency models in memory of the storage network.

Abstract: A method includes identifying, by a dispersed storage (DS) processing unit of a plurality of DS processing units of a dispersed storage network (DSN), a DSN operation to be performed that will adversely interfere with processing of DSN access requests. The method further includes sending, by the DS processing unit, a suspension request to a process balancing module of the DSN where the suspension request is for temporarily suspending processing of DSN access requests. The method further includes determining, by the process balancing module, whether sufficient processing resources of active DS processing units of the plurality of DS processing units are available for processing DSN access requests on behalf of the DS processing unit. When determined that sufficient processing resources are available, the method further includes granting the suspension request, and allocating DSN access requests of the DS processing unit to one ore of the active DS processing units.

Abstract: A record processing and storage system is operable to receive a plurality of row data that each indicate a batch number. Each of the plurality of row data is added to a pending row data pool. A plurality of pages is generated from the plurality of row data via a plurality of processing core resources. Each processing core resource in the plurality of processing core resources processing a corresponding subset of the plurality of row data by retrieving, in each time slice of a plurality of time slices, one row data from the pending row data pool with a most favorably ordered batch number of row data in the pending row data pool. Each processing core resource further processes the one row data in the in each time slice to participate in generation of at least one of the plurality of pages.

Abstract: A record processing and storage system is operable to receive a plurality of labeled row data from a data source. Each labeled row data of the plurality of labeled row data includes at least one record and a corresponding row number of a plurality of row numbers. A plurality of pages are generated from records included in the labeled row data. The plurality of pages are stored via a page storage system. A plurality of page metadata corresponding to the plurality of pages is generated, where each of the plurality of page metadata is generated based on at least corresponding one row number of at least one labeled row data with records included in a corresponding one of the plurality of pages. Deduplication of duplicated records included the plurality of pages is facilitated based on the plurality of page metadata.

Abstract: A method for rebuilding data in a distributed storage network (DSN) including a plurality of storage units. A rebuilding module of the DSN selects an address range associated with memory of a storage unit of the DSN, and initiates execution of data rebuilding activities for the selected address range. The rebuilding module determines a first rebuilding rate of the storage unit for a timeframe subsequent to initiating the execution of the data rebuilding activities. The rebuilding module further operates to pause the execution of the data rebuilding activities for a pause timeframe, and determine a second rebuilding rate of the storage unit for the pause timeframe. In response to determining that the second rebuilding rate compares favorably to the first rebuilding rate, the rebuilding module resumes execution of the data rebuilding activities. If the comparison is unfavorable, a second address range is selected for data rebuilding activities.

Abstract: A method includes, when a first batch rebuild threshold is met, a computing device of a dispersed storage network (DSN) determining whether target storage units of a set of storage units of the DSN are available, where a data segment of data is dispersed storage error encoded into a set of encoded data slices, a number of encoded data slices that need rebuilding corresponds to the first batch rebuild threshold, and the number of encoded data slices is to be stored in the target storage units. When a target storage unit is unavailable, the method continues by delaying a batch rebuild process until the target storage unit is available or a second batch rebuild threshold is met. When the target storage unit becomes available before the second batch rebuild threshold is met, the method continues by executing the batch rebuild process for the number of encoded data slices.

Abstract: A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. A computing device receives a data access request for an encoded data slice (EDS) associated with a data object. The computing device compares a slice name of the data access request with slice names stored within RAM. When the data access request slice name compares unfavorably with those stored slice names, the computing device transmits an empty data access response that includes no EDS to the other computing device without needing to access a hard disk drive (HDD) that stores EDSs. Alternatively, the computing device transmits a data access response that includes the EDS.

Abstract: A database system operates by: receiving a plurality of row data associated with a first data source; identifying a subset of row data from the plurality of row data that includes only ones of the plurality of row data that compare favorably to maintained row durability data; generating at least one page from ones of the plurality of row data included in the subset of row data; storing the at least one page in long term storage; generating updated row durability data indicating a least favorably ordered row number of a plurality of row numbers corresponding to the subset of row data based on storing the at least one page in long term storage; and updating the maintained row durability data to indicate the least favorably ordered row number of the updated row durability data.

Abstract: A method for execution by a record processing and storage system includes receiving a plurality of records and corresponding row numbers. Pages are generated from the received records. Page metadata is generated for each page that includes row number span data based on row numbers of the records included in each page. Pairs of pages are identified in the plurality of pages based on having row number span data in their page metadata that include a row number span overlap. For each pair of pages, row number span data is updated for a first page in the pair by removing the row number span overlap with a second page in the pair. Reads of pages are performed based on their row number span data. Only records of each first page of each pair of pages having row numbers that are within the updated row number span data are read.

Abstract: A method for execution by a computing device of a dispersed or distributed storage network begins or continues by, for a data access request, accessing a plurality of estimated efficiency models of a plurality of processing units of the storage network, where an estimated efficiency model of the plurality of estimated efficiency models includes a list of estimated efficiency probabilities, and the list of estimated efficiency probabilities corresponds to a list of data access request types for a processing unit of the plurality of processing units. The method continues by selecting one of the processing units from the plurality of processing units based on the plurality of estimated efficiency models to produce a selected processing unit. The method continues by sending the data access request to the selected processing unit for execution.

Abstract: A method includes identifying an existing logical storage vault having existing dispersed storage coding properties for vault transformation, where a first set of storage units support the existing logical storage vault, and a data object of first data objects stored within the first set of storage units is stored as a first plurality of sets of encoded data slices in accordance with the existing dispersed storage coding properties. The method includes identifying a new logical storage vault having new dispersed storage coding properties, wherein storage units support the new logical storage vault. The method includes transforming the first data objects from being in accordance with the existing dispersed storage coding properties to being in accordance with the new dispersed storage coding properties to produce transformed first data objects. The method includes storing the transformed first data objects in the new logical storage vault supported by the storage units.

Abstract: A plurality of encoded data slices are stored at locations within a memory, and a list of pointers to the encoded data slices is maintained. A request to delete a particular encoded data slice is received. In response to receiving the request, a particular location within the memory used to store the particular encoded data slice is determined based on the list of pointers, and an updated list of pointers is generated indicating that the particular encoded data slice is no longer stored at the particular location within the memory.

Abstract: A method includes determining, by a computing device of a storage network, a storage inconsistency exists for a set of encoded data slices based on an indicator of a difference list, where the difference list is regarding storage of the set of encoded data slices within a set of storage units of the storage network and where a data segment is error encoded into the set of encoded data slices. The method further includes determining a storage resolution of a plurality of storage resolutions for the storage inconsistency based on the one or more indicators. The method further includes facilitating the storage resolution to resolve the storage inconsistency for the set of encoded data slices.

Abstract: A method begins with a processing module obtaining a data object from a user and identifying a plurality of virtual storage vaults (VSVs) for storage of the data object. The method continues with the processing module determining dispersal error encoding parameters associated with a first virtual storage vault and error encoding the data object in accordance with those dispersal error encoding parameters to produce a corresponding plurality of sets of encoded data slices (EDSs), issuing write slice requests for the EDSs and receiving EDS information from at least some of the plurality of VSVs. When the EDS information indicates that a threshold number of VSVs have successfully stored the plurality of sets of encoded data slices within a synchronization timeframe, the method continues with the processing module issuing a favorable response to the user, or when a threshold number of VSVs have not successfully stored the plurality of sets of encoded data slices initiating a retry process.

Abstract: A method for execution by storage network begins by identifying a subset of encoded data slices stored in a set of storage units for enhanced access and interpreting registry data to determine a listing type. The method continues by generating listing slice names for the subset of encoded data slices based on the listing type and sending the listing slice names to the set of storage units, where each storage unit updating storage and/or listing of at least some of the subset of encoded data slices associated with the storage unit based on the listing type and each storage unit issuing slice access information to a requesting entity in response to one or more slice access requests in accordance with the listing type for some of the subset of encoded data slices and the requesting entity receiving slice access information from at least some of the set of storage units.

Abstract: A method begins by a processing module receiving a request to store a data object in distributed storage (DS) units. The processing module generates and transmits a proposal message that includes a preferred source name, and a proposal attempt identifier to a plurality of DS units. The processing module then receives a proposal message acceptance response from at least one of the plurality of DS units and when the proposal message response indicates that no other proposal messages have been received by at least one of the plurality of DS units, retains the preferred source name included within the proposal message as a persistent value for the source name.

Abstract: A method for execution by a record processing and storage system includes receiving a plurality of records and corresponding row numbers. Pages are generated from the received records. Page metadata is generated for each page that includes row number span data based on row numbers of the records included in each page. Pairs of pages are identified in the plurality of pages based on having row number span data in their page metadata that include a row number span overlap. For each pair of pages, row number span data is updated for a first page in the pair by removing the row number span overlap with a second page in the pair. Reads of pages are performed based on their row number span data. Only records of each first page of each pair of pages having row numbers that are within the updated row number span data are read.

Abstract: A method includes receiving a set of difference lists from a set of storage units of the DSN, where the set of storage units store a plurality of sets of encoded data slices, wherein a first difference list identifies first encoded data slices that have first indicators that are different than corresponding first indicators of the first encoded data slices included in a shared common list. The method continues by determining, for a set of encoded data slices of the plurality of sets of encoded data slices, whether a storage inconsistency exists based on one or more indicators associated with the encoded data slice included in the set of difference lists. When the storage inconsistency exists, the method continues by flagging for rebuilding encoded data slices of the set of encoded data slices associated with the storage inconsistency.

Abstract: A distributed storage network (DSN) stores sets of encoded data slices in sets of storage units. A first storage unit assigned to store an encoded data slice included in a set of encoded data slices transmits a rebuild request associated with the storage error to a second storage unit. The second storage unit generates the rebuilt encoded data slice in response to the rebuild request, and transmits the rebuilt encoded data slice back to the first storage unit, which stores the rebuilt encoded data slice.

Abstract: A method begins by a processing module receiving a request to store a data object in distributed storage (DS) units. The processing module generates and transmits a proposal message that includes a preferred source name, and a proposal attempt identifier to a plurality of DS units. The processing module then receives a proposal message acceptance response from at least one of the plurality of DS units and when the proposal message response indicates that no other proposal messages have been received by at least one of the plurality of DS units, retains the preferred source name included within the proposal message as a persistent value for the source name.