Abstract: A method for execution by a compaction management system includes determining observed compaction information based on compaction observed in at least one storage device during an observed timeframe. An estimated compaction rate is generated for a first future timeframe based on the observed compaction information. An updated ingest rate is generated for the first future timeframe based on a current ingest rate and the estimated compaction rate. A first proper subset of a set of data to be written to the at least one storage device is generated based on the updated ingest rate. Storage of the first proper subset in the at least one storage device is facilitated during the first future timeframe. Storage of a remaining proper subset of the set of data in an elastic buffer is facilitated during the first future timeframe, where the elastic buffer utilizes a memory of the compaction management system.

Abstract: A method for processing partial tasks in a distributed storage network (DSN) includes receiving a partial task request message for a DS execution unit, where the partial task request message includes corresponding partial tasks. The method continues by processing each partial task request message in accordance with the processing parameters to produce task request slice groupings, generating slices, such that each message is directed at a corresponding DS execution unit, and the sending the slice groupings and the task request slice groupings to the selected DS execution units for storage therein. The method continues by retrieving at least a decode threshold number of task response slices of one or more task response slice groupings from the DS execution units, decoding the task response slices, retrieving at least a decode threshold number of partial result slices, and decoding the partial results slices and processing the partial results to produce a result.

Abstract: An integrity processing unit includes rebuild modules to rebuild one or more encoded data slices in a dispersed storage network (DSN) memory unit. The rebuild modules determine a rebuild rate of the DSN memory unit, and, based on the rebuild rate, a rebuild rate status of the DSN memory unit. When the rebuild rate status is a high rebuild rate status, the rebuild rate to the DSN memory unit is reduced and, when the rebuild rate is not zero, the one or more encoded data slices are rebuilt in the DSN memory unit.

Abstract: A method includes determining an encoding modification for a set of encoded data slices where a data segment of data is dispersed storage error encoded into the set of encoded data slices based on dispersed storage error encoding parameters. The method further includes determining a plurality of tasks for executing the encoding modification, where the encoding modification includes altering one or more parameters of the dispersed storage error encoding parameters. The method further includes assigning a first task of the plurality of tasks to a first storage unit and assigning remaining tasks of the plurality of tasks to a set of storage units. The method further includes executing, by the first storage unit and at least some storage units of the set of storage units, the first task and the remaining tasks of the plurality of tasks, respectively, to produce a modified set of encoded data slices.

Abstract: A method comprises encoding, by a DS processing unit of a dispersed storage network (DSN), a data segment using an information dispersal algorithm with a first pillar width number to produce a set of encoded data slices. The method continues by generating a set of DSN addresses for the set of encoded data slices based on the first pillar width number, a second pillar width number and a DSN address mapping function. The method continues by identifying, based on the DSN addresses, a first group of storage units of a set of storage units that includes the second width pillar number of storage units and where the first group of storage units includes the first pillar width number of storage units. The method continues by sending the set of encoded data slices to the first group of storage units in accordance with the set of DSN addresses.

Abstract: A method for execution by a computing device of a dispersed storage network includes identifying a DSN address for data to be accessed within a DSN. A plurality of storage cohorts associated with the data to be accessed are identified. Ranked scoring information for each storage cohort obtaining ranked scoring information for each storage cohort of the plurality of storage cohorts with regards to the data to be accessed is obtained. A storage cohort of the plurality of storage cohorts is selected based on the ranked scoring information. Slice access requests are issued to a set of storage units of the selected storage cohort for the data to be accessed. Slice access responses received from at least one of the set of storage units of the selected storage cohort are processed.

Abstract: A distributed storage network (DSN) includes a DSN memory and a distributed storage (DS) processing unit. The DS processing unit determines a particular revision of source data for persistence, and generates a protocol message associated with the particular revision of source data. The protocol message is generated based on a balance between durability and persistence associated with the particular revision. The protocol message is transmitted from the DS processing unit to the DSN memory, and indicates to the DSN memory a particular persistence protocol. The DSN memory process data slices corresponding to the particular revision of source data in accordance with the particular persistence protocol.

Abstract: A method begins or continues by a computing device of a dispersed storage network (DSN) determining an error with a first write request in a first zone of a plurality of zones of a memory device of a storage unit of DSN, where the first zone includes a first set of sequential blocks that are in a first logical and physical location of the memory device. The method continues with the computing device abandoning pending write requests to the first zone. The method continues with reassigning the first write request to a second zone of the memory device, where the second zone includes a second set of sequential blocks that are in a second logical and physical location of the memory device.

Abstract: A method for execution by a processing system includes determining to retrieve index data for a set of objects stored by an object storage system. An index type is selected from a plurality of index. A request message indicating a request for the index data of one of a plurality of indices stored by the object storage system corresponding to the index type is generated, and the request message is transmitted to the object storage system. The index data for the set of objects is received in a response message from the object storage system. The index data includes a plurality of pairs that each include a key and a value, where the keys correspond to the index type, and where the value of each pair includes metadata of one of the set of objects. The index data is stored in local memory in response to receiving the index data.

Abstract: A method for execution by a dispersed storage and task (DST) execution unit includes generating a first compaction object by performing a compaction function on a first previously compacted object and first data slices that compare favorably to a cold access threshold. Storage of the first previously compacted object in a cold memory region is replaced with the first compacted object, and the first data slices are removed from prior storage in different memory regions. A second compacted object is generated by performing the compaction function on a second previously compacted object and second data slices that compare favorably to a hot access threshold. Storage of the second previously compacted object in a hot memory region is replaced with the second compacted object, and the second data slices are removed from prior storage in different memory regions.

Abstract: A migration agent, which is part of a distributed storage network, identifies one or more data objects stored as sets of encoded data slices in a first storage pool, and determines, for each of those data objects, whether to migrate corresponding sets of encoded data slices from the first storage pool to another storage pool. For at least one of the data objects, the migration agent determines to migrate a set of encoded data slices, and issues a set of MoveSlice requests to storage units included in the first storage pool, from which the data slices will be migrated. In response to the MoveSlice requests, the migration agent receives MoveSlice responses from the storage units in the first storage pool. If a threshold number of favorable MoveSlice responses is received, the migration agent facilitates deletion of the migrated encoded data slices from the first storage pool.

Abstract: A method for execution by a processing system in dispersed storage and task network (DSTN) that includes a processor, includes: identifying a slice name of a slice in error of a set of slices stored in a set of dispersed storage (DS) units; identifying a number of slice errors of the set of slices; generating a queue entry that includes the slice name of the slice in error, a rebuilding task indicator, an identity of the set of slices, and the number of slice errors; identifying a rebuilding queue based on the number of slice errors, wherein the rebuilding queue is associated with one of: the set of DS units or another set of DS units; and facilitating storing the queue entry in the identified rebuilding queue.

Abstract: A method for execution by a dispersed storage and task (DST) processing unit includes determining a first optimal slice size requirement is determined for a first independent data element (IDE). The first IDE is split into a first plurality of IDEs based on determining the first IDE compares unfavorably to the first optimal slice size requirement. An error coding function is performed on the first plurality of IDEs to produce a set of encoded slices for each of the first plurality of IDEs for transmission to storage units. A subset of additional IDEs are selected to be merged into a merged IDE based on a second optimal slice size requirement, and continuous content of the subset of additional IDEs is concatenated to produce the merged IDE. The error coding function is performed on the merged IDE to produce a second set of encoded slices for transmission to storage units.

Abstract: A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. For example, the computing device generates a prioritized request that includes at least one of a task for execution or a priority level based on information stored within a storage unit (SU) of a plurality of storage units (SUs) implemented within the DSN. Note that the information corresponds to a data object that is related to a set of encoded data slices (EDSs) that are distributedly stored within the DSN. The computing device then transmits the prioritized request to the SU and receives, from the SU, a response to the prioritized request.

Abstract: A method includes obtaining a plurality of sets of write requests regarding a plurality of sets of encoded data slices. During a first time period, the method continues by identifying a set of storage units of a dispersed storage network and identifying a first namespace subrange of a namespace associated with the set of storage units. The method continues by issuing a first set of write requests of the plurality of sets of write requests to a first subset of storage units of the set of storage units regarding first sets of encoded data slices to be stored in the first namespace subrange. The method continues by issuing a second set of write requests of the plurality of sets of write requests to a second subset of storage units of the set of storage units regarding second sets of encoded data slices to be stored in the first namespace subrange.

Abstract: A method for execution by a computing device of a dispersed storage network (DSN). The method begins by dispersed storage error encoding a data segment of data into a set of encoded data slices, wherein data blocks of the data segment are arranged into a data matrix, wherein the data matrix is multiplied by an encoding matrix to create the set of encoded data slices. The method continues by arranging data blocks of an encoded data slice into a second data matrix and multiplying the second data matrix with a second encoding matrix to produce a set of transmission data slices. The method continues with identifying a plurality of paths for transmitting a decode threshold number of transmission data slices to a storage unit of the DSN. The method continues by transmitting the decode threshold number of transmission data slices to the storage unit via the plurality of paths.

Abstract: A method for execution by a dispersed storage and task (DST) processing unit includes determining a first optimal slice size requirement is determined for a first independent data element (IDE). The first IDE is split into a first plurality of IDEs based on determining the first IDE compares unfavorably to the first optimal slice size requirement. An error coding function is performed on the first plurality of IDEs to produce a set of encoded slices for each of the first plurality of IDEs for transmission to storage units. A subset of additional IDEs are selected to be merged into a merged IDE based on a second optimal slice size requirement, and continuous content of the subset of additional IDEs is concatenated to produce the merged IDE. The error coding function is performed on the merged IDE to produce a second set of encoded slices for transmission to storage units.

Abstract: A method for execution by a dispersed storage and task (DST) processing unit includes determining a first optimal slice size requirement is determined for a first independent data element (IDE). The first IDE is split into a first plurality of IDEs based on determining the first IDE compares unfavorably to the first optimal slice size requirement. An error coding function is performed on the first plurality of IDEs to produce a set of encoded slices for each of the first plurality of IDEs for transmission to storage units. A subset of additional IDEs are selected to be merged into a merged IDE based on a second optimal slice size requirement, and continuous content of the subset of additional IDEs is concatenated to produce the merged IDE. The error coding function is performed on the merged IDE to produce a second set of encoded slices for transmission to storage units.

Abstract: A method includes dividing dispersed storage error encoding of a data object into a plurality of operations based on at least one of the data object and available computing devices for executing the dispersed storage error encoding. The method further includes allocating the plurality of operations to the available computing devices, where a first encoding operation of the plurality of encoding operations is allocated to a first computing device of the available computing devices. The method further includes coordinating execution of the plurality of operations by the available computing devices to dispersed storage error encode the data object into a plurality of sets of encoded data slices and a corresponding plurality of sets of slice names, and write the plurality of sets of encoded data slices based on the corresponding plurality of sets of slice names to a set of storage units.

Abstract: A method for execution by a storage unit of a dispersed storage network includes updating a rebuilding task list based on detecting at least one storage error associated with storage of encoded data slices in a set of storage units that includes the storage unit. An encoded data slice is rebuilt based on the rebuilding task list and an affinity with the encoded data slice. The rebuilding task list is again updated based on detecting execution of at least one task of the rebuilding task list. The rebuilding task list is further updated based on detecting expiration of an execution time frame between sequential tasks of the rebuilding task list.