Abstract: A method includes: receiving, by a computing device, data for storage in a dispersed storage network; writing, by the computing device, the data to a first location; generating, by the computing device, a first pointer to the first location; receiving, by the computing device, updated data that is an updated version of the data; writing, by the computing device, the updated data to a second location; generating, by the computing device, a second pointer to the second location; and deleting, by the computing device, the first pointer at a time after writing the updated data and generating the second pointer.

Abstract: Systems and methods for proactive transfer of stored data between storage zones to avoid anticipated bit rot are provided. In embodiments, a method includes: determining that one or more quality prediction parameters of a storage zone of a data storage device meet a predetermined threshold for user access or adjacency to another storage zone determined to be unhealthy; and initiating a proactive refreshing of the storage zone based on the determining that the storage zone meets the predetermined threshold, the proactive refreshing of the storage zone including: reading all data in the storage zone; determining that no errors have occurred during the reading of the data; and based on the determination that no errors have occurred, moving all of the data of the storage zone to a new storage zone.

Abstract: A method includes: receiving, by a computing device, a data slice for storage in a dispersed storage network; predicting, by the computing device, a modification frequency associated with the data slice; and storing, by the computing device, the data slice in one of a first type zone of a data storage and a second type zone of the data storage based on the predicted modification frequency.

Abstract: Methods and systems for compression of aging data during compaction are disclosed. A method includes: ingesting a plurality of data objects into a dispersed storage network (DSN); determining that a compaction threshold is met for a storage medium in the DSN; and compacting the storage medium, the compaction including, for each of the plurality of data objects: determining a number of times the data object has been compacted; in response to the number of times the data object has been compacted exceeding a predetermined threshold, compressing the data object and rewriting the compressed data object to a new area on a storage medium; and in response to the number of times the data object has been compacted not exceeding the predetermined threshold, rewriting the data object to the new area on the storage medium without compressing the data object.

Abstract: A failure detection and correction module (FDCM) uses statistical measurement to detect failures in a distributed computing system caused by hardware, software, workflow, deployment, environmental factors, etc. in a component of the computing system, the computing system, or multiple computing systems and produces corrective actions. The FDCM identifies issues from various components, correlates the estimated failures in each level of components and rolls up failures and estimated failures from each level of components to system level estimations of failures, reevaluates the system reliability factors, readjusts the system reliability and system functions from the adjusted reliability factors, and produces intelligent corrective actions to improve both system reliability and the system efficiency. Corrective action includes changing slice storing parameters and rebuild priorities on a dispersed storage system.

Abstract: Systems and methods for managing conflicting background tasks in a dispersed storage network are provided. In embodiments, a method includes: gathering scheduled future task data for scheduled future tasks from a plurality of task scheduling modules within a dispersed storage network, wherein the scheduled future tasks are tasks associated with stored data objects; monitoring the scheduled future task data for scheduling conflicts based on stored rules; determining that a scheduling conflict exists between a first future task of the scheduled future tasks and a second future task of the scheduled future tasks; issuing instructions to at least one of the plurality of task scheduling modules to update the first future task or the second future task based on the scheduling conflict; and updating, by the at least one of the plurality of task scheduling modules, the first future task or the second future task based on the instructions.

Abstract: A method for execution by a dispersed storage and task (DST) processing unit includes determining a first optimal slice size requirement is determined for a first independent data element (IDE). The first IDE is split into a first plurality of IDEs based on determining the first IDE compares unfavorably to the first optimal slice size requirement. An error coding function is performed on the first plurality of IDEs to produce a set of encoded slices for each of the first plurality of IDEs for transmission to storage units. A subset of additional IDEs are selected to be merged into a merged IDE based on a second optimal slice size requirement, and continuous content of the subset of additional IDEs is concatenated to produce the merged IDE. The error coding function is performed on the merged IDE to produce a second set of encoded slices for transmission to storage units.

Abstract: A method begins by obtaining a set of load level information regarding a set of dispersed storage (DS) processing units of a plurality of sets of DS processing units of the DSN. The method continues by determining whether the first DS processing unit has a load imbalance based on the set of load level information. When the first DS processing unit has the load imbalance, the method continues by determining whether to delegate a first access request of one or more access requests to another DS processing unit. When determining to delegate the first access request, the method continues by determining a delegate DS processing unit of the set of DS processing units based on the set of load level information. The method continues by instructing the first DS processing unit to send the first access request to the delegate DS processing unit for processing.

Abstract: A method begins or continues by a computing device of a dispersed storage network (DSN) determining an error with a first write request in a first zone of a plurality of zones of a memory device of a storage unit of DSN, where the first zone includes a first set of sequential blocks that are in a first logical and physical location of the memory device. The method continues with the computing device abandoning pending write requests to the first zone. The method continues with reassigning the first write request to a second zone of the memory device, where the second zone includes a second set of sequential blocks that are in a second logical and physical location of the memory device.

Abstract: A method for execution by a dispersed storage and task (DST) execution unit includes generating a first compaction object by performing a compaction function on a first previously compacted object and first data slices that compare favorably to a cold access threshold. Storage of the first previously compacted object in a cold memory region is replaced with the first compacted object, and the first data slices are removed from prior storage in different memory regions. A second compacted object is generated by performing the compaction function on a second previously compacted object and second data slices that compare favorably to a hot access threshold. Storage of the second previously compacted object in a hot memory region is replaced with the second compacted object, and the second data slices are removed from prior storage in different memory regions.

Abstract: A method includes determining, by a storage unit, to place a pointer in a hash table that is stored in local memory of the storage unit. The method further includes executing, by the storage unit, a hash function on addressing information relating to the pointer to produce a hash resultant. The method further includes executing, by the storage unit, a second function on related data to the addressing information or to the pointer to produce an auxiliary resultant. The method further includes storing, by the storage unit, the hash resultant and the auxiliary resultant as an entry in the hash table for the pointer, wherein, when a hash collision occurs with respect to the hash resultant, the storage unit utilizes the auxiliary resultant to resolve the hash collision.

Abstract: A method for execution by a compaction management system includes determining observed compaction information based on compaction observed in at least one storage device during an observed timeframe. An estimated compaction rate is generated for a first future timeframe based on the observed compaction information. An updated ingest rate is generated for the first future timeframe based on a current ingest rate and the estimated compaction rate. A first proper subset of a set of data to be written to the at least one storage device is generated based on the updated ingest rate. Storage of the first proper subset in the at least one storage device is facilitated during the first future timeframe. Storage of a remaining proper subset of the set of data in an elastic buffer is facilitated during the first future timeframe, where the elastic buffer utilizes a memory of the compaction management system.

Abstract: A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method begins by detecting a failure (partial or full) of a dispersed storage (DS) unit memory device within the DSN. The method continues by tracking namespace ranges of encoded data slices intended for access in the failed DS unit memory device. The method continues by redirecting access of the encoded data slices reflected by the namespace ranges from the failed DS unit memory device to a failover DS unit memory device.

Abstract: A processing module of a memory storage unit includes an interface configured to interface and communicate with a communication system, a memory that stores operational instructions, and processing circuitry operably coupled to the interface and to the memory that is configured to execute the operational instructions to manage data stored using append-only formatting. When the processing module determines that a section of the memory includes invalid data and the amount of invalid data compares unfavorably to a predetermined limit, the processing module determines a rate for execution of a compaction routine for the section of memory, where the rate is based on a proportion, integral and derivative (PID) function that is based on a target usage level of the memory and a current usage level of the memory.

Abstract: A distributed storage network (DSN) includes a DSN memory and a distributed storage (DS) processing unit. The DS processing unit determines a particular revision of source data for persistence, and generates a protocol message associated with the particular revision of source data. The protocol message is generated based on a balance between durability and persistence associated with the particular revision. The protocol message is transmitted from the DS processing unit to the DSN memory, and indicates to the DSN memory a particular persistence protocol. The DSN memory process data slices corresponding to the particular revision of source data in accordance with the particular persistence protocol.

Abstract: A method begins or continues by a computing device of a dispersed storage network (DSN) determining an error with a first write request in a first zone of a plurality of zones of a memory device of a storage unit of DSN, where the first zone includes a first set of sequential blocks that are in a first logical and physical location of the memory device. The method continues with the computing device abandoning pending write requests to the first zone. The method continues with reassigning the first write request to a second zone of the memory device, where the second zone includes a second set of sequential blocks that are in a second logical and physical location of the memory device.

Abstract: A method for execution by a storage unit includes identifying, from a plurality of memory devices of the storage unit, a first memory device that is designated for storage of a first data slice to be written of the storage unit based on determining a slice name of the first data slice compares favorably to a namespace assigned to the first memory device. A function is performed on the slice name of the first data slice to identify a second memory device from the plurality of memory devices of the storage unit for temporary storage the first data slice in response to determining that the first memory device is unavailable and the first data slice is stored in the second memory device in response. The first data slice is migrated to storage in the first memory device in response to determining that the first memory device is available.

Abstract: A method for execution by a storage unit includes identifying, from a plurality of memory devices of the storage unit, a first memory device that is designated for storage of a first data slice to be written of the storage unit based on determining a slice name of the first data slice compares favorably to a namespace assigned to the first memory device. A function is performed on the slice name of the first data slice to identify a second memory device from the plurality of memory devices of the storage unit for temporary storage the first data slice in response to determining that the first memory device is unavailable and the first data slice is stored in the second memory device in response. The first data slice is migrated to storage in the first memory device in response to determining that the first memory device is available.

Abstract: A method for execution by a dispersed storage and task (DST) execution unit includes generating a first compaction object by performing a compaction function on a first previously compacted object and first data slices that compare favorably to a cold access threshold. Storage of the first previously compacted object in a cold memory region is replaced with the first compacted object, and the first data slices are removed from prior storage in different memory regions. A second compacted object is generated by performing the compaction function on a second previously compacted object and second data slices that compare favorably to a hot access threshold. Storage of the second previously compacted object in a hot memory region is replaced with the second compacted object, and the second data slices are removed from prior storage in different memory regions.

Abstract: A method for execution by a storage unit includes identifying, from a plurality of memory devices of the storage unit, a first memory device that is designated for storage of a first data slice to be written of the storage unit based on determining a slice name of the first data slice compares favorably to a namespace assigned to the first memory device. A function is performed on the slice name of the first data slice to identify a second memory device from the plurality of memory devices of the storage unit for temporary storage the first data slice in response to determining that the first memory device is unavailable and the first data slice is stored in the second memory device in response. The first data slice is migrated to storage in the first memory device in response to determining that the first memory device is available.