Abstract: A processing module of a memory storage unit includes an interface configured to interface and communicate with a communication system, a memory that stores operational instructions, and processing circuitry operably coupled to the interface and to the memory that is configured to execute the operational instructions to manage data stored using append-only formatting. When the processing module determines that a section of the memory includes invalid data and the amount of invalid data compares unfavorably to a predetermined limit, the processing module determines a rate for execution of a compaction routine for the section of memory, where the rate is based on a proportion, integral and derivative (PID) function that is based on a target usage level of the memory and a current usage level of the memory.

Abstract: A method begins by receiving a storage request to store one or more portions of a data object and determining whether the one or more portions are stored within a dispersed storage network memory. When stored, the method continues by determining a retrieval demand for a portion of the one or more portions. When below a first threshold, the method continues by indicating that an existing copy of the one or more portions will be used for retrieval requests for the one or more portions. When above the first threshold, the method continues by copying the existing copy to create a first one or more portion copies and indicating the one or more portions will be used for retrieval requests from a first group of user computing devices and indicating the first one or more portion copies will be used for retrieval requests from a second group.

Abstract: A storage unit (SU) includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and processing circuitry operably coupled to the interface and to the memory. The processing circuitry is configured to execute the operational instructions to perform various operations and functions. The SU determines to rebuild an encoded data slice (EDS) of a set of EDSs based on a storage error of the EDS and selects combining units to perform receiving of slice representations to enable rebuilding of the EDS. The SU facilitates receiving, by the combining units and/or by the SU, slice representations of a decode threshold number of EDSs and receives and combines a combined partial slice from a combining unit of the combining units and combines the combined partial slice and any additional representation of the decode threshold number of EDSs to generate a rebuilt EDS.

Abstract: A processing module of a memory storage unit includes an interface configured to interface and communicate with a communication system, one or more memory devices, and processing circuitry operably coupled to the interface and to the memory that is configured to receive an access request from a distributed storage network (DSN) client and determine whether a quality of service (QoS) identity for the DSN client is associated with the access request. Based on a determination that a QoS identity is associated with the access request, the processing module determines a QoS level associated with the access request, where the QoS level is based at least in part on the QoS identity that is associated with the access request. The processing module maps the QoS level to a storage layer priority for the access request and transmits the storage layer priority for execution by at least one of the one or more memory devices.

Abstract: A distributed storage network (DSN) includes a DSN memory and a distributed storage (DS) processing unit. The DS processing unit determines a particular revision of source data for persistence, and generates a protocol message associated with the particular revision of source data. The protocol message is generated based on a balance between durability and persistence associated with the particular revision. The protocol message is transmitted from the DS processing unit to the DSN memory, and indicates to the DSN memory a particular persistence protocol. The DSN memory process data slices corresponding to the particular revision of source data in accordance with the particular persistence protocol.

Abstract: A method for execution by a storage unit of a dispersed storage network (DSN) includes receiving a data slice for storage. A first bin that includes the data slice is generated and stored in a first location of a memory device of the storage unit, and a bin pointer that includes a reference to the first location is generated. A revision of the data slice is later received, and a second bin that includes the revised data slice is generated and stored in a second location of the memory device. A modified bin pointer is generated by editing the bin pointer to include a reference to the second location. A back pointer that references the first location is generated in response to commencing writing of the revised data slice. The back pointer is deleted in response to determining that the revised data slice has reached a finalized write stage.

Abstract: A method for execution within a dispersed storage network (DSN), where the method begins by calculating, utilizing a first integrity check value function, an integrity check value of a first type for each encoded data slice of a set of encoded data slices to produce a corresponding set of integrity check values. The method continues by issuing, via a network, one or more sets of write slice requests 1-n to a set of storage units 1-n within the DSN, where the one or more sets of write slice requests include a plurality of sets of the encoded data slices and a corresponding plurality of sets of the integrity check values. The method continues, when verifying integrity of a received encoded data slice, by a storage unit calculating, utilizing a second integrity check value function, an integrity check value of a second type for the encoded data slice.

Abstract: A distributed storage network (DSN) includes a plurality of distributed storage task execution (DST EX) units, each which includes a plurality of distributed storage (DS) memories configured to store encoded data slices. A DST EX unit detects a physical storage failure associated with a first DS memory included in the DST EX unit, wherein the first DS memory is associated with a sub-DSN address range. An available spare DS memory included in the DST EX unit is selected, and slice mapping information is updated to associate the available spare DS memory with the sub-DSN address range. The DST EX unit facilitates rebuilding one or more encoded data slices associated with the sub-DSN address range to produce one or more rebuilt encoded data slices. The DST EX unit facilitates storage of the rebuilt encoded data slices in the available spare DS memory.

Abstract: A method includes determining, by a storage unit, to place a pointer in a hash table that is stored in local memory of the storage unit. The method further includes executing, by the storage unit, a hash function on addressing information relating to the pointer to produce a hash resultant. The method further includes executing, by the storage unit, a second function on related data to the addressing information or to the pointer to produce an auxiliary resultant. The method further includes storing, by the storage unit, the hash resultant and the auxiliary resultant as an entry in the hash table for the pointer, wherein, when a hash collision occurs with respect to the hash resultant, the storage unit utilizes the auxiliary resultant to resolve the hash collision.

Abstract: A storage unit (SU) includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and processing circuitry operably coupled to the interface and to the memory. The SU determines to rebuild an encoded data slice (EDS) of a set of EDSs based on a storage error of the EDS and selects combining units to enable rebuilding of the EDS. The SU also selects participating storage units (SUs) that store at least the decode threshold number of EDSs and facilitates establishing pairwise encryption keys between appropriate components. The SU facilitates the combining units and the SU receiving encrypted representations of the decode threshold number of EDSs of the set of EDSs. The SU combines an encrypted combined partial slice and any additional encrypted representation to generate an encrypted rebuilt EDS and decrypts the encrypted rebuilt EDS to generate a rebuilt EDS.

Abstract: A method for execution by a storage unit of a dispersed storage network (DSN) includes receiving a data slice for storage. A first bin that includes the data slice is generated and stored in a first location of a memory device of the storage unit, and a bin pointer that includes a reference to the first location is generated. A revision of the data slice is later received, and a second bin that includes the revised data slice is generated and stored in a second location of the memory device. A modified bin pointer is generated by editing the bin pointer to include a reference to the second location. A back pointer that references the first location is generated in response to commencing writing of the revised data slice. The back pointer is deleted in response to determining that the revised data slice has reached a finalized write stage.

Abstract: A method for use in a dispersed storage network operates to select a recovery of selected ones of one or more first sets of encoded data slices in response to detecting a storage error associated with the selected ones of the one or more first sets of encoded data slices; issue requests for a second decode threshold number of encoded data slices of selected ones of one or more second sets of encoded data slices corresponding to the selected ones of the one or more first sets of encoded data slices; decode the second decode threshold number of encoded data slices to produce recovered data in response to receiving the second decode threshold number of encoded data slices; encode the recovered data utilizing first IDA parameters associated with the first IDA to produce one or more rebuilt encoded data slices corresponding to the selected ones of the one or more first sets of encoded data slices; and facilitate storage of the one or more rebuilt encoded data slices.

Abstract: A distributed storage network (DSN) includes a DSN memory and a distributed storage (DS) processing unit. The DS processing unit determines a particular revision of source data for persistence, and generates a protocol message associated with the particular revision of source data. The protocol message is generated based on a balance between durability and persistence associated with the particular revision. The protocol message is transmitted from the DS processing unit to the DSN memory, and indicates to the DSN memory a particular persistence protocol. The DSN memory process data slices corresponding to the particular revision of source data in accordance with the particular persistence protocol.

Abstract: A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. The computing device receives a data access request for a data object and determines a first revision number of a corresponding set of EDSs stored among first SU(s) and a second revision number of a corresponding trimmed copy of the set of EDSs stored among second SU(s). When the second revision number compares favorably to the first revision number, the computing device issues the data access request to the first SU(s) and/or the second SU(s) and issues the data access request for the data object to only the first SU(s) when it doesn't.

Abstract: A method for use in a dispersed storage network operates to select a recovery of selected ones of one or more first sets of encoded data slices in response to detecting a storage error associated with the selected ones of the one or more first sets of encoded data slices; issue requests for a second decode threshold number of encoded data slices of selected ones of one or more second sets of encoded data slices corresponding to the selected ones of the one or more first sets of encoded data slices; decode the second decode threshold number of encoded data slices to produce recovered data in response to receiving the second decode threshold number of encoded data slices; encode the recovered data utilizing first IDA parameters associated with the first IDA to produce one or more rebuilt encoded data slices corresponding to the selected ones of the one or more first sets of encoded data slices; and facilitate storage of the one or more rebuilt encoded data slices.

Abstract: A method for use in a dispersed storage network operates to select a recovery of selected ones of one or more first sets of encoded data slices in response to detecting a storage error associated with the selected ones of the one or more first sets of encoded data slices; issue requests for a second decode threshold number of encoded data slices of selected ones of one or more second sets of encoded data slices corresponding to the selected ones of the one or more first sets of encoded data slices; decode the second decode threshold number of encoded data slices to produce recovered data in response to receiving the second decode threshold number of encoded data slices; encode the recovered data utilizing first IDA parameters associated with the first IDA to produce one or more rebuilt encoded data slices corresponding to the selected ones of the one or more first sets of encoded data slices; and facilitate storage of the one or more rebuilt encoded data slices.

Abstract: A method begins by receiving a storage request to store one or more portions of a data object and determining whether the one or more portions are stored within a dispersed storage network memory. When stored, the method continues by determining a retrieval demand for a portion of the one or more portions. When below a first threshold, the method continues by indicating that an existing copy of the one or more portions will be used for retrieval requests for the one or more portions. When above the first threshold, the method continues by copying the existing copy to create a first one or more portion copies and indicating the one or more portions will be used for retrieval requests from a first group of user computing devices and indicating the first one or more portion copies will be used for retrieval requests from a second group.

Abstract: A method for execution by a dispersed storage and task (DST) processing unit includes receiving a write request that includes a data object. Characterization data is generated by determining a data type of the data object. Object constraint data is generated by determining a compressibility of the data type and a processing cost of the data type. Optimized trade-off data is generated by optimizing a plurality of trade-off constraints based on the object constraint data. A compression algorithm is selected from a plurality of compression algorithm options based on the optimized trade-off data. A compressed data object is generated by performing the selected compression algorithm on the data object. A plurality of data slices are generated for transmission to a plurality of storage units for storage by performing an information dispersal algorithm on the compressed data object.

Abstract: A method begins by receiving a storage request to store one or more portions of a data object. The method continues by determining whether the one or more portions are stored within a dispersed storage network memory. When stored, the method continues by determining a retrieval demand for each portion. When the retrieval demand for a first portion is below a first threshold, the method continues by indicating that an existing copy of the first portion will be used for retrieval requests of the first portion. When the retrieval demand is above the first threshold, the method continues by copying the existing copy to create a first portion copy and indicating that the existing copy will be used for retrieval requests from a first group of user computing devices and indicating that the first portion copy will be used for retrieval requests from a second group of user computing devices.

Abstract: A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. The computing device receives a data access request for a data object and determines a first revision number of a corresponding set of EDSs stored among first SU(s) and a second revision number of a corresponding trimmed copy of the set of EDSs stored among second SU(s). When the second revision number compares favorably to the first revision number, the computing device issues the data access request to the first SU(s) and/or the second SU(s) and issues the data access request for the data object to only the first SU(s) when it doesn't.