Abstract: A method includes identifying an independent data object of a plurality of independent data objects for retrieval from dispersed storage network (DSN) memory. The method further includes determining a mapping of the plurality of independent data objects into a data matrix, wherein the mapping is in accordance with the dispersed storage error encoding function. The method further includes identifying, based on the mapping, an encoded data slice of the set of encoded data slices corresponding to the independent data object. The method further includes sending a retrieval request to a storage unit of the DSN memory regarding the encoded data slice. When the encoded data slice is received, the method further includes decoding the encoding data slice in accordance with the dispersed storage error encoding function and the mapping to reproduce the independent data object.

Abstract: A method for execution by an integrity processing unit includes performing a deterministic function on data for storage to produce an integrity value. The data and the integrity value are combined in accordance with a combining function to produce a data package. The processing system determines an encryption approach in response to determining to encrypt the data package. The data package is encrypted in accordance with the encryption approach to produce a secure package. The secure package is encoded to produce a set of slices. The set of slices is decoded to reproduce the secure package. The secure package is decrypted to reproduce the data package. The data package is de-combined in to generate reproduced data and a received integrity value. The deterministic function is performed on the data to produce a calculated integrity value, and the received integrity value is compared to the calculated integrity value.

Abstract: A method for execution by a dispersed storage and task (DST) processing unit includes generating contention level data by evaluating an update contention level in response to determining to update an entry of a node of a dispersed hierarchical index. The update of the node is executed when the contention level data indicates that the update contention level is favorable. An index update request is generated for transmission to an index update unit via a network when the contention level data indicates that the update contention level is unfavorable.

Abstract: A method for execution in a dispersed storage network operates to determine one or more slice names of one or more slices and determine whether to establish a new access policy corresponding to the one or more slices. When the new access policy is to be established, the method determines a timestamp; determines a new access policy; and sends the new access policy and the timestamp to one or more storage units that store the one or more slices.

Abstract: Migrating workloads between a plurality of execution environments, including: identifying, in dependence upon on characteristics of a workload, one or more execution environments that can support the workload; determining, for each execution environment, costs associated with supporting the workload on the execution environment; selecting, in dependence upon the costs associated with supporting the workload on each the execution environments, a target execution environment for supporting the workload; and executing the workload on the target execution environment.

Abstract: Ensuring the appropriate utilization of system resources using weighted workload based, time-independent scheduling, including: receiving an I/O request associated with an entity; determining whether an amount of system resources required to service the I/O request is greater than an amount of available system resources in a storage system; responsive to determining that the amount of system resources required to service the I/O request is greater than the amount of available system resources in the storage system: queueing the I/O request in an entity-specific queue for the entity; detecting that additional system resources in the storage system have become available; and issuing an I/O request from an entity-specific queue for an entity that has a highest priority, where a priority for each entity is determined based on the amount of I/O requests associated with the entity and a weighted proportion of resources designated for use by the entity.

Abstract: A method begins by a processing module receiving a checked write slice request from a requesting entity. The method continues by determining that locally stored encoded data slices do not include the requested encoded data slice. The method continues by identifying an alternate location for the requested encoded data slice. The method continues by determining whether the alternate location is associated with storage of the encoded data slice. The method continues when the alternate location is associated with the storage of the encoded data slice, by issuing a favorable checked write slice response to a requesting entity. The method can include facilitating transfer of the requested encoded data slice from the alternate location to the storage unit for storage.

Abstract: A migration agent, which is part of a distributed storage network, identifies one or more data objects stored as sets of encoded data slices in a first storage pool, and determines, for each of those data objects, whether to migrate corresponding sets of encoded data slices from the first storage pool to another storage pool. For at least one of the data objects, the migration agent determines to migrate a set of encoded data slices, and issues a set of MoveSlice requests to storage units included in the first storage pool, from which the data slices will be migrated. In response to the MoveSlice requests, the migration agent receives MoveSlice responses from the storage units in the first storage pool. If a threshold number of favorable MoveSlice responses is received, the migration agent facilitates deletion of the migrated encoded data slices from the first storage pool.

Abstract: A method for extending data lifetime for reference in deduplication is provided. The method includes determining that a quantity of user data has at least a threshold amount of data that is re-created in a storage system. The method includes protecting at least portions of the quantity of user data from erasure by garbage collection in the storage system during a predetermined time interval, wherein the protected at least portions are available for data deduplication of further user data in the storage system during the predetermined time interval.

Abstract: A method for execution by a dispersed storage and task (DST) processing unit includes generating a first access request for transmission via a network to a first one of a plurality of storage units in a dispersed storage network (DSN). A first access response is received via the network from the first one of the plurality of storage units that includes a first access time duration. Access duration data is updated to include the first access time duration received from the first one of the plurality of storage units. A subset of storage units is selected from the plurality of storage units based on comparing a plurality of access time durations corresponding to the plurality of storage units included in the access duration data to perform a second data access. At least one second access request is generated for transmission via the network to the subset of storage units.

Abstract: A method for hardware assisted data lookup in a storage unit is provided. The method includes formatting data in at least one of a plurality of data formats for storage in the storage unit. The method includes configuring a logic unit with one or more parameters associated with the plurality of data formats and identifying incoming data with the one or more parameters as an instruction for execution.

Abstract: A method for execution by a rebuilding module includes detecting that less than a pillar width number of encoded data slices of a common revision are retrievable from a set of storage units. A decode threshold number of encoded data slices are retrieved and decoded to reproduce a data segment. The data segment is encoded to produce at least one encoded data slice and storage of the at least one encoded data slice in the set of storage units is facilitated in accordance with the common revision when determining to rebuild the at least one encoded data slice. The data segment is encoded to reproduce the set of encoded data slices and storage of the reproduced set of encoded data slices is facilitated in the set of storage units in accordance with a new revision when determining to not rebuild the at least one encoded data slice.

Abstract: A method for administration through files in a storage cluster is provided. The method includes distributing user data, including files, throughout a plurality of storage nodes having non-volatile solid-state memory, through erasure coding. The plurality of storage nodes are housed within a single chassis that couples the storage nodes as the storage cluster. The method includes receiving at the storage cluster a command to write a file or read a file, the file having a filename. The method includes determining whether the filename is reserved and determining an administrative action according to the file, responsive to determining that the filename is reserved. The method includes initiating the administrative action. A storage system is also provided.

Abstract: A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and processing circuitry operably coupled to the interface and to the memory. The processing circuitry is configured to execute the operational instructions to perform various operations and functions. The computing device determine status of a set of memory devices of one or more sets of storage units (SUs) that distributedly store a set of encoded data slices (EDSs) within the DSN. The computing device detects one or more memory devices thereof that are failing and determines a number of available EDSs. When an insufficient number of EDSs are available, the computing device issues urgent read slice request(s) to recover additional available EDS(s) and facilitate temporary storage thereof within the DSN.

Abstract: A method of stateful load balancing, performed by a storage system is provided. The method includes obtaining a request identifier for a request received by the storage system through a network and calculating a version identifier for provider selection, based on the request identifier. The method includes determining a provider for the request, based on the request identifier and the version identifier, and sending the request to one of a plurality of nodes of the storage system as the determined provider.

Abstract: A method of applying a data format in a direct memory access transfer is provided. The method includes distributing user data throughout a plurality of storage nodes through erasure coding, wherein the plurality of storage nodes are housed within a single chassis that couples the storage nodes as a cluster, each of the plurality of storage nodes having nonvolatile solid-state memory for user data storage. The method includes reading a self-describing data portion from a first memory of the nonvolatile solid-state memory and extracting a destination from the self-describing data portion. The method includes writing data, from the self-describing data portion, to a second memory of the nonvolatile solid-state memory according to the destination.

Abstract: A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and processing circuitry operably coupled to the interface and to the memory. The processing circuitry is configured to execute the operational instructions to perform various operations and functions. The computing device detects memory error(s) associated with a plurality of sets of memory devices of sets of storage unit(s) (SU(s)) within the DSN that distributedly store a set of encoded data slices (EDSs). The computing device facilitates detection of EDS error(s) associated with the memory error(s). For a set of memory devices, the computing device establishes a corresponding memory replacement priority level and facilitates replacement of corresponding memory device(s) associated with the EDS error(s) based on the corresponding memory replacement priority level.

Abstract: A method for deleting a set of keys from a storage server is provided. The method includes generating a probabilistic data structure for a first set of keys and for each key in a second set of keys, determining whether a key of the second set of keys is found in the probabilistic data structure. The method includes identifying the key as a candidate for deletion if the key is not found in the probabilistic data structure. A system is also provided.

Abstract: A method of scalable garbage collection includes receiving an indication to perform a garbage collection process on a section of a database of a storage array comprising a plurality of storage devices. The method further includes determining, by a processing device of a storage array controller of the storage array, whether the section corresponds to any check-pointed data set. The method further includes, if the section does not correspond to any check-pointed data set: performing the garbage collection process on the section. The method further includes, if the section does correspond to a check-pointed data set: performing, by the processing device, a scalable garbage collection process on the section.

Abstract: A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. A computing device receives a data access request for an encoded data slice (EDS) associated with a data object. The computing device compares a slice name of the data access request with slice names stored within RAM. When the data access request slice name compares unfavorably with those stored slice names, the computing device transmits an empty data access response that includes no EDS to the other computing device without needing to access a hard disk drive (HDD) that stores EDSs. Alternatively, the computing device transmits a data access response that includes the EDS.