Abstract: Systems and methods of deduplication aware scalable content placement am described. A method may include receiving data to be stored on one or more nodes of a storage array and calculating a plurality of hashes corresponding to the data. The method further includes determining a first subset of the plurality of hashes, determining a second subset of the plurality of hashes of the first subset, and generating a node candidate placement list. The method may further include sending the first subset to one or more nodes represented on the node candidate placement list and receiving, from the nodes represented on the node candidate placement list, characteristics corresponding to the nodes represented on the candidate placement list. The method may further include identifying one of the one or more nodes represented on the candidate placement list m view of the characteristic and sending the data to the identified node.

Abstract: A storage controller coupled to a storage array comprising one or more storage devices receive a request to write data to one of the storage devices. The storage controller determines a first data block on the storage device comprising a list of deallocated data blocks on the storage device, the list comprising a block number of each deallocated data block and an access operation count value at which each deallocated data block was deallocated. The storage controller identifies a second data block from the list of deallocated data blocks on the storage device based on a corresponding access operation count value from the list and writes the data to the second data block.

Abstract: Methods for verifying rebuilt data in a dispersed storage network (DSN). In various examples, a storage unit of the DSN receives a rebuilt encoded data slice of a set of encoded data slices and a set of corresponding integrity values. An integrity function is performed on the set of integrity values to calculate an integrity value of the set of integrity values, which is then compared to a locally stored value. If the values match, the set of integrity values is validated. Another integrity function is performed on the rebuilt encoded data slice to produce a calculated integrity value of the received encoded data slice. The encoded data slice is validated and stored when the calculated integrity value compares favorably to the corresponding value of the validated set of integrity values. Proof of authenticity information from other storage units may be employed to further validate the set of integrity values.

Abstract: A method of efficient rekey in a transparent decrypting storage array includes receiving an instruction to rekey data on a storage array, wherein the instruction identifies first encryption information and second encryption information. The method further includes decrypting, by a processing device of a storage array controller, the data using the first encryption information to generate decrypted data. The method further includes encrypting the decrypted data using the second encryption information to generate encrypted data.

Abstract: A storage unit (SU) includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and processing circuitry operably coupled to the interface and to the memory. The processing circuitry is configured to execute the operational instructions to perform various operations and functions. The SU, of a first SU set, receives a partially encoded slice request to restore a set of encoded data slices (EDSs) that are based on first dispersed storage error coding function parameters. The SU generates a decode threshold number of partially EDSs based on the first and second dispersed storage error coding function parameters. The SU then outputs the decode threshold number of partially EDSs to a second SU set to undergo selective combination respectively to generate new EDSs for storage within the SUs of the second decode threshold number of SUs of the second SU set.

Abstract: Managing storage device evacuation that includes a plurality of storage devices, including: detecting, by the storage system, an occurrence of a storage device evacuation event associated with a source storage device within a write group, wherein the write group is a subset of storage devices storing a data set; responsive to detecting the occurrence of the storage device evacuation event, identifying, by the storage system, a target storage device for receiving data stored on the source storage device; and migrating, by the storage system, the data stored on the source storage device to the target storage device.

Abstract: A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. For example, the computing device generates a prioritized request that includes at least one of a task for execution or a priority level based on information stored within a storage unit (SU) of a plurality of storage units (SUs) implemented within the DSN. Note that the information corresponds to a data object that is related to a set of encoded data slices (EDSs) that are distributedly stored within the DSN. The computing device then transmits the prioritized request to the SU and receives, from the SU, a response to the prioritized request.

Abstract: Offloading data storage to a decentralized storage network, including: identifying a plurality of decentralized storage networks that the storage system can utilize for storing data; selecting, in dependence upon characteristics of each decentralized storage network and requirements associated with storing the data, one or more decentralized storage networks for storing the data; and initiating storage of the data on the selected one of more decentralized storage networks.

Abstract: Locally providing cloud storage array services for a plurality of storage systems within a data center by receiving a request from an operating system level virtualization service; and determining, among the plurality of storage systems, an implementation of the request from the operating system level virtualization service, among the plurality of storage systems.

Abstract: A method for erasure detection in a storage cluster is provided. The method includes establishing a connection, via a network, of a storage unit to one of a plurality of storage nodes of a storage cluster and determining, for at least one page of a storage memory of the storage unit, that the at least one page is erased. The storage unit is one of a plurality of storage units configured to store user data in memory of the storage units in accordance with direction from the plurality of storage nodes. The method includes communicating from the storage unit to the one of the plurality of storage nodes that the at least one page is erased.

Abstract: A method of operating a storage unit having non-volatile random-access memory (NVRAM) and solid-state memory is provided. The method includes transferring contents of the NVRAM to the solid-state memory, in response to detecting a power loss. The method includes during the transferring, having each of a plurality of channels in parallel, reading one or more of a plurality of logical unit numbers (LUNs) each corresponding to a portion of the NVRAM, performing an XOR of data of each of the one or more of the plurality of LUNs with data of a preceding LUN, and writing results of the XOR to the solid-state memory.

Abstract: A method for utilizing a decentralized agreement protocol for data access operations in a dispersed storage network (DSN) including a legacy storage unit pool and one or more non-legacy storage unit pools. In response to obtaining a DSN address associated with a data access request, a DSN computing device(s) generates ranked scoring information for the legacy storage unit pool and the non-legacy storage unit pool, the ranked scoring information based at least in part on the DSN address, a first location weight associated with the legacy storage unit pool, and a second location weight associated with the non-legacy storage unit pool. In an embodiment, the first location weight decreases over time (e.g., as a generation of legacy storage units is retired). Based on the ranked scoring information, at least one of the legacy storage unit pool or the non-legacy storage unit pool is selected for processing the data access request.

Abstract: A method for execution by a computing device includes, receiving, from a requesting device, a request for a data segment of a data object that is or is to be stored in storage units of a content delivery network. The method further includes determining whether the data segment is stored in a cache memory of the content delivery network or in the storage units. When stored in the cache memory, the method includes retrieving the cached data segment, and sending it to the requesting device. When stored in the storage units, the method includes, sending read requests regarding the data segment to the storage units, receiving, in response to the read requests, at least a decode threshold number of encoded data slices, decoding the at least the decode threshold number of encoded data slices to reproduce the data segment, and sending the data segment to the requesting device.

Abstract: A system and method for performing copy offload operations. When a copy offload operation from a first volume (pointing to a first medium) to a second volume (pointing to a second medium) is requested, the copy offload operation is performed without accessing the data being copied. A third medium is created, and the first medium is recorded as the underlying medium of the third medium. The first volume is re-pointed to the third medium. Also, a fourth medium is created, the second volume is re-pointed to the fourth medium, and the second medium is recorded as the underlying medium of the targeted range of the fourth medium. All other ranges of the fourth medium have the second medium as their underlying medium.

Abstract: Dynamically adjusting an error correction effort level of a storage device, including: receiving, from a storage array controller, an error correction effort level to perform when attempting to read data from the storage device; identifying that an attempt to read the data resulted in an error; and determining whether an amount of error correction effort level required to attempt to correct the error exceeds the error correction effort level to perform when attempting to read data from the storage device.

Abstract: In one implementation, a method comprises storing verification data and erasure codes separately in a plurality of storage devices. The method further comprises determining, by a processing device, whether data to be written to the plurality of storage devices is lost or corrupted using the verification data and the erasure codes.

Abstract: A method of automatic load rebalancing includes determining that a new storage device was added to a storage array comprising a plurality of storage devices, wherein the new storage device is distinct from the plurality of storage devices. The method further includes, in response to the determining, identifying a first shard on a first storage device of the plurality of storage devices, wherein the first storage device has a fullness metric that is equal to or exceeds a fullness threshold. The method further includes moving, by a processing device of a storage array controller of the storage array, the first shard from the first storage device to the new storage device.

Abstract: A method includes detecting, by a vault management device, a failed storage unit common to a first vault and a second vault. The first vault is associated with a first set of storage units and the second vault is associated with a second set of storage units. The failed storage unit is in each of the first and second sets of storage units. The method further includes identifying a number of non-failed storage units of the first and second sets of storage units and comparing the number of non-failed storage units with first and second decode threshold numbers to determine a failure impact level. The first decode threshold number is associated with the first vault and the second decode threshold number is associated with the second vault. The method further includes determining a failure abatement approach based on the failure impact level and facilitating the failure abatement approach.

Abstract: A method includes identifying an independent data object of a plurality of independent data objects for retrieval from dispersed storage network (DSN) memory. The method further includes determining a mapping of the plurality of independent data objects into a data matrix, wherein the mapping is in accordance with the dispersed storage error encoding function. The method further includes identifying, based on the mapping, an encoded data slice of the set of encoded data slices corresponding to the independent data object. The method further includes sending a retrieval request to a storage unit of the DSN memory regarding the encoded data slice. When the encoded data slice is received, the method further includes decoding the encoding data slice in accordance with the dispersed storage error encoding function and the mapping to reproduce the independent data object.

Abstract: Servicing I/O operations in a virtual storage system, including: receiving, by the virtual storage system, a request to write data to the virtual storage system; storing, within staging memory provided by one or more virtual drives of the virtual storage system, both the data and an erasure code based on the data; and migrating, from the staging memory to more durable data storage provided by a cloud services provider, at least a portion of data stored within the staging memory without migrating the erasure code based on the data.