Abstract: A storage system includes a plurality of solid-state storage devices and a storage group controller. The storage group controller receives, from a central storage controller, a command comprising information associated with a RAID rebuild operation to reconstruct data stored at the set of solid-state storage devices. In response to receiving the information associated with the RAID rebuild operation, the storage group controller reads other data and parity data stored associated with the data to be reconstructed at the set of solid-state storage devices based on the information associated with the RAID rebuild operation. Upon reading the other data and the parity data stored at the set of solid-state storage devices, the storage group controller reconstructs the data based on the other data, the parity data and the information associated with the RAID rebuild operation and transmits, to the central storage controller, the reconstructed data.

Abstract: Applying a rate limit across a plurality of storage systems, including: determining a rate limit for paired storage systems; receiving, by a first storage system, an amount of I/O operations serviced by the second storage system during a previous predetermined period of time; determining whether the amount of I/O operations serviced by the second storage system is less than half of the rate limit for the paired storage systems; if so, setting local a rate limit for a next predetermined period of time for the first storage system to the difference between the rate limit for the paired storage systems and the amount of I/O operations serviced by the second storage system during the previous predetermined period of time; and otherwise, setting a local rate limit for a next predetermined period of time for the first storage system to half of the rate limit for the paired storage systems.

Abstract: Policy based path management, including: identifying a plurality of data communications paths between a storage system and an entity that issues input/output (‘I/O’) operations to the storage system; selecting, in dependence upon a path selection policy, a subset of data communications paths from the plurality of data communications paths between the entity and the storage system; and issuing, from the entity to the storage system, I/O operations using only the subset of data communications paths.

Abstract: A storage cluster is provided. The storage cluster includes a plurality of storage nodes within a chassis. The plurality of storage nodes has flash memory for storage of user data and is configured to distribute the user data and metadata throughout the plurality of storage nodes such that the storage nodes can access the user data with a failure of two of the plurality of storage nodes. Each of the storage nodes is configured to generate at least one address translation table that maps around defects in the flash memory on one of a per flash package basis, per flash die basis, per flash plane basis, per flash block basis, per flash page basis, or per physical address basis. Each of the plurality of storage nodes is configured to apply the at least one address translation table to write and read accesses of the user data.

Abstract: A method for adjustable error correction in a storage cluster is provided. The method includes determining health of a non-volatile memory of a non-volatile solid-state storage unit of each of a plurality of storage nodes in a storage cluster on a basis of per flash package, per flash die, per flash plane, per flash block, or per flash page. The determining is performed by the storage cluster. The plurality of storage nodes is housed within a chassis that couples the storage nodes as the storage cluster. The method includes adjusting erasure coding across the plurality of storage nodes based on the health of the non-volatile memory and distributing user data throughout the plurality of storage nodes through the erasure coding. The user data is accessible via the erasure coding from a remainder of the plurality of storage nodes if any of the plurality of storage nodes are unreachable.

Abstract: Synchronizing snapshots between storage systems, including: receiving, from a source storage system, an identification of a snapshot to be replicated to a destination storage system; identifying, from hint information stored on the destination storage system, a most recent version of the snapshot that is stored on the destination storage system; issuing, to the source storage system, a request for an identification of the differences between the snapshot to be replicated to the destination storage system and the most recent version of the snapshot that is stored on the destination storage system; receiving, from the source storage system, the identification of the differences; and issuing a request to transfer, from the source storage system to the destination storage system, data associated with the differences.

Abstract: A method for accessing a file in a storage system is provided. The method includes determining, for each file chunk of the file, an authority among differing storage nodes of the storage system and receiving from the authority having ownership of the file chunk, location information for the file chunk. The method includes accessing file chunks of the file as directed by each of the determined authorities.

Abstract: Servicing I/O operations in a cloud-based storage system, including: receiving, by the cloud-based storage system, a request to write data to the cloud-based storage system; storing, in solid-state storage of the cloud-based storage system, the data; storing, in object storage of the cloud-based storage system, the data; detecting that at least some portion of the solid-state storage of the cloud-based storage system has become unavailable; identifying data that was stored in the portion of the solid-state storage of the cloud-based storage system that has become unavailable; retrieving, from object storage of the cloud-based storage system, the data that was stored in the portion of the solid-state storage of the cloud-based storage system that has become unavailable; and storing, in solid-state storage of the cloud-based storage system, the retrieved data.

Abstract: A method includes segmenting, by a computing device of a dispersed storage network (DSN), a data object into data segments. The method further includes encrypting, by the computing device, the data segments using encryption keys to produce encrypted data segments. The method further includes dispersed storage error encoding, by the computing device, the encryption keys using a key dispersed storage error encoding function to produce a set of encoded key slices. The method further includes appending, by the computing device, at least a decode threshold number of encoded key slices of the set of encoded key slices to at least some of the encrypted data segments in accordance with an appending approach to produce secure packages. The method further includes dispersed storage error encoding, by the computing device, the secure packages to produce sets of encoded data slices.

Abstract: A system includes a solid-state storage array having a plurality of solid-state storage devices and a storage controller coupled to the solid-state storage array, the storage controller including a processing device, the processing device to select a segment height based on erase block sizes of the plurality of solid-state storage devices. The processing device is further to program a data segment using the segment height to a data stripe across two or more of the plurality of solid-state storage devices and store the segment height in metadata associated with the data segment.

Abstract: A method includes identifying an independent data object of a plurality of independent data objects for retrieval from dispersed storage network (DSN) memory. The method further includes determining a mapping of the plurality of independent data objects into a data matrix, wherein the mapping is in accordance with the dispersed storage error encoding function. The method further includes identifying, based on the mapping, an encoded data slice of the set of encoded data slices corresponding to the independent data object. The method further includes sending a retrieval request to a storage unit of the DSN memory regarding the encoded data slice. When the encoded data slice is received, the method further includes decoding the encoding data slice in accordance with the dispersed storage error encoding function and the mapping to reproduce the independent data object.

Abstract: A method of operating a solid-state storage drive is provided. The method includes encrypting, in the solid-state storage drive, first data by a first key, and writing the encrypted first data into solid-state storage memory of the solid-state storage drive. The method includes encrypting, in the solid-state storage drive, second data by a second key, and writing the encrypted second data into the solid-state storage memory of the solid-state storage drive, wherein at least one of the first key or the second key is supplied from a source external to the solid-state storage drive.

Abstract: A method for storage cluster expansion is provided. The method includes distributing user data throughout a storage cluster as directed by each of a plurality of authorities in the storage cluster. Each of the plurality of authorities has a plurality of wards, and each of the plurality of wards has ownership of a range of the user data. The method includes splitting one of the plurality of authorities, as a parent authority, into at least two child authorities and assigning a first subset of the plurality of wards of the parent authority to one of the at least two child authorities, and a second subset of the plurality of wards of the parent authority to another one of the at least two child authorities.

Abstract: Writing data in a storage system that includes a first type of storage device and a second type of storage device, including: selecting, for one or more unprocessed write requests, a target storage device type from the first type of storage device and the second type of storage device; issuing a first group of write requests to the first type of storage device, the first group of write requests addressed to one or more locations selected in dependence upon an expected address translation to be performed by the first type of storage device; and issuing a second group of write requests to the second type of storage device, the second group of write requests addressed to one or more locations selected in dependence upon a layout of memory in the second type of storage device.

Abstract: An exemplary security threat monitoring system receives performance metric data representative of a performance metric for a storage system, applies the performance metric data as an input to an unsupervised machine learning model, and identifies, based on an output of the unsupervised machine learning model, an anomaly in the performance metric data.

Abstract: A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and processing circuitry operably coupled to the interface and to the memory. The processing circuitry is configured to execute the operational instructions to perform various operations and functions. The computing device detects a total number of errors that is associated with a set of memory devices of one or more sets of storage units (SUs) within a DSN that distributedly store a set of encoded data slices (EDSs). When the total number of errors compares unfavorably to a priority error threshold level, the computing device indicates that a minimum number of error-free EDSs are available of the set of EDSs. The computing device also selects a mechanism for data retention process from a plurality of mechanisms for data retention process and executes it.

Abstract: A method includes determining, by a computing device of a dispersed storage network (DSN), routing path performance information of a set of routing paths with respect to a set of storage units of the DSN. The method further includes adjusting a pillar width to decode threshold ratio of a dispersed storage error encoding function when the routing path performance information deviates from a performance threshold. The performance threshold includes a first error rate threshold and a second error rate threshold. The method further includes dispersed storage error encoding a data object using the adjusted pillar width to decode threshold ratio to produce a plurality of sets of encoded data slices. The method further includes sending the plurality of sets of encoded data slices to the set of storage units via the set of routing paths for storage therein.

Abstract: A method begins by a processing module dispersed storage error encoding a data segment to produce a set of encoded data slices and dispersed storage error encoding metadata associated with the data segment to produce a set of encoded metadata slices. The method continues with the processing module creating a set of data slice names for the set of encoded data slices and creating a set of metadata slice names based on the set of data slice names. The method continues with the processing module sending the set of encoded data slices and the set of data slice names to a dispersed storage network (DSN) memory for storage therein. The method continues with the processing module sending the set of encoded metadata slices and the set of metadata slice names to the DSN memory for storage therein.

Abstract: A method for execution by a computing device includes detecting that an incremental storage cohort has been added to a storage generation to produce an updated plurality of storage cohorts of an updated storage generation, where each storage cohort includes a set of storage units. For each storage cohort, a slice listing process is initiated to identify a plurality of DSN addresses associated with storage of data objects within the each storage cohort. For each DSN address, ranked scoring information is obtained for the each storage cohort of the updated plurality of storage cohorts. One storage cohort is identified based on the ranked scoring information. When the identified storage cohort is different than another storage cohort associated with current storage of encoded data slices associated with the DSN address of the identified storage cohort, a migration process is initiated to migrate the encoded data slices to the identified storage cohort.

Abstract: Coalescing write operations in a cloud-based storage system including receiving, from a storage controller application of the cloud-based storage system, a first plurality of write operations, wherein each of the first plurality of write operations comprises a respective write to a storage volume; coalescing the first plurality of write operations into a plurality of coalesced write operations, wherein each of the coalesced write operations are configured to effect two or more of the first plurality of write operations; and performing the plurality of coalesced write operations on the storage volume.