Abstract: Managing connectivity to synchronously replicated storage systems, including: identifying a plurality of storage systems across which a dataset is synchronously replicated; identifying a host that can issue I/O operations directed to the dataset; identifying a plurality of data communications paths between the host and the plurality of storage systems across which a dataset is synchronously replicated; identifying, from amongst the plurality of data communications paths between the host and the plurality of storage systems across which a dataset is synchronously replicated, one or more optimal paths; and issuing, to the host, an identification of the one or more optimal paths.

Abstract: Applying a rate limit across a plurality of storage systems, including: determining a rate limit for paired storage systems; receiving, by a first storage system, an amount of I/O operations serviced by the second storage system during a previous predetermined period of time; determining whether the amount of I/O operations serviced by the second storage system is less than half of the rate limit for the paired storage systems; if so, setting local a rate limit for a next predetermined period of time for the first storage system to the difference between the rate limit for the paired storage systems and the amount of I/O operations serviced by the second storage system during the previous predetermined period of time; and otherwise, setting a local rate limit for a next predetermined period of time for the first storage system to half of the rate limit for the paired storage systems.

Abstract: A plurality of storage nodes in a single chassis is provided. The plurality of storage nodes in the single chassis is configured to communicate together as a storage cluster. Each of the plurality of storage nodes includes nonvolatile solid-state memory for user data storage. The plurality of storage nodes is configured to distribute the user data and metadata associated with the user data throughout the plurality of storage nodes such that the plurality of storage nodes maintain the ability to read the user data, using erasure coding, despite a loss of two of the plurality of storage nodes. A plurality of compute nodes is included in the single chassis, each of the plurality of compute nodes is configured to communicate with the plurality of storage nodes. A method for accessing user data in a plurality of storage nodes having nonvolatile solid-state memory is also provided.

Abstract: Automatic storage system configuration for mediation services that includes: determining that a particular storage system of the storage systems is not configured to request mediation from a mediation target for mediation between storage systems synchronously replicating a dataset; requesting, by the particular storage system from a configuration service, configuration information indicating one or more service handles for a mediation service; and configuring, in dependence upon the one or more service handles received from the configuration service, a mediation handler to communicate with the mediation service responsive to detecting a communication fault with one of the storage systems.

Abstract: A method begins by a dispersed storage (DS) processing module storing a data portion in a dispersed storage network (DSN), where the data portion is associated with a set of data index keys, a version index key, and with a set of portion index keys. The method continues with the DS processing module traversing, based on the set of data index keys, a plurality of index structures to determine whether an object retrieval structure exists for the data portion. When the object retrieval structure exists, the method continues with the DS processing module updating the object retrieval structure by determining whether a version record exists that corresponds to the version index key and when the version record exists, creating, within the version record, a new data portion record for the data portion to include an identifier of the data portion and the set of portion index keys.

Abstract: A method includes identifying characteristics of a plurality of resources of one or more resource levels, where a data segment of a data object is one of a plurality of data types and is stored in memory of a dispersed storage network (DSN) according to a distributed agreement protocol (DAP). The method continues by generating a plurality of sets of DAP configuration information for the plurality of resource levels based on the characteristics. The method continues by receiving a data access request regarding a data segment and determining a data type of the data segment. When the data type is the first data type, the method continues with obtaining a first set of DAP configuration information and performing one or more distributed agreement protocol functions utilizing an identifier of the data and the first set of DAP configuration information to select storage units of the DSN.

Abstract: Data transformation offloading in an artificial intelligence infrastructure that includes one or more storage systems and one or more graphical processing unit (‘GPU’) servers, including: storing, within the storage system, a dataset; identifying, in dependence upon one or more machine learning models to be executed on the GPU servers, one or more transformations to apply to the dataset; and generating, by the storage system in dependence upon the one or more transformations, a transformed dataset.

Abstract: Consistently selecting a group of datasets during recovery, including: for each of a plurality of datasets: generating a respective identifier corresponding to a respective dataset, wherein the respective identifier is different from each other identifier; and storing the respective identifier to represent successful storage of the respective dataset; and responsive to recovering from a system interruption: extracting identifier information from all readable datasets; and generating, in dependence upon the extracted identifiers, a group of datasets from among the readable datasets.

Abstract: Workload planning with quality-of-service (‘QoS’) integration, including: determining, for each of a plurality of entities, one or more QoS thresholds associated with the entity; determining, for each of the plurality of entities, one or more resource utilization levels associated with the entity; and determining, in dependence upon the one or more QoS thresholds associated with the plurality of entities, the one or more resource utilization levels associated with the plurality of entities, and one or more overprovisioning factors associated with the storage system, a risk factor that identifies the likelihood that one or more system resources will be overconsumed by the plurality of entities.

Abstract: A computing device includes an interface configured to interface and communicate with a dispersed or distributed storage network (DSN), a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. The computing device receives first samples corresponding to inputs that characterize configuration of the DSN and receives second samples corresponding to outputs that characterize system behavior of the DSN. The computing device then processes the first and samples to generate a DSN model to generate predictive performance of the outputs based on various values of the inputs. In some instances, the DSN model is based on a neural network model that employs the inputs that characterize the configuration of the DSN and generates the outputs that characterize system behavior of the DSN.

Abstract: Data transformation caching in an artificial intelligence infrastructure that includes one or more storage systems and one or more graphical processing unit (‘GPU’) servers, including: identifying, in dependence upon one or more machine learning models to be executed on the GPU servers, one or more transformations to apply to a dataset; generating, in dependence upon the one or more transformations, a transformed dataset; storing, within one or more of the storage systems, the transformed dataset; receiving a plurality of requests to transmit the transformed dataset to one or more of the GPU servers; and responsive to each request, transmitting, from the one or more storage systems to the one or more GPU servers without re-performing the one or more transformations on the dataset, the transformed dataset.

Abstract: A method begins by a processing module of a dispersed storage network (DSN) receiving a DSN retrieval request regarding a data object and performing a scoring function using properties of the DSN retrieval request and properties of DSN memory of the DSN to produce a storage scoring resultant. The method continues with the processing module identifying a set of primary storage units based on the storage scoring resultant and sending a set of retrieval requests to the set of primary storage units. When a primary storage unit does not provide a favorable response, using the storage scoring resultant to identify an alternative storage unit. When the alternative storage unit is identified, sending a corresponding retrieval request to the alternative storage unit.

Abstract: Systems and methods for rebuilding data in a dispersed storage network (DSN) involve identifying a rebuilding process for a set of DS units requiring additional resources, identifying one or more tasks of the rebuilding process, identifying identified DS units of another set of DS units to support the one or more tasks and assigning the one or more tasks to the identified DS units. The systems and methods further involve receiving one or more rebuilding results and facilitating completion of the one or more tasks utilizing the one or more rebuilding results. The systems and method may further involve partitioning the one or more tasks to produce partial tasks, assigning the partial tasks to identified DS units, receiving rebuilding partial results and facilitating completion of one or more tasks utilizing the one or more rebuilding partial results.

Abstract: A method for processing partial tasks in a distributed storage network (DSN) includes receiving a partial task request message for a DS execution unit, where the partial task request message includes corresponding partial tasks. The method continues by processing each partial task request message in accordance with the processing parameters to produce task request slice groupings, generating slices, such that each message is directed at a corresponding DS execution unit, and the sending the slice groupings and the task request slice groupings to the selected DS execution units for storage therein. The method continues by retrieving at least a decode threshold number of task response slices of one or more task response slice groupings from the DS execution units, decoding the task response slices, retrieving at least a decode threshold number of partial result slices, and decoding the partial results slices and processing the partial results to produce a result.

Abstract: A system and method for performing replication using mediums. A snapshot ‘S’ is selected for replication, and the anchor medium ‘M’ of S is identified. The replica storage array ‘R’ to which M is being replicated determines a list of medium extents which are available. R sends this list to an original storage array ‘O’ and O generates a list of medium extents which need to be sent to R. Only those medium extents corresponding to M which R does not already have will be sent from O to R. Also, R can obtain medium extents from other source storage arrays during the replication process.

Abstract: A method for execution by a computing device of a dispersed storage network includes dispersed storage error encoding a data segment to produce a set of encoded data slices. Storage of more than a write threshold number of encoded data slices of the set of encoded data slices in a set of storage units is facilitated. A plurality of favorably stored encoded data slices is identified. A number of stored encoded data slices for deletion is determined, and the number of stored encoded data slices for deletion is selected from the plurality of favorably stored encoded data slices. Deletion of the selected number of stored encoded data slices for deletion is facilitated.

Abstract: A method for execution by a dispersed storage and task (DST) processing unit includes obtaining audit records for an audit object and determining when the audit object is complete. When the audit object is complete, aggregating the audit records of the audit object within the audit object by generating the audit object to include the audit records; generating identifier (ID) information and generating integrity information. Fields of the audit object are populated with the audit records, the ID information, and the integrity information and a name of the audit object is determined for storage of the audit object and the name of the audit object in a dispersed storage network (DSN).

Abstract: A method includes a first computing device encrypting a portion of a data matrix based on a set of encryption keys to produce an encrypted data matrix. The method further includes the first computing device sending the encrypted data matrix to a second computing device. The method further includes the second computing device dispersed storage error encoding the data matrix to produce a set of encrypted encoded data slices. The method further includes the second computing device sending the set of encrypted encoded data slices to a set of storage units of the DSN for storage therein.

Abstract: A method begins with determining a memory device of a dispersed storage network (DSN) has an issue, where a data segment of a data object is dispersed storage error encoded to produce a set of encoded data slices that is stored in a set of storage units of the DSN, where a storage unit includes a plurality of memory devices that includes the memory device, and where the memory device stores an encoded data slice of the set of encoded data slices. The method continues by identifying an issue with the memory device and by identifying sets of encoded data slices. The method continues by generating an additional encoded data slice for each of the sets of encoded data slices to produce a group of encoded data slices and storing the group of encoded data slices in at least one other memory device of the DSN.

Abstract: An indication to replicate, for a second storage resource, a volume that is stored at a first storage resource may be received. A snapshot of the volume that is stored at the second storage resource may be identified. Data with operations to data blocks of the volume may be identified. A sequence number range, containing one or more sequence numbers, associated with the snapshot of the volume that is stored at the second resource may be received. Furthermore, a portion of the data associated with the operations to the data blocks of the volume based on the sequence number range that is associated with the snapshot of the volume that is stored at the second resource may be received to replicate the volume at the second storage resource.