Abstract: A method for snapshots with crash consistency is provided. The method includes indicating intent to create a snapshot of storage system contents associated with an existing snapshot copy identifier, responsive to confirmation that all of a plurality of distributed agents are in a snapshot creation mode. The method includes responding, with a new snapshot copy identifier that replaces the existing snapshot copy identifier, to one or more inquiries from the plurality of distributed agents regarding one or more I/O operations, after the indicating the intent to create the snapshot. The method includes creating the snapshot using the existing snapshot copy identifier and committing the snapshot, responsive to receiving responses from the plurality of distributed agents that service is complete for all I/O operations that started before the indicating the intent to create the snapshot or that are associated with the existing snapshot copy identifier.

Abstract: An exemplary method includes sending a storage query to a plurality of storage nodes of a data storage system, the storage query including a data identifier of a data instance and an inquiry as to which of the storage nodes can store the data instance; receiving, in response to the storage query, a plurality of responses from a subset of storage nodes included in the plurality of storage nodes and that have at least a predetermined minimum amount of free storage space, the responses including information about each of the storage nodes included in the subset; selecting, based on the information included in the responses, multiple storage nodes included in the subset; and sending the data instance and the data identifier to the selected storage nodes for storage by the selected storage nodes.

Abstract: A method for execution by a computing device of a storage network includes obtaining at least a “T” number of encoded data slices of a set of encoded data slices, where a plurality of data chunks are all-or-nothing encoded in accordance with distributed data storage parameters to produce the set of encoded data slices. The method further includes decoding a first section of the at least the “T” number of encoded data slices to recover a first data chunk of a plurality of data chunks. The method further includes decoding a second section of the at least the “T” number of encoded data slices to recover a second data chunk of the plurality of data chunks.

Abstract: A method for execution by a computing device of a storage network includes determining an encoded data slice reduction scheme for a set of encoded data slices stored in a set of storage units of the storage network, where a data segment of data is encoded into the set of encoded data slices in accordance with encoding parameters, and where the encoding parameters include a pillar width number and a decode threshold number. The method further includes maintaining storage of the set of encoded data slices in accordance with the encoded data slice reduction scheme, where the maintaining storage includes keeping a number of encoded data slices of the set of encoded data slices equal to or greater than the decode threshold number and less than the pillar width number.

Abstract: A method for execution by one or more storage units of a dispersed storage network (DSN). The method begins by receiving, at a first storage unit, a request for a partial task. The method continues by generating a slice request, to one or more additional storage units, when the first storage unit does not contain all encoded data slices required to execute the partial task. The method continues by receiving the at least one additional encoded data slice from the one or more additional storage units and performing the partial task on the first encoded data slice and the at least one additional encoded data slice to produce at least partial results.

Abstract: Deploying client-specific applications in a storage system utilizing redundant system resources, including: identifying a redundant controller in the storage system, wherein the storage system includes at least a first controller and the redundant controller; and executing one or more applications on the redundant controller, wherein the one or more applications are executed in a container.

Abstract: Modifying storage distribution in a storage system that includes one or more storage devices, including: detecting, for a storage device among the one or more storage devices, that a storage capacity of the storage device is different from a storage capacity of another storage device of the one or more storage devices, and responsive to detecting that the storage capacity for the storage device is different from the storage capacity of the other storage devices of the one or more storage devices, modifying a distribution of shards of data for a data stripe among the one or more storage devices.

Abstract: Methods and apparatus for selection of memory devices in a distributed storage network. In an example, a computing device receives a data object for storage and selects a set of storage nodes of a plurality of sets of storage nodes for storing the data object. Selection of the set of storage nodes includes determining storage attributes associated with each set of storage nodes of the plurality of sets of storage nodes. Selection of the set of storage nodes additionally includes determining a storage preference associated with the data object, and comparing the storage preference with the storage attributes of the plurality of sets of storage nodes to determine a best match. Following selection of a set of storage nodes, the computing device facilitates storage of the data object in the selected set of storage nodes.

Abstract: Cloning storage systems in a cloud computing environment, including: receiving a request to create a cloud-based storage system; retrieving, from cloud-based object storage, one or more objects to include in the cloud-based storage system; and creating the cloud-based storage system, including storing, in block storage of the cloud-based storage system, data contained in the one or more objects retrieved from the cloud-based object storage.

Abstract: Storage layer data obfuscation, including: determining a subset of a dataset to obfuscate in accordance with a security policy; generating, based at least in part on the security policy, an obfuscated snapshot of the dataset that is representative of the dataset with the subset of the dataset obfuscated; and sending, to a target computer system, the obfuscated snapshot from which a restored version of the dataset includes the subset of the dataset obfuscated.

Abstract: Batch building for artificial intelligence workflows, including: issuing, responsive to a request for a batch of data objects, requests to a data repository for multiple data objects stored among one or more directories; selecting, in accordance with a batch building policy, a subset of data objects based on one or more responses to the requests; and providing, to the artificial intelligence workflow, a batch of data objects that includes the subset of data objects.

Abstract: A method includes receiving data for storage and encoding the data to produce a plurality of data slices. Metadata is determined for a data slice of the plurality of data slices. The metadata is stored in a metadata storage tree. The metadata storage tree is stored via a first plurality of memory devices of a first memory type. The data slice is stored in a slice storage location in a second plurality of memory devices of a second memory type. The slice storage location is indicated by the metadata. The first memory type has a higher performance level than the second memory type based on a utilization approach.

Abstract: A method for tracking valid and invalid sequence numbers in a storage system, performed by a processor, is provided. The method includes establishing a table as a key value store in memory in the storage system. The table has sequence numbers as keys and represents valid sequence numbers and invalidated sequence numbers of an open-ended sequence relating to storage of data or metadata in the storage system. The method includes adding to the table an entry that records a first plurality of consecutive sequence numbers, as a first range-valued key associated with a first value indicating the first plurality of consecutive sequence numbers is valid. The method includes adding to the table an entry that records a deletion of a second plurality of consecutive sequence numbers, as a second range-valued key associated with a second value indicating the second plurality of consecutive sequence numbers is invalid.

Abstract: Protecting an encryption key for data stored in a storage system that includes a plurality of storage devices, including: reading, from at least a majority of the storage devices, a portion of an apartment key; reconstructing the apartment key using the portions of the apartment key read by the majority of the storage devices; unlocking the main portion of each of the storage devices utilizing the apartment key; reading, from the main portion of one of the storage devices, a portion of a third-party resource access key; requesting, from the third-party resource utilizing the third-party resource access key, an encryption key; receiving, from the third-party resource, the encryption key; and decrypting the data stored on the storage devices utilizing the encryption key.

Abstract: A scheduling system for a memory controller is provided. The system includes a scheduler configurable to receive a plurality of operation requests from a plurality of masters. The scheduler is configurable to form a sequence of one or more phases from each of the operation requests. The scheduler is configurable to arbitrate the plurality of operation requests and the one or more phases through one or more configurable policies. The system includes a sequencer configurable to receive the one or more phases and communicate with at least two flash memory devices having differing types of flash memory device interfaces through a plurality of channels.

Abstract: Rehabilitating storage devices in a storage array that includes a plurality of storage devices, including: receiving a request to rehabilitate a storage device that is operating outside of a defined range of expected operating parameters; selecting, from a hierarchy of rehabilitative actions that can be performed on the storage device, a rehabilitative action to perform on a storage device in dependence upon information describing a number of times that one or more of the rehabilitative actions have been performed on the storage device; and initiating execution of the selected rehabilitative action.

Abstract: A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and processing circuitry operably coupled to the interface and to the memory. The processing circuitry is configured to execute the operational instructions to perform various operations and functions. The computing device determines a number storage units that have been provisioned on the storage network but has not been assigned for storage of data. The computing device then determines a storage utilization level for each of a plurality of active storage units associated with the storage network and based on the storage utilization level, determines whether to activate one or more inactive storage units. The computing device is configured to determine at least a portion of the number of inactive storage units for activation and issue an assignment message to the at least a portion of the number of inactive storage units.

Abstract: Replication handling during storage system transportation, including: replicating, between a first storage system and a second storage system that are both located at a first physical location, a dataset; connecting, by the first storage system and responsive to movement of the first storage system to an intermediate physical location, to a communication network available at the intermediate physical location; and continuing, between the first storage system at the intermediate physical location and the second storage system at the first physical location, replication of the dataset over the communication network available at the intermediate physical location.

Abstract: A method of orchestrating a virtual storage system, the method comprising: determining a change to one or more resource demands; determining, based on the change to the one or more resource demands, one or more modifications to one or more virtual components included as part of a virtual storage system architecture of a virtual storage system within a cloud computing environment; and initiating, responsive to the change to the one or more resource demands, the one or more modifications to the one or more virtual components included as part of the virtual storage system architecture of the virtual storage system.

Abstract: Systems, methods, and computer readable storage mediums for using a cloud assist service to generate a read-only GUI to view the status of a storage system. An authorized user can login remotely to the cloud assist service to view the read-only GUI of the storage system. The read-only GUI will present a view of the status of the storage system that is similar to a local GUI available to users connecting directly to the storage system. Responses used to generate the local GUI are recorded and simulated by the cloud assist service to create a familiar read-only GUI.