Abstract: Providing highly available application-specific storage by a cloud-based storage system, including: detecting that a component within a cloud-based storage system that supports an application has become unavailable; and selecting a replacement component within the cloud-based storage system to support the application, wherein at least a portion of a dataset associated with the application is stored as blocks within block storage resources in the cloud-based storage system and also stored as objects within object storage resources in the cloud-based storage system.

Abstract: A method includes receiving a request to write a data block to a volume resident on a multi-tenant storage array, wherein the request is associated with a first tenant of the multi-tenant storage array, and determining whether the data block matches an existing data block on the multi-tenant storage array, wherein the existing block corresponds to a second tenant. In response to determining that the decrypted data block matches the existing data block: encrypting the existing data block with a shared volume encryption key; encrypting the shared volume encryption key with a first tenant encryption key and providing the shared volume encryption key encrypted with the first tenant encryption key to the first tenant; and encrypting the shared volume encryption key with a second tenant encryption key and providing the shared volume encryption key encrypted with the second tenant encryption key to the second tenant.

Abstract: Ensuring reproducibility in an artificial intelligence infrastructure that includes one or more storage systems and one or more graphical processing unit (‘GPU’) servers, including: identifying, by a unified management plane, one or more transformations applied to a dataset by the artificial intelligence infrastructure, wherein applying the one or more transformations to the dataset causes the artificial intelligence infrastructure to generate a transformed dataset; storing, within the one or more storage systems, information describing the dataset, the one or more transformations applied to the dataset, and the transformed dataset; identifying, by the unified management plane, one or more machine learning models executed by the artificial intelligence infrastructure using the transformed dataset as input; and storing, within the one or more storage systems, information describing one or more machine learning models executed using the transformed dataset as input.

Abstract: A plurality of data blocks stored at a plurality of solid-state storage devices are identified. A portion of data is read from each data block of the plurality of data blocks. A corresponding property is determined for each data block of the plurality of data blocks based on reading the portion of the data. A set of data blocks from the plurality of data blocks is identified, wherein each data block of the set of data blocks is associated with a first corresponding property. The set of data blocks is stored at a data segment.

Abstract: Methods and apparatus for selecting data access resources in a distributed storage network (DSN) are disclosed. In various embodiments, a processing module(s) receives a data access request including a data identifier. Addressing information is determined for the data access request. In addition, the processing module obtains DSN configuration information associated with a plurality of access resources that interface with DSN memory. Ranked scoring information is determined for the plurality of access resources based, at least in part, on the DSN configuration information and the addressing information. In an example, a distributed agreement protocol module is utilized to calculate the ranked scoring information as a processing capacity of a respective access resource divided by a negative log of a normalized deterministic function value based on the addressing information and an identifier. The ranked scoring information is then utilized to select an access resource for processing of the data access request.

Abstract: Emulating persistent reservations in a cloud-based storage system, including: acquiring, by a first storage controller, in response to a request for a Small Computer Systems Interface (SCSI) persistent reservation, a first lease for an area of storage of a cloud-based storage system; and storing, by the first storage controller, a first identifier for the first lease in a predefined portion of the area of storage, wherein the predefined portion of the area of storage is accessible to a second storage controller.

Abstract: A method of operating a storage system is provided. The method includes establishing a security context between a client and the storage system, the security context comprising a single ticket for multiple nodes within the storage system. The method includes distributing a first request to a first blade within the storage system and distributing a second request to a second blade within the storage system. The distributing the first request and the second request includes determining a node for handling the first request and the second request based on data within the single ticket.

Abstract: A storage system with garbage collection prioritizes data segments for garbage collection in the storage memory. Priority of each data segment relates to amount of processing device work for garbage collection of the data segment versus benefit relating to amount of dead data to be reclaimed by garbage collection of the data segment. Garbage collection is performed according to the priority of each data segment.

Abstract: Compressing data in dependence upon characteristics of a storage system, including: receiving an amount of processing resources available in the storage system; receiving an amount of space available in the storage system; and selecting, in dependence upon the priority for conserving the amount of processing resources and the amount of space, a data compression algorithm to utilize to compress the data.

Abstract: A method of applying a data format in a direct memory access transfer is provided. The method includes distributing user data throughout a plurality of storage nodes through erasure coding, wherein the plurality of storage nodes are housed within a single chassis that couples the storage nodes as a cluster, each of the plurality of storage nodes having nonvolatile solid-state memory for user data storage. The method includes reading a self-describing data portion from a first memory of the nonvolatile solid-state memory and extracting a destination from the self-describing data portion. The method includes writing data, from the self-describing data portion, to a second memory of the nonvolatile solid-state memory according to the destination.

Abstract: A system for cloud-based file services, comprising: a plurality of single-tenant file system nodes configured to provide file system access to an object store via a plurality of multitenant storage nodes; the plurality of multitenant storage nodes sharing access to the object store; and one or more management nodes configured to provision resources for the plurality of single-tenant file system nodes and the plurality of multitenant storage nodes.

Abstract: Remote data management, including: receiving, at a data storage service from a client device and via a remote data management application program interface (API), a data object; receiving, at the data storage service from the client device and via the remote data management API, a message including an indication of updated metadata and including an identification of a particular data object stored by the data storage service; and generating, in accordance with the updated metadata received from the client device, a modified data object that is a reduced version of the particular data object specified by the message received from the client device, wherein the reduced version of the particular data object is stored by the data storage service.

Abstract: Methods for RAID stripe update parity calculation are disclosed. The methods include: receiving a first portion of data of a RAID stripe for writing to a first memory location of a plurality of solid state drives. A first parity value for the first portion of data of the RAID stripe is calculated. The methods include receiving a second portion of data of the RAID stripe for writing to a second memory location that is different from the first memory location. A second parity value is calculated based upon the second portion of the data and the first parity value Based on successfully writing the second portion of data of the RAID stripe, the first parity value is replaced with the second parity value.

Abstract: A method includes all-or-nothing (AONT) encoding a first data chunk of a plurality of data chunks in accordance with distributed data storage parameters to produce a first set of AONT encoded data pieces, where the distributed data storage parameters include a T number that corresponds to a minimum number AONT encoded data pieces needed to recover a data chunk of the plurality of data chunks, and where the first set of AONT encoded data pieces includes the T number of AONT encoded data pieces. The method further includes facilitating storage of the first set of AONT encoded data pieces in a set of storage units of the storage network, where the set of storage units includes the T number of storage units.

Abstract: A method for execution by a computing device of a storage network begins or continues by generating a plurality of estimated efficiency models associated with a plurality of processing units of the storage network, where an estimated efficiency model of the plurality of estimated efficiency models includes estimated efficiency probabilities, and where the estimated efficiency probabilities correspond to data access request types for a processing unit of the plurality of processing units. The method further includes storing the plurality of estimated efficiency models in memory of the storage network.

Abstract: In one implementation, a system resource is added to a storage system, for a resource-preserving upgrade. An upgrade component is coupled to the storage system as a temporary storage system shelf. Storage drives are moved from the storage system to the upgrade component. One or more storage controllers of the upgrade component are promoted to take over data services from the storage system.

Abstract: A storage cluster includes a plurality of storage nodes. Each of the plurality of storage nodes includes nonvolatile solid-state memory and each of the plurality of storage nodes is configured to cooperate with others of the plurality of storage nodes having differing storage capacities in applying erasure coding. The plurality of storage nodes are configured to distribute the user data and metadata throughout the plurality of storage nodes.

Abstract: Offloading device management responsibilities from a storage device in an array of storage devices, including: retrieving, from the storage device, control information describing the state of one or more memory blocks in the storage device; and performing, in dependence upon the control information, a storage device management operation.

Abstract: A method of operating a remote procedure call cache in a storage cluster is provided. The method includes receiving a remote procedure call at a first storage node having solid-state memory and writing information, relating to the remote procedure call, to a remote procedure call cache of the first storage node. The method includes mirroring the remote procedure call cache of the first storage node in a mirrored remote procedure call cache of a second storage node. A plurality of storage nodes and a storage cluster are also provided.

Abstract: A system and method for performing garbage collection. A system includes a storage medium, a first table including entries which map a virtual address to locations in the storage medium, and a second table with entries which include a reverse mapping of a physical address in a data storage medium to one or more virtual addresses. A storage controller is configured to perform garbage collection. During garbage collection, the controller is configured to identify one or more entries in the second table which correspond to a segment to be garbage collected. In response to determining the first table includes a valid mapping for a virtual address included in an entry of the one of the one or more entries, the controller is configured to copy data from a first location identified in the entry to a second location in the data storage medium, and reclaim the first storage location.