Abstract: Ensuring the appropriate utilization of system resources using weighted workload based, time-independent scheduling, including: receiving an I/O request associated with an entity; determining whether an amount of system resources required to service the I/O request is greater than an amount of available system resources in a storage system; responsive to determining that the amount of system resources required to service the I/O request is greater than the amount of available system resources in the storage system: queueing the I/O request in an entity-specific queue for the entity; detecting that additional system resources in the storage system have become available; and issuing an I/O request from an entity-specific queue for an entity that has a highest priority, where a priority for each entity is determined based on the amount of I/O requests associated with the entity and a weighted proportion of resources designated for use by the entity.

Abstract: A system, apparatus, and product for allowing access to a partially replicated dataset, including: detecting, by a target storage system, that a dataset will be replicated from a source storage system to the target storage system; exposing, by the target storage system, the dataset for user operations prior to receiving the entire dataset; receiving, by the target storage system, a request to read a portion of the dataset that has not been received by the target storage system; and retrieving, by the target storage system, the portion of the dataset.

Abstract: A storage network method begins by determining to initiate periodic integrity checks for a plurality of data objects stored in the storage network, where a data object is dispersed error encoded in accordance with dispersed error encoding parameters to produce a plurality of sets of encoded data slices. The method continues by determining a frequency for performing the periodic integrity checks and determining, based on the frequency for performing the periodic integrity checks, a portion of a data object for a first integrity check, where the portion includes one or more encoded data slices of the plurality of sets of encoded data slices. The method then continues by issuing a list data request for the portion of the data object, receiving a list data response for the portion of the one or more data objects and determining whether the list data response indicates a data error for the portion of the one or more data objects.

Abstract: A computing device includes an interface configured to interface and communicate with a storage network, a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. A computing device receives a data access request for an encoded data slice associated with a data object, determines whether the encoded data slice is stored in the first memory and in response to a determination that the encoded data slice is not stored in the first memory, issues another data access request for the encoded data slice to a second memory, where the first memory includes access characteristics that are faster than the second memory. When a data access response including the encoded data slice is received from the second memory, a response including the encoded data slice is transmitted.

Abstract: A processor-based method for locating data and metadata closely together in a storage system is provided. The method includes writing a first range of a file and a first metadata relating to attributes of the file into at least one segment controlled by a first authority of the file. The method includes delegating, by the first authority, a second authority for a second range of the file, and writing the second range of the file and second metadata relating to the attributes of the file into at least one segment controlled by the second authority.

Abstract: A storage network operates by: identifying a set of data objects having a common trait from a plurality of data objects for storage via the storage network; combining the set of data objects to produce a concatenated data object; encoding the concatenated data object in accordance with a dispersed encoding function to produce a set of encoded data blocks; and generating a mapping of the set of data objects to the set of encoded data blocks, wherein the mapping facilitates retrieval from the storage network of individual data objects of the set of data objects.

Abstract: A storage system is provided. The storage system includes a plurality of storage nodes, each of the plurality of storage nodes having a plurality of storage units with storage memory. The system includes a first network coupling the plurality of storage nodes and a second network coupled to at least a subset of the plurality of storage units of each of the plurality of storage nodes such that one of the plurality of storage units of a first one of the plurality of storage nodes can initiate or relay a command to one of the plurality of storage units of a second one of the plurality of storage nodes via the second network without the command passing through the first network.

Abstract: Transitioning leadership in a cluster of nodes, including: initiating, by two or more nodes among a cluster of nodes, a leadership transition, wherein: a first node transmits a first secret key identifier to each of the other nodes in the cluster of nodes; and a second node transmits a second secret key identifier to each of the other nodes in the cluster of nodes; updating, by each node and based at least in part on a resolution policy, the current secret key identifier to be the second secret key identifier instead of the first secret key identifier; and transitioning, based at least in part on the second secret key identifier being selected to be the current secret key identifier, the second node to be a leader node of the cluster of nodes.

Abstract: A storage system performs data replication with a recovery point objective (RPO). The storage system replicates data at intervals through data transfers over a network. The storage system determines bandwidth of the network. The storage system determines the intervals for replicating the data, based on size of data transfers, network bandwidth, and the recovery point objective.

Abstract: A method is provided. The method includes receiving a set of data blocks to be stored in a storage system. The storage system includes a plurality of non-volatile memory modules. The method also includes generating a set of segments based on the set of data blocks. A respective segment comprising portions of one or more erase blocks. The method further includes writing the set of segments to the non-volatile memory modules based on orderings of the portions of the one or more erase blocks.

Abstract: An illustrative method includes detecting a request to perform an overwrite operation with respect to a non-header portion of a file stored by a storage system and determining, based on the detecting the request, that data stored by the storage system is possibly being targeted by a security threat.

Abstract: A method for proactively rebuilding user data in a plurality of storage nodes of a storage cluster is provided. The method includes distributing user data and metadata throughout the plurality of storage nodes such that the plurality of storage nodes can read the user data, using erasure coding, despite loss of two of the storage nodes. The method includes determining that one of the storage nodes is unreachable and determining to rebuild the user data for the one of the storage nodes that is unreachable. The method includes reading the user data across a remainder of the plurality of storage nodes, using the erasure coding and writing the user data across the remainder of the plurality of storage nodes, using the erasure coding. A plurality of storage nodes within a single chassis that can proactively rebuild the user data stored within the storage nodes is also provided.

Abstract: Managing disaster recovery to a cloud computing environment, including: receiving recovery objectives associated with a dataset that is stored in a primary storage system; creating a cloud-based storage system; suspending the cloud-based storage system; and resuming, periodically and based on the recovery objectives, the cloud-based storage system, including refreshing a copy of the dataset that is maintained by the cloud-based storage system.

Abstract: A method of processing transactions associated with a command in a storage system is provided. The method includes receiving, at a first authority of the storage system, a command relating to user data. The method includes sending a transaction of the command, from the first authority to a second authority of the storage system, wherein a token accompanies the transaction and writing data in accordance with the transaction as permitted by the token into a partition that is allocated to the second authority in a storage device of the storage system.

Abstract: A system and method of uniform sampling in an append-only database to determine temporal distribution of data.

Abstract: Protecting an encryption key for data stored in a storage system that includes a plurality of storage devices, including: reading, from at least a majority of the storage devices, a portion of an apartment key; reconstructing the apartment key using the portions of the apartment key read by the majority of the storage devices; unlocking the main portion of each of the storage devices utilizing the apartment key; reading, from the main portion of one of the storage devices, a portion of a third-party resource access key; requesting, from the third-party resource utilizing the third-party resource access key, an encryption key; receiving, from the third-party resource, the encryption key; and decrypting the data stored on the storage devices utilizing the encryption key.

Abstract: A storage system has NVRAM (nonvolatile random-access memory), storage memory that includes SLC (single level cell) flash memory and QLC (quad level cell) flash memory, and a processor. The processor performs a method that includes selecting one of a plurality of write paths for incoming data, and writing the incoming data via the selected write path. A first write path includes writing to NVRAM, writing from NVRAM to SLC flash memory and writing from SLC flash memory to QLC flash memory. A second write path includes writing to NVRAM and writing from NVRAM to QLC flash memory, bypassing SLC flash memory. A third write path includes writing to SLC flash memory, bypassing NVRAM, and writing from SLC flash memory to QLC flash memory.

Abstract: A hyperscale artificial intelligence and machine learning infrastructure includes a plurality of racks, where: at least one or more of the racks include one or more GPU servers; at least one or more of the racks include one or more storage systems; each of the racks include one or more switches coupled to at least one switch in another rack; and the one or more GPU servers are configured to execute one or more artificial intelligence or machine learning applications, wherein data stored within the one or more storage systems is used as input to the one or more artificial intelligence or machine learning applications.

Abstract: A storage system performs garbage collection, with data compression, in storage memory. The system obtains hash results from data segments. The system determines similarity of content of data segments, based on the hash results. The system performs data compression of live data of two or more data segments that have similarity of content meeting a similarity threshold. The system writes the compressed live data of the two or more data segments into the storage memory.

Abstract: Providing storage tailored for a storage consuming application, including: identifying, for an application that utilizes storage resources within a cloud-based storage system, one or more storage performance characteristics associated with the application; comparing the storage performance characteristics of the application that were identified with storage performance characteristics of storage resources of one or more cloud-based storage systems; and selecting, based on the comparing, one or more storage resources within the one or more cloud-based storage systems to provide storage services to the application.