Abstract: A system, apparatus, and product for allowing access to a partially replicated dataset, including: detecting, by a target storage system, that a dataset will be replicated from a source storage system to the target storage system; exposing, by the target storage system, the dataset for user operations prior to receiving the entire dataset; receiving, by the target storage system, a request to read a portion of the dataset that has not been received by the target storage system; and retrieving, by the target storage system, the portion of the dataset.

Abstract: A method of cache management, the method comprising: identifying, among a plurality of storage items, storage items having an access count above a first threshold to generate a set of storage items; identifying, among the set of storage items, storage items having an updated access count above a second threshold to generate a subset of storage items, wherein, for each storage item, the updated access count is dependent upon a number of accesses subsequent to generating the set of storage items; and adding the storage items of the subset of storage items to a cache.

Abstract: Using machine learning to dynamically select data reduction techniques, including: generating data describing resource usage in a storage system; determining, by a machine learning model and based on the data, a change to one or more data reduction processes of the storage system; and applying the change to the one or more data reduction processes.

Abstract: A storage system performs garbage collection, with data compression, in storage memory. The system obtains hash results from data segments. The system determines similarity of content of data segments, based on the hash results. The system performs data compression of live data of two or more data segments that have similarity of content meeting a similarity threshold. The system writes the compressed live data of the two or more data segments into the storage memory.

Abstract: A storage system switching between mediation models within a storage system, where the switching between mediation models includes: determining, among one or more of the plurality of storage systems, a change in availability of a mediator service, wherein one or more of the plurality of storage systems are configured to request mediation from the mediator service in response to a fault; and communicating, among the plurality of storage systems and responsive to determining the change in availability of the mediator service, a fault response model to be used as an alternate to the mediator service among one or more of the plurality of storage systems.

Abstract: Providing Quality of Service (QOS) for replicating datasets including: receiving, by a target data repository from a source data repository, a checkpoint describing one or more updates to one or more datasets stored in the source data repository and the target data repository; adding, by the target data repository, the checkpoint to a first queue for checkpoints directed to one or more volumes in the target data repository, wherein the first queue is included in a plurality of queues for the target data repository; selecting, by the target data repository, one or more queues from the plurality of queues; and servicing an operation from each of the selected one or more queues.

Abstract: A method for managing processing power in a storage system is provided. The method includes providing a plurality of blades, each of a first subset having a storage node and storage memory, and each of a second, differing subset having a compute-only node. The method includes distributing authorities across the plurality of blades, to a plurality of nodes including at least one compute-only node, wherein each authority has ownership of a range of user data.

Abstract: Methods, computer systems, and computer readable medium are described for low latency reads using cached deduplicated data, including: receiving a request to read data from a storage system; query, using a generated hash value associated with the request to read data, one or more deduplication tables that corresponds to the hash value; and responsive to determining that the one or more deduplication tables includes an entry that corresponds to the hash value, using a mapping contained in the entry to perform the requested to read data, wherein the mapping includes a pointer to a physical location where at least a portion of the data is stored.

Abstract: A method for proactively rebuilding user data in a plurality of storage nodes of a storage cluster is provided. The method includes distributing user data and metadata throughout the plurality of storage nodes such that the plurality of storage nodes can read the user data, using erasure coding, despite loss of two of the storage nodes. The method includes determining that one of the storage nodes is unreachable and determining to rebuild the user data for the one of the storage nodes that is unreachable. The method includes reading the user data across a remainder of the plurality of storage nodes, using the erasure coding and writing the user data across the remainder of the plurality of storage nodes, using the erasure coding. A plurality of storage nodes within a single chassis that can proactively rebuild the user data stored within the storage nodes is also provided.

Abstract: A method for proactively rebuilding user data in a plurality of storage nodes of a storage cluster in a single chassis is provided. The method includes distributing user data and metadata throughout the plurality of storage nodes such that the plurality of storage nodes can read the user data, using erasure coding, despite loss of two of the plurality of storage nodes. The method includes determining to rebuild the user data for one of the plurality of storage nodes in the absences of an error condition. The method includes rebuilding the user data for the one of the plurality of storage nodes. A plurality of storage nodes within a single chassis that can proactively rebuild the user data stored within the storage nodes is also provided.

Abstract: Creating a modifiable cloned image of a dataset, including: generating, based on a dataset stored within a source data repository, a tracking copy of replica data on a target data repository; generating, after receiving an indication to begin accepting modifications to the tracking copy of the replica data, a cloned image of the dataset that is modifiable without modifying the tracking copy of the replica data; and responsive to a storage operation directed to the target data repository, modifying the cloned image of the dataset without modifying the tracking copy of the replica data.

Abstract: A system and method for offset protection data in a RAID array. A computer system comprises client computers and data storage arrays coupled to one another via a network. A data storage array utilizes solid-state drives and Flash memory cells for data storage. A storage controller within a data storage array is configured to store user data in a first page of a first storage device of the plurality of storage devices; generate intra-device protection data corresponding to the user data, and store the intra-device protection data at a first offset within the first page. The controller is further configured to generate inter-device protection data corresponding to the first page, and store the inter-device protection data at a second offset within a second page in a second storage device of the plurality of storage devices, wherein the first offset is different from the second offset.

Abstract: A system, method, and product for flexible RAID layouts in a storage system, including: determining a reliability of an individual storage device of a plurality of storage devices, the individual storage device containing a plurality of portions of a Redundant Array of Independent Disks (RAID) stripe in a storage system, wherein the RAID stripe includes user data and inter-device parity data; detecting a change in the reliability of the individual storage device that contains the portion corresponding to the RAID stripe; and changing an amount of intra-device protection corresponding to the RAID stripe by decreasing, in the RAID stripe, an amount of space used to store the inter-device protection data.

Abstract: Preparing data for deduplication including: generating, by a storage system for a compressed data block, a padded compressed data block by padding the compressed data block to conform to a fixed block size, wherein the fixed block size is greater than a size of the compressed data block; storing, in the storage system, the padded compressed data block beginning at a block boundary of a storage device in the storage system; and performing block-based deduplication on the storage system, wherein the block-based deduplication determines whether the padded compressed data block matches one or more other padded compressed data blocks stored in the storage system.

Abstract: A system and method comprising: receiving a request to write data stored at a first range of a first volume to a second range of a second volume, where first metadata for the first range of the first volume is associated with a range of physical addresses where the data is stored in the storage system; and responsive to receiving the request: creating second metadata for the second range of the second volume, wherein the second metadata is associated with the range of physical addresses where the data is stored in the storage system; and associating the second volume with the second metadata.

Abstract: End-to-end encryption in a storage system with multi-tenancy, includes: performing deduplication on a first tenant dataset, the first tenant dataset including data encrypted using a first storage system encryption key; and performing deduplication on a second tenant dataset, the second tenant dataset including data encrypted using a second storage system encryption key, where deduplication is not performed between the first and second tenant datasets.

Abstract: Intra-block error correction including generating reconstructed sub-portions of a portion of a dataset that is distributed into a plurality of portions that are stored on different storage devices of the storage system, wherein the dataset indicates a first number of errors that is greater than a second number of parity data portions of the dataset, determining that a sub-portion of the dataset does not match a corresponding reconstructed sub-portion of the dataset; and based on the determination, replacing the sub-portion of the dataset with the reconstructed sub-portion of the dataset.

Abstract: Supporting multiple replication schemes across distinct network layers, including: replicating, over a first type of network messaging layer, data between a first storage system and a second storage system; selecting a different messaging layer for data replication; and replicating, over a second type of network messaging layer, data between the first storage system and the second storage system.

Abstract: Replicating data using inferred trust, including: receiving, by a first storage system from a computing device, data encrypted using a first encryption key; decrypting, by the first storage system, the encrypted data using the first encryption key; encrypting, by the first storage system, the decrypted data using a second encryption key; storing, on the first storage system, the data encrypted using the second encryption key; sending, from the first storage system to the second storage system, the data; and servicing, by the second storage system, an input/output (‘I/O’) operation directed to the data.

Abstract: A method for deleting a set of keys from a storage server is provided. The method includes generating a probabilistic data structure for a first set of keys and for each key in a second set of keys, determining whether a key of the second set of keys is found in the probabilistic data structure. The method includes identifying the key as a candidate for deletion if the key is not found in the probabilistic data structure. A system is also provided.