Abstract: A storage system is provided. The storage system includes a plurality of storage units, each having a controller and solid-state storage memory. The storage system further includes one or more first pathways that couple processing devices of a plurality of storage nodes and is configured to couple to a network external to the storage system and one or more second pathways that couple the plurality of storage nodes to the plurality of storage units, wherein the one or more second pathways enable multiprocessing applications.

Abstract: A system and method comprising: receiving a request to write data stored at a first range of a first volume to a second range of a second volume, where first metadata for the first range of the first volume is associated with a range of physical addresses where the data is stored in the storage system; and responsive to receiving the request: creating second metadata for the second range of the second volume, wherein the second metadata is associated with the range of physical addresses where the data is stored in the storage system; and associating the second volume with the second metadata.

Abstract: Continuing to service a dataset after prevailing in mediation, including: requesting, by a first storage system, mediation from a mediation service that is configured to resolve which storage system continues to service a dataset after the triggering event, wherein the dataset is synchronously replicated by a plurality of storage systems that includes the first storage system and a second storage system; and detaching the second storage system from a plurality of storage systems synchronously replicating the dataset.

Abstract: Prioritizing garbage collection based on the extent to which data is deduplicated, including: determining, for one or more data elements, a number of deduplicated references to each data element; storing, for each of the data elements, the data element in an area of the storage device that contains other data elements with a similar number of deduplicated references; and adjusting a garbage collection schedule for the storage device, wherein garbage collection operations are performed more frequently on areas of the storage device that contain data elements with a relatively low number of deduplicated references.

Abstract: Projecting capacity utilization for snapshots includes identifying one or more data release patterns of a storage system; identifying a snapshot policy; and generating, based on the one or more data release patterns and the snapshot policy, an estimate of an impact of the snapshot policy on a capacity of the storage system.

Abstract: A system, apparatus, and product for allowing access to a partially replicated dataset, including: detecting, by a target storage system, that a dataset will be replicated from a source storage system to the target storage system; exposing, by the target storage system, the dataset for user operations prior to receiving the entire dataset; receiving, by the target storage system, a request to read a portion of the dataset that has not been received by the target storage system; and retrieving, by the target storage system, the portion of the dataset.

Abstract: A method of cache management, the method comprising: identifying, among a plurality of storage items, storage items having an access count above a first threshold to generate a set of storage items; identifying, among the set of storage items, storage items having an updated access count above a second threshold to generate a subset of storage items, wherein, for each storage item, the updated access count is dependent upon a number of accesses subsequent to generating the set of storage items; and adding the storage items of the subset of storage items to a cache.

Abstract: Using machine learning to dynamically select data reduction techniques, including: generating data describing resource usage in a storage system; determining, by a machine learning model and based on the data, a change to one or more data reduction processes of the storage system; and applying the change to the one or more data reduction processes.

Abstract: A storage system performs garbage collection, with data compression, in storage memory. The system obtains hash results from data segments. The system determines similarity of content of data segments, based on the hash results. The system performs data compression of live data of two or more data segments that have similarity of content meeting a similarity threshold. The system writes the compressed live data of the two or more data segments into the storage memory.

Abstract: A storage system switching between mediation models within a storage system, where the switching between mediation models includes: determining, among one or more of the plurality of storage systems, a change in availability of a mediator service, wherein one or more of the plurality of storage systems are configured to request mediation from the mediator service in response to a fault; and communicating, among the plurality of storage systems and responsive to determining the change in availability of the mediator service, a fault response model to be used as an alternate to the mediator service among one or more of the plurality of storage systems.

Abstract: Providing Quality of Service (QOS) for replicating datasets including: receiving, by a target data repository from a source data repository, a checkpoint describing one or more updates to one or more datasets stored in the source data repository and the target data repository; adding, by the target data repository, the checkpoint to a first queue for checkpoints directed to one or more volumes in the target data repository, wherein the first queue is included in a plurality of queues for the target data repository; selecting, by the target data repository, one or more queues from the plurality of queues; and servicing an operation from each of the selected one or more queues.

Abstract: A method for managing processing power in a storage system is provided. The method includes providing a plurality of blades, each of a first subset having a storage node and storage memory, and each of a second, differing subset having a compute-only node. The method includes distributing authorities across the plurality of blades, to a plurality of nodes including at least one compute-only node, wherein each authority has ownership of a range of user data.

Abstract: Methods, computer systems, and computer readable medium are described for low latency reads using cached deduplicated data, including: receiving a request to read data from a storage system; query, using a generated hash value associated with the request to read data, one or more deduplication tables that corresponds to the hash value; and responsive to determining that the one or more deduplication tables includes an entry that corresponds to the hash value, using a mapping contained in the entry to perform the requested to read data, wherein the mapping includes a pointer to a physical location where at least a portion of the data is stored.

Abstract: A method for proactively rebuilding user data in a plurality of storage nodes of a storage cluster is provided. The method includes distributing user data and metadata throughout the plurality of storage nodes such that the plurality of storage nodes can read the user data, using erasure coding, despite loss of two of the storage nodes. The method includes determining that one of the storage nodes is unreachable and determining to rebuild the user data for the one of the storage nodes that is unreachable. The method includes reading the user data across a remainder of the plurality of storage nodes, using the erasure coding and writing the user data across the remainder of the plurality of storage nodes, using the erasure coding. A plurality of storage nodes within a single chassis that can proactively rebuild the user data stored within the storage nodes is also provided.

Abstract: Creating a modifiable cloned image of a dataset, including: generating, based on a dataset stored within a source data repository, a tracking copy of replica data on a target data repository; generating, after receiving an indication to begin accepting modifications to the tracking copy of the replica data, a cloned image of the dataset that is modifiable without modifying the tracking copy of the replica data; and responsive to a storage operation directed to the target data repository, modifying the cloned image of the dataset without modifying the tracking copy of the replica data.

Abstract: A method for proactively rebuilding user data in a plurality of storage nodes of a storage cluster in a single chassis is provided. The method includes distributing user data and metadata throughout the plurality of storage nodes such that the plurality of storage nodes can read the user data, using erasure coding, despite loss of two of the plurality of storage nodes. The method includes determining to rebuild the user data for one of the plurality of storage nodes in the absences of an error condition. The method includes rebuilding the user data for the one of the plurality of storage nodes. A plurality of storage nodes within a single chassis that can proactively rebuild the user data stored within the storage nodes is also provided.

Abstract: A system and method for offset protection data in a RAID array. A computer system comprises client computers and data storage arrays coupled to one another via a network. A data storage array utilizes solid-state drives and Flash memory cells for data storage. A storage controller within a data storage array is configured to store user data in a first page of a first storage device of the plurality of storage devices; generate intra-device protection data corresponding to the user data, and store the intra-device protection data at a first offset within the first page. The controller is further configured to generate inter-device protection data corresponding to the first page, and store the inter-device protection data at a second offset within a second page in a second storage device of the plurality of storage devices, wherein the first offset is different from the second offset.

Abstract: A system, method, and product for flexible RAID layouts in a storage system, including: determining a reliability of an individual storage device of a plurality of storage devices, the individual storage device containing a plurality of portions of a Redundant Array of Independent Disks (RAID) stripe in a storage system, wherein the RAID stripe includes user data and inter-device parity data; detecting a change in the reliability of the individual storage device that contains the portion corresponding to the RAID stripe; and changing an amount of intra-device protection corresponding to the RAID stripe by decreasing, in the RAID stripe, an amount of space used to store the inter-device protection data.

Abstract: Preparing data for deduplication including: generating, by a storage system for a compressed data block, a padded compressed data block by padding the compressed data block to conform to a fixed block size, wherein the fixed block size is greater than a size of the compressed data block; storing, in the storage system, the padded compressed data block beginning at a block boundary of a storage device in the storage system; and performing block-based deduplication on the storage system, wherein the block-based deduplication determines whether the padded compressed data block matches one or more other padded compressed data blocks stored in the storage system.

Abstract: A system and method comprising: receiving a request to write data stored at a first range of a first volume to a second range of a second volume, where first metadata for the first range of the first volume is associated with a range of physical addresses where the data is stored in the storage system; and responsive to receiving the request: creating second metadata for the second range of the second volume, wherein the second metadata is associated with the range of physical addresses where the data is stored in the storage system; and associating the second volume with the second metadata.