Abstract: A method, computer program product, and computing system for receiving a flush request for a metadata page stored in a storage array of a multi-node storage system. The flush request may be queued on a flush request lock queue on at least one node of the multi-node storage system. One or more flush requests may be processed, via multiple nodes of the multi-node storage system, on the metadata page based upon, at least in part, the flush request lock queue.

Abstract: At least one processing device comprises a processor and a memory coupled to the processor. The at least one processing device is configured to implement adaptive flow control in conjunction with processing of input-output operations in a storage system. The adaptive flow control comprises a first feedback loop in which a window size defining an amount of concurrent processing of the input-output operations in the storage system is adjusted responsive to a measured latency for processing of one or more of the input-output operations. The adaptive flow control further comprises a second feedback loop in which at least one latency threshold used to control adjustment of the window size in the first feedback loop is adjusted. The at least one processing device illustratively comprises at least one processing core of a multi-core storage node of a distributed storage system.

Abstract: A method, computer program product, and computing system for assigning a plurality of unique sequential identifiers to a plurality of tablets in a cache memory system. One or more metadata deltas associated with a metadata page stored in a storage array may be written to the plurality of tablets in the cache memory system. Each metadata delta stored in at least one tablet of the plurality of tablets may be written to the metadata page stored in the storage array, thus defining one or more destage tablets. A largest unique sequential identifier from the plurality of unique sequential identifiers assigned to the one or more destage tablets, may be written to the storage array, thus defining a current tablet identifier for the metadata page.

Abstract: A method of operating a computing device for processing data is provided. The method includes (a) monitoring a set of performance characteristics of the processing of the data; (b) periodically calculating, using a predefined set of coefficients, a linear combination of the monitored set of performance characteristics to yield a combined metric; and (c) upon detecting that the combined metric exceeds a threshold while operating in a first processing mode, transitioning from operating in the first processing mode to operating in a second processing mode. (1) The second processing mode has a higher bandwidth than the first processing mode, and (2) processing of data in the second processing mode is less robust than processing of data in the first processing mode. An apparatus, system, and computer program product for performing a similar method are also provided.

Abstract: Techniques performed by a computing device of storing data in a data storage system are provided. A method includes (a) storing references to write commands within entries of a first chained hash table (CHT), the first CHT being pointed to by a first data structure representative of a logical disk; (b) keeping track of a load factor of the first CHT during operation; and (c) in response to determining that the load factor of the first CHT has transitioned outside of predetermined bounds: (1) creating a second CHT and a second data structure representative of the logical disk, the second CHT being pointed to by the second data structure; (2) linking the second data structure to the first data structure via a linked list; and (3) storing references to new write commands directed at the logical disk within entries of the second CHT rather than the first CHT.

Abstract: Techniques performed by a computing device of storing data in a data storage system are provided. A method includes (a) storing references to write commands within entries of a first chained hash table (CHT), the first CHT being pointed to by a first data structure representative of a logical disk; (b) keeping track of a load factor of the first CHT during operation; and (c) in response to determining that the load factor of the first CHT has transitioned outside of predetermined bounds: (1) creating a second CHT and a second data structure representative of the logical disk, the second CHT being pointed to by the second data structure; (2) linking the second data structure to the first data structure via a linked list; and (3) storing references to new write commands directed at the logical disk within entries of the second CHT rather than the first CHT.

Abstract: A method, computer program product, and computing system for executing a plurality of IO traces on a storage system. At least one vertical flow and at least one horizontal flow associated with the at least one vertical flow may be defined for the plurality of IO traces. A hierarchical representation of the plurality of IO traces may be generated with the at least one vertical flow and the at least one horizontal flow associated with the at least one vertical flow defined for the plurality of IO traces.

Abstract: Processing a read request to read metadata from an entry of a metadata page may include: determining whether the metadata page is cached; responsive to determining the metadata page is cached, obtaining the first metadata from the cached metadata page; responsive to determining the metadata page is not cached, determining whether the requested metadata is in a metadata log of metadata changes stored in a volatile memory; and responsive to determining the metadata is the metadata log of metadata changes stored in the volatile memory, obtaining the requested metadata from the metadata log. Processing a write request that overwrites an existing value of a metadata page with an updated value may include: recording a metadata change in the metadata log that indicates to update the metadata page with the updated value; and performing additional processing during destaging that uses the existing value prior to overwriting it with the updated value.

Abstract: Load balancing may include: receiving I/O workloads of storage server entities that service I/O operations received for logical devices, wherein each logical device has an owner that is one of the storage server entities that processes I/O operations directed to the logical device; determining normalized I/O workloads corresponding to the I/O workloads of the storage server entities; determining, in accordance with utilization criteria, imbalance criteria and the normalized I/O workloads, whether to rebalance the I/O workloads of the storage server entities; and responsive to determining to rebalance the I/O workloads of the storage server entities, performing processing to alleviate a detected I/O workload imbalance between two storage server entities. The processing may include moving logical device from a first storage server entity to a second storage server entity; and transferring ownership of the logical device from the first to the second storage server entity.

Abstract: Processing may be performed in accordance with a policy to assign roles of winner and loser between two nodes. The roles may be used in connection with deadlock resolution processing. A deadlock or potential deadlock may be detected between the two nodes performing processing for two transactions In response to detecting the deadlock or potential deadlock, using a current state may be used to determine whether to perform the deadlock resolution processing to resolve the deadlock or potential deadlock. The current state may indicate whether assignment of the winner and loser roles between the two nodes is in progress. Responsive to the current state indicating that processing is not in progress to assign roles of winner and loser between the two nodes, the current state may be used perform deadlock resolution processing to resolve the deadlock or potential deadlock. The current state may denote which node is the current winner.

Abstract: Techniques for reconstructing or building metadata pages in storage nodes that have a delta-log based architecture. The techniques include walking “up” an ancestor chain of a metadata page, detecting the most recent delta update for a metadata entry of the metadata page, writing the most recent delta update to a location of the metadata entry in the metadata page, setting a bitmap entry corresponding to the location of the metadata entry in the metadata page, detecting a less recent delta update for the metadata entry of the metadata page, and, having previously set the bitmap entry corresponding to the location of the metadata entry in the metadata page, avoiding writing the less recent delta update to the location of the metadata entry in the metadata page. In this way, the need to save in memory the entire ancestor chain of the metadata page can be eliminated.

Abstract: A method comprises maintaining stripe metadata corresponding to stripes of data stored on a plurality of storage devices. The stripe metadata comprises a first timestamp, a second timestamp and a read count for each stripe. The method comprises obtaining a read request associated with a given stripe and having an associated timestamp and determining whether or not a time period for the given stripe has been exceeded. The method comprises updating the stripe metadata corresponding to the given stripe based at least in part on the determination and the associated timestamp and determining a read access frequency of the given stripe based at least in part on the read count, the first timestamp and the second timestamp of the updated stripe metadata. The method comprises performing at least one of a defragmentation process and a rebuild process based at least in part on the determined read access frequency.

Abstract: An apparatus comprises a processing device. The processing device is configured to persistently store metadata pages on a plurality of storage devices. The metadata pages are organized into buckets. The processing device is configured to access a given metadata page based at least in part on a bucket identifier where the given metadata page corresponds to a given logical volume. The bucket identifier comprises a first portion comprising an indication of a given bucket range that corresponds to the given logical volume and a second portion comprising an indication of an offset into the given bucket range that corresponds to a grouping of buckets that correspond to the given logical volume. The grouping of buckets corresponds to the given logical volume. The bucket identifier further comprises a third portion comprising an indication of an offset into the grouping of buckets that corresponds to the bucket comprising the given metadata page.

Abstract: An apparatus comprises a processing device. The processing device is configured to persistently store metadata pages on a plurality of storage devices. The metadata pages are organized into buckets. The processing device is configured to access a given metadata page based at least in part on a bucket identifier where the given metadata page corresponds to a given logical volume. The bucket identifier comprises a first portion comprising an indication of a given bucket range that corresponds to the given logical volume and a second portion comprising an indication of an offset into the given bucket range that corresponds to a grouping of buckets that correspond to the given logical volume. The grouping of buckets corresponds to the given logical volume. The bucket identifier further comprises a third portion comprising an indication of an offset into the grouping of buckets that corresponds to the bucket comprising the given metadata page.

Abstract: A method, computer program product, and computing system for during a high IOPs period, receiving content to be written to a storage system; storing the content to a specific location within a storage device associated with the storage system; updating a temporary map to include an entry that defines the specific location of the content; and during a subsequent period, binding the content with respect to the storage device.

Abstract: Techniques for estimating performance metrics of standalone or clustered storage systems. The techniques include receiving a request from a storage client for an estimated capacity or capability of a storage system to handle a specified workload pattern within a specified periodicity interval, in which the estimated capacity or capability of the storage system is represented by a headroom metric. The techniques further include, in response to the request from the storage client, obtaining a value of the headroom metric for the specified periodicity interval using a performance model characterized by at least a peak load reserve (PLR) metric and a long-term load reserve (LLR) metric, in which the obtained value of the headroom metric corresponds to the minimum of respective values of at least the PLR metric and the LLR metric. The techniques further include upgrading, scaling-up, and/or scaling-out the storage system based on the value of the headroom metric.

Abstract: A storage system in one embodiment comprises storage nodes, an address space, address mapping sub-journals and write cache data sub-journals. Each address mapping sub-journal corresponds to a slice of the address space, is under control of one of the storage nodes and comprises update information corresponding to updates to an address mapping data structure. Each write cache data sub journal is under control of the one of the storage nodes and comprises data pages to be later destaged to the address space. A given storage node is configured to store write cache metadata in a given address mapping sub journal that is under control of the given storage node. The write cache metadata corresponds to a given data page stored in a given write cache data sub-journal that is also under control of the given storage node.

Abstract: In a method of flushing cached data in a data storage system, instances of a working-set structure (WSS) are used over a succession of operating periods to organize cached data for storing to the persistent storage. In each operating period, leaf structures of the WSS are associated with respective address ranges of a specified size. Between operating periods, a structure-tuning operation is performed to adjust the specified size and thereby dynamically adjust a PD-to-leaf ratio of the WSS, including (1) comparing a last-period PD-to-leaf ratio to a predetermined ratio range, (2) when the ratio is below the predetermined ratio range, increasing the specified size for use in a next operating period, and (3) when ratio is above the predetermined ratio range, then decreasing the specified size for use in the next operating period.

Abstract: A method, computer program product, and computer system for acquiring, by a first node, local locks of the first node associated with a metadata log transaction, wherein the first node acquires the local locks of the first node prior to sending a commit message to a second node. The second node may acquire local locks of the second node associated with the metadata log transaction, wherein the second node acquires the local locks of the second node based upon, at least in part, receiving the commit message from the first node.

Abstract: A method, computer program product, and computer system for creating, by a computing device, a child leaf of a snapshot associated with a parent leaf of a volume. At least one entry associated with one of the parent leaf and the child leaf may be added to a delta log. Only delta changes are written to the child leaf. One of a first destaging technique and a second destaging technique may be used for the child leaf.