Abstract: A method, computer program product, and computing system for determining a respective past activity level associated with a plurality of storage objects. The plurality of storage objects are divided into a plurality of storage object groups based upon, at least in part, the respective past activity level associated with the plurality of storage objects. Input/output (IO) performance data for a first storage object group of the plurality of storage object groups is forecasted using a first machine learning model. IO performance data for a second storage object group of the plurality of storage object groups is forecasted using a statistical method.

Abstract: A technique for performing load balancing between storage nodes includes generating a first performance metric for volumes accessed through a first storage node and generating a second performance metric for volumes accessed through a second storage node. The volumes accessed through the first storage node include a set of volumes that belong to a NAS (network-attached storage) file server hosted by the first storage node. In response to detecting a load imbalance based on the first performance metric and the second performance metric, the technique further includes moving the NAS file server from the first storage node to the second storage node and hosting the set of volumes that belong to the NAS file server from the second storage node.

Abstract: A method, computer program product, and computing system for processing a plurality of input/output (IO) operations on a plurality of storage objects of a storage system. The plurality of storage objects may be divided into a plurality of storage object groups based upon, at least in part, the plurality of IO operations processed on the plurality of storage objects. Each storage object group may be associated with an IO machine learning model selected from a plurality of IO machine learning models, thus defining a plurality of storage object group-specific IO machine learning models. IO performance data may be forecasted for the plurality of storage objects using the plurality of storage object group-specific IO machine learning models.

Abstract: A data storage system includes a tuner that obtains data samples for data storage operations of workloads and calculates feature measures for a set of features of the data storage operations over aggregation intervals of an operating period. It further (1) applies a cluster analysis to the feature measures to define a set of clusters, and assigns the feature measures to the clusters, and (2) applies a classification analysis to the feature measures labelled by their clusters to identify dominating features of each cluster, and generates workload profiles for the clusters based on the dominating features, and then automatically adjusts configurable processing mechanisms (e.g., caching or tiering) based on the workload profiles and performance or efficiency goals.

Abstract: Techniques for forecasting temperatures of storage objects in a storage system using machine learning (ML). The techniques can include forecasting at least one temperature of a storage object using at least one ML model, modifying storage of the storage object based on the at least one temperature of the storage object, and, having modified storage of the storage object, obtaining at least one performance metric associated with the storage object. The techniques can further include, based on the performance metric(s), varying a frequency of forecasting the at least one temperature of the storage object, retraining the at least one ML model used in forecasting the at least one temperature, and/or adjusting at least one operational parameter of the system. The techniques provide increased accuracy over known statistical approaches to forecasting temperatures of storage objects, leading to increased performance gains in terms of IO latency, IO operations per second, and bandwidth.

Abstract: A method, computer program product, and computing system for processing historical input/output (IO) performance data associated with one or more storage objects of a storage system. A smoothing model may be applied on at least a portion of the historical IO performance data to generate forecast IO performance data. The forecast IO performance data may be compared to observed IO performance data to generate one or more performance differentials. A normal IO performance range may be generated based upon, at least in part, the one or more performance differentials. One or more IO performance anomalies may be detected based upon, at least in part, the normal IO performance range.

Abstract: A method, computer program product, and computing system for processing historical input/output (IO) performance data associated with one or more storage objects of a storage system. A plurality of IO modeling systems may be trained using the historical IO performance data. Modeling performance information may be determined for the plurality of IO modeling systems across the historical IO performance data. A forecast score may be determined for each IO modeling system based on the modeling performance information for the plurality of IO modeling systems. A subset of the plurality of IO modeling systems may be selected based upon the forecast score for each IO modeling system. The at least one IO modeling system may be trained using the historical IO performance data. IO performance data may be forecasted using the at least one trained IO modeling system from the subset of the plurality of IO modeling systems.

Abstract: Techniques for detecting impactful performance anomalies in storage systems. The techniques include obtaining, for each performance metric of a storage system's workload, a training set of series diffs based on a threshold. Each diff represents a difference between an observed value from an observed set of time series values for the performance metric and a normalized value from a corresponding set of normalized time series values. The techniques include applying the training set of series diffs for each performance metric to an unsupervised anomaly detection algorithm and running the algorithm to identify potentially impactful anomalies in a multi-dimensional search space. The techniques include identifying impactful anomalies from among the potentially impactful anomalies that exceed an anomaly score.

Abstract: Techniques are used for balancing load on a storage system according to multiple variables. The techniques may be used to provide, among other things, defining, across at least two variables, a balance constraint for a load on a storage system. Among a set of transfers of volumes from one node to another node in the storage system, a transfer of a volume that minimizes the distance between the load and an ideal balanced state of the storage system is identified. The identified transfer of a volume is added to a combination of transfers of volumes. Whether the combination of transfers of volumes meets the balance constraint is determined. If the combination meets the balance constraint, the combination is selected as a solution to balance the load.

Abstract: A system and method improve caching efficiency in a data storage system by performing machine learning processes on metadata relating to extents of data blocks, rather than individual blocks themselves. Thus, once the storage devices are divided into extents, various metadata regarding access to the blocks within each extent are aggregated, and per-extent features are extracted. These features are used to train a data regression model that is subsequently used to infer a most likely “hotness” value for each extent at a future time. These predicted values, which may be further classified as e.g. “hot”, “warm”, and “cold” using thresholds, are used to implement the cache replacement policy. Embodiments scale to large and multi-layered caches, and may avoid common caching problems like thrashing, by adjusting the extent size. Policy goal functions may be optimized by dynamically adjusting the classification thresholds.

Abstract: A method, computer program product, and computing system for receiving a plurality of input/output (IO) requests at a storage system. One or more IO properties may be extracted from the plurality of IO requests. The one or more IO properties may be processed, using one or more machine learning models, to define an access temperature value for one or more storage objects of the storage system. The one or more storage objects may be tiered between a plurality of storage tiers of the storage system, based upon, at least in part, the access temperature values defined for the one or more storage objects of the storage system and one or more tiering policies.

Abstract: IO traces on a high-speed memory that provides temporary storage for multiple storage volumes are stored in a trace buffer. IO operations on different storage volume are considered separate workloads on the high-speed memory. Periodically, the IO traces are processed to extract workload features for each workload. The workload features are stored in a feature matrix, and the workload features from multiple IO trace buffer processing operations are aggregated over time. A HDBSCAN unsupervised clustering machine learning process is used to create a set of four workload clusters and an outlier cluster. A dominant feature of each workload cluster is used to set a policy for the workload cluster. IO percentages for clusters with the same policies are used to set minimum sizes for policy regions in the high-speed memory. Histograms based on the workloads are used to determine segmentation rules specifying slot sizes for the policy regions.

Abstract: Data units of a dataset may be compressed by clustering the data units into clusters, selecting a reference unit for each unit cluster, and compressing data units of each unit cluster using the reference unit of the unit cluster as a dictionary. The computational efficiency of the clustering algorithm may be improved by not applying it to data units themselves, but rather to hash values of the data units, where the hash values have a much smaller size than the data units. The hash function may be a locality-sensitive hash (LSH) function. The reference unit of a cluster may be determined in any of a variety of ways, for example, by selecting a centroid or exemplar of the cluster. Clusters, including their references values, may be indexed in a cluster index (e.g., a Faiss index), which may be searched to assign future added or modified data units to clusters.

Abstract: Techniques are used for balancing load on a storage system according to multiple variables. The techniques may be used to provide, among other things, defining, across at least two variables, a balance constraint for a load on a storage system. Among a set of transfers of volumes from one node to another node in the storage system, a transfer of a volume that minimizes the distance between the load and an ideal balanced state of the storage system is identified. The identified transfer of a volume is added to a combination of transfers of volumes. Whether the combination of transfers of volumes meets the balance constraint is determined. If the combination meets the balance constraint, the combination is selected as a solution to balance the load.

Abstract: A method, computer program product, and computing system for receiving a plurality of input/output (IO) requests at a storage system. One or more IO properties may be extracted from the plurality of IO requests. The one or more IO properties may be processed, using one or more machine learning models, to define an access temperature value for one or more storage objects of the storage system. The one or more storage objects may be tiered between a plurality of storage tiers of the storage system, based upon, at least in part, the access temperature values defined for the one or more storage objects of the storage system and one or more tiering policies.

Abstract: Load balancing may include: receiving I/O workloads of storage server entities that service I/O operations received for logical devices, wherein each logical device has an owner that is one of the storage server entities that processes I/O operations directed to the logical device; determining normalized I/O workloads corresponding to the I/O workloads of the storage server entities; determining, in accordance with utilization criteria, imbalance criteria and the normalized I/O workloads, whether to rebalance the I/O workloads of the storage server entities; and responsive to determining to rebalance the I/O workloads of the storage server entities, performing processing to alleviate a detected I/O workload imbalance between two storage server entities. The processing may include moving logical device from a first storage server entity to a second storage server entity; and transferring ownership of the logical device from the first to the second storage server entity.

Abstract: A method, computer program product, and computing system for receiving a first set of input/output (IO) requests for one or more storage objects. One or more IO properties may be extracted from the first set of IO requests. Metadata may be associated with the one or more storage objects using one or more machine learning models based upon, at least in part, the one or more IO properties extracted from the first set of IO requests, thus defining storage object metadata. One or more IO processing rules may be enabled based upon, at least in part, the storage object metadata. A subsequent set of IO requests may be received. Processing of the subsequent set of IO requests on the one or more storage objects may be optimized based upon, at least in part, the storage object metadata and the one or more IO processing rules associated with the one or more storage objects.

Abstract: Techniques for estimating performance metrics of standalone or clustered storage systems. The techniques include receiving a request from a storage client for an estimated capacity or capability of a storage system to handle a specified workload pattern within a specified periodicity interval, in which the estimated capacity or capability of the storage system is represented by a headroom metric. The techniques further include, in response to the request from the storage client, obtaining a value of the headroom metric for the specified periodicity interval using a performance model characterized by at least a peak load reserve (PLR) metric and a long-term load reserve (LLR) metric, in which the obtained value of the headroom metric corresponds to the minimum of respective values of at least the PLR metric and the LLR metric. The techniques further include upgrading, scaling-up, and/or scaling-out the storage system based on the value of the headroom metric.

Abstract: Techniques are used for balancing load on a storage system according to multiple variables. The techniques may be used to provide, among other things, defining, across at least two variables, a balance constraint for a load on a storage system. Among a set of transfers of volumes from one node to another node in the storage system, a transfer of a volume that minimizes the distance between the load and an ideal balanced state of the storage system is identified. The identified transfer of a volume is added to a combination of transfers of volumes. Whether the combination of transfers of volumes meets the balance constraint is determined. If the combination meets the balance constraint, the combination is selected as a solution to balance the load.

Abstract: A system and method improve caching efficiency in a data storage system by performing machine learning processes on metadata relating to extents of data blocks, rather than individual blocks themselves. Thus, once the storage devices are divided into extents, various metadata regarding access to the blocks within each extent are aggregated, and per-extent features are extracted. These features are used to train a data regression model that is subsequently used to infer a most likely “hotness” value for each extent at a future time. These predicted values, which may be further classified as e.g. “hot”, “warm”, and “cold” using thresholds, are used to implement the cache replacement policy. Embodiments scale to large and multi-layered caches, and may avoid common caching problems like thrashing, by adjusting the extent size. Policy goal functions may be optimized by dynamically adjusting the classification thresholds.