Abstract: Systems, methods, and machine-readable media for monitoring a storage system and assigning performance service levels to workloads running on nodes within a cluster are disclosed. A performance manager may estimate the performance demands of each workload within the cluster and assign a performance service level to each workload according to the performance requirements of the workload, and further taking into account an overall budgeting framework. The estimates are performed using historical performance data for each workload. A performance service level may include a service level object, a service level agreement, and latency parameters. These parameters may provide a ceiling to the number of operations per second that a workload may use without guaranteeing the use of the operations per second, a guaranteed number of operations per second that a workload may use before being throttled, and define the permitted delay in completing a request to the workload.

Abstract: Systems, methods, and machine-readable media for monitoring a storage system and correcting demand imbalances among nodes in a cluster are disclosed. A performance manager for the storage system may detect performance imbalances that occur over a period of time. When operating below an optimal performance capacity, the manager may cause a volume to be moved from a node with a high load to a node with a lower load to achieve a preventive result. When operating at or near optimal performance capacity, the manager may cause a QOS limit to be imposed to prevent the workload from exceeding the performance capacity, to achieve a proactive result. When operating abnormally, the manager may cause a QOS limit to be imposed to throttle the workload to bring the node back within the optimal performance capacity of the node, to achieve a reactive result. These actions may be performed independently, or in cooperation.

Abstract: Systems, methods, and machine-readable media for monitoring a storage system and assigning performance service levels to workloads running on nodes within a cluster are disclosed. A performance manager may estimate the performance demands of each workload within the cluster and assign a performance service level to each workload according to the performance requirements of the workload, and further taking into account an overall budgeting framework. The estimates are performed using historical performance data for each workload. A performance service level may include a service level object, a service level agreement, and latency parameters. These parameters may provide a ceiling to the number of operations per second that a workload may use without guaranteeing the use of the operations per second, a guaranteed number of operations per second that a workload may use before being throttled, and define the permitted delay in completing a request to the workload.

Abstract: Systems, methods, and machine-readable media for monitoring a storage system and correcting demand imbalances among nodes in a cluster are disclosed. A performance manager for the storage system may detect performance imbalances that occur over a period of time. When operating below an optimal performance capacity, the manager may cause a volume to be moved from a node with a high load to a node with a lower load to achieve a preventive result. When operating at or near optimal performance capacity, the manager may cause a QOS limit to be imposed to prevent the workload from exceeding the performance capacity, to achieve a proactive result. When operating abnormally, the manager may cause a QOS limit to be imposed to throttle the workload to bring the node back within the optimal performance capacity of the node, to achieve a reactive result. These actions may be performed independently, or in cooperation.

Abstract: Systems, methods, and machine-readable media for monitoring a storage system and assigning performance service levels to workloads running on nodes within a cluster are disclosed. A performance manager may estimate the performance demands of each workload within the cluster and assign a performance service level to each workload according to the performance requirements of the workload, and further taking into account an overall budgeting framework. The estimates are performed using historical performance data for each workload. A performance service level may include a service level object, a service level agreement, and latency parameters. These parameters may provide a ceiling to the number of operations per second that a workload may use without guaranteeing the use of the operations per second, a guaranteed number of operations per second that a workload may use before being throttled, and define the permitted delay in completing a request to the workload.

Abstract: Systems, methods, and machine-readable media for monitoring a storage system and correcting demand imbalances among nodes in a cluster are disclosed. A performance manager for the storage system may detect performance imbalances that occur over a period of time. When operating below an optimal performance capacity, the manager may cause a volume to be moved from a node with a high load to a node with a lower load to achieve a preventive result. When operating at or near optimal performance capacity, the manager may cause a QOS limit to be imposed to prevent the workload from exceeding the performance capacity, to achieve a proactive result. When operating abnormally, the manager may cause a QOS limit to be imposed to throttle the workload to bring the node back within the optimal performance capacity of the node, to achieve a reactive result. These actions may be performed independently, or in cooperation.

Abstract: Methods and systems for inter-cluster storage system monitoring and analysis are provided. The method includes monitoring a non-volatile memory delay center for a first storage cluster having a first node and a second node configured to operate as a first high availability pair, where data for a write request to write data to the first node is also written to the second node as well as to a second cluster having a third node and a fourth node, where the third node and the fourth node are also configured to operate as a second high availability pair to store the data for the write request at one or both of the third and fourth node. The non-volatile memory delay center is used to monitor and detect latency due to any delay caused by a non-volatile memory of the first node used as a write cache.

Abstract: Methods and systems for monitoring quality of service (QOS) data for a plurality of storage volumes are provided. QOS data is collected for the plurality of storage volumes and includes a response time in which each of the plurality of storage volumes respond to an input/output (I/O) request. An expected range for future QOS data based on the collected QOS data is generated. The process then determines a deviation of each potential bully storage volume of a resource used by any victim storage volume, where the deviation of each bully storage volume is based on a number of current I/O requests (IOPS) that are processed by each potential bully storage volume, a forecasted value of TOPS and a predicted upper threshold TOPS value for each potential bully storage volume; and filters the potential bully storage volumes based on an impact of each potential bully storage volume.

Abstract: Methods and systems for monitoring quality of service (QOS) data for a plurality of storage volumes from a storage operating system of a storage system are provided. A performance manager collects the QOS data from the storage operating system and the QOS data includes a response time in which each of the plurality of storage volumes respond to an input/output (I/O) request. An expected range for future QOS data is generated based on the collected QOS data. The QOS data is monitored for each storage volume for determining whether a current QOS data for each storage volume is within the expected range.

Abstract: Methods and systems for monitoring quality of service (QOS) data for a plurality of storage volumes are provided. QOS data is collected for the plurality of storage volumes and includes a response time in which each of the plurality of storage volumes respond to an input/output (I/O) request. The process determines an average of N collected QOS data points at any given time; and iteratively analyzes each QOS data point to detect if a step-up or a step-down function has occurred, where a step-up function represents an unpredictable increase in value of a data point and a step-down function is an unpredictable decrease in value of the data point. A subset of the N QOS data points based on when the step-up function or step-down function occurs is selected for analysis and an expected range for future QOS data based on the subset of the N QOS data points is generated.

Abstract: Methods and systems for identifying a victim storage volume from among a plurality of storage volumes based on a comparison of current Quality of Service (QOS) data with a dynamic threshold value that is based on historical QOS collected data for the plurality of storage volumes are provided. A performance manager collects the current and historical QOS data from a storage operating system of the storage system, which includes a response time in which each of the plurality of storage volumes respond to an input/output (I/O) request. The current and historical QOS data for the resources used by the victim storage volume are retrieved and compared with the current QOS data of each resource to an expected range based on the historical QOS data. Another storage volume is identified as a bully when its usage of a resource in contention contributes to creating the victim storage volume.

Abstract: Methods and systems for inter-cluster storage system monitoring and analysis are provided. The method includes monitoring a non-volatile memory delay center for a first storage cluster having a first node and a second node configured to operate as a first high availability pair, where data for a write request to write data to the first node is also written to the second node as well as to a second cluster having a third node and a fourth node, where the third node and the fourth node are also configured to operate as a second high availability pair to store the data for the write request at one or both of the third and fourth node. The non-volatile memory delay center is used to monitor and detect latency due to any delay caused by a non-volatile memory of the first node used as a write cache.

Abstract: Methods and systems for monitoring quality of service (QOS) data for a plurality of storage volumes from a storage operating system of a storage system are provided. A performance manager collects the QOS data from the storage operating system and the QOS data includes a response time in which each of the plurality of storage volumes respond to an input/output (I/O) request. An expected range for future QOS data is generated based on the collected QOS data. The QOS data is monitored for each storage volume for determining whether a current QOS data for each storage volume is within the expected range.

Abstract: Methods and systems for monitoring quality of service (QOS) data for a plurality of storage volumes are provided. QOS data is collected for the plurality of storage volumes and includes a response time in which each of the plurality of storage volumes respond to an input/output (I/O) request. An expected range for future QOS data based on the collected QOS data is generated. The process then determines a deviation of each potential bully storage volume of a resource used by any victim storage volume, where the deviation of each bully storage volume is based on a number of current I/O requests (IOPS) that are processed by each potential bully storage volume, a forecasted value of TOPS and a predicted upper threshold TOPS value for each potential bully storage volume; and filters the potential bully storage volumes based on an impact of each potential bully storage volume.

Abstract: Methods and systems for monitoring quality of service (QOS) data for a plurality of storage volumes are provided. QOS data is collected for the plurality of storage volumes and includes a response time in which each of the plurality of storage volumes respond to an input/output (I/O) request. The process determines an average of N collected QOS data points at any given time; and iteratively analyzes each QOS data point to detect if a step-up or a step-down function has occurred, where a step-up function represents an unpredictable increase in value of a data point and a step-down function is an unpredictable decrease in value of the data point. A subset of the N QOS data points based on when the step-up function or step-down function occurs is selected for analysis and an expected range for future QOS data based on the subset of the N QOS data points is generated.

Abstract: Methods and systems for identifying a victim storage volume from among a plurality of storage volumes based on a comparison of current Quality of Service (QOS) data with a dynamic threshold value that is based on historical QOS collected data for the plurality of storage volumes are provided. A performance manager collects the current and historical QOS data from a storage operating system of the storage system, which includes a response time in which each of the plurality of storage volumes respond to an input/output (I/O) request. The current and historical QOS data for the resources used by the victim storage volume are retrieved and compared with the current QOS data of each resource to an expected range based on the historical QOS data. Another storage volume is identified as a bully when its usage of a resource in contention contributes to creating the victim storage volume.

Abstract: The present invention discloses a system and method for adaptive adjustment of bit budget based on the content of the input image sequence. In one embodiment, two encoders are employed in a dual pass encoding system. A first encoder receives the input image sequence and encodes each frame of the image sequence using a standard or any predefined encoding algorithms. Specifically, by encoding the image sequence using the first encoder, the first encoder is able to assess the complexity of each picture in the image sequence, e.g., by measuring the number of bits needed to encode each picture. This complexity information serves as look-ahead information for a second encoder.