Abstract: Systems and methods are described for using a Deep Reinforcement Learning (DRL) agent to automatically tune Quality of Service (QoS) settings of a distributed storage system (DSS). According to one embodiment, a DRL agent is trained in a simulated environment to select QoS settings (e.g., a value of one or more of a minimum IOPS parameter, a maximum IOPS parameter, and a burst IOPS parameter). The training may involve placing the DRL agent into every feasible state representing combinations of QoS settings, workload conditions, and system metrics for a period of time for multiple iterations, and rewarding the DRL agent for selecting QoS settings that minimize an objective function based on a selected measure of system load. The trained DRL agent may then be deployed to one or more DSSs to constantly update QoS settings so as to minimize the selected measure of system load.

Abstract: Systems and methods are described for using a Deep Reinforcement Learning (DRL) agent to automatically tune Quality of Service (QoS) settings of a distributed storage system (DSS). According to one embodiment, a DRL agent is trained in a simulated environment to select QoS settings (e.g., a value of one or more of a minimum IOPS parameter, a maximum IOPS parameter, and a burst IOPS parameter). The training may involve placing the DRL agent into every feasible state representing combinations of QoS settings, workload conditions, and system metrics for a period of time for multiple iterations, and rewarding the DRL agent for selecting QoS settings that minimize an objective function based on a selected measure of system load. The trained DRL agent may then be deployed to one or more DSSs to constantly update QoS settings so as to minimize the selected measure of system load.

Abstract: Systems and methods for automated tuning of Quality of Service (QoS) settings of volumes in a distributed storage system are provided. According to one embodiment, one or more characteristics of a workload of a client to which a storage node of multiple storage nodes of the distributed storage system is exposed are monitored. After a determination has been made that a characteristic meets or exceeds a threshold, (i) information regarding multiple QoS settings assigned to a volume of the storage node utilized by the client is obtained, (ii) a new value of a burst IOPS setting of the multiple QoS settings is calculated by increasing a current value of the burst IOPS setting by a factor dependent upon a first and a second QoS setting of the multiple QoS settings, and (iii) the new value of the burst IOPS setting is assigned to the volume for the client.

Abstract: Systems and methods are provided for learning normal behavior for user roles of an application running within a cluster of container orchestration platform and based thereon proactively taking action responsive to suspicious events. According to one embodiment, an event data stream is created by an API server of the cluster. The data for each event includes information regarding a request made to an API exposed by the API server with which the event is associated and a user of the application by which the event was initiated. The data is augmented with a role associated with the user and an anomaly threshold for the role. Normal behavior is learned by an ML algorithm of respective user roles by processing the augmented data. When an anomaly score associated with a particular event is output by the ML algorithm that exceeds the anomaly threshold, a predefined or configurable action is triggered.

Abstract: Systems and methods for automated tuning of Quality of Service (QoS) settings of volumes in a distributed storage system are provided. According to one embodiment, responsive to a predetermined event, information regarding a multiple QoS settings assigned to a volume of a distributed storage system that is being utilized by a client are obtained. A difference between a first QoS setting of the multiple QoS settings and a second QoS setting of the multiple QoS settings is determined. Responsive to determining the difference is less than a threshold a new value of the first QoS setting or a third QoS setting of the multiple QoS settings that is greater than a respective current value of the first QoS setting or the third QoS setting is determined and assigned to the volume for the client.

Abstract: Systems and methods for enhancing the representation of outliers in a distribution of telemetry data of a monitored system are provided. According to one embodiment, telemetry data of the monitored system may be continuously collected. Frequency values representing a frequency of occurrence of corresponding telemetry data of the collected telemetry data may be generated by aggregating the collected telemetry data. As the vast majority of telemetry data is expected to represent a normal operating state of the system and relatively few, if any, of the telemetry data (e.g., outliers) will be indicative of one or more events of significance, the resulting distribution of the frequency values is highly skewed. In order to facilitate visualization of the distribution that accentuates the outliers, display characteristics may be calculated for the frequency values by applying a visualization model based on a weighted combination of multiple data transformations to each of the frequency values.

Abstract: Systems and methods for automated tuning of Quality of Service (QoS) settings of volumes in a distributed storage system are provided. According to one embodiment, responsive to a predetermined event, information regarding a multiple QoS settings assigned to a volume of a distributed storage system that is being utilized by a client are obtained. A difference between a first QoS setting of the multiple QoS settings and a second QoS setting of the multiple QoS settings is determined. Responsive to determining the difference is less than a threshold a new value of the first QoS setting or a third QoS setting of the multiple QoS settings that is greater than a respective current value of the first QoS setting or the third QoS setting is determined and assigned to the volume for the client.

Abstract: Systems and methods are described for using a Deep Reinforcement Learning (DRL) agent to automatically tune Quality of Service (QoS) settings of a distributed storage system (DSS). According to one embodiment, a DRL agent is trained in a simulated environment to select QoS settings (e.g., a value of one or more of a minimum IOPS parameter, a maximum IOPS parameter, and a burst IOPS parameter). The training may involve placing the DRL agent into every feasible state representing combinations of QoS settings, workload conditions, and system metrics for a period of time for multiple iterations, and rewarding the DRL agent for selecting QoS settings that minimize an objective function based on a selected measure of system load. The trained DRL agent may then be deployed to one or more DSSs to constantly update QoS settings so as to minimize the selected measure of system load.

Abstract: Systems and methods for use a multi-model block capacity forecasting approach are provided to predict when a distributed storage system will reach a fullness threshold. According to one embodiment, given a time series telemetry dataset collected from multiple distributed storage systems, a forecasting algorithm trains multiple time series forecasting models (e.g., Simple linear regression (SLR), Autoregressive Integrated Moving Average (ARIMA), Generalized additive model (GAM), and/or others) for each of the distributed storage systems. The best performing time series forecasting model is then independently selected for each of the distributed storage systems based on a respective performance metric (e.g., root mean squared error) associated with the time series forecasting models.

Abstract: Systems and methods are described for using a Deep Reinforcement Learning (DRL) agent to automatically tune Quality of Service (QoS) settings of volumes in a distributed storage system (DSS). According to one embodiment, a DRL agent is trained in a simulated environment to select QoS settings (e.g., a value of one or more of a minimum IOPS parameter, a maximum IOPS parameter, and a burst IOPS parameter). The training may involve placing the DRL agent into every feasible state representing combinations of QoS settings, workload conditions, and system metrics for a period of time for multiple iterations, and rewarding the DRL agent for selecting QoS settings that minimize an objective function based on a selected measure of system load. The trained DRL agent may then be deployed to one or more DSSs to constantly update QoS settings so as to minimize the selected measure of system load.

Abstract: Systems and methods for enhancing the representation of outliers in a distribution of telemetry data of a monitored system are provided. According to one embodiment, telemetry data of the monitored system may be continuously collected. Frequency values representing a frequency of occurrence of corresponding telemetry data of the collected telemetry data may be generated by aggregating the collected telemetry data. As the vast majority of telemetry data is expected to represent a normal operating state of the system and relatively few, if any, of the telemetry data (e.g., outliers) will be indicative of one or more events of significance, the resulting distribution of the frequency values is highly skewed. In order to facilitate visualization of the distribution that accentuates the outliers, display characteristics may be calculated for the frequency values by applying a visualization model based on a weighted combination of multiple data transformations to each of the frequency values.