Abstract: A system and method for managing application performance includes a storage controller including a memory containing machine readable medium comprising machine executable code having stored thereon instructions for performing a method of managing application performance and a processor coupled to the memory. The processor is configured to execute the machine executable code to receive storage requests from a plurality of first applications via a network interface, manage QoS settings for the storage controller and the first applications, and in response to receiving an accelerate command associated with a second application from the first applications, increase a first share of a storage resource allocated to the second application, decrease unlocked second shares of the storage resource of the first applications, and lock the first share. The storage resource is a request queue or a first cache. In some embodiments, the second application is a throughput application or a latency application.

Abstract: A system and method for scheduling the pre-loading of long-term data predicted to be requested in future time epochs into a faster storage tier are disclosed. For each epoch into the future, which may be on the order of minutes or hours, data chunks which may be accessed are predicted. Intersections are taken between predicted data chunks, starting with the furthest predicted epoch in the future, ranging back to the next future epoch. These are then intersected with adjacent results, on up a hierarchy until an intersection is taken of all of the predicted epochs. Commands are generated to preload the data chunks predicted to have the most recurring accesses, and the predicted data chunks are pre-loaded into the cache. This proceeds down the load order until either the last predicted data set is pre-loaded or it is determined that the cache has run out of space.

Abstract: A system and method for managing application performance includes a storage controller including a memory containing machine readable medium comprising machine executable code having stored thereon instructions for performing a method of managing application performance and a processor coupled to the memory. The processor is configured to execute the machine executable code to receive storage requests from a plurality of first applications via a network interface, manage QoS settings for the storage controller and the first applications, and in response to receiving an accelerate command associated with a second application from the first applications, increase a first share of a storage resource allocated to the second application, decrease unlocked second shares of the storage resource of the first applications, and lock the first share. The storage resource is a request queue or a first cache. In some embodiments, the second application is a throughput application or a latency application.

Abstract: A system and method for scheduling the pre-loading of long-term data predicted to be requested in future time epochs into a faster storage tier are disclosed. For each epoch into the future, which may be on the order of minutes or hours, data chunks which may be accessed are predicted. Intersections are taken between predicted data chunks, starting with the furthest predicted epoch in the future, ranging back to the next future epoch. These are then intersected with adjacent results, on up a hierarchy until an intersection is taken of all of the predicted epochs. Commands are generated to preload the data chunks predicted to have the most recurring accesses, and the predicted data chunks are pre-loaded into the cache. This proceeds down the load order until either the last predicted data set is pre-loaded or it is determined that the cache has run out of space.

Abstract: A system and method for managing application performance includes a storage controller including a memory containing machine readable medium comprising machine executable code having stored thereon instructions for performing a method of managing application performance and a processor coupled to the memory. The processor is configured to execute the machine executable code to receive storage requests from a plurality of first applications via a network interface, manage QoS settings for the storage controller and the first applications, and in response to receiving an accelerate command associated with a second application from the first applications, increase a first share of a storage resource allocated to the second application, decrease unlocked second shares of the storage resource of the first applications, and lock the first share. The storage resource is a request queue or a first cache. In some embodiments, the second application is a throughput application or a latency application.

Abstract: A system and method for determining I/O performance headroom that accounts for a real-world workload is provided. In some embodiments, a computing device is provided that is operable to identify a data transaction received by a storage system and directed to a storage device. The computing system identifies an attribute of the data transaction relating to a performance cost of the data transaction and queries a performance profile to determine a benchmark performance level for the storage device. The computing system determines a benchmark performance level for the storage system based on the benchmark performance level for the storage device and compares a metric of the performance of the data transaction with the storage system benchmark performance level to determine remaining headroom of the storage system.

Abstract: Systems and methods for efficient input/output (I/O) workload capture are provided. For example, in one aspect, a machine implemented method includes: opening a network socket for listening to a connection request from a computing device; accepting the connection request from the computing device over the network socket; enabling selective data collection based on a network connection with the computing device over the network socket, where the network connection based selective data collection includes obtaining information regarding a plurality of input/output (I/O) requests and responses and performance information of a storage server for processing the I/O requests; sub-sampling the network connection based collected data; and sending at least a portion of the network connection based collected data over the network socket connection to the computing device.

Abstract: A system and method of cache monitoring in storage systems includes storing storage blocks in a cache memory. Each of the storage blocks is associated with status indicators. As requests are received at the cache memory, the requests are processed and the status indicators associated with the storage blocks are updated in response to the processing of the requests. One or more storage blocks are selected for eviction when a storage block limit is reached. As ones of the selected one or more storage blocks are evicted from the cache memory, the block counters are updated based on the status indicators associated with the evicted storage blocks. Each of the block counters is associated with a corresponding combination of the status indicators. Caching statistics are periodically updated based on the block counters.

Abstract: A system and method for recognizing data access patterns in large data sets and for preloading a cache based on the recognized patterns is provided. In some embodiments, the method includes receiving a data transaction directed to an address space and recording the data transaction in a first set of counters and in a second set of counters. The first set of counters divides the address space into address ranges of a first size, whereas the second set of counters divides the address space into address ranges of a second size that is different from the first size. One of a storage device or a cache thereof is selected to service the data transaction based on the first set of counters, and data is preloaded into the cache based on the second set of counters.

Abstract: A system and method of cache monitoring in storage systems includes storing storage blocks in a cache memory. Each of the storage blocks is associated with status indicators. As requests are received at the cache memory, the requests are processed and the status indicators associated with the storage blocks are updated in response to the processing of the requests. One or more storage blocks are selected for eviction when a storage block limit is reached. As ones of the selected one or more storage blocks are evicted from the cache memory, the block counters are updated based on the status indicators associated with the evicted storage blocks. Each of the block counters is associated with a corresponding combination of the status indicators. Caching statistics are periodically updated based on the block counters.

Abstract: Systems and methods for efficient input/output (I/O) workload capture are provided. For example, in one aspect, a machine implemented method includes: opening a network socket for listening to a connection request from a computing device; accepting the connection request from the computing device over the network socket; enabling selective data collection based on a network connection with the computing device over the network socket, where the network connection based selective data collection includes obtaining information regarding a plurality of input/output (I/O) requests and responses and performance information of a storage server for processing the I/O requests; sub-sampling the network connection based collected data; and sending at least a portion of the network connection based collected data over the network socket connection to the computing device.

Abstract: Described herein is a system and method for dynamically managing service-level objectives (SLOs) for workloads of a cluster storage system. Proposed states/solutions of the cluster may be produced and evaluated to select one that achieves the SLOs for each workload. A planner engine may produce a state tree comprising nodes, each node representing a proposed state/solution. New nodes may be added to the state tree based on new solution types that are permitted, or nodes may be removed based on a received time constraint for executing a proposed solution or a client certification of a solution. The planner engine may call an evaluation engine to evaluate proposed states, the evaluation engine using an evaluation function that considers SLO, cost, and optimization goal characteristics to produce a single evaluation value for each proposed state. The planner engine may call a modeler engine that is trained using machine learning techniques.

Abstract: A system and method for determining I/O performance headroom that accounts for a real-world workload is provided. In some embodiments, a computing device is provided that is operable to identify a data transaction received by a storage system and directed to a storage device. The computing system identifies an attribute of the data transaction relating to a performance cost of the data transaction and queries a performance profile to determine a benchmark performance level for the storage device. The computing system determines a benchmark performance level for the storage system based on the benchmark performance level for the storage device and compares a metric of the performance of the data transaction with the storage system benchmark performance level to determine remaining headroom of the storage system.

Abstract: A system and method for managing application performance includes a storage controller including a memory containing machine readable medium comprising machine executable code having stored thereon instructions for performing a method of managing application performance and a processor coupled to the memory. The processor is configured to execute the machine executable code to receive storage requests from a plurality of first applications via a network interface, manage QoS settings for the storage controller and the first applications, and in response to receiving an accelerate command associated with a second application from the first applications, increase a first share of a storage resource allocated to the second application, decrease unlocked second shares of the storage resource of the first applications, and lock the first share. The storage resource is a request queue or a first cache. In some embodiments, the second application is a throughput application or a latency application.

Abstract: A system and method for recognizing data access patterns in large data sets and for preloading a cache based on the recognized patterns is provided. In some embodiments, the method includes receiving a data transaction directed to an address space and recording the data transaction in a first set of counters and in a second set of counters. The first set of counters divides the address space into address ranges of a first size, whereas the second set of counters divides the address space into address ranges of a second size that is different from the first size. One of a storage device or a cache thereof is selected to service the data transaction based on the first set of counters, and data is preloaded into the cache based on the second set of counters.

Abstract: Described herein is a system and method for dynamically managing service-level objectives (SLOs) for workloads of a cluster storage system. Proposed states/solutions of the cluster may be produced and evaluated to select one that achieves the SLOs for each workload. A planner engine may produce a state tree comprising nodes, each node representing a proposed state/solution. New nodes may be added to the state tree based on new solution types that are permitted, or nodes may be removed based on a received time constraint for executing a proposed solution or a client certification of a solution. The planner engine may call an evaluation engine to evaluate proposed states, the evaluation engine using an evaluation function that considers SLO, cost, and optimization goal characteristics to produce a single evaluation value for each proposed state. The planner engine may call a modeler engine that is trained using machine learning techniques.

Abstract: A system and method of cache monitoring in storage systems includes storing storage blocks in a cache memory. Each of the storage blocks is associated with status indicators. As requests are received at the cache memory, the requests are processed and the status indicators associated with the storage blocks are updated in response to the processing of the requests. One or more storage blocks are selected for eviction when a storage block limit is reached. As ones of the selected one or more storage blocks are evicted from the cache memory, the block counters are updated based on the status indicators associated with the evicted storage blocks. Each of the block counters is associated with a corresponding combination of the status indicators. Caching statistics are periodically updated based on the block counters.

Abstract: Example embodiments provide various techniques for modeling network storage environments. To model a particular storage environment, component models that are associated with the components of the storage environment are loaded. Each component model is programmed to mathematically simulate one or more components of the storage environment. A system model is then composed from the component models and this system model is configured to simulate the storage environment.

Abstract: Example embodiments provide various techniques for modeling network storage environments. To model a particular storage environment, component models that are associated with the components of the storage environment are loaded. Each component model is programmed to mathematically simulate one or more components of the storage environment. A system model is then composed from the component models and this system model is configured to simulate the storage environment.

Abstract: Described herein is a system and method for dynamically managing service-level objectives (SLOs) for workloads of a cluster storage system. Proposed states/solutions of the cluster may be produced and evaluated to select one that achieves the SLOs for each workload. A planner engine may produce a state tree comprising nodes, each node representing a proposed state/solution. New nodes may be added to the state tree based on new solution types that are permitted, or nodes may be removed based on a received time constraint for executing a proposed solution or a client certification of a solution. The planner engine may call an evaluation engine to evaluate proposed states, the evaluation engine using an evaluation function that considers SLO, cost, and optimization goal characteristics to produce a single evaluation value for each proposed state. The planner engine may call a modeler engine that is trained using machine learning techniques.