Abstract: It is detected that a metric associated with a first workload has breached a first threshold. It is determined that the first workload and a second workload access the same storage resources, wherein the storage resources are associated with a storage server. It is determined that the metric is impacted by the first workload and the second workload accessing the same storage resources. A candidate solution is identifier. An estimated impact of a residual workload is determined based, at least in part, on the candidate solution. A level of caching of at least one of the first workload or the second workload is adjusted based, at least in part, on the estimated impact of the residual workload.

Abstract: Data in a storage system is deduplicated after receiving from at least one writing entity requests for a plurality of write operations for a corresponding plurality of data blocks in a storage object. The received blocks are buffered and sorted in order and a sequence of clumps is created from the buffered blocks, where each clump comprises a grouping of at least one of the sorted, buffered blocks. A boundary is determined between at least one pair of clumps based at least in part on the content of at least one of the buffered blocks, and it is then determined whether at least one of the clumps is a duplicate of a previously stored clump.

Abstract: It is detected that a metric associated with a first workload has breached a first threshold. It is determined that the first workload and a second workload access the same storage resources, wherein the storage resources are associated with a storage server. It is determined that the metric is impacted by the first workload and the second workload accessing the same storage resources. A candidate solution is identifier. An estimated impact of a residual workload is determined based, at least in part, on the candidate solution. A level of caching of at least one of the first workload or the second workload is adjusted based, at least in part, on the estimated impact of the residual workload.

Abstract: It is detected that a metric associated with a first workload has breached a first threshold. It is determined that the first workload and a second workload access the same storage resources, wherein the storage resources are associated with a storage server. It is determined that the metric is impacted by the first workload and the second workload accessing the same storage resources. A candidate solution is identifier. An estimated impact of a residual workload is determined based, at least in part, on the candidate solution. A level of caching of at least one of the first workload or the second workload is adjusted based, at least in part, on the estimated impact of the residual workload.

Abstract: It is detected that a metric associated with a first workload has breached a first threshold. It is determined that the first workload and a second workload access the same storage resources, wherein the storage resources are associated with a storage server. It is determined that the metric is impacted by the first workload and the second workload accessing the same storage resources. A candidate solution is identifier. An estimated impact of a residual workload is determined based, at least in part, on the candidate solution. A level of caching of at least one of the first workload or the second workload is adjusted based, at least in part, on the estimated impact of the residual workload.

Abstract: Data in a storage system is deduplicated after receiving from at least one writing entity requests for a plurality of write operations for a corresponding plurality of data blocks in a storage object. The received blocks are buffered and sorted in order and a sequence of clumps is created from the buffered blocks, where each clump comprises a grouping of at least one of the sorted, buffered blocks. A boundary is determined between at least one pair of clumps based at least in part on the content of at least one of the buffered blocks, and it is then determined whether at least one of the clumps is a duplicate of a previously stored clump.

Abstract: A method and system for grouping and compressing system logs are described herein. A generation module residing and executing on a computer system may be configured to generate system logs containing system data important for support purposes describing the computer system, and system log metadata for describing the system logs. A grouping module residing and executing on a computer system may be configured to group system logs based on the system log metadata. Each group of system logs may comprise two or more system logs having the same or similar system log metadata. A data container is produced for each group of system logs and the data container is compressed. System logs having the same or similar system log metadata are more likely to have similar data, thus resulting in a higher degree of compression when grouped and compressed together.

Abstract: Data in a storage system is deduplicated after receiving from at least one writing entity requests for a plurality of write operations for a corresponding plurality of data blocks in a storage object. The received blocks are buffered and sorted in order and a sequence of clumps is created from the buffered blocks, where each clump comprises a grouping of at least one of the sorted, buffered blocks. A boundary is determined between at least one pair of clumps based at least in part on the content of at least one of the buffered blocks, and it is then determined whether at least one of the clumps is a duplicate of a previously stored clump.

Abstract: Technology is described for actively responding to data storage traffic. The technology can provide an application program interface; receive, via the application program interface, from an application, a command to query a data storage attribute associated with a virtual data storage component; query the associated virtual data storage component; and return to the application a value for the data storage attribute.

Abstract: Technology is described for a profile-based lifecycle management for data storage servers. The technology can receive a profile, monitor events emitted by devices of the data storage system, determine based on the monitored events that a device of the storage system matches the indicated condition, and perform the action corresponding to the indicated condition, wherein the action includes managing data stored by the data storage system. The received profile can indicate a condition and an action corresponding to the condition.

Abstract: Described herein is a system and method for dynamically managing service-level objectives (SLOs) for workloads of a cluster storage system. Proposed states/solutions of the cluster may be produced and evaluated to select one that achieves the SLOs for each workload. A planner engine may produce a state tree comprising nodes, each node representing a proposed state/solution. New nodes may be added to the state tree based on new solution types that are permitted, or nodes may be removed based on a received time constraint for executing a proposed solution or a client certification of a solution. The planner engine may call an evaluation engine to evaluate proposed states, the evaluation engine using an evaluation function that considers SLO, cost, and optimization goal characteristics to produce a single evaluation value for each proposed state. The planner engine may call a modeler engine that is trained using machine learning techniques.

Abstract: Data in a storage system is deduplicated after receiving from at least one writing entity requests for a plurality of write operations for a corresponding plurality of data blocks in a storage object. The received blocks are buffered and sorted in order and a sequence of clumps is created from the buffered blocks, where each clump comprises a grouping of at least one of the sorted, buffered blocks. A boundary is determined between at least one pair of clumps based at least in part on the content of at least one of the buffered blocks, and it is then determined whether at least one of the clumps is a duplicate of a previously stored clump.

Abstract: A dynamic caching technique adaptively controls copies of data blocks stored within caches (“cached copies”) of a caching layer distributed among servers of a distributed data processing system. A cache coordinator of the distributed system implements the dynamic caching technique to increase the cached copies of the data blocks to improve processing performance of the servers. Alternatively, the technique may decrease the cached copies to reduce storage capacity of the servers. The technique may increase the cached copies when it detects local and/or remote cache bottleneck conditions at the servers, a data popularity condition at the servers, or a shared storage bottleneck condition at the storage system. Otherwise, the technique may decrease the cached copies at the servers.

Abstract: It is detected that a metric associated with a first workload has breached a first threshold. It is determined that the first workload and a second workload access the same storage resources, wherein the storage resources are associated with a storage server. It is determined that the metric is impacted by the first workload and the second workload accessing the same storage resources. A candidate solution is identifier. An estimated impact of a residual workload is determined based, at least in part, on the candidate solution. A level of caching of at least one of the first workload or the second workload is adjusted based, at least in part, on the estimated impact of the residual workload.

Abstract: Described herein is a system and method for dynamically managing service-level objectives (SLOs) for workloads of a cluster storage system. Proposed states/solutions of the cluster may be produced and evaluated to select one that achieves the SLOs for each workload. A planner engine may produce a state tree comprising nodes, each node representing a proposed state/solution. New nodes may be added to the state tree based on new solution types that are permitted, or nodes may be removed based on a received time constraint for executing a proposed solution or a client certification of a solution. The planner engine may call an evaluation engine to evaluate proposed states, the evaluation engine using an evaluation function that considers SLO, cost, and optimization goal characteristics to produce a single evaluation value for each proposed state. The planner engine may call a modeler engine that is trained using machine learning techniques.

Abstract: Described herein is a system and method for an efficient cache warm-up. The system and method may copy data blocks from a primary storage device to a cache memory device. The system and method may identify a subset of data blocks stored on the primary storage device as candidate data blocks for copying to the cache memory device during a cache warm-up period. A cost effectiveness for copying the candidate data blocks to the cache memory device may be determined. In some embodiments, the cost effectiveness may be calculated based on one or more latency values associated with the primary storage device and the cache memory device. The candidate data blocks may be copied to the cache memory device based on the cost effectiveness.

Abstract: Embodiments of the systems and techniques described here can leverage several insights into the nature of workload access patterns and the working-set behavior to reduce the memory overheads. As a result, various embodiments make it feasible to maintain running estimates of a workload's cacheability in current storage systems with limited resources. For example, some embodiments provide for a method comprising estimating cacheability of a workload based on a first working-set size estimate generated from the workload over a first monitoring interval. Then, based on the cacheability of the workload, a workload cache size can be determined. A cache then can be dynamically allocated (e.g., change, possibly frequently, the cache allocation for the workload when the current allocation and the desired workload cache size differ), within a storage system for example, in accordance with the workload cache size.

Abstract: System and method for processing cache metadata from a plurality of cache devices. One or more storage systems may store data blocks to a set of storage devices, each storage system hosting and operating one or more cache devices for caching data blocks of the storage devices. Each cache device may host and implement an agent engine that collects and stores cache metadata for the cache device, the cache metadata describing caching operations of the cache device. A monitor engine receives and aggregates “raw cache metadata” from each of a plurality of cache devices and processes the raw cache metadata to produce processed cache metadata comprising normalized cache metadata (based on activity rate of a cache device), weighted cache metadata, or normalized weighted cache metadata. The processed cache metadata is used by a cache prediction engine to select data blocks for caching to one or more of the cache devices.

Abstract: It is detected that a metric associated with a first workload has breached a first threshold. It is determined that the first workload and a second workload access the same storage resources, wherein the storage resources are associated with a storage server. It is determined that the metric is impacted by the first workload and the second workload accessing the same storage resources. In response to a determination that the metric is impacted by the first workload and the second workload accessing the same storage resources, a first virtual cache appliance is instantiated and one of the first workload or the second workload is routed through the virtual cache appliance. Routing one of the first workload or the second workload through the first virtual cache appliance causes the first virtual cache appliance to cache data associated with the storage resources.

Abstract: Described herein is a system and method for dynamically managing service-level objectives (SLOs) for workloads of a cluster storage system. Proposed states/solutions of the cluster may be produced and evaluated to select one that achieves the SLOs for each workload. A planner engine may produce a state tree comprising nodes, each node representing a proposed state/solution. New nodes may be added to the state tree based on new solution types that are permitted, or nodes may be removed based on a received time constraint for executing a proposed solution or a client certification of a solution. The planner engine may call an evaluation engine to evaluate proposed states, the evaluation engine using an evaluation function that considers SLO, cost, and optimization goal characteristics to produce a single evaluation value for each proposed state. The planner engine may call a modeler engine that is trained using machine learning techniques.