Autonomic self-optimization of protection storage
A network system comprises a first component, a second component, and a system for self-optimization. The system for self-optimization comprises an interface and a processor. The interface is configured to receive a performance indication from the first component. The processor is configured to determine a configuration change for the second component according to a set of configuration rules.
Latest EMC IP HOLDING COMPANY LLC Patents:
- Authentication methods and apparatus for generating digital signatures
- Kubernetes security isolation based on classified information and risk levels
- Supporting multiple operations in transaction logging for a cloud-enabled file system
- System and a method for generating live VM images with an extended recovery range
- Techniques for determining compression tiers and using collected compression hints
In a complex networked system, installation of a data protection system is a complicated and arduous task, often leading to a non-optimal data protection system configuration. Network system conditions can change over time, further bringing the data protection system configuration away from the ideal system configuration.
Various embodiments of the invention are disclosed in the following detailed description and the accompanying drawings.
The invention can be implemented in numerous ways, including as a process; an apparatus; a system; a composition of matter; a computer program product embodied on a computer readable storage medium; and/or a processor, such as a processor configured to execute instructions stored on and/or provided by a memory coupled to the processor. In this specification, these implementations, or any other form that the invention may take, may be referred to as techniques. In general, the order of the steps of disclosed processes may be altered within the scope of the invention. Unless stated otherwise, a component such as a processor or a memory described as being configured to perform a task may be implemented as a general component that is temporarily configured to perform the task at a given time or a specific component that is manufactured to perform the task. As used herein, the term ‘processor’ refers to one or more devices, circuits, and/or processing cores configured to process data, such as computer program instructions.
A detailed description of one or more embodiments of the invention is provided below along with accompanying figures that illustrate the principles of the invention. The invention is described in connection with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. Numerous specific details are set forth in the following description in order to provide a thorough understanding of the invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the invention is not unnecessarily obscured.
Autonomic self-optimization of protection storage is disclosed. A network system comprises a first component; a second component; and a system for self-optimization, comprising an interface configured to receive a performance indication from the first component; and a processor configured to determine a configuration change for the second component according to a set of configuration rules. The system for self-optimization additionally comprises a memory coupled to the processor and configured to provide the processor with instructions.
In some embodiments, a network system includes a plurality of network components (e.g., server systems, client systems, networking systems, etc.). A protection storage system for the network system comprises a set of network components configured to protect data on the network system (e.g., ensure that all network data is properly stored and backed up). A system for self-optimization monitors a set of performance indications for the protection storage system in order to ensure that the protection storage system is configured correctly. In various embodiments, the performance indications comprise host attached application configurations, host attached application versions, number of host attached applications using the protection storage, read and write throughput statistics, network connectivity and performance statistics, system software version information, protection storage functionality and performance, error and log files for infrastructure devices, host applications and protection storage software, or any other appropriate performance indications. A set of configuration rules stored by the system for self-optimization describes configuration changes to the system that are indicated to be made by the performance indications. The set of configuration rules describe protection storage best practices (e.g., best practices that a protection storage system administrator would typically attempt to implement). In various embodiments, configuration rules comprise host block size modification rules, stream modification rules, bandwidth optimization rules, application specific optimization rules, recovery time objective (RTO) optimization rules, recovery point objective (RPO) optimization rules, host kernel parameter modification rules, data type target optimization rules, system leveling optimization rules, file routing optimization rules, or any other appropriate configuration rules. Configuration rules describe rules to modify the configuration of the network component reporting the performance indication as well as configurations of other components attached to the network. In some embodiments, the system for self-optimization is used to perform an initial configuration of a protection storage system as it is installed as part of a network system. In some embodiments, the system for self-optimization is used to keep the system optimally configured as network conditions change. The system for self-optimization determines configuration changes according to the configuration rules and provides the configuration changes to the network components.
In some embodiments, the autonomic self-optimization of protection storage system gets indications from one system component and based on best practices and heuristic rules adjusts another system component.
In some embodiments, part of providing a true data protection solution is the ability to do replication for remote relocation of disaster recovery requirements. Often changes in the original design or use of network bandwidth can have a impact on replication causing lag and out of compliance scenarios, placing the business at risk. The ASOPS system will have the ability to recognize any changes in the environment that may cause a lag in replication and either remedy it, (Rule1) or provide and alert, (Rule2):
-
- Input=Network bandwidth availability, remote system information, data synchronicity information
- Output=Automatic modifications in replication and bandwidth usage
- Rule1=If network bandwidth utilization changes between replication systems then adjust replication job, start time, duration time, bandwidth throttling, etc., to meet business requirements for SLO
- Rule2=If data between replication sites is out of sync then check bandwidth current usage, historic usage and generate a detailed alert to user
In some embodiments, today it is well understood which data formats deduplicate well and which do not. These known formats can be coded into the system such that they can be tracked and rerouted to the most optimal, cost effective storage type available to the ASOPS (Rule1). Over time the ASOPS can learn to reroute for optimization based on data predictive compression ratios (Rule2):
-
- Output=traditional storage (TS), deduplicating storage (DS)
- Range1=all known data formats that do not deduplicate well, including but not limited to, .geoTIFF, .jpg, jpeg, .gif, and SRTM, to name a few.
- Rule1: If data format for a data set is within Range1, then route to TS; else route to DS
- Range2=data that has a predicted compression ratio that is less than 20:1 compression ratio
- Rule2: If data format for a data set is within Range2, then route to TS; else route to DS
In some embodiments, with many servers, optimum performance can only be obtained when using jumbo frames between them and the storage systems, (Rule1). Also, each operating system vendor has its own recommendations for system optimizations. This knowledge can be learned by the ASOPS and implemented in an autonomic way, (Rule2):
-
- Output=Block Size/frame size is modified, host operating system is modified
- Range1=Target device=dedup-storage, traditional disk, tape
- Rule1: If target is target device is n, then change write-block-size to match target device.
In some embodiments, the ASOPS system will be able to recognize the maximum possible performance the business can achieve and auto-configure itself to be able to meet the SLO or backup window. Taking into account the available systems and infrastructure, the ASOPS will disallow an unreasonable SLO to be used or configured as well as have the ability to notify an admin when an SLO becomes unobtainable.
The data collection points articulate details regarding application settings, dataset sizes, data types, host memory, CPU, primary storage configuration, disk drive speeds, RAID configuration, network cards and settings, connectivity, available bandwidth, connection hops, switch settings and configuration and available protection storage resources like streams count limitations. This data is then correlated within the ASOPS and tested against an SLO to validate if the SLO can be satisfied (Rule1). If not the ASOPS system will make recommendations (Rule2).
-
- Input=Environmental information as described above
- Output=SLO that supports the infrastructure and business
- Range1=User defined Data Protection window, Recover Time Objective, Recover Point Objective, Operational Window
- Range2=Measured max speeds of infrastructure data points
- Rule1=If infrastructure data point results fall outside Range1then alert user
- Rule2=If Range2 falls outside user defined SLO then ASOPS recommends possible SLO options based on Range2
In some embodiments, in any given customer environment an application can be configured in a customized fashion. This can have an impact on the data protection policies. The ASOPS should be able to discover these customizations and adapt itself to operate most optimally for protection of the application.
-
- Range1=Application backup options
- Range2=Application configuration options
- Rule1=If application backup options are other than default, then ASOPS adapt to new conditions
- Rule2=If application configuration is other than default, then ASOPS adapts to new conditions
Although the foregoing embodiments have been described in some detail for purposes of clarity of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and not restrictive.
Claims
1. A network system, comprising:
- a first component system including a first set of one or more systems;
- a second component system including a second set of one or more systems, the second component system being a different type of system than the first component system, wherein the different type comprises a client system, or a networking system, or a storage system; and
- a system for self-optimization, comprising: an interface configured to receive a performance indication from the first component system, wherein the performance indication comprises a file compression ratio problem comprising a file not compressing at a desired compression ratio after deduplication, and an indication that a number of input streams to a set of storage systems is below a threshold; and a hardware processor configured to determine a configuration change for the second component system and the first component system based on the received performance indication according to a set of configuration rules, wherein the first component system, the second component system, and the system for self-optimization are physically separate systems in the network system.
2. A network system as in claim 1, wherein the performance indication comprises a bandwidth problem of a backup not finishing in time.
3. A network system as in claim 1, wherein the configuration change for the second component system comprises adding bandwidth.
4. A network system as in claim 1, wherein the first component system comprises the client system and the second component system comprises the networking system.
5. A network system as in claim 1, wherein the first component system comprises the storage system and the second component system comprises the client system.
6. A network system as in claim 1, wherein the configuration change for the second component system comprises routing to the storage system not performing deduplication.
7. A network system as in claim 1, wherein the performance indication comprises a data write speed below a data write speed threshold.
8. A network system as in claim 1, wherein the first component system comprises the storage system performing a backup and the second component system comprises the client system sending data for backup.
9. A network system as in claim 1, wherein the configuration change for the second component system comprises changing a block size.
10. A network system as in claim 1, wherein the configuration change for the second component system comprises changing a backup time.
11. A network system as in claim 1, wherein the first component system comprises the storage system storing backup data and the second component system comprises the client system sending backup data.
12. A method for self-optimization, comprising:
- receiving a performance indication from a first component system including a first set of one or more systems; and
- determining, using a hardware processor, a configuration change for a second component system including a second set of one or more systems and the first component system based on the received performance indication according to a set of configuration rules,
- wherein the performance indication comprises a file compression ratio problem comprising a file not compressing at a desired compression ratio after deduplication and an indication that a number of input streams to a set of storage systems is below a threshold,
- wherein the first component system, the second component system, and the hardware processor are physically separate systems in a network system, the second component system being a different type of system than the first component system, and wherein the different type comprises a client system, or a networking system, or a storage system.
13. A computer program product for self-optimization, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
- receiving a performance indication from a first component system including a first set of one or more systems; and
- determining, using a hardware processor of the computer, a configuration change for a second component system including a second set of one or more systems and the first component system based on the received performance indication according to a set of configuration rules,
- wherein the performance indication comprises a file compression ratio problem comprising a file not compressing at a desired compression ratio after deduplication and an indication that a number of input streams to a set of storage systems is below a threshold,
- wherein the first component system, the second component system, and the hardware processor of the computer are physically separate systems in a network system, the second component system being a different type of system than the first component system, and wherein the different type comprises a client system, or a networking system, or a storage system.
6006289 | December 21, 1999 | James |
8228804 | July 24, 2012 | Ninan |
8650163 | February 11, 2014 | Harnik |
9209950 | December 8, 2015 | Luo |
9258771 | February 9, 2016 | Esch |
20030110250 | June 12, 2003 | Schnitzer |
20040073641 | April 15, 2004 | Minhazuddin |
20070294319 | December 20, 2007 | Mankad |
20120317300 | December 13, 2012 | Huang |
20130266310 | October 10, 2013 | Fox |
20140095439 | April 3, 2014 | Ram |
20140120940 | May 1, 2014 | Ren |
20140136685 | May 15, 2014 | Chan |
20140281361 | September 18, 2014 | Park |
20140289080 | September 25, 2014 | Ippolito |
20140304239 | October 9, 2014 | Lewis |
20140365449 | December 11, 2014 | Chambliss |
20150234710 | August 20, 2015 | Berrington |
Type: Grant
Filed: Mar 28, 2014
Date of Patent: Jul 17, 2018
Assignee: EMC IP HOLDING COMPANY LLC (Hopkinton, MA)
Inventors: Steven Weller (Discovery Bay, CA), Mark Twomey (Cork)
Primary Examiner: Richard G Keehn
Application Number: 14/229,536
International Classification: G06F 15/177 (20060101); H04L 12/24 (20060101); G06F 3/06 (20060101); G06F 17/30 (20060101); G06F 11/14 (20060101);