METHOD FOR TRAFFIC LOAD BALANCING
A method for balancing traffic load is disclosed. The method comprises: monitoring, by a stacking device in a distribution layer, traffic in an aggregation link between the stacking device and a core layer; when a traffic unbalance is monitored in the aggregation link, finding by said stacking device a data flow which causes said traffic unbalance from data flows transferred over the aggregation link; and determining by said stacking device an incoming interface through which the found data flow enters said stacking device, reallocating a new MAC address to the incoming interface, and sending the new MAC address allocated to the incoming interface to a terminal server, so as to enable said traffic unbalance to transit to traffic balance by way of said terminal server sending a data flow with the new MAC address as the destination MAC address.
Latest HANGZHOU H3C TECHNOLOGIES CO., LTD. Patents:
With the increasing dependence of an enterprise on information access, the influences of a data center on the daily business of the enterprise are also increasing. Once a fault occurs to the data center in the enterprise, the normal operation of the enterprise's daily business will be greatly impacted, causing significant losses to the enterprise. Generally speaking, when a fault occurs to the data center, the losses suffered by the enterprise fall into the following categories: impact on the daily enterprise work (e.g. employees cannot work as normal), direct property loss (e.g. loss of orders, compensation to the loss of the enterprise's partners, etc.), damages to the enterprise's reputation (e.g. loss of some clients). Therefore, the reliability has become an important aspect in evaluating the quality of a data center.
The accompanying drawings illustrate various examples of various aspects of the present disclosure. It will be appreciated that the illustrated element boundaries (e.g., boxes, groups of boxes, or other shapes) in the figures represent one example of the boundaries. It will be appreciated that in some examples one element may be designed as multiple elements or that multiple elements may be designed as one element. In some examples, an element shown as an internal component of another element may be implemented as an external component and vice versa.
In order to improve reliability of a data center, a network in a data center usually employs a redundant networking solution with dual nodes and dual links. Additionally, the link aggregation technology and stacking technology can be employed to improve utilization of network devices and links. The link aggregation technology and the stacking technology are described below.
Link Aggregation Technology
Link aggregation, also referred to as Ethernet link aggregation, is to form a logical aggregation link by aggregating a plurality of physical Ethernet ports together. That is to say, a plurality of physical links in the same aggregation link are considered as one logical link. Wherein, the plurality of physical links that constitute the aggregation link are referred to as member links. Through link aggregation, data traffic can be shared among the member links within the aggregation link so that the bandwidth is increased and each member link within the same aggregation link provides dynamic backup to each other, which improves connection reliability.
The aggregation link can be divided into two types: a load-sharing aggregation link and a non-load-sharing aggregation link. The load-sharing aggregation link is currently the dominant application mode. Wherein, the aggregation load-sharing mode of the load-sharing aggregation link can be selectively configured. The load sharing for traffic within an aggregation group can be flexibly achieved by changing the aggregation load-sharing mode.
Stacking Technology
The so-called stacking technology is to cascade a plurality of devices together via stacking ports, and to virtualize them into a “distributed device” after configuration. For ease of description, this “virtual device” is also referred to as a stacking device and each device that constitutes the stacking device is referred to as a member device, with the connection between each member device being referred to as a stacking link. Through the stacking technology, cooperation, centralized management and uninterrupted maintenance of a plurality of member devices can be achieved.
In the stacking technology there exists a concept of local forwarding first. This concept of local forwarding first specifically refers to the following: a data stream received by a member device in the stacking device will be forwarded preferentially through an outgoing interface of the present member device, and only when the present member device does not have an outgoing interface, the received data stream will be sent to another member device through a stacking link to be forwarded over the outgoing interface of this another member device. For example, two switches (e.g. switch 1 and switch 2) cascade to form a stacking device, wherein both the downlink and the uplink of the stacking device support link aggregation. A data stream received by a member device in the stacking device such as switch 1 will be forwarded preferentially through its outgoing interface (i.e. the outgoing interface of switch 1). Only when switch 1 does not have an outgoing interface, the received data stream will be forwarded to another member device such as switch 2 through a stacking link between switch 1 and switch 2, and the data stream will be forwarded over the outgoing interface of switch 2. By way of local forwarding first, a data stream normally does not pass the stacking link. As such, member devices in a stacking device may not need to set many stacking ports, which lowers the cost for the device, and because a data stream is forwarded through a minimum number of devices, the forwarding delay can be reduced.
The above has described the link aggregation technology and the stacking technology. Networking of a data center that employs the link aggregation technology and the stacking technology will be described below.
The networking of a data center is typically divided into three layers: a core layer, an access layer and a distribution layer between them, wherein devices in the distribution layer cascade to form a stacking device, devices in the core layer cascade to form a stacking device and the access layer does not employ the stacking technology. Devices in each layer (i.e. access devices in the access layer, the stacking device in the distribution layer and the stacking device in the core layer) are connected through aggregation links. There is at least one aggregation link between the stacking device in the distribution layer and the stacking device in the core layer.
In a typical networking mode, the access devices in the access layer respectively connect to more than one terminal servers and the devices in the distribution layer act as gateways and routes run between the distribution layer and the core layer.
In this kind of networking, a data stream sent by a terminal server first arrives at the access layer. When an access device in the access layer receives the data stream, it performs a Hash operation on the configured load balancing algorithm and the destination MAC address of this data stream, selects a member link from the aggregation link used for forwarding this data stream according to the result of the Hash operation (i.e. to select an aggregation outgoing port for forwarding this data stream), and then forwards this data stream to the distribution layer through this member link.
In the following, certain examples are described in detail with reference to the drawings.
With reference to
As shown in
block 101, the stacking device in the distribution layer monitors traffic in an aggregation link between the stacking device and a core layer;
block 102, when a traffic unbalance is monitored in the aggregation link, the stacking device finds a data flow which causes said traffic unbalance from data flows transferred over the aggregation link; and
block 103, the stacking device determines an incoming interface through which the found data flow enters itself, reallocates a new MAC address to the incoming interface, and sends the new MAC address allocated to the incoming interface to a terminal server, so as to enable traffic unbalance to transit to traffic balance by way of said terminal server sending a data flow with the new MAC address as the destination MAC address.
In this example, with regard to block 101, the monitored aggregation link is at least one aggregation link designated from the aggregation links between the stacking device and the core layer, and the operation of designation can be performed by a user according to experiences or actual situations.
With reference to
With reference to
The stacking device then finds data flows transferred over the member link L1a from the aggregation link L1, selects N data flows from the found data flows that have a relatively large traffic, and determines the selected N data flows as the data flows that cause traffic unbalance in the aggregation link L1.
In this example, the first set threshold value is independent from the second set threshold value. Both threshold values can be set according to experiences or actual situations and the present disclosure is not limited in this regard.
Additionally, in the above example, the number N can also be set according to experiences or actual situations. For instance, N can take the value of 10.
With reference to
In this example, the above described incoming interface is specifically a VLAN virtual interface. The MAC address of the VLAN virtual interface is the destination MAC address that is used by the terminal server to forward a data stream to the distribution layer through an access device, and this MAC address is allocated to this VLAN virtual interface by a member device of the stacking device in the distribution layer such as a switch in the distribution layer when it creates the VLAN virtual interface. Thus, this MAC address can be modified, that is to say, the MAC address of the VLAN virtual interface can be reallocated.
With reference now to
It can be seen from the above description that, the stacking device sending the new MAC address allocated to the incoming interface to the terminal server as described in block 103 of
With reference to
In terms of probability, when an access device determines a member link based on the new MAC address, the determined member link will be different from the previously determined member link due to the difference between the new MAC address and the old MAC address that was based on (i.e. the MAC address used before the new MAC address is allocated), thus ensuring that the data stream forwarded this time and the previous data streams arrive at different member devices of the stacking device in the distribution layer. In this way, because the member devices of the stacking device in the distribution layer have the characteristic of local forwarding first, when a member device different from the previous ones receives the data stream, it will preferentially forward the received data stream to the core layer via its own outgoing port.
This may help to avoid the situation where traffic in the member links of the aggregation link between the stacking device in the distribution layer and the core layer is unbalanced (for instance, in the aggregation link, a member link can be overloaded and starts to lose packets, while the other member links can be under loaded) due to data streams with different traffic being transmitted from different terminal servers or different business applications to the stacking device in the distribution layer, and thus help improve the quality of user service and forwarding performance of the whole system.
With reference to
In addition, devices between those layers in
Following the process of
When the stacking device 1 monitors that the occupied bandwidth of a member link (hereinafter, taking LCA for example) from the aggregation link L (here, the percentage of the occupied bandwidth can be used) exceeds a first set threshold value and the difference between this occupied bandwidth and the occupied bandwidth of another member link LDB (here, the percentage of the occupied bandwidth can be used) exceeds a second set threshold value, the stacking device determines that there is serious traffic unbalance in the aggregation link L and it needs to be adjusted.
Based on this, the stacking device 1 finds N data flows with a relatively large traffic from LCA. Then, for each data flow, the stacking device 1 finds an incoming interface (i.e. VLAN virtual interface) through which the data flow enters said stacking device by unicast reverse path forwarding (URPF) based on the source IP address of the data flow and reallocates a new MAC address to the VLAN virtual interface corresponding to the data stream, wherein the allocation can be implemented in particular by: randomly allocating an MAC address from the MAC address pool corresponding to the distribution layer, as long as it is different from the previous MAC address of this VLAN virtual interface. In this way, VLAN virtual interfaces corresponding to the N data streams can be reallocated an MAC address.
Next, the stacking device 1 sends a free ARP message over the VLAN virtual interface to which a new MAC address is allocated. The access device which receives said free ARP message learns the new MAC address from the free APR message and forwards said free ARP message to a terminal server to which it accesses. And the terminal server obtains the new MAC address from the free ARP message and uses the obtained new MAC address to update an APR entry. When the terminal server sends again a data stream to the VLAN virtual interface to which a new MAC address has been allocated, it uses the new MAC address as the destination address of this data stream. When an access device in the access layer receives the data stream sent again by the terminal server, it will perform Hash operation once again on the new destination MAC address and the load balancing algorithm and selects a member link from the aggregation link used for forwarding this data stream according to the result of this Hash operation. In terms of probability, the member link selected by the access device will be different from the previously determined member link due to the difference between destination MAC addresses. Then the access device sends this data stream to the distribution layer through this selected member link.
When a member device in the stacking device 1 in the distribution layer receives this data stream, it preferentially sends it through its local outgoing port. Based on the above described fact that the member link selected by the access device will be different from the previously determined member link due to the difference between destination MAC addresses, it follows that compared with previously sent data streams, this data stream arrives at a different member device in the stacking device 1 in the distribution layer. For instance, device C in the stacking device 1 is previously reached and device D is reached this time. As such, device D sends the received data stream out over the member link LDB in the aggregation link L, which on one hand reduces loads on LCA and on the other hand tends to balance loads between LDB and LCA.
It can be seen that, in terms of probability, the terminal server using the new MAC address as the destination MAC address to send data streams can break the phenomenon that abnormal traffic unbalance exists between member links in the aggregation link, and make this phenomenon transit to a better aspect (i.e. the traffic load between member links of the aggregation link is balanced).
Of course, if the stacking device 1 once again monitors that traffic unbalance occurs in the aggregation link between itself and the core layer at a later time, the above method can be repeated. That is to say, the method according to the above examples is a cyclic, continuously optimizing, and automatic procedure.
In the following an apparatus according to an example of the present disclosure will be described.
With reference to
The apparatus 800 is a stacking device in a distribution layer. As shown in
The monitoring unit 801 is used to monitor traffic in an aggregation link between the apparatus 800 and a core layer.
The searching unit 802 is used, when said monitoring unit 801 monitors traffic unbalance in the aggregation link, to find a data flow which causes the traffic unbalance from data flows transferred over the aggregation link.
The allocating unit 803 further comprises a determining subunit 804, an allocating subunit 805 and a sending subunit 806, wherein, said determining subunit 804 is used to determine an incoming interface through which the data flow found by said searching unit 802 enters said apparatus 800; said allocating subunit 805 is used to reallocate a new MAC address to the incoming interface determined by said determining subunit 804; and said sending subunit 806 is used to send the new MAC address allocated to the incoming interface determined by said determining subunit 804 to a terminal server, so as to enable traffic unbalance to transit to traffic balance by way of said terminal server sending a data flow with the new MAC address as the destination MAC address.
With reference to
According to an example, the link-unbalance determining subunit 907 is used to determine that there is traffic unbalance in the aggregation link when the bandwidth occupied by a member link from the aggregation link exceeds a first set threshold value and the difference between bandwidths occupied by this member link and another member link exceeds a second set threshold value. Other components in
With reference to
According to an example, the data flow searching subunit 1008 is used to find a data flow transferred over a member link from the aggregation link the occupied bandwidth of which exceeds the first set threshold value and the difference between the occupied bandwidth of which and that of another member link exceeds the second set threshold value, select a set number N of data flows from the found data flows according to a principle of choosing data flows with a large traffic, and determine the selected data flows as the data flows that cause traffic unbalance. Other components in
In an example, said determining subunit 804, 904 or 1004 is used to, determine for each found data flow the source IP address of the data flow, and find an incoming interface of the data flow by unicast reverse path forwarding (URPF) based on the determined source IP address; and said incoming interface is a virtual local area network (VLAN) virtual interface.
According to an example, said sending subunit 806, 906 or 1006 sends a free address resolution protocol (ARP) message to an access layer over the incoming interface to which the new MAC address is allocated, so that an access device in the access layer which receives said free ARP message forwards said free ARP message to a terminal server to which it accesses.
According to an example said apparatus 800, 900 or 1000 further comprises a plurality of member devices which are cascaded to form said apparatus (not shown in
In this example, when receiving a data flow forwarded by an access device in the access layer, each member device selects a member link from the aggregation link used to forward the data flow according to a principle of local forwarding first, and forwards the data flow to the core layer via the selected member link. Here, the destination MAC address of the data flow is the new MAC address, and the access device forwards the data flow to said member device through the following operations: performing a hash operation on the new MAC address which is used as the destination MAC address of the data flow and the load balancing algorithm, selecting a member link from the aggregation link used to forward the data flow according to the result of the hash operation, and forwarding the data flow to the distribution layer using the selected member link.
It can be seen from the above description that, according to an example of the present disclosure, an access device in the access layer selects a member link from an aggregation link based on the destination MAC address of a data stream sent from a terminal server, and forwards the data stream to the distribution layer through this selected member link.
A stacking device in the distribution layer will select a member link from an aggregation link according to the principle of local forwarding first to forward the received data stream to the core layer, and monitors traffic in the aggregation link between the stacking device and the core layer. When a traffic unbalance is monitored in the aggregation link, the stacking device finds a data flow which causes said traffic unbalance from data flows transferred over the aggregation link, determines an incoming interface corresponding to the found data flow, reallocates a new MAC address to the incoming interface, and sends the new MAC address allocated to the incoming interface to a terminal server through an access device. In this way, when the terminal server subsequently sends a data flow to the incoming interface that has been allocated the new MAC address, it uses this new MAC address as the destination MAC address for sending the data stream to an access device. When the access device receives this data stream, it uses this new MAC address to select a member link from the aggregation link and forwards this data stream to the distribution layer through this selected member link. And when a member device of the stacking device in the distribution layer receives the data stream, it selects a member link from the aggregation link according to the principle of local forwarding first and forwards this data stream to the core layer through this selected member link. In terms of probability, the member link determined by the access device based on the new MAC address would be different from the member link that was previously determined based on the old MAC address (i.e. the MAC address used before the new MAC address is allocated). This ensures that the two different data streams forwarded by the access device (i.e. the data stream with the new MAC address as its destination MAC address and the data stream with the old MAC address as its destination MAC address) arrive at different member devices of the stacking device in the distribution layer. According to the principle of local forwarding first, this will alleviate the phenomenon that abnormal traffic unbalance exists among member links in the aggregation link between the stacking device in the distribution layer and the core layer, and make this phenomenon transit to a better aspect (i.e. the traffic load between member links of the aggregation link is balanced).
From the above depiction of the implementation mode, the above examples can be implemented by hardware, software or firmware or a combination thereof. For example the various methods, processes, modules and functional units described herein may be implemented by a processor (the term processor is to be interpreted broadly to include a CPU, processing unit, ASIC, logic unit, or programmable gate array etc.) The processes, methods and functional units may all be performed by a single processor or split between several processers. They may be implemented as machine readable instructions executable by one or more processors. Further the teachings herein may be implemented in the form of a software product. The computer software product is stored in a storage medium and comprises a plurality of instructions for making a computer device (which can be a personal computer, a server or a network device, etc.) implement the method recited in the examples of the present disclosure.
The figures are only illustrations of an example, wherein the modules or procedure shown in the figures are not necessarily essential for implementing the present disclosure. Moreover, the sequence numbers of the above examples are only for description, and do not indicate an example is more superior to another.
Those skilled in the art can understand that the modules in the device in the example can be arranged in the device in the example as described in the example, or can be alternatively located in one or more devices different from that in the example. The modules in the aforesaid example can be combined into one module or further divided into a plurality of sub-modules.
Claims
1. A method for balancing traffic load, wherein the method comprises:
- monitoring, by a stacking device in a distribution layer, traffic in an aggregation link between the stacking device and a core layer;
- when a traffic unbalance is monitored in the aggregation link, finding by said stacking device a data flow which causes said traffic unbalance from data flows transferred over the aggregation link; and
- determining by said stacking device an incoming interface through which the found data flow enters said stacking device, reallocating a new MAC address to the incoming interface, and sending the new MAC address allocated to the incoming interface to a terminal server, so as to enable said traffic unbalance to transit to traffic balance by way of said terminal server sending a data flow with the new MAC address as the destination MAC address.
2. The method according to claim 1, wherein monitoring said traffic unbalance in the aggregation link includes:
- determining that there is traffic unbalance in the aggregation link when the occupied bandwidth of a member link from the aggregation link exceeds a first set threshold value and the difference between bandwidths occupied by this member link and another member link exceeds a second set threshold value.
3. The method according to claim 2, wherein finding a data flow which causes the traffic unbalance from data flows transferred over the aggregation link comprises:
- finding a data flow transferred over a member link from the aggregation link the occupied bandwidth of which exceeds the first set threshold value and the difference between the occupied bandwidth of which and that of another member link exceeds the second set threshold value; and
- selecting a set number N of data flows from the found data flows according to a principle of choosing data flows with a large traffic, and determining the selected data flows as the data flows that cause traffic unbalance.
4. The method according to claim 1, wherein said determining by the stacking device an incoming interface through which the found data flow enters the stacking device comprises:
- for each data flow found, determining by said stacking device the source IP address of the data flow, and finding the incoming interface of the data flow by unicast reverse path forwarding (URPF) based on the determined source IP address; wherein said incoming interface is a virtual local area network (VLAN) virtual interface.
5. The method according to claim 1, wherein said sending by the stacking device the new MAC address allocated to the incoming interface to the terminal server comprises:
- sending by said stacking device a free address resolution protocol (ARP) message to an access layer over the incoming interface to which the new MAC address is allocated; and forwarding, by an access device in the access layer which receives said free ARP message, said free ARP message to a terminal server to which it accesses.
6. The method according to claim 1, wherein said enabling traffic unbalance to transit to traffic balance by way of said terminal server sending a data flow with the new MAC address as the destination MAC address comprises:
- sending by the terminal server a data flow with the destination MAC address being the new MAC address to an access device to which it accesses;
- after receiving the data flow sent by the terminal server, performing by said access device a hash operation on the destination MAC address of the data flow and the load balancing algorithm, selecting a member link from the aggregation link used to forward the data flow according to the result of the hash operation, and forwarding the data flow to the distribution layer via the selected member link; and
- when receiving the data flow, selecting, by a member device of the stacking device in the distribution layer, a member link from the aggregation link used to forward the data flow according to a principle of local forwarding first, and forwarding the data flow to the core layer via the selected member link.
7. A traffic load balancing apparatus, wherein the apparatus is a stacking device in a distribution layer and comprises:
- a monitoring unit to monitor traffic in an aggregation link between said apparatus and a core layer;
- a searching unit to find, when said monitoring unit monitors traffic unbalance in the aggregation link, a data flow which causes the traffic unbalance from data flows transferred over the aggregation link;
- an allocating unit comprising a determining subunit, an allocating subunit and a sending subunit, wherein,
- said determining subunit is to determine an incoming interface through which the data flow found by said searching unit enters said apparatus;
- said allocating subunit is to reallocate a new MAC address to the incoming interface determined by said determining subunit; and
- said sending subunit is to send the new MAC address allocated to the incoming interface determined by said determining subunit to a terminal server, so as to enable said traffic unbalance to transit to traffic balance by way of said terminal server sending a data flow with the new MAC address as the destination MAC address.
8. The apparatus according to claim 7, wherein said searching unit comprises:
- a link-unbalance determining subunit to determine that there is traffic unbalance in the aggregation link when the bandwidth occupied by a member link from the aggregation link exceeds a first set threshold value and the difference between bandwidths occupied by this member link and another member link exceeds a second set threshold value.
9. The apparatus according to claim 8, wherein said searching unit further comprises:
- a data flow searching subunit to find a data flow transferred over a member link from the aggregation link the occupied bandwidth of which exceeds the first set threshold value and the difference between the occupied bandwidth of which and that of another member link exceeds the second set threshold value, select a set number N of data flows from the found data flows according to a principle of choosing data flows with a large traffic, and determine the selected data flows as the data flows that cause traffic unbalance.
10. The apparatus according to claim 7, wherein said determining subunit is to, determine for each found data flow the source IP address of the data flow, and find an incoming interface of the data flow by unicast reverse path forwarding (URPF) based on the determined source IP address;
- wherein said incoming interface is a virtual local area network (VLAN) virtual interface.
11. The apparatus according to claim 7, wherein said sending subunit sends a free address resolution protocol (ARP) message to an access layer over the incoming interface to which the new MAC address is allocated, so that an access device in the access layer which receives said free ARP message forwards said free ARP message to a terminal server to which it accesses.
12. The apparatus according to claim 7, wherein said apparatus further comprises a plurality of member devices which are cascaded to form said apparatus;
- when receiving a data flow forwarded by an access device in the access layer, each member device selects a member link from the aggregation link used to forward the data flow according to a principle of local forwarding first, and forwards the data flow to the core layer via the selected member link; and
- wherein the destination MAC address of the data flow is the new MAC address, and the access device forwards the data flow to said member device through the following operations: performing a hash operation on the new MAC address which is used as the destination MAC address of the data flow and the load balancing algorithm, selecting a member link from the aggregation link used to forward the data flow according to the result of the hash operation, and forwarding the data flow to the distribution layer using the selected member link.
Type: Application
Filed: Apr 6, 2012
Publication Date: Jan 16, 2014
Patent Grant number: 9614768
Applicant: HANGZHOU H3C TECHNOLOGIES CO., LTD. (Hangzhou)
Inventor: Wei Li (Beijing)
Application Number: 13/979,449
International Classification: H04L 12/803 (20060101);