- Cisco Community
- Technology and Support
- Data Center and Cloud
- Data Center and Cloud Knowledge Base
- Deploying GSS and ACE in least loaded GSLB based on VIP conn number
- Subscribe to RSS Feed
- Mark as New
- Mark as Read
- Bookmark
- Subscribe
- Printer Friendly Page
- Report Inappropriate Content
- Subscribe to RSS Feed
- Mark as New
- Mark as Read
- Bookmark
- Subscribe
- Printer Friendly Page
- Report Inappropriate Content
11-01-2010 02:46 PM - edited 08-28-2017 10:22 PM
Introduction
The purpose of this document is to present two scenarios in deploying GSS and ACE to achieve least loaded GSLB that is linked to the amount of current connections on ACE VIP. Cisco ACE working as loadbalancer helps to decide which server should serve a client request. The job of the load balancer is to select the server that can successfully fulfill the client request and do so in the shortest amount of time without overloading either the server or the server farm as a whole.
The Problem
Currently ACE calculates the load value (numeric value in range 2-255) based on current number of in-service real server vs. total number of configured real servers. More info can be found here. This value is then reported back to GSS via KAL-AP so it can utilize least loaded method when deciding how to resolve a DNS request. However, this does not necessarily give a true picture on how much are the VIP really loaded.
Possible Solutions
To overcome this shortcoming the following two deplyment types can be used:
1) Using max-conn and KAL-AP
On ACE it is possible to configure connection limit per each real server with the "conn-limit max <maxconns> min <minconns>" command. If the number of connections reaches the specified max value the server will be put to MAXCONNS state and ACE will no longer send connections to it. This lasts until connection number falls below min value. More info about this can be found here.
While server is in MAXCONNS state, this also means that it is out of service and because of that the ACE will increase the load value it advertises via KAL-AP.
gss#show statistics keepalive kalap all IP: 10.86.232.193 Keepalive => 10.86.232.190 Status: ONLINE Keepalive Type: Standard Tag: "fun" Primary Circuit: 10.86.232.190 Load: 85 <<<< 1 out of 3 reals not available Circuit Transitions: 0 VIP Failovers: 0 Packets Sent: 29 Packets Received: 25 Positive Probe: 25 Negative Probe: 3 Transitions: 2 GID: 131 LID: 3 ace# show serverfarm detail --------------------------------- ----------connections----------- real weight state current total failures ---+---------------------+------+------------+----------+----------+--------- rserver: 10-degtluv4200-SignalIduna 32.32.32.32:0 1 OUTOFSERVICE 0 1 6 description : - max-conns : 200000 , out-of-rotation count : 0 min-conns : 200000 conn-rate-limit : - , out-of-rotation count : - bandwidth-rate-limit : - , out-of-rotation count : - retcode out-of-rotation count : - load value : 0 rserver: C 10.86.232.143:0 8 OPERATIONAL 2 2 0 description : - max-conns : - , out-of-rotation count : - min-conns : - conn-rate-limit : - , out-of-rotation count : - bandwidth-rate-limit : - , out-of-rotation count : - retcode out-of-rotation count : - load value : 0 rserver: vm1 10.86.232.157:0 8 OPERATIONAL 0 1 0 description : - max-conns : 2 , out-of-rotation count : 0 min-conns : 2 conn-rate-limit : - , out-of-rotation count : - bandwidth-rate-limit : - , out-of-rotation count : - retcode out-of-rotation count : - load value : 0 >>>>>>>>> hit maxconns <<<<<<<<<<
ace# show serverfarm detail --------------------------------- ----------connections----------- real weight state current total failures ---+---------------------+------+------------+----------+----------+---------
rserver: 10-degtluv4200-SignalIduna 32.32.32.32:0 1 OUTOFSERVICE 0 1 6 description : - max-conns : 200000 , out-of-rotation count : 0 min-conns : 200000 conn-rate-limit : - , out-of-rotation count : - bandwidth-rate-limit : - , out-of-rotation count : - retcode out-of-rotation count : - load value : 0 rserver: C 10.86.232.143:0 8 OPERATIONAL 2 2 0 description : - max-conns : - , out-of-rotation count : - min-conns : - conn-rate-limit : - , out-of-rotation count : - bandwidth-rate-limit : - , out-of-rotation count : - retcode out-of-rotation count : - load value : 0 rserver: vm1 10.86.232.157:0 8 MAXCONNS 2 3 0 <<server in MAXCONNS state description : - max-conns : 2 , out-of-rotation count : 1 min-conns : 2 conn-rate-limit : - , out-of-rotation count : - bandwidth-rate-limit : - , out-of-rotation count : - retcode out-of-rotation count : - load value : 0 gss#show statistics keepalive kalap all IP: 10.86.232.193 Keepalive => 10.86.232.190 Status: ONLINE Keepalive Type: Standard Tag: "fun" Primary Circuit: 10.86.232.190 Load: 170 <<<<< load is updated Circuit Transitions: 0 VIP Failovers: 0 Packets Sent: 30 Packets Received: 26 Positive Probe: 26 Negative Probe: 3 Transitions: 2
If GSS has two VIPs with the same load value it will resolve DNS requests in round-robin fashion. As soon as one of the load values becomes higher, the GSS will resolve all new DNS requests to the remaining VIP(s) with lower load value (if there are more than one they will be round-robined). This lasts until the load value equalizes (when round-robin resumes) or until the other VIP becomes more favorable.
If there is a backup serverfarm (usually a "sorry" serverfarm) and the VIP has lost all associated servers the ACE can be configured to advertise load value of 255 making it least favorable VIP. ACE A2(3.2) release is needed for this feature for this feature and it is described here.
If both VIPs go to backup server farm, both will advertise load value of 255 and GSS will now again round robin answers. In this outcome the system has reached maximum capacity and it can no longer serve requests. Any new requests can be redirected to the sorry page hosted on the backup serverfarm.
2) Using scripted SNMP KAL
ACE currently supports an OID that gives connection number per context. This means that a VIP needs to be isolated into a separate ACE context for the GSS to be able to pool data for that one VIP.
=============================== Relevant ACE config: class-map type management match-any management 2 match protocol snmp any policy-map type management first-match management class management permit interface vlan 666 ip address 1.1.1.1 255.255.255.0 service-policy input management no shutdown snmp-server community public group Network-Monitor snmp-server host 2.2.2.2 public ------------------------------- Relevant GSS config: shared-keepalive scripted-kal 1.1.1.1 kal-name ikovacev snmp-scalar oid 1.3.6.1.4.1.9.9.254.1.1.1.1.3.1 community public return-load answer vip 1.1.1.1 location TAC manual-reactivation disable activate keepalive type scripted-kal kal-name ikovacev max-load 20 use-load enable ===============================
In this example max-load is set to 20, which means if number connections in a context goes above it, the VIP will be marked as OFFLINE:
**Load below the threshold: cdn-ace-1/Admin# sh conn count total current connections : 17 cdn-gss-4491-1.cisco.com#sh statistics keepalive scripted-kal ikovacev IP: 1.1.1.1 Keepalive => 1.1.1.1 Status: ONLINE Keepalive Type: Shared, Standard Kal Name: test Scripted Kal Type: SNMP_scalar OID: 1.3.6.1.4.1.9.9.254.1.1.1.1.3.1 Community Name: public Return Type: Load Use Load: Yes Load: 216 Max VIP Load: 20 No. of Execution: 2 Positive Probe: 2 Negative Probe: 0 Transitions: 1 VIP GID: 215 LID: 6 Keepalive GID: 218 **Load goes over the limit: cdn-ace-1/Admin# sh conn count total current connections : 25 cdn-gss-4491-1.cisco.com#sh statistics keepalive scripted-kal ikovacev IP: 1.1.1.1 Keepalive => 1.1.1.1 Status: OFFLINE Keepalive Type: Shared, Standard Kal Name: test Scripted Kal Type: SNMP_scalar OID: 1.3.6.1.4.1.9.9.254.1.1.1.1.3.1 Community Name: public Return Type: Load Use Load: Yes Load: 255 Max VIP Load: 20 No. of Execution: 3 Positive Probe: 3 Negative Probe: 0 Transitions: 2 VIP GID: 215 LID: 6 Keepalive GID: 218
Please note that GSS displays the load proportionality in the 2-255 interval. The formula is:
<load value> = <current connection count> x 253 / <configured max-load>
Related Information
ACE behavior with static sticky and rserver down situation
How to configure an ACL for Inter-GSS communication
Catch-all VIP with forward or transparent serverfarm options
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:
