ACE30: Rserver is Not Loadbalancing

[Irvan Tambunan]

Hi All,

I have ACE30 with multiple context. On some context, load-balacing is not working. Rserver connection is not equal. Client is accessing from VIP. Below is the log:

ACE/ccq# show serverfarm crm_fase1

   Codes: L - local,   R - remote

 serverfarm     : crm_fase1, type: HOST
 total rservers : 2
 state          : ACTIVE
 DWS state      : DISABLED
 ---------------------------------
                                                ----------connections-----------
       real                  weight state        current    total      failures
   ---+---------------------+-----+------------+----------+----------+---------
   rserver: crm1
       10.6.82.156:5555      8   OPERATIONAL     122        75685756   35392
   rserver: crm2
       10.6.82.157:5555      8   OPERATIONAL     473        85749330   17836

ACE/ccq#              show service-policy crm_fase1_policy detail

Status     : ACTIVE
Description: -----------------------------------------
Interface: vlan 658
  service-policy: crm_fase1_policy
    class: l4_crm_fase1
     VIP Address:    Protocol:  Port:
     10.6.82.119     tcp        eq    5555
      loadbalance:
        L7 loadbalance policy: crm_fase1_loadpolicy
        VIP Route Metric     : 77
        VIP Route Advertise  : DISABLED
        VIP ICMP Reply       : ENABLED-WHEN-ACTIVE
        VIP State: INSERVICE
        VIP DWS state: DWS_DISABLED
        Persistence Rebalance: DISABLED
        curr conns       : 584       , hit count        : 199508586
        dropped conns    : 45516     
        conns per second    : 0         
        client pkt count : 1263810860, client byte count: 2244901805384       
        server pkt count : 1778561657, server byte count: 5666664200031       
        conn-rate-limit      : 0         , drop-count : 0         
        bandwidth-rate-limit : 0         , drop-count : 0         
        L7 Loadbalance policy : crm_fase1_loadpolicy
          class/match : class-default
            LB action: :
               sticky group: sticky_crm_fase1
                  primary serverfarm: crm_fase1
                    state:UP
                  backup serverfarm : -
            hit count        : 199463204
            dropped conns    : 6465866   
            compression      : off
      compression:
        bytes_in  : 0                          bytes_out : 0                   
        Compression ratio : 0.00%
                Gzip: 0               Deflate: 0         
      compression errors:
        User-Agent  : 0               Accept-Encoding    : 0         
        Content size: 0               Content type       : 0         
        Not HTTP 1.1: 0               HTTP response error: 0         
        Others      : 0         

The problem is when rserver crm2 is down, some user seem to cannot login to system. Could you explain this behaviour.

I also attach my configuration and output from "show conn"

Thanks.

Irvan.

[Kanwaljeet Singh]

Hi Irvan,

You have failaction purge so i am not sure why some users will fail to connect once the real server fails. If the user tries to connect again, it should be loadbalanced to other real server in the serverfarm. What do you see in "show conn address <client ip>" during the problem?

Also, by default loadbalancing happens via round robin which is not a very good way to loadbalance. You can use least connections predictor and see if the unequal loadbalance situation improves.

You have sticky based on source and destination. This can be a problem when lot of users are coming from the same IP i.e behind the proxy or a NAT device. You can try changing "sticky method" as well. 

Regards,

Kanwal

Note: Please mark answers if they are helpful.

[Irvan Tambunan]

Hi Kanwal,

When rserver crm2 down, some users cannot login, but others is normal. This situation was weird. It has to be handled by rserver crm1.

Could give me script for least connections predictor?

Since, accessing the server via VIP only internally, then users are coming from their IP (not via proxy or NAT).

Thanks.

Irvan.

[Kanwaljeet Singh]

Hi Irvan,

For troubleshooting the issue when one rserver goes down, i would need more outputs during the problem itself.

For configuration of predictor least connections, you just need to do it under the serverfarm:

serverfarm host crm_fase1
  failaction purge

 predictor leastconns---------------------->Here is what you need to configure.

rserver crm1 5555
    probe crm_fase1_port
    inservice
  rserver crm2 5555
    probe crm_fase1_port
    inservice

Regards,

Kanwal

Note: Please mark the answers if they are helpful.

[Irvan Tambunan]

Hi Kanwal,

In attachment, you can see output from "show conn".

For sticky configuration, i am using /24 subnet mask. Is it good for production environtment? Or i have to use only /32? What do you think?

As before, you said to change "sticky method". Could you elaborate this statement?

Thanks.

Irvan.

[Irvan Tambunan]

Hi All,

This case solved by changing subnet mask from /24 to /32 on stickiness configuration.

Thanks.

[usman ali dar]

hey Kanwal,

hope you are doing great, i need some advise which is regarding VIP what if i do not state VIP INSERVICE Command and only VIP ICMP-REPLY

[Kanwaljeet Singh]

Hi Usman,

Interesting but i never did that myself.  Loadbalance vip icmp-reply will mean that even if serverfarm has failed the VIP will reply. "Loadbalance vip icmp-reply active" will mean that only if serverfarm is operational you will have the reply from VIP. But in both the cases loadbalance vip inservice should be there. This command makes the VIP active. If it is not there, i doubt that loadbalance vip icmp-reply will work. 

Regards,

Kanwal

Note: Please mark answers if they are helpful.

[usman ali dar]

yes Kanwal you are absolutely right i checked it with and without and saw that VIP ICMP-REPLY was good.

but even with vip inservice i see that i can work with http but cant terminate SSL on load balancer but its giving me a error like 

Unable to make a secure connection to the server. This may be a problem with the server, or it may be requiring a client authentication certificate that you don't have.

Error code: ERR_SSL_PROTOCOL_ERROR..

can you help please 

[Kanwaljeet Singh]

Hi Usman,

Share the configuration you have in place for SSL termination. What do you get on the client side packet captures while attempting to connect?

Regards,

Kanwal

Note: Please mark answers if they are helpful.

[usman ali dar]

hey Kanwal the issue is resolved i think it was my bad somehow the other one of my class map was associated with wrong policy map which was creating an error space but i really appriciat your help by which i am keep improving myself.

thanks and best regards

[Kanwaljeet Singh]

Hi Usman,

Happy to be of help! Thank you for nice words.

Regards,

Kanwal

Note: Please mark answers if they are helpful.