Application Slowness through ACE 4710

[swizzguy3]

We are struggling with an issue where a user can run a report directly from the server but not from ACE loadbalanced url.

• Report run via Individual Web server URL’s
  
  • The report takes less than 20 minutes (average 15 minutes) to fetch and return the data. This is observed 9 out of 10 times.
    
• Report run via ACE Load Balanced URL
  
  • The report keeps on running for more than 20 minutes and never completes. The front end keeps showing report is running.
    
• The data in general when tested directly by running queries against the database (bypassing the platform) completes in 15-18 minutes
  
• The network connectivity for each and every ports involved (Loadbalancer/Servers) have been throulgly checked.
  

Please advise what is the best way to troubleshoot this issue.

Thx

Ahmad

[Cesar Roque]

Hi Muhammad,

Paste your configuration to have a better picture. 

Is the same behavior with any browser or OS?

---------------------
Cesar R
ANS Team

[Jorge Bejarano]

Ahmad,

Is this for http/https?

Yes, please paste your current configuration.

Jorge

[swizzguy3]

Here is my current configurations.

--------------------------------------------------

probe http ping.htm

  port 80

  interval 5

  faildetect 2

  request method head url /ping.htm

  expect status 200 200

rserver host ttdcsrv1617

  ip address 10.36.172.65

  inservice

rserver host ttdcsrv1618-sandapp

  ip address 10.36.172.58

  inservice

rserver host ttdcsrv1714-sandapp

  ip address 10.36.172.41

  inservice

rserver host ttdcsrv1960

  ip address 10.36.188.69

  inservice

serverfarm host appbo

  probe ping.htm

  rserver ttdcsrv1617 80

    inservice

  rserver ttdcsrv1618-sandapp 80

    inservice

  rserver ttdcsrv1714-sandapp 80

    inservice

  rserver ttdcsrv1960 80

    inservice

parameter-map type connection 3600-seconds

  set timeout inactivity 3600

sticky ip-netmask 255.255.255.255 address source appbo

  timeout 20

  serverfarm appbo

class-map match-all appbo

  2 match virtual-address 10.36.253.245 tcp eq www

policy-map type loadbalance first-match appbo-l7slb

  class class-default

    sticky-serverfarm appbo

policy-map multi-match int63

  class appbo

    loadbalance vip inservice

    loadbalance policy appbo-l7slb

    loadbalance vip icmp-reply active primary-inservice

    connection advanced-options 3600-seconds

[Jorge Bejarano]

Ahmad,

What version are you using?

Can you upload these outputs?

# show service-policy int63 class-map appbo detail

# show stats http

#show resource usage

#show license brief

Also the configuration for the interface.

Has this ever worked before?

Why do you have a low timeout for your sticky configuration?

If you clear the cookies of the browser and turn off some of the servers to test only with one, do you have the same behavior?

In your tests, are you trying to the same type of query to your database? I mean do you see the problem if you try to update/delete/select in the database?

What are the servers? Oracle, SQL server, MS Access?

Jorge

[swizzguy3]

Hello Jorge,

  Here is my response below.

What version are you using? Version A4(2.3a)

Can you upload these outputs?

# show service-policy int63 class-map appbo detail

Status     : ACTIVE

Description: -----------------------------------------

Interface: vlan 1 62 63 155 409

  service-policy: int63

    class: appbo

     VIP Address:    Protocol:  Port:

     170.116.253.245 tcp        eq    80  

      loadbalance:

        L7 loadbalance policy: appbo-l7slb

        VIP ICMP Reply       : ENABLED-WHEN-PRIMARY-SF-UP

        VIP State: INSERVICE

        VIP DWS state: DWS_DISABLED

        Persistence Rebalance: DISABLED

        curr conns       : 0         , hit count        : 136348   

        dropped conns    : 21       

        conns per second    : 0        

        client pkt count : 4579400   , client byte count: 1054651106         

        server pkt count : 6006908   , server byte count: 7506886155         

        conn-rate-limit      : -         , drop-count : -        

        bandwidth-rate-limit : -         , drop-count : -        

        L7 Loadbalance policy : appbo-l7slb

          class/match : class-default

            LB action: :

               sticky group: appbo

                  primary serverfarm: appbo

                    state:UP

                  backup serverfarm : -

            hit count        : 136348   

            dropped conns    : 12       

            compression      : off

      compression:

        bytes_in  : 0                          bytes_out : 0                  

        Compression ratio : 0.00%

                Gzip: 0               Deflate: 0        

      compression errors:

        User-Agent  : 0               Accept-Encoding    : 0        

        Content size: 0               Content type       : 0        

        Not HTTP 1.1: 0               HTTP response error: 0        

        Others      : 0        

        Parameter-map(s):

          3600-seconds

# show stats http

+------------------------------------------+

+-------------- HTTP statistics -----------+

+------------------------------------------+

LB parse result msgs sent : 455403292  , TCP data msgs sent       : 2434371041

Inspect parse result msgs : 0          , SSL data msgs sent       : 1740205587

                      sent

TCP fin msgs sent         : 40338385   , TCP rst msgs sent:       : 22697825 

Bounced fin msgs sent     : 3083341    , Bounced rst msgs sent:   : 2062455  

SSL fin msgs sent         : 53021042   , SSL rst msgs sent:       : 89469    

Drain msgs sent           : 260995432  , Particles read           : 749867347

Reuse msgs sent           : 0          , HTTP requests            : 413618855

Reproxied requests        : 232446464  , Headers removed          : 0        

Headers inserted          : 105893583  , HTTP redirects           : 1485493  

HTTP chunks               : 144637122  , Pipelined requests       : 42911599 

HTTP unproxy conns        : 305776225  , Pipeline flushes         : 942      

Whitespace appends        : 720        , Second pass parsing      : 0        

Response entries recycled : 42882784   , Analysis errors          : 0        

Header insert errors      : 49         , Max parselen errors      : 35941    

Static parse errors       : 261401     , Resource errors          : 0        

Invalid path errors       : 0          , Bad HTTP version errors  : 117      

Headers rewritten         : 49         , Header rewrite errors    : 0        

SSL headers inserted      : 0          , SSL header insert errors : 0        

SSL spoof headers deleted : 0         , Unproxy msgs sent         : 305776279

HTTP passthrough stat     : 0        

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Has this ever worked before? YES but the issue is Intermittent

Why do you have a low timeout for your sticky configuration? NOT SURE. WILL HAVE TO CHECK WITH APP OWNERS IF WE CAN INCREASE THAT

If you clear the cookies of the browser and turn off some of the servers to test only with one, do you have the same behavior? Will try to perform this test with app owners

In your tests, are you trying to the same type of query to your database? I mean do you see the problem if you try to update/delete/select in the database? No Refresh is pretty quick.

What are the servers? Oracle, SQL server, MS Access? ORAGLE 11G.

Thanks for your help!

[Jorge Bejarano]

Ahmad,

Can you try something like below under your policy multimatch

policy-map multi-match int63 under your class

appbo?

parameter-map type http test

  persistence-rebalance

  set header-maxparse-length 65535

  set content-maxparse-length 65535

  length-exceed continue

Jorge