Ace 20 total number of concurrent sessions sudden drop

[Zarahelll]

Good morning to you all

I need your advice on a behaviour I´m detecting on my ACE 20.

I´m monitoring the total number of concurrent sessions of my ACE 20 (using Cacti), and from time to time, with no discernable pattern, I see an instant drop of sessions to half...I don´t detect any disturbance with our traffic and service, I have no complaints, but it's a very accentuated drop.

I´m able to get 1 or 2 days withouth any suddent drop of connections, and then for no reason I pass from 500.000 to 200.000 sessions in a minute. Then they gradually go up again.

I´ve seen in ACE´s session table that she keeps a great number of half-open, or closed sessions, and those are counted as part of concurrent sessions. Is there any flush on ACE´s table when she reaches a certain number of closed TCP sessions or something like that?

I apreciate your help.

Best regards

[sivaksiv]

Hi,

you can configure timeout for half-closed conns on ACE using parameter map. The default is 3600 secs.

host1/C1(config-parammap-conn)# parameter-map type connection TCP

host1/C1(config-parammap-conn)# set tcp timeout half-closed 900

http://www.cisco.com/en/US/docs/app_ntwk_services/data_center_app_services/ace_appliances/vA1_7_/configuration/security/guide/tcpipnrm.html#wp1072427

HTH,

Siva

[Zarahelll]

sivaksiv

Thank you for your feedback. I don´t believe half-closed conns to be the issue here. I used it as an example of entries that are present in ACE´s session table besides open sessions. But the behaviour doesn´t fit with a specific timeout in

half-closed conns.

Like I mentioned, I'm able to get by 2 or 3 days without any sudden drop in connections, and all of the sudden it crashes for a minute. Besided, the crash is very accentuated...sometimes it drops from 500.000 to 200.000 sessions...I don´t believe I have 300.000 half-closed conns...besides, it it was a matter of timeout values, the sessions would drop over a space of time (since the users don´t make requests simultaneously), and that´s not what happens...they all drop

simultaneously

[Jorge Bejarano · Accepted solution]

Hello Zarahell.

• Are these connections related to a specific virtual address or with several?
• Are these idle connections?
• What type of traffic is this?
• Is this happening for one context or some of them?
• Is this happening during normal operations or under a higher peak of traffic?
• What about #show resource usage all?

Jorge

[Zarahelll]

Jorge Bejarano

Thank you for your reply.

• Are these connections related to a specific virtual address or with several?

          It´s been happening with a couple of VIPs (not all of them, just some)

• Are these idle connections?

           Not sure, so far I´ve only caught it because I´m graphing the total  concurrent sessions. When it happens its too           late to find what  type of connections were dropped.

• What type of traffic is this?

          Mostly TCP. Web traffic through TCP 80 and 443

• Is this happening for one context or some of them?

          1 Context. I only have 1 context active.

• Is this happening during normal operations or under a higher peak of traffic?

           No discernible patterns. It can occur in higher peak times, or not. No  operational work being performed on the           ACE at those times  (configuration, replication, etc)

• What about #show resource usage all?

# show resource usage all

                                                     Allocation

        Resource         Current       Peak        Min        Max       Denied

-------------------------------------------------------------------------------

Context: CONTEXT1

  conc-connections         502309     631220          0    7999900          0

  mgmt-connections            164        704          0      99900          0

  proxy-connections          2192       9176          0    1048574          0

  xlates                        0          0          0    1048574          0

  bandwidth             107966766  272639303          0 1122500016          0

    throughput          107952639  272511428          0  998750016          0

    mgmt-traffic rate       14127     127875          0  123750000          0

  connections rate           5954       9351          0     324900          0

  ssl-connections rate          0          0          0       1000          0

  mac-miss rate                 0          4          0       2000          0

  inspect-conn rate             0          7          0       6000          0

  acl-memory              3347896    3347896          0   70844416          0

  sticky                        0          0          0          0          0

  regexp                     1146       1146          0    1048576          0

  syslog buffer           3563520    3563520          0    4194304          0

  syslog rate                   7        106          0     100000          0

[Jorge Bejarano]

Hello Zarahell,

It will be good to get those details which I mentioned before and go deeper with this internally. You may need to check the serverfarm details and ask your users what they are experiencing. Packets captures may help.

Please monitor the behavior and let us know.

Jorge

[Zarahelll]

Jorge

That´s the main issue...withouth any discernible patterns, I have no idea when it´s going to happen. It can go 1 or 2 days without any problems and then it just happens...at the same time it can also happen once or twice a day.

The traffic level I have for my farms is elevated...I don´t have enough server disk size to place an IDS performing a captures during 2 or 3 days in the event it happens =S...for me to capture the packets it has to happen at the exact moment it happens, and I have no idea of when it will.

My user´s don´t complain, but since this is a random behaviour spaced in a couple of hours/days, it might just be a case of the user thinking he lost the connection, and then performed a refresh on the webpage.

Is there any known caveat or issue with Version A2(3.5) [build 3.0(0)A2(3.5)] that may fit the described behaviour?

[Jorge Bejarano]

Hello Zarahell,

It is hard to say, if we do not have enough evidence.

Perhaps, you can clear the serverfarm stats and then check the outputs after a while and monitor the behavior.

Jorge