cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2154
Views
0
Helpful
4
Replies

Ace - connection reset (Error 101)

nicolasxy
Level 1
Level 1

Hi, I have a problem with a Cisco ACE, after approximately an hour being in production, for all new connections

it gives the message: connection reset. The message on any web browser is: connection reset (Error 101)

It blocks any backend server (Apache).I get same error also when I try to connect direcly to the backend address.

This error saturates the connections on the servers (in the log of the DB I found error connection reset)

Without ACE all work fine, it's not a load traffic issue.

It seems like once opened a connection the ace does not close it anymore!

But the graphical snmp servers do not report the increase in connections, what is mistake ?

The balancer manages two physical servers and is configured in stickyness mode

Please find attached the configuration

---------------------------------

logging enable

logging timestamp

logging trap 4

logging buffered 3

logging host 172.16.0.2 udp/514 format emblem

access-list ANY line 8 extended permit icmp any any

access-list ANY line 16 extended permit ip any any

probe http HTTP_PROBE1

  request method get url /index.php

  expect status 200 206

  expect status 300 307

  expect status 400 417

probe tcp PROBE_TCP

  interval 30

rserver host 03a.it

  ip address 172.16.0.1

  conn-limit max 50000 min 40000

  inservice

rserver host 03b.it

  ip address 172.16.0.2

  conn-limit max 50000 min 40000

  inservice

serverfarm host FARM_WEB

  predictor leastconns

probe HTTP_PROBE1

  rserver 03a.it

    inservice

  rserver 03b.it

    inservice

parameter-map type http HTTP_PARAMETER_MAP

  persistence-rebalance

sticky http-cookie session StickyGroup1

  timeout 3600

  serverfarm FARM_WEB

class-map type management match-all ICMP-ALLOW_CLASS

  2 match protocol icmp source-address x.x.x.x

class-map match-all L4-WEB-IP

  2 match virtual-address x.x.x.x tcp eq www

class-map type management match-all REMOTE_ACCESS

  2 match protocol ssh any

policy-map type management first-match REMOTE_MGMT_ALLOW_POLICY

  class REMOTE_ACCESS

    permit

policy-map type loadbalance http first-match WEB_L7_POLICY

  class class-default

    sticky-serverfarm StickyGroup1

    insert-http x-forward header-value "%is"

policy-map multi-match WEB-to-vIPs

  class L4-WEB-IP

    loadbalance vip inservice

    loadbalance policy WEB_L7_POLICY

    loadbalance vip icmp-reply active

    nat dynamic 1 vlan 2541

    appl-parameter http advanced-options HTTP_PARAMETER_MAP

interface vlan 125

  ip address

  access-group input ANY

  service-policy input REMOTE_MGMT_ALLOW_POLICY

  service-policy input WEB-to-vIPs

  no shutdown

interface vlan 254

  ip address

  access-group input ANY

  nat-pool

  service-policy input REMOTE_MGMT_ALLOW_POLICY

  no shutdown

---------------------------------

At the moment this happens, the simultaneous connections (command: show conn) on the server are around 350

the CPU load is 2%

sticky database has approximately 24000 records.

Log level is set to 4. But no error report.

Do you need more info to resolve the problem?

Thank you

Best Regards

N.

4 Replies 4

Jorge Bejarano
Level 4
Level 4

Hello Nicolas,

Can you upload these files zipped?

#show serverfarm FARM_WEB

#show serverfarm FARM_WEB detail

#show stats http

#show stats loadbalance

#show resource usage all

#show service-policy WEB-to-vIPs class-map L4-WEB-IP

#show probe HTTP_PROBE1

#show probe HTTP_PROBE1 detail

Jorge

Please find attached the output of commands you request.

They have been executed a few hours after the problem occurred.

I hope you find them useful

Do you need any others ?

Many Thanks

Hello Nicolas,

I wonder if you can include these values:

parameter-map type http HTTP_PARAMETER_MAP

  case-insensitive

  persistence-rebalance

  set header-maxparse-length 65535

  set content-maxparse-length 65535

  length-exceed continue

  parsing non-strict

I also noticed a lot of errors which might be caused also due to these denied under the #show resource usage all which may indicate you are reaching the license limits, but you should discuss it with your Cisco SE, please see below:

                                               Allocation

        Resource         Current       Peak        Min        Max       Denied

-------------------------------------------------------------------------------

Context: vrack254

  conc-connections              4       1267      60000      60000          0

  mgmt-connections              2         28        748        748          0

  proxy-connections             0       1255       7864       7864          0

  xlates                        0          0       7864       7864          0

  bandwidth                   572    3824781    3740624  127490624    1416859

    throughput                 96    3712886    3740624    3740624    1416859

    mgmt-traffic rate         476     111895          0  123750000          0

  connection rate               1       1729       4500       4500          0

  ssl-connections rate          0          0        224        224          0

  mac-miss rate                 0         15         16         16          4

  inspect-conn rate             0          0       1800       1800          0

  http-comp rate                0          0    5898240    5898240          0

  to-cp-ipcp rate               0         11         36         36          0

  acl-memory                 8216      10568     744800     744800          0

  sticky                    22978      22978      31456      31456          0

  regexp                       19         23       7864       7864          0

  syslog buffer             30720      30720      30720      30720          0

  syslog rate                   0          6        750        750          0

Can you upload the specific error which you are getting also?

Jorge

Hello,

     my "Cisco SE" says my bandwitch limit is 30Mbps.

In my snmp data from cisco ace, I see that the total traffic on the balancer is:

  1,708MB/s(13,664Mbps)

- IN: 1,55MB/s (12,4Mbps)

- OUT: 158kB/s (1,264Mbps)

I do not understand which is the  unit of measure of the command "show resource usage all"

                                                Allocation

        Resource         Current       Peak        Min        Max       Denied

-------------------------------------------------------------------------------

bandwidth                   572    3824781    3740624  127490624    1416859

    throughput                 96    3712886    3740624    3740624    1416859

    mgmt-traffic rate         476     111895          0  123750000          0

Can you tell me if they are byte? So I can understand the values from "Denied" and "Max"

Last question: How can I undestand if I reached the 30Mbps limit ?

Many Thanks

Nicolas