ACE- 4710 graceful disconnect timeout (no FIN ACK)

samarjitdas · ‎09-12-2011

I have below 4 real servers into a single serverfarm and serverfarm is being reported by huge number of drop. Below is the log coming in the ACE buffer.Please suggest what may be the pobable reason for such behavior.

Sep 13 2011 11:31:47 : %ACE-3-251010: Health probe failed for server 172.18.104.128 on port 8001, graceful disconnect timeout (no FIN ACK)

Sep 13 2011 11:32:05 : %ACE-3-251010: Health probe failed for server 172.18.104.126 on port 8001, graceful disconnect timeout (no FIN ACK)

Sep 13 2011 11:32:08 : %ACE-3-251010: Health probe failed for server 172.18.104.125 on port 8001, graceful disconnect timeout (no FIN ACK)

Sep 13 2011 11:32:09 : %ACE-3-251010: Health probe failed for server 172.18.104.127 on port 8001, graceful disconnect timeout (no FIN ACK)

Cesar Roque · ‎09-13-2011

Hello Samarjit,

Please copy and paste the serverfarm, rserver and probe configuration.

Also paste the output of the command:

#show serverfarm {farm-name} detail

Cesar R

--------------------- Cesar R ANS Team

samarjitdas · ‎09-14-2011

Hi Plz find required config,

=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2011.09.14 13:37:31 =~=~=~=~=~=~=~=~=~=~=~=

APP-ACE/Admin# sh serverfarm App_MIS

serverfarm : App_MIS, type: HOST

total rservers : 9

---------------------------------

----------connections-----------

real weight state current total failures

---+---------------------+------+------------+----------+----------+---------

rserver: Application-Server-123

172.18.104.123:8001 8 OPERATIONAL 135 65809 460

rserver: Application-Server-124

172.18.104.124:8001 8 OPERATIONAL 135 69809 124

rserver: Application-Server-125

172.18.104.125:8001 8 PROBE-FAILED 135 69956 56

rserver: Application-Server-126

172.18.104.126:8001 8 OPERATIONAL 135 71015 27

rserver: Application-Server-127

172.18.104.127:8001 8 OPERATIONAL 135 73187 33

rserver: Application-Server-128

172.18.104.128:8001 8 OPERATIONAL 135 69613 33

rserver: Application-Server-129

172.18.104.129:8001 8 OPERATIONAL 135 75545 74

rserver: Application-Server-42

172.18.104.42:8001 8 OPERATIONAL 134 73487 35

rserver: Application-Server-46

172.18.104.46:8001 8 OPERATIONAL 134 78237 48

APP-ACE/Admin# sh serverfarm App_MIS detail

serverfarm : App_MIS, type: HOST

total rservers : 9

active rservers: 8

description : Application Zone Load Balancer for Weblogic

state : ACTIVE

predictor : LEASTCONNS

slowstart : 0 secs

failaction : -

back-inservice : 0

partial-threshold : 0

num times failover : 0

num times back inservice : 0

total conn-dropcount : 0

Probe(s) :

ICMP-ICMP-Probe, type = ICMP

TCP-8001, type = TCP

---------------------------------

----------connections-----------

real weight state current total failures

---+---------------------+------+------------+----------+----------+---------

rserver: Application-Server-123

172.18.104.123:8001 8 OPERATIONAL 133 65812 460

description : -

max-conns : - , out-of-rotation count : -

min-conns : -

conn-rate-limit : - , out-of-rotation count : -

bandwidth-rate-limit : - , out-of-rotation count : -

retcode out-of-rotation count : -

rserver: Application-Server-124

172.18.104.124:8001 8 OPERATIONAL 134 69810 124

description : -

max-conns : - , out-of-rotation count : -

min-conns : -

conn-rate-limit : - , out-of-rotation count : -

bandwidth-rate-limit : - , out-of-rotation count : -

retcode out-of-rotation count : -

rserver: Application-Server-125

172.18.104.125:8001 8 PROBE-FAILED 135 69956 56

description : -

max-conns : - , out-of-rotation count : -

min-conns : -

conn-rate-limit : - , out-of-rotation count : -

bandwidth-rate-limit : - , out-of-rotation count : -

retcode out-of-rotation count : -

rserver: Application-Server-126

172.18.104.126:8001 8 OPERATIONAL 133 71015 27

description : -

max-conns : - , out-of-rotation count : -

min-conns : -

conn-rate-limit : - , out-of-rotation count : -

bandwidth-rate-limit : - , out-of-rotation count : -

retcode out-of-rotation count : -

rserver: Application-Server-127

172.18.104.127:8001 8 OPERATIONAL 135 73187 33

description : -

max-conns : - , out-of-rotation count : -

min-conns : -

conn-rate-limit : - , out-of-rotation count : -

bandwidth-rate-limit : - , out-of-rotation count : -

retcode out-of-rotation count : -

rserver: Application-Server-128

172.18.104.128:8001 8 OPERATIONAL 133 69622 33

description : -

max-conns : - , out-of-rotation count : -

min-conns : -

conn-rate-limit : - , out-of-rotation count : -

bandwidth-rate-limit : - , out-of-rotation count : -

retcode out-of-rotation count : -

rserver: Application-Server-129

172.18.104.129:8001 8 OPERATIONAL 131 75553 74

description : -

max-conns : - , out-of-rotation count : -

min-conns : -

conn-rate-limit : - , out-of-rotation count : -

bandwidth-rate-limit : - , out-of-rotation count : -

retcode out-of-rotation count : -

rserver: Application-Server-42

172.18.104.42:8001 8 OPERATIONAL 133 73489 35

description : -

max-conns : - , out-of-rotation count : -

min-conns : -

conn-rate-limit : - , out-of-rotation count : -

bandwidth-rate-limit : - , out-of-rotation count : -

retcode out-of-rotation count : -

rserver: Application-Server-46

172.18.104.46:8001 8 OPERATIONAL 133 78244 48

description : -

max-conns : - , out-of-rotation count : -

min-conns : -

conn-rate-limit : - , out-of-rotation count : -

bandwidth-rate-limit : - , out-of-rotation count : -

retcode out-of-rotation count : -

serverfarm host App_MIS

description Application Zone Load Balancer for Weblogic

predictor leastconns

probe ICMP-ICMP-Probe

probe TCP-8001

rserver Application-Server-123 8001

inservice

rserver Application-Server-124 8001

inservice

rserver Application-Server-125 8001

inservice

rserver Application-Server-126 8001

inservice

rserver Application-Server-127 8001

inservice

rserver Application-Server-128 8001

inservice

rserver Application-Server-129 8001

inservice

rserver Application-Server-42 8001

inservice

rserver Application-Server-46 8001

inservice

APP-ACE/Admin#sh running-config probe

Generating configuration....

probe icmp ICMP-ICMP-Probe

interval 2

faildetect 2

passdetect interval 60

probe tcp TCP-8001

port 8001

interval 2

faildetect 2

passdetect interval 15

passdetect count 2

open 1

Andrew Nam · ‎09-13-2011

Samarjit,

The log message is indicating the server doesn't respond with FIN/ACK to the FIN
ACE sends when trying to close the probe connection. A few possibilities for this.
1. the server fails to repond
2. packet lost between ACE and the server during tcp tearing down process
3. slow reponse from the server.

The best way to isolate this is to take packet capture.

Regardless of above, you can configure the ACE to forcefully terminate the connection as below : For example, enter: host1/Admin(config-probe-tcp)# connection term forced https://www.cisco.com/en/US/docs/interfaces_modules/services_modules/ace/v3. 00_A2/configuration/slb/guide/probe.html

regards

Andrew

gulanticus · ‎08-04-2015

Andrew

Is there an additional command to be used if the connection term forced command does not resolve the "no SYN ACK" issue?