ASA 5510 randomly reset TCP connections

Eugene.alekseev · ‎01-25-2017

Hello everyone.

I have a problem with my Cisco ASA 5510 with Security Plus license.

Few days ago suddenly users in my network issued a problem with big file downloading - download was failed in random cases.

After investigating this problem I figured out that ASA randomly sends TCP RST message to client and server and connection lost.

There is no IPS or protocol inspection configured on device. Only NAT (PAT), ACL and Site-to-Site VPN.

I've tried to mark a "TCP state bypass" in default global service policy - problem passed away, but I observed that number of connections and xlate translation grew up immediatly and stuck at 130k point (usual it about 30-40k). After it ASA stops accept new connection (there is a connection limit for my ASA). I turned off bypassing TCP state and all return to the previous state.

There is no errors in ASA log when it resets TCP connection - only usual teardown messages like "teardown dynamic TCP connection..."

Can anybody help me in this case?

Ajay Saini · ‎01-25-2017

TCP state bypass needs to be specific to the traffic rather than for all the traffic. If you identify the set of interested traffic through access-list, you can apply MPF using that access-list so that all other traffic is still inspected and doesn't fall under tcp-state-bypass.

To identify the root cause, captures and syslogs would be an ideal way. If you have them from the time of issue, please feel free to attach to this post.

-

AJ

Eugene.alekseev · ‎01-25-2017

ASA syslog looks like

6|Jan 25 2017|16:25:11|305012|192.168.1.10|14299|1.1.1.1|14299|Teardown dynamic TCP translation from Local-net-if:192.168.1.10/14299 to Router-GW-if:1.1.1.1/14299 duration 0:00:30
6|Jan 25 2017|16:25:03|305012|192.168.1.10|14265|1.1.1.1|14265|Teardown dynamic TCP translation from Local-net-if:192.168.1.10/14265 to Router-GW-if:1.1.1.1/14265 duration 0:00:46
6|Jan 25 2017|16:25:01|305012|192.168.1.10|14273|1.1.1.1|14273|Teardown dynamic TCP translation from Local-net-if:192.168.1.10/14273 to Router-GW-if:1.1.1.1/14273 duration 0:00:36
6|Jan 25 2017|16:24:55|305011|192.168.1.10|14338|1.1.1.1|14338|Built dynamic TCP translation from Local-net-if:192.168.1.10/14338 to Router-GW-if:1.1.1.1/14338
6|Jan 25 2017|16:24:55|305011|192.168.1.10|14337|1.1.1.1|14337|Built dynamic TCP translation from Local-net-if:192.168.1.10/14337 to Router-GW-if:1.1.1.1/14337
6|Jan 25 2017|16:24:54|305012|192.168.1.10|14272|1.1.1.1|14272|Teardown dynamic TCP translation from Local-net-if:192.168.1.10/14272 to Router-GW-if:1.1.1.1/14272 duration 0:00:30
6|Jan 25 2017|16:24:47|305012|192.168.1.10|14264|1.1.1.1|14264|Teardown dynamic TCP translation from Local-net-if:192.168.1.10/14264 to Router-GW-if:1.1.1.1/14264 duration 0:00:30
6|Jan 25 2017|16:24:41|305011|192.168.1.10|14300|1.1.1.1|14300|Built dynamic TCP translation from Local-net-if:192.168.1.10/14300 to Router-GW-if:1.1.1.1/14300
6|Jan 25 2017|16:24:41|305011|192.168.1.10|14299|1.1.1.1|14299|Built dynamic TCP translation from Local-net-if:192.168.1.10/14299 to Router-GW-if:1.1.1.1/14299
6|Jan 25 2017|16:24:39|305012|192.168.1.10|14205|1.1.1.1|14205|Teardown dynamic TCP translation from Local-net-if:192.168.1.10/14205 to Router-GW-if:1.1.1.1/14205 duration 0:00:53

where 192.168.1.10 - client IP

1.1.1.1 - external ASA IP

192.168.1.1 - internal ASA IP

I forgot to say - there is a lot of "TCP segment of a reassembled PDU" and "TCP Dup ACK 14901#XYZ" messages.

In attach a capture from Wireshark at local host.

Ajay Saini · ‎01-25-2017

These captures are in txt format. Not really useful. Captures taken on ASA ingress and egress will be certainly useful. Another thing that we can check is if there are out-of-order packets coming to the ASA from either lan side or ISP side and if we have L7 inspection turned ON like http inspection, IPS etc.

"TCP segment of a reassembled PDU" and "TCP Dup ACK 14901#XYZ" type messages are generic and a complete wireshark format capture from egress and ingress can throw more light.

-

AJ