02-29-2012 08:15 AM - edited 03-11-2019 03:36 PM
Hello
We have a setup with a MS-TMG - ASA (8.2.4(4) in routing mode) - (internal) Router - FWSM - Router - Exchange with NLB. We have now the problem that IMAPS is not really working through this setup. It works from internal (without ASA and TMG inbetween), but not reliably through the internet.
There is a rule on the ASA which permits the ports from the TMG to the Exchange NLB address.
We opened a case with Microsoft and they told us that not all tcp-syn packets are received by the Exchange server which were sent by the TMG.
Thus I sniffed on the ASA with a packet capture and indeed, a lot of syn packets were on the interface to the TMG, but not anymore on the interface to the internal router.
I don't find anything really helpful in the ASA log and am at a loss.
This ASA also filters all other internet<->company traffic, so there's a lot of stuff running.
Maybe it's dropped in the ASP, or is the capture maybe not valid?
Here the show asp drop:
ASA01-Internet# sh asp drop
Frame drop:
Invalid TCP Length (invalid-tcp-hdr-length) 1
Reverse-path verify failed (rpf-violated) 319
Flow is denied by configured rule (acl-drop) 477077
First TCP packet not SYN (tcp-not-syn) 10212
TCP data send after FIN (tcp-data-past-fin) 41
TCP failed 3 way handshake (tcp-3whs-failed) 824
TCP RST/FIN out of order (tcp-rstfin-ooo) 1419
TCP SEQ in SYN/SYNACK invalid (tcp-seq-syn-diff) 6
TCP SYNACK on established conn (tcp-synack-ooo) 1
TCP packet SEQ past window (tcp-seq-past-win) 821
TCP invalid ACK (tcp-invalid-ack) 331
TCP Out-of-Order packet buffer full (tcp-buffer-full) 393
TCP Out-of-Order packet buffer timeout (tcp-buffer-timeout) 1538
TCP RST/SYN in window (tcp-rst-syn-in-win) 4228
TCP dup of packet in Out-of-Order queue (tcp-dup-in-queue) 500
TCP packet failed PAWS test (tcp-paws-fail) 23039
Early security checks failed (security-failed) 13
DNS Inspect id not matched (inspect-dns-id-not-matched) 148
FP L2 rule drop (l2_acl) 4674
Dropped pending packets in a closed socket (np-socket-closed) 26
Last clearing: 16:43:13 CEST Feb 29 2012 by xxxxxxxxxxxx
Flow drop:
Flow is denied by access rule (acl-drop) 56
NAT failed (nat-failed) 342
Inspection failure (inspect-fail) 6552
Last clearing: 16:43:13 CEST Feb 29 2012 by xxxxxxxxxxx
ASA01-Internet#
I hope somebody could help me to debug this better
Thanks,
Patrick
02-29-2012 08:30 AM
Patrick,
Could you give us a peak at the capture? Also, could you provide the config relevant to this connection (any translation statements, ACLs, policies, etc).
As you can tell from your 'show ASP drop', there are many reasons why the ASA drops packets. The above may help us understand why in this case they are being dropped.
Thanks!
Joey
02-29-2012 08:39 AM
Hi Joey
Here the captures showing ingress and egress traffic. It was captured through the ASDM simultaneous. As you can see there are many more TCP Syn packets in the ingress as on the egress (I hope the pictures are visible).
The configuration is a tad more complicated to post, as it's our live config.
02-29-2012 08:38 AM
Hi Patrick,
Please check which of the counters in the 'show asp drop' increment after the intrested traffic passes through ASA. It would be better, if you configure captures for the specific tcp traffic and asp drops, then check which packets are getting dropped.
Following link helps you with packt capture configuration :
http://www.cisco.com/en/US/docs/security/asa/asa82/command/reference/c1.html#wp2129312
Thanks
Narayana
03-01-2012 04:47 AM
That is what I tried, but it did not really work out.
The previous capture was done with these commands (through ASDM):
! DMZ_Transit
! Apply ingress capture on the DMZ_Transit interface.
capture asdm_cap_ingress match tcp src_ip 255.255.255.255 dst_ip 255.255.255.255 eq 993
capture asdm_cap_ingress packet-length 1522 circular-buffer buffer 1524288
capture asdm_cap_ingress interface DMZ_Transit
! inside
! Apply egress capture on the inside interface.
capture asdm_cap_egress match tcp src_ip 255.255.255.255 dst_ip 255.255.255.255 eq 993
capture asdm_cap_egress packet-length 1522 circular-buffer buffer 1524288
capture asdm_cap_egress interface inside
And as you can see on the capture, around 9/10 of all TCP-SYN were not anymore on the inside, which were on the DMZ_Transit.
03-01-2012 07:24 AM
Hi Patrick,
As mentioned by Joey please provide all relevant configurations for this traffic flow and then the capture outputs from the CLI of ASA on both interfaces. That would help in understanding this better.
Narayana
03-05-2012 09:19 AM
Talked with my external support in the mean time. They checked the whole config and think that it might be a software bug. I'll update now (soon) to 8.2.5.22(interims) and hope that this will help.
I'll try to keep you updated.
04-05-2012 08:17 AM
I've opened now a TAC case with Cisco and so far it indeed looks to be a software bug. Will try to keep you updated on the progression.
05-03-2012 02:24 AM
Small update. TAC case still open.
I did make a discovery now though, it seems the affected packets get dropped because of 'tcp-rst-syn-in-win' in the ASP path. When I explicit capture that asp drop reason, then I more or less have only the affected packets!
Now I just need a solution.
05-07-2012 12:48 AM
The issue is solved!
The reason for the drops was a timeout issue on our ASA. We once set the tcp connection timeout to 6 hours (from the default of 1) which is fairly high. It seems now that the TMG had a lower timeout for tcp connections and thus killed some connections from it's table after they timeouted. Then the TMG started to re-use the tcp ports, which our ASA still had in an existing connection, so the asa dropped the valid, but for the ASA duplicate, TCP Syn packets.
After chaning the timeout on the ASA to a much lower time (currently on 7 minutes for IMAP connections, but we are still experimenting with the timeout) it is now working as it should!
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide