Having problems accessing a certain HTTPS site through ASA

jomar050485 · ‎11-19-2015

Hello All,

I'm having problems accessing a certain site via HTTPS. I see some weird stuff happening.

First I'm seeing this in the logs:

Nov 19 2015 10:36:29: %ASA-4-419002: Duplicate TCP SYN from inside:10.10.138.34/61730 to outside:12.23.45.56/443 with different initial sequence number
Nov 19 2015 10:36:29: %ASA-4-419002: Duplicate TCP SYN from inside:10.127.40.102/62249 to outside:12.23.45.56/443 with different initial sequence number
Nov 19 2015 10:36:29: %ASA-4-419002: Duplicate TCP SYN from inside:10.10.137.38/55599 to outside:12.23.45.56/443 with different initial sequence number
Nov 19 2015 10:36:30: %ASA-4-419002: Duplicate TCP SYN from inside:10.10.137.50/55204 to outside:12.23.45.56/443 with different initial sequence number
Nov 19 2015 10:36:31: %ASA-4-419002: Duplicate TCP SYN from inside:10.10.137.20/56259 to outside:12.23.45.56/443 with different initial sequence number
Nov 19 2015 10:36:32: %ASA-4-419002: Duplicate TCP SYN from inside:10.127.40.118/62617 to outside:12.23.45.56/443 with different initial sequence number
Nov 19 2015 10:36:32: %ASA-4-419002: Duplicate TCP SYN from inside:10.10.138.34/61730 to outside:12.23.45.56/443 with different initial sequence number
Nov 19 2015 10:36:32: %ASA-4-419002: Duplicate TCP SYN from inside:10.127.40.102/62249 to outside:12.23.45.56/443 with different initial sequence number
Nov 19 2015 10:36:32: %ASA-4-419002: Duplicate TCP SYN from inside:10.10.137.38/55599 to outside:12.23.45.56/443 with different initial sequence number
Nov 19 2015 10:36:34: %ASA-4-733100: [ 12.23.45.56] drop rate-1 exceeded. Current burst rate is 2 per second, max configured rate is 10; Current average rate is 8 per second, max configured rate is 5; Cumulative total count is 10030
Nov 19 2015 10:36:34: %ASA-4-419002: Duplicate TCP SYN from inside:10.10.126.42/50391 to outside:12.23.45.56/443 with different initial sequence number
Nov 19 2015 10:36:35: %ASA-4-419002: Duplicate TCP SYN from inside:10.127.40.118/62617 to outside:12.23.45.56/443 with different initial sequence number
Nov 19 2015 10:36:36: %ASA-4-419002: Duplicate TCP SYN from inside:10.10.137.50/55204 to outside:12.23.45.56/443 with different initial sequence number
Nov 19 2015 10:36:36: %ASA-4-419002: Duplicate TCP SYN from inside:10.10.138.35/57317 to outside:12.23.45.56/443 with different initial sequence number
Nov 19 2015 10:36:37: %ASA-4-419002: Duplicate TCP SYN from inside:10.10.126.42/50391 to outside:12.23.45.56/443 with different initial sequence number
Nov 19 2015 10:36:39: %ASA-4-419002: Duplicate TCP SYN from inside:10.10.127.53/51445 to outside:12.23.45.56/443 with different initial sequence number
Nov 19 2015 10:36:39: %ASA-4-419002: Duplicate TCP SYN from inside:10.10.138.35/57317 to outside:12.23.45.56/443 with different initial sequence number

Then my drop capture is seeing this:

2337: 10:33:08.495625 12.23.45.56.443 > 10.10.138.50.55428: S 953294807:953294807(0) ack 1957104558 win 14600 <mss 1380>
2346: 10:33:09.040036 10.10.126.37.60781 > 12.23.45.56.443: R 3312837979:3312837979(0) ack 1489666041 win 0
2351: 10:33:09.521579 10.127.40.149.49917 > 12.23.45.56.443: R 2492934361:2492934361(0) ack 1599332666 win 0
2363: 10:33:10.694910 12.23.45.56.443 > 10.10.138.50.55428: S 953294807:953294807(0) ack 1957104558 win 14600 <mss 1380>
2375: 10:33:11.497990 12.23.45.56.443 > 10.10.127.47.48819: S 2216496751:2216496751(0) ack 254941459 win 14600 <mss 1380>
2400: 10:33:14.891235 12.23.45.56.443 > 10.10.138.50.55428: S 953294807:953294807(0) ack 1957104558 win 14600 <mss 1380>
2405: 10:33:15.414193 10.10.127.14.55259 > 12.23.45.56.443: R 1642612023:1642612023(0) ack 2147749739 win 0
2412: 10:33:16.463675 10.10.137.31.60199 > 12.23.45.56.443: R 2289985330:2289985330(0) ack 1619959172 win 0
2417: 10:33:16.850847 10.127.30.72.49512 > 12.23.45.56.443: R 910793492:910793492(0) ack 1139747253 win 0
2430: 10:33:18.475424 10.10.129.39.49225 > 12.23.45.56.443: F 1225527575:1225527575(0) ack 2840927136 win 257
2432: 10:33:18.696069 10.10.129.39.49225 > 12.23.45.56.443: R 1225527576:1225527576(0) ack 2840927370 win 0
2453: 10:33:21.632566 10.10.138.34.61652 > 12.23.45.56.443: R 2749495861:2749495861(0) ack 4160180661 win 0
2461: 10:33:22.576645 10.10.129.37.60331 > 12.23.45.56.443: F 3178188495:3178188495(0) ack 2192264854 win 255
2462: 10:33:22.809543 10.10.129.37.60331 > 12.23.45.56.443: R 3178188496:3178188496(0) ack 2192265088 win 0
2465: 10:33:23.038724 10.10.127.191.64041 > 12.23.45.56.443: R 2913968222:2913968222(0) ack 2465891620 win 0
2509: 10:33:26.408059 12.23.45.56.443 > 10.10.127.101.63587: R 808568565:808568565(0) ack 1610972180 win 35
2512: 10:33:26.571549 10.127.40.101.61201 > 12.23.45.56.443: R 2178944903:2178944903(0) ack 3538513005 win 0
2525: 10:33:27.692041 10.10.126.34.51009 > 12.23.45.56.443: R 64803698:64803698(0) ack 1744988619 win 0
2527: 10:33:28.061032 10.10.127.3.64701 > 12.23.45.56.443: R 390092785:390092785(0) ack 1627169847 win 0
2540: 10:33:29.795323 10.10.127.164.60229 > 12.23.45.56.443: R 2591560456:2591560456(0) ack 1325297103 win 0
2575: 10:33:33.449546 10.10.127.35.51854 > 12.23.45.56.443: R 56633680:56633680(0) ack 854237420 win 0
2578: 10:33:33.849901 10.10.137.48.58042 > 12.23.45.56.443: R 2136357540:2136357540(0) ack 1544543548 win 0
2605: 10:33:36.926587 12.23.45.56.443 > 10.10.127.101.63600: R 3318437017:3318437017(0) ack 2472933263 win 102
2618: 10:33:38.586624 10.10.127.60.56920 > 12.23.45.56.443: R 1677284339:1677284339(0) ack 2898862314 win 0
2631: 10:33:39.581894 10.10.137.45.59256 > 12.23.45.56.443: R 3257315637:3257315637(0) ack 2249249901 win 0
2636: 10:33:40.124718 10.10.126.38.53323 > 12.23.45.56.443: R 458562104:458562104(0) ack 52028260 win 0
2668: 10:33:42.548555 12.23.45.56.443 > 10.10.126.36.50165: R 2622513396:2622513396(0) ack 2759037920 win 29480
2681: 10:33:44.554963 10.10.129.34.64007 > 12.23.45.56.443: R 1455789499:1455789499(0) ack 2554980939 win 0
2692: 10:33:45.877273 10.10.138.50.55410 > 12.23.45.56.443: R 2709192364:2709192364(0) ack 3842025307 win 0
2701: 10:33:46.869446 10.10.127.4.53668 > 12.23.45.56.443: R 45419687:45419687(0) ack 2566465679 win 0
2708: 10:33:48.779409 10.10.127.47.48889 > 12.23.45.56.443: R 2335721534:2335721534(0) ack 3393837489 win 0
2712: 10:33:49.624723 10.10.137.40.63474 > 12.23.45.56.443: R 4143594416:4143594416(0) ack 3488932962 win 0
2725: 10:33:51.598266 10.10.137.50.55191 > 12.23.45.56.443: R 1426374098:1426374098(0) ack 862528743 win 0
2726: 10:33:51.619718 10.10.126.43.49696 > 12.23.45.56.443: R 117455973:117455973(0) ack 4101789803 win 0
2761: 10:33:57.192143 10.10.126.35.57316 > 12.23.45.56.443: R 724845598:724845598(0) ack 133881962 win 0
2767: 10:33:57.600860 10.10.127.39.44447 > 12.23.45.56.443: R 3602776651:3602776651(0) ack 4209052478 win 0
2778: 10:33:58.179495 10.10.129.30.52819 > 12.23.45.56.443: F 1478242291:1478242291(0) ack 174420596 win 258
2779: 10:33:58.291504 10.10.129.30.52819 > 12.23.45.56.443: R 1478242292:1478242292(0) ack 174420830 win 0
2788: 10:33:59.164481 10.10.126.42.50375 > 12.23.45.56.443: R 220369676:220369676(0) ack 3291575909 win 0
2789: 10:33:59.193562 12.23.45.56.443 > 10.127.40.150.51398: R 4151411183:4151411183(0) ack 3445629529 win 32489
2791: 10:33:59.472662 10.10.138.33.51359 > 12.23.45.56.443: R 3028996532:3028996532(0) ack 3562474004 win 0
2800: 10:34:00.831728 10.10.138.35.57311 > 12.23.45.56.443: R 3636728935:3636728935(0) ack 1686995308 win 0
2811: 10:34:02.313429 10.10.127.163.59971 > 12.23.45.56.443: R 719270918:719270918(0) ack 330353705 win 0
2816: 10:34:03.308516 10.10.127.53.51429 > 12.23.45.56.443: R 4106643337:4106643337(0) ack 3873838035 win 0
2863: 10:34:09.797016 10.10.126.185.55050 > 12.23.45.56.443: R 1399584795:1399584795(0) ack 3191627796 win 0
2919: 10:34:12.754767 10.10.137.54.54140 > 12.23.45.56.443: R 2567272396:2567272396(0) ack 1671021254 win 0
2935: 10:34:14.238253 10.10.137.52.63151 > 12.23.45.56.443: R 1488636143:1488636143(0) ack 1229048983 win 0
2965: 10:34:18.732368 10.10.127.3.51347 > 12.23.45.56.443: R 1220010538:1220010538(0) ack 1946760772 win 0
2970: 10:34:19.717721 10.10.137.39.50204 > 12.23.45.56.443: R 1763366074:1763366074(0) ack 1105566694 win 0
2994: 10:34:22.660015 10.10.127.56.49515 > 12.23.45.56.443: R 1543710137:1543710137(0) ack 92945924 win 0
2998: 10:34:23.270478 10.10.126.185.55052 > 12.23.45.56.443: R 3592034307:3592034307(0) ack 625726751 win 0
3007: 10:34:24.885925 10.10.137.30.49260 > 12.23.45.56.443: R 200618218:200618218(0) ack 2971978347 win 0
3051: 10:34:30.699106 10.10.129.44.63066 > 12.23.45.56.443: F 1864636218:1864636218(0) ack 1100890674 win 258
3052: 10:34:30.851716 10.10.129.44.63066 > 12.23.45.56.443: R 1864636219:1864636219(0) ack 1100890908 win 0
3083: 10:34:33.898497 10.10.127.41.55961 > 12.23.45.56.443: R 3001237182:3001237182(0) ack 3415937120 win 0
3169: 10:34:44.078639 10.10.127.31.53449 > 12.23.45.56.443: R 1327461638:1327461638(0) ack 4250936559 win 0
3175: 10:34:45.004287 10.127.40.102.62232 > 12.23.45.56.443: R 1062731457:1062731457(0) ack 3120750080 win 0
3176: 10:34:45.318007 12.23.45.56.443 > 10.127.40.106.52021: R 2232345755:2232345755(0) ack 2506533944 win 35
3206: 10:34:48.546266 10.10.127.78.54327 > 12.23.45.56.443: R 1969310806:1969310806(0) ack 3476719781 win 0
3262: 10:34:54.561158 10.10.127.88.62467 > 12.23.45.56.443: R 3042169563:3042169563(0) ack 3270415806 win 0
3278: 10:34:55.953655 10.10.126.185.55046 > 12.23.45.56.443: R 3418591050:3418591050(0) ack 2787662311 win 0

What's going on? If I leave the site trying to connect, it might load after 3 mins. It loads fine on the other side of the firewall.

Any ideas?

Rishabh Seth · ‎11-19-2015

Hi,

Here is an explanation for the syslog message from the syslog definition:

This is an excerpts from the syslog guide:

419002

Error Message %ASA-4-419002: Received duplicate TCP SYN from in_interface : src_address / src_port to out_interface: dest_address / dest_port with different initial sequence number.

Explanation A duplicate TCP SYN was received during the three-way-handshake that has a different initial sequence number from the SYN that opened the embryonic connection. This may indicate that SYNs are being spoofed. This message occurs in Release 7.0.4.1 and later.

in_interface —The input interface
src_address —The source IP address of the packet
src_port —The source port of the packet
out_interface —The output interface
dest_address —The destination IP address of the packet
dest_port —The destination port of the packet

Recommended Action None required.

http://www.cisco.com/c/en/us/td/docs/security/asa/syslog-guide/syslogs/logmsgs1.html

>> Regarding thedrop captures, we do not see any reason for drop. You can try clearing the asp drop countersa and then collect multiple outputs of show asp drop and check counters for different categories to find some relation with the traffic drops.

Let us know if you are using any firewall services modules ips/sfr/cx for the traffic in question. Also share the version of ASA you are using.

Thanks,

RS

jomar050485 · ‎11-19-2015

Hello,

Thanks for the response. I'm running 9.1(2) with no IPS/SFR/CX just the base ASA system.

I'll see if the ASP drops increase anywhere. I'm seeing some Slowpath security checks failed (sp-security-failed) drops. Going to investigate it a little more.

Rishabh Seth · ‎11-19-2015

Try capturing traffic from one host to a particular destination on ASA's inside and outside interface.

Analyze captures and see if you see any changes introduced by ASA in the captured traffic.

Does this issue happens while accessing all the websites or there is some specific website that shows this behavior.

You can also check proxy servers or load balancers if present in your network.

Thanks,

RS