06-24-2011 02:55 AM - edited 03-11-2019 01:50 PM
I’ve learned something today and would like to share it with others.
Problem
User 192.168.1.1 complaint that he can’t telnet to IP Address 10.10.10.10 anymore.
There is no response from 10.10.10.10 and the telnet session just disconnected. He was able to telnet to the same server last week.
Troubleshooting Steps
I did a quick check on the firewall and found the following error message.
ASA#sh log | i 10.10.10.10
Jun 24 2011 02:19:48 172.16.1.1 : %ASA-2-106001: Inbound TCP connection denied from 10.10.10.10/23 to 192.168.1.1/1068 flags SYN ACK on interface outside
Jun 24 2011 02:19:54 172.16.1.1 : %ASA-2-106001: Inbound TCP connection denied from 10.10.10.10/23 to 192.168.1.1/1068 flags SYN ACK on interface outside
Jun 24 2011 02:20:06 172.16.1.1 : %ASA-2-106001: Inbound TCP connection denied from 10.10.10.10/23 to 192.168.1.1/1068 flags SYN ACK on interface outside
ASA#
Seems like the firewall has dropped the packet. Quickly I checked the firewall rules to make sure it’s there
access-list acl-out extended permit tcp host 192.168.1.1 host 10.10.10.10 eq telnet
Yes, the ACL is there.
Then, I run a packet-tracer to make sure the firewall rule is working fine.
ASA# packet-tracer input outside tcp 192.168.1.1 1024 10.10.10.10 23
Phase: 1
Type: ACCESS-LIST
Subtype:
Result: ALLOW
Config:
Implicit Rule
Additional Information:
MAC Access list
Phase: 2
Type: FLOW-LOOKUP
Subtype:
Result: ALLOW
Config:
Additional Information:
Found no matching flow, creating a new flow
Phase: 3
Type: UN-NAT
Subtype: static
Result: ALLOW
Config:
static (inside,outside) 10.10.10.10 10.10.10.10 netmask 255.255.255.255
match ip inside host 10.10.10.10 outside any
static translation to 10.10.10.10
translate_hits = 0, untranslate_hits = 68
Additional Information:
NAT divert to egress interface inside
Untranslate 10.10.10.10/0 to 10.10.10.10/0 using netmask 255.255.255.255
Phase: 4
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group acl-out in interface outside
access-list acl-out extended permit tcp host 192.168.1.1 host 10.10.10.10 eq telnet
Additional Information:
Phase: 5
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Phase: 6
Type: AAA
Subtype: aaa-auth
Result: ALLOW
Config:
aaa authentication match acl-auth-out outside RSA
Additional Information:
Phase: 7
Type: FOVER
Subtype: standby-update
Result: ALLOW
Config:
Additional Information:
Phase: 8
Type: NAT
Subtype: rpf-check
Result: ALLOW
Config:
static (inside,outside) 10.10.10.10 10.10.10.10 netmask 255.255.255.255
match ip inside host 10.10.10.10 outside any
static translation to 10.10.10.10
translate_hits = 0, untranslate_hits = 68
Additional Information:
Phase: 9
Type: NAT
Subtype: host-limits
Result: ALLOW
Config:
static (inside,outside) 10.10.10.10 10.10.10.10 netmask 255.255.255.255
match ip inside host 10.10.10.10 outside any
static translation to 10.10.10.10
translate_hits = 0, untranslate_hits = 68
Additional Information:
Phase: 10
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Phase: 11
Type: FLOW-CREATION
Subtype:
Result: ALLOW
Config:
Additional Information:
New flow created with id 124969, packet dispatched to next module
Result:
input-interface: outside
input-status: up
input-line-status: up
output-interface: inside
output-status: up
output-line-status: up
Action: allow
ASA#
Based on packet-tracer, I can see that it allow this traffic. Hmm...
I also googled the error message and found the following information.
106001
Error Message %PIX|ASA-2-106001: Inbound TCP connection denied from IP_address/port
to IP_address/port flags tcp_flags on interface interface_name
Explanation This is a connection-related message. This message occurs when an attempt to connect to an inside address is denied by your security policy. Possible tcp_flags values correspond to the flags in the TCP header that were present when the connection was denied. For example, a TCP packet arrived for which no connection state exists in the security appliance, and it was dropped. The tcp_flags in this packet are FIN and ACK.
The tcp_flags are as follows:
•ACK—The acknowledgment number was received.
•FIN—Data was sent.
•PSH—The receiver passed data to the application.
•RST—The connection was reset.
•SYN—Sequence numbers were synchronized to start a connection.
•URG—The urgent pointer was declared valid.
Recommended Action None required.
http://www.cisco.com/en/US/docs/security/asa/asa70/system/message/logmsgs.html
Since there is no information that can help me, I did a packet capture and found this.
9 packets captured1: 13:37:05.331449 192.168.1.1.1337 > 10.10.10.10.23: S 2194094828:2194094828(0) win 64512 <[|tcp]>
2: 13:37:05.331785 10.10.10.10.23 > 192.168.1.1.1337: S 1102669092:1102669092(0) ack 2194094829 win 8192 <[|tcp]>
3: 13:37:05.332273 10.10.10.10.23 > 192.168.1.1.1337: S 1102669092:1102669092(0) ack 2194094829 win 8192 <[|tcp]>
4: 13:37:08.370662 192.168.1.1.1337 > 10.10.10.10.23: S 2194094828:2194094828(0) win 64512 <[|tcp]>
5: 13:37:11.326124 10.10.10.10.23 > 192.168.1.1.1337: S 1102669092:1102669092(0) ack 2194094829 win 8192 <[|tcp]>
6: 13:37:11.326704 10.10.10.10.23 > 192.168.1.1.1337: S 1102669092:1102669092(0) ack 2194094829 win 8192 <[|tcp]>
7: 13:37:14.386469 192.168.1.1.1337 > 10.10.10.10.23: S 2194094828:2194094828(0) win 64512 <[|tcp]>
8: 13:37:23.325804 10.10.10.10.23 > 192.168.1.1.1337: S 1102669092:1102669092(0) ack 2194094829 win 8192 <[|tcp]>
9: 13:37:23.326322 10.10.10.10.23 > 192.168.1.1.1337: S 1102669092:1102669092(0) ack 2194094829 win 8192 <[|tcp]>
9 packets shown
TCP 3 way handshake is not completed due to last ACK packet from user is not sent to the server 10.10.10.10.
I’ll let you guys figure out what is the answer for this, I’ll share the solution later
06-24-2011 03:43 AM
Could you please paste the captures in pcap format, I want to eliminate any assymetric routing issues.
Thanks,
Varun
10-06-2011 02:34 AM
Hey, did you ever get to the bottom of this?
What was the solution, what did you learn??!
I'm having the same problem at the moment. In theory I've ruled assymetric routing, NAT, Server actually listening on specific port and ACL's all out, yet I'm still getting a Sev 2 Message in my syslog;
2 Oct 06 2011 10:32:14 106001 [SOURCE IP] 1800 [DESTINATION IP] 1433 Inbound TCP connection denied from [SOURCE IP]/1800 to [DESTINATION IP]/1433 flags SYN on interface [Interface that Source IP lives on]
halp!
10-06-2011 04:08 AM
Hi Guys,
I figured out my issue. It was to do with Security Levels and that the traffic I was looking at was trying to go from one Interface to another both with the same Security Level and that I did not have the "enable traffic between two or more interfaces which are configured with same security levels" tick box checked.
I managed to work around this by elevating the security level of one of the interfaces to be 5 higher (and thus different) from the other and traffic is now allowed to pass freely as per my ACL's intended.
Cheers and sorry for digging up an old topic!
11-17-2011 12:05 AM
Hi,
Did you forget to post your solution ?
The only root cause I am thinkg is relating to asymetric rooting as well. I don't understand how we can got this message is other case...
Rgd,
/Fabien
11-17-2011 12:27 AM
It could not be Asymmetric cuz you can see the SYN and SYN/ACK.packet reaching the ASA on the capture.
I hardly think that
TCP 3 way handshake is not completed due to last ACK packet from user is not sent to the server 10.10.10.10.
Was the problem, as the SYN-ACK never reached the client because it is dropped by the firewall. Lack of info.
Mike
11-17-2011 12:54 AM
Yes, you have right. SYN/ACK reach FW and but does not through it...
Fabien
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide