Solved: Re: ASA route ping but not TCP packets between two internal LANs - Page 3

atishin · ‎01-06-2012

I need to connect two internal LANs each of which has ASA as a firewall to outside. One has ASA 5505 with two interfaces and another - ASA 5510 with three interfaces. I managed to pass echo packets from one internal LAN to another, but not the TCP packets. It must be something simple that I missed. Any help will be highly appreciated! Here is the network diagram:

Here is Config from ASA 5510 (i removed obvious settings to save space):

interface Ethernet0/0
 nameif outside
 security-level 0
 ip address YY.YY.YY.YY 255.255.255.224 
!
interface Ethernet0/1
 nameif inside
 security-level 100
 ip address 192.168.35.1 255.255.255.0 
!
interface Ethernet0/2
 nameif a-02
 security-level 100
 ip address 192.168.30.250 255.255.255.0 
!
interface Ethernet0/3
 shutdown
 no nameif
 no security-level
 no ip address
!
ftp mode passive
clock timezone EST -5
clock summer-time EDT recurring
dns server-group DefaultDNS
 domain-name latista.local
same-security-traffic permit inter-interface
same-security-traffic permit intra-interface
access-list outside_access_in extended permit icmp any any inactive 
access-list a-02_access_in extended permit ip any any 
access-list a-02_access_in extended permit icmp any any inactive 
access-list inside_access_in extended permit ip any any 
access-list inside_access_in extended permit icmp any any inactive 
access-list inside_access_out extended permit ip any any 
access-list inside_access_out extended permit icmp any any inactive 
access-list inside_nat0_outbound_1 extended permit ip 192.168.35.0 255.255.255.0 192.168.30.0 255.255.255.0 
access-list inside_nat0_outbound extended permit ip 192.168.35.0 255.255.255.0 192.168.30.0 255.255.255.0 
access-list a-02_nat0_outbound extended permit ip 192.168.30.0 255.255.255.0 192.168.35.0 255.255.255.0 
access-list a-02_nat0_outbound_1 extended permit ip 192.168.30.0 255.255.255.0 192.168.35.0 255.255.255.0 
pager lines 24
logging enable
logging asdm informational
mtu outside 1500
mtu inside 1500
mtu a-02 1500
mtu management 1500
no failover
icmp unreachable rate-limit 1 burst-size 1
no asdm history enable
arp timeout 14400
nat (inside) 0 access-list inside_nat0_outbound
nat (inside) 0 access-list inside_nat0_outbound_1 outside
nat (a-02) 0 access-list a-02_nat0_outbound
nat (a-02) 0 access-list a-02_nat0_outbound_1 outside
access-group outside_access_in in interface outside
access-group inside_access_in in interface inside
access-group inside_access_out out interface inside
access-group a-02_access_in in interface a-02
!
router rip
 version 1
!
route outside 0.0.0.0 0.0.0.0 205.251.79.33 1
route inside 192.168.30.0 255.255.255.0 192.168.30.250 1
timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
timeout tcp-proxy-reassembly 0:01:00
timeout floating-conn 0:00:00
dynamic-access-policy-record DfltAccessPolicy
http server enable
http 192.168.1.0 255.255.255.0 management
http 192.168.35.0 255.255.255.0 inside
http 67.208.89.64 255.255.255.224 outside
http 4.26.115.0 255.255.255.240 outside
http 192.168.30.0 255.255.255.0 a-02
http 192.168.20.0 255.255.255.0 a-02
http 96.255.26.199 255.255.255.255 outside
no snmp-server location
no snmp-server contact
snmp-server enable traps snmp authentication linkup linkdown coldstart
crypto ipsec security-association lifetime seconds 28800
crypto ipsec security-association lifetime kilobytes 4608000
telnet timeout 5
ssh timeout 5
console timeout 0
!
threat-detection basic-threat
threat-detection statistics access-list
threat-detection statistics tcp-intercept rate-interval 30 burst-rate 400 average-rate 200
webvpn
!
class-map inspection_default
 match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
 parameters
  message-length maximum client auto
  message-length maximum 512
policy-map global_policy
 class inspection_default
  inspect dns preset_dns_map 
  inspect ftp 
  inspect h323 h225 
  inspect h323 ras 
  inspect rsh 
  inspect rtsp 
  inspect esmtp 
  inspect sqlnet 
  inspect skinny  
  inspect sunrpc 
  inspect xdmcp 
  inspect sip  
  inspect netbios 
  inspect tftp 
  inspect ip-options 
  inspect http 
!
service-policy global_policy global

On another ASA (ASA 5505) I only configured the Routing and NAT Exemption. Here is that portion:

same-security-traffic permit intra-interface

access-list inside_nat0_outbound extended permit ip NET_COLO_INT 255.255.255.0 192.168.35.0 255.255.255.0 
access-list inside_nat0_outbound extended permit ip 192.168.35.0 255.255.255.0 NET_COLO_INT 255.255.255.0

route inside 192.168.35.0 255.255.255.0 192.168.30.250 1

Please help!

Julio Carvajal · ‎01-06-2012

The configuration is fine, Lets see a capture in here,

Can you create one just like the one we used on the other ASA, also can you create the ASP capture and attach the files.

Regards,

Julio

Julio Carvajal
Senior Network Security and Core Specialist
CCIE #42930, 2xCCNP, JNCIP-SEC

atishin · ‎01-06-2012

Julio,

Attached is the PCAP from ASA 5505 (since only one interface involved on ASA5505 - there is only one file). ASP response is following (I was using 30.6 server instead of 30.3 on 30.0 network):

Result of the command: "show capture asp | include 192.168.30.6"

1: 22:01:06.418710 802.1Q vlan#1 P0 192.168.30.6.2806 > 192.168.30.1.443: F 1271193653:1271193653(0) ack 1272526702 win 65285 Drop-reason: (tcp-not-syn) First TCP packet not SYN

Result of the command: "show capture asp | include 192.168.35.2"

2: 22:01:09.345258 802.1Q vlan#1 P0 192.168.30.14.135 > 192.168.35.2.3852: S 2211736781:2211736781(0) ack 2112571713 win 8192 Drop-reason: (tcp-not-syn) First TCP packet not SYN

4: 22:01:21.341794 802.1Q vlan#1 P0 192.168.30.14.135 > 192.168.35.2.3852: R 2211736782:2211736782(0) win 0

Julio Carvajal · ‎01-07-2012

Hello,

When you did the capture, how was the defautl gateway of the server innitiating the connection?

Edit: The drop reason says assymetric routing, but those drop messages are not involved with the communication we are looking for.

Again 192.168.30.6 is sendig the reset packet.

Please provide following output

packet-tracer input inside tcp 192.168.30.6 1025 192.168.35.2 3389

Julio

Julio Carvajal
Senior Network Security and Core Specialist
CCIE #42930, 2xCCNP, JNCIP-SEC

atishin · ‎01-07-2012

Julio,

On 192.168.30.6 the default gateway is 192.168.30.1 (e.g. ASA 5505's inside interface).

Here is trace from ASA 5505:

Result of the command: "packet-tracer input inside tcp 192.168.30.6 1025 192.168.35.2 3389"

Phase: 1

Type: CAPTURE

Subtype:

Result: ALLOW

Config:

Additional Information:

MAC Access list

Phase: 2

Type: ACCESS-LIST

Subtype:

Result: ALLOW

Config:

Implicit Rule

Additional Information:

MAC Access list

Phase: 3

Type: FLOW-LOOKUP

Subtype:

Result: ALLOW

Config:

Additional Information:

Found no matching flow, creating a new flow

Phase: 4

Type: ROUTE-LOOKUP

Subtype: input

Result: ALLOW

Config:

Additional Information:

in 192.168.35.0 255.255.255.0 inside

Phase: 5

Type: ACCESS-LIST

Subtype: log

Result: ALLOW

Config:

access-group inside_access_in in interface inside

access-list inside_access_in extended permit ip any any

Additional Information:

Phase: 6

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Phase: 7

Type: NAT-EXEMPT

Subtype:

Result: ALLOW

Config:

match ip inside NET_COLO_INT 255.255.255.0 inside 192.168.35.0 255.255.255.0

NAT exempt

translate_hits = 5734, untranslate_hits = 0

Additional Information:

Phase: 8

Type: NAT-EXEMPT

Subtype: rpf-check

Result: ALLOW

Config:

match ip inside 192.168.35.0 255.255.255.0 inside NET_COLO_INT 255.255.255.0

NAT exempt

translate_hits = 0, untranslate_hits = 4300

Additional Information:

Phase: 9

Type: NAT

Subtype:

Result: ALLOW

Config:

nat (inside) 1 NET_COLO_INT 255.255.255.0

match ip inside NET_COLO_INT 255.255.255.0 inside any

dynamic translation to pool 1 (No matching global)

translate_hits = 33, untranslate_hits = 0

Additional Information:

Phase: 10

Type: NAT

Subtype: host-limits

Result: ALLOW

Config:

nat (inside) 1 NET_COLO_INT 255.255.255.0

match ip inside NET_COLO_INT 255.255.255.0 inside any

dynamic translation to pool 1 (No matching global)

translate_hits = 33, untranslate_hits = 0

Additional Information:

Phase: 11

Type: HOST-LIMIT

Subtype:

Result: ALLOW

Config:

Additional Information:

Phase: 12

Type: ACCESS-LIST

Subtype: log

Result: ALLOW

Config:

access-group inside_access_out out interface inside

access-list inside_access_out extended permit ip any any

Additional Information:

Phase: 13

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Phase: 14

Type: FLOW-CREATION

Subtype:

Result: ALLOW

Config:

Additional Information:

New flow created with id 19228897, packet dispatched to next module

Result:

input-interface: inside

input-status: up

input-line-status: up

output-interface: inside

output-status: up

output-line-status: up

Action: allow

Julio Carvajal · ‎01-07-2012

Hello,

The packet tracer again let us know everything is well configured on the ASA..

Hmm Can you try using 2 different host that the ones you have used before ( I know you already changed the Linux box) and please make sure the windows firewall is disabled just in case..

Regards,

Julio Carvajal
Senior Network Security and Core Specialist
CCIE #42930, 2xCCNP, JNCIP-SEC

atishin · ‎01-07-2012

Windows firewall is disabled (entire service) on all our servers.

I am not sure what you ask me to do with other server - do you want me to run packet tracer and use 30.2 instead of 30.6?

Julio Carvajal · ‎01-08-2012

Hello,

Good to know Firewall is disabled.. At this point I can let you know the problem is not the ASA. Why this:

1-ICMP traffic is traversing the ASA

2-ASP captures are not showing the ASA to be dropping those connections

3- On all the captures on both ASAs we see the traffic traversing the interfaces so the asa is doing the right thing.

4-On all the captures on the ASA we see a RST packet comming from one server

5-On a capture on the server we see the server sending the RST packet without talking to the ASA.

So I think on the ASA side, we have troubleshoot it!

Julio

Julio Carvajal
Senior Network Security and Core Specialist
CCIE #42930, 2xCCNP, JNCIP-SEC

atishin · ‎01-08-2012

Julio,

I think I found explanation of my case - see this link: http://www.8-p.org/wiki/doku.php?id=asahairpinning

As I understand - when TCP packet originated from 30.0 net - it is routed via ASA5505 towards ASA5510, but when ASA5510 ack - it sends reply directly to source host in 30.0 net instead of sening it via ASA5505 and hence breaks the 3-way handshake. They call it hairpinning, I guess.

As a solution - they suggest that on ASA5510 I need to create a route-map which will instruct ASA5510 to have a next hop as ASA5505 (e.g. 192.168.30.1) and not the source host in 30.0 net. Now I need some help of how to create such route-map since I am not that deep in ASA8.4 programming. Thanks!

atishin · ‎01-08-2012

OK. Finally I managed to work it out. I disabled TCP State for that route (PBR is not available in ASA as I misassumed in previous suggesion). Here is what I did - just for sake of someone who might bump into similar problem:

ASA(config)#access-list STATE_BYPASS_ACL extended permit tcp 192.168.30.0 255.255.255.0 192.168.35.0 255.255.255.0ASA(config)#class-map STATE_BYPASS_CMAPASA(config-cmap)#match access-list STATE_BYPASS_ACLASA(config-cmap)#exit
ASA(config)#policy-map STATE_BYPASS_PMAPASA(config-pmap)#class STATE_BYPASS_CMAPASA(config-pmap-c)#set connection advanced-options tcp-state-bypassASA(config-pmap-c)#exit
ASA(config)#service-policy STATE_BYPASS_PMAP interface inside

Julio - thank you a lot for the helping me to troubleshoot this issue!

Julio Carvajal · ‎01-08-2012

Hello Anatoly,

So the TCP state-bypass did it!

Great to hear everything is working, please mark the question as answered so as you said future users can view the solution of this.

Regards,

Julio

Julio Carvajal
Senior Network Security and Core Specialist
CCIE #42930, 2xCCNP, JNCIP-SEC