12-27-2011 12:20 PM - edited 03-11-2019 03:07 PM
Hi everyone, I was hoping that I could get some help for an issue we are having and i'm about to rip my hair out.
Out setup consists of two locations, one ASA 5505 (security license) \ at our main site and the other is a remote site with a cisco 1921 acting as the edge device.
The ASA 5505 acting as an edge device at our main site. From our ISP, two vlans come into the ASA (one for public internet traffic, one for the remote site, set up as a VC). The main site and the remote site are both separate LAN subnets, with a third subnet acting as a serial between the two locations.
At our main site, the ASA can access the public internet just fine, it can also ping to the gateway address on the 1921 (for their lan10.34.60.245: below) and receive a reply. This tells me that the Route commands are all set up fine as well as NAT translations to the public internet. The idea in this case is to have the remote site send all data back to the ASA5505 (think of the VC as one long cable connecting the two) and the ASA will handle the actual public internet connectivity as well as allowing connectivity to their private LAN (to access servers). I.E both LANs need to be able to talk to eachother.
the remote site as we stand now is able to ping the other end of the serial IP (10.1.1.1) but that's it. It can't ping the main site LAN gateway and it can't ping anything on the public internet.
I've narrowed the problem down to something on this ASA that isn't allowing these private LANs to communicate however I have no idea what it is. Any help would be very much appreciated.
here is some info to help
REMOTE SITE LAN =
10.34.60.0/24 (gateway is 10.34.60.245)
Serial IP on the VC: 10.1.1.0/30
ASA sh run:
interface Vlan1
nameif inside
security-level 100
ip address 10.25.102.245 255.255.255.0
!
interface Vlan783
nameif Internet
security-level 0
ip address X.X.X.X 255.255.255.252
!
interface Vlan789
nameif remotesite
security-level 100
ip address 10.1.1.1 255.255.255.252
!
interface Ethernet0/0
switchport trunk allowed vlan 783,789
switchport mode trunk
speed 10
duplex full
ftp mode passive
clock timezone EST -5
clock summer-time EDT recurring
dns server-group DefaultDNS
domain-name workgroup
same-security-traffic permit inter-interface
same-security-traffic permit intra-interface
access-list ingress extended deny ip 0.0.0.0 0.255.255.255 any
access-list ingress extended deny ip 127.0.0.0 255.255.255.0 any
access-list ingress extended deny ip 169.254.0.0 255.255.0.0 any
access-list ingress extended deny ip 172.16.0.0 255.255.240.0 any
access-list ingress extended deny ip 192.0.2.0 255.255.255.0 any
access-list ingress extended deny ip 192.168.0.0 255.255.0.0 any
access-list ingress extended deny ip 224.0.0.0 255.255.255.224 any
access-list ingress extended permit ip any any
access-list inside_nat0_outbound extended permit ip 10.25.102.0 255.255.255.0 10.34.60.0 255.255.255.0
access-list egress extended permit ip any any
access-list remotesite extended permit ip any any
pager lines 24
logging buffer-size 40960
mtu inside 1500
mtu Internet 1500
mtu remotesite1500
no failover
icmp unreachable rate-limit 1 burst-size 1
icmp permit any inside
icmp permit any Internet
icmp permit any remotesite
no asdm history enable
arp timeout 14400
global (inside) 11 interface
global (Internet) 1 interface
global (remotesite) 10 interface
nat (inside) 0 access-list inside_nat0_outbound
nat (inside) 1 10.25.102.0 255.255.255.0
nat (inside) 10 0.0.0.0 0.0.0.0
nat (remotesite) 1 10.34.60.0 255.255.255.0
nat (remotesite) 11 0.0.0.0 0.0.0.0
static (inside,remotesite) 10.25.102.100 10.25.102.100 netmask 255.255.255.255
static (inside,remotesite) 10.1.1.100 10.1.1.100 netmask 255.255.255.255
access-group egress in interface inside
access-group ingress in interface Internet
access-group remotesite in interface remotesite
route Internet 0.0.0.0 0.0.0.0 204.186.244.193 1
route inside 10.25.102.0 255.255.255.0 10.25.102.145 1
route remotesite10.34.60.0 255.255.255.0 10.1.1.2 1
timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
timeout tcp-proxy-reassembly 0:01:00
dynamic-access-policy-record DfltAccessPolicy
aaa authentication ssh console LOCAL
no snmp-server location
no snmp-server contact
snmp-server enable traps snmp authentication linkup linkdown coldstart
crypto ipsec security-association lifetime seconds 28800
crypto ipsec security-association lifetime kilobytes 4608000
ssh timeout 5
console timeout 0
01-06-2012 02:13 PM
Hi,
Nat exemption or Identity nat would be the option!
Idendity nat would be :
static (remotesite,remotesite) 10.34.60.0 10.34.60.0 netmask 255.255.255.0
You can give it a try and let me know.
If this does not work. please provide the entire config ( of course change the ips for security purposes)
Julio
01-09-2012 05:34 AM
Ok, so as far as I can tell it isn't working at this moment, but it very well still may be working. The reason I say this is because right now I am sending ping requests from the remote site router to the LAN gateway on the ASA, which are failing. Therefore the ping request is coming from 10.1.1.2 (The serial int of the Router and ASA) rather than the actual LAN of 10.34.60.0. So right now it may be working fine, but I can't really test unless I have a host on the LAN of 10.34.60.0 ping a host on 10.25.102.0. Is that sound logic? I'll post the entire config below in case any other errors occur.
I see this error in sh log:
Jan 09 2012 01:08:16: %ASA-5-305013: Asymmetric NAT rules matched for forward and reverse flows; Connection for udp src inside:10.25.102.20/59874 dst remotesite:10.34.60.106/161 denied due to NAT reverse path failure
ASA Version 8.2(2)
!
hostname ciscoasa
domain-name workgroup
enable password xxxxxxxxxxxxx
passwd xxxxxxxxxxxxxxx
names
!
interface Vlan1
nameif inside
security-level 100
ip address 10.25.102.245 255.255.255.0
!
interface Vlan783
nameif Internet
security-level 0
ip address x.x.x.194 255.255.255.252
!
interface Vlan789
nameif remotesite
security-level 100
ip address 10.1.1.1 255.255.255.252
!
interface Ethernet0/0
switchport trunk allowed vlan 783,789
switchport mode trunk
speed 10
duplex full
!
interface Ethernet0/1
!
ftp mode passive
dns server-group DefaultDNS
domain-name workgroup
same-security-traffic permit inter-interface
same-security-traffic permit intra-interface
access-list ingress extended deny ip 0.0.0.0 0.255.255.255 any
access-list ingress extended deny ip 127.0.0.0 255.255.255.0 any
access-list ingress extended deny ip 169.254.0.0 255.255.0.0 any
access-list ingress extended deny ip 172.16.0.0 255.255.240.0 any
access-list ingress extended deny ip 192.0.2.0 255.255.255.0 any
access-list ingress extended deny ip 192.168.0.0 255.255.0.0 any
access-list ingress extended deny ip 224.0.0.0 255.255.255.224 any
access-list ingress extended permit tcp any host x.x.x.194 eq telnet
access-list ingress extended permit icmp any host x.x.x.194
access-list ingress extended deny ip any host x.x.x.194
access-list ingress extended permit ip any any
access-list inside_nat0_outbound extended permit ip 10.25.102.0 255.255.255.0 10.34.60.0 255.255.255.0
access-list egress extended permit ip any any
access-list remotesite extended permit ip any any
access-list capout extended permit ip host 10.34.60.62 any
access-list capout extended permit ip host 10.25.102.245 any
access-list capin extended permit ip host 10.25.102.60 any
pager lines 24
logging enable
logging timestamp
logging buffer-size 40960
logging monitor debugging
logging buffered debugging
logging trap informational
logging asdm informational
mtu inside 1500
mtu Internet 1500
mtu remotesite1500
no failover
icmp unreachable rate-limit 1 burst-size 1
icmp permit any inside
icmp permit any Internet
icmp permit any remotesite
no asdm history enable
arp timeout 14400
global (inside) 11 interface
global (Internet) 1 interface
global (remotesite) 10 interface
nat (inside) 0 access-list inside_nat0_outbound
nat (inside) 1 10.25.102.0 255.255.255.0
nat (inside) 1 0.0.0.0 0.0.0.0
nat (remotesite) 2 10.64.30.0 255.255.255.0
nat (remotesite) 1 0.0.0.0 0.0.0.0
static (inside,remotesite) 10.25.102.100 10.25.102.100 netmask 255.255.255.255
static (inside,remotesite) 10.1.1.100 10.1.1.100 netmask 255.255.255.255
static (remotesite,Internet) x.x.x.194 10.1.1.2 netmask 255.255.255.255
static (remotesite,remotesite) 10.34.60.0 10.34.60.0 netmask 255.255.255.0
access-group egress in interface inside
access-group ingress in interface Internet
access-group remotesite in interface remotesite
route Internet 0.0.0.0 0.0.0.0 204.186.244.193 1
route inside 10.25.102.0 255.255.255.0 10.25.102.145 1
route remotesite10.34.60.0 255.255.255.0 10.1.1.2 1
timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
timeout tcp-proxy-reassembly 0:01:00
dynamic-access-policy-record DfltAccessPolicy
aaa authentication ssh console LOCAL
no snmp-server location
no snmp-server contact
snmp-server enable traps snmp authentication linkup linkdown coldstart
crypto ipsec security-association lifetime seconds 28800
crypto ipsec security-association lifetime kilobytes 4608000
ssh x.x.x.0 255.255.255.0 Internet
ssh timeout 60
console timeout 0
threat-detection basic-threat
threat-detection statistics access-list
no threat-detection statistics tcp-intercept
!
class-map inspection_default
match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
parameters
message-length maximum 512
policy-map global_policy
class inspection_default
inspect ftp
inspect h323 h225
inspect h323 ras
inspect rsh
inspect rtsp
inspect esmtp
inspect sqlnet
inspect skinny
inspect sunrpc
inspect xdmcp
inspect sip
inspect netbios
inspect tftp
!
prompt hostname context
call-home
profile CiscoTAC-1
no active
destination address http https://tools.cisco.com/its/service/oddce/services/DDCEService
destination address email callhome@cisco.com
destination transport-method http
subscribe-to-alert-group diagnostic
subscribe-to-alert-group environment
subscribe-to-alert-group inventory periodic monthly
subscribe-to-alert-group configuration periodic monthly
subscribe-to-alert-group telemetry periodic daily
Cryptochecksum:6d7f16bf3d9680b2b458c7ad9d6e1f62
: end
01-09-2012 10:31 AM
Hello John,
You are doing the identity nat just from the remote site to the inside, not from the inside to the remote site.
Please do the following:
no global (remotesite) 10 interface
static (inside,remotesite) 10.25.102.0 10.25.102.0 netmask 255.255.255.0
and give it a try...
Regards,
01-09-2012 10:51 AM
This is what the NAT configs look like right now.
global (inside) 11 interface
global (Internet) 1 interface
nat (inside) 0 access-list inside_nat0_outbound
nat (inside) 1 10.25.102.0 255.255.255.0
nat (inside) 1 0.0.0.0 0.0.0.0
nat (remotesite) 2 10.64.30.0 255.255.255.0
nat (remotesite) 1 0.0.0.0 0.0.0.0
static (inside,remotesite) 10.25.102.100 10.25.102.100 netmask 255.255.255.255
static (inside,remotesite) 10.1.1.100 10.1.1.100 netmask 255.255.255.255
static (remotesite,Internet) x.x.x.194 10.1.1.2 netmask 255.255.255.255
static (remotesite,remotesite) 10.34.60.0 10.34.60.0 netmask 255.255.255.0
static (inside,remotesite) 10.25.102.0 10.25.102.0 netmask 255.255.255.0
It's still not working but at this point I may have found out why. The customer is telling me that right now this ASA plugs into a switch and right now there is also a second ASA plugged into that switch with the same default gateway configured. This second ASA will be replaced, but considering the switch's arp table hasn't been cleared, it would explain why a ping fails from the remotesite when I try to ping a host with 10.25.102.x because the switch returns the request to the other ASA. This would explain why I can't ping certain hosts, but it doesn't explain why I can't ping 10.25.102.245 from the remotesite router (10.1.1.2)
01-09-2012 10:59 AM
Hello John,
That is correct, let me know what happens when the second ASA gets replaced.
Julio
01-10-2012 07:26 AM
tests were unsuccessful.
Here is my NAT Table as of now:
global (inside) 11 interface
global (Internet) 1 interface
nat (inside) 0 access-list inside_nat0_outbound
nat (inside) 1 10.25.102.0 255.255.255.0
nat (inside) 1 0.0.0.0 0.0.0.0
nat (remotesite) 2 10.64.30.0 255.255.255.0
nat (remotesite) 1 0.0.0.0 0.0.0.0
static (remotesite,Internet) 204.186.113.194 10.1.1.2 netmask 255.255.255.255
static (inside,remotesite) 10.25.102.100 10.25.102.100 netmask 255.255.255.255
static (remotesite,remotesite) 10.34.60.0 10.34.60.0 netmask 255.255.255.0
static (inside,remotesite) 10.25.102.0 10.25.102.0 netmask 255.255.255.0
static (inside,remotesite) 10.1.1.100 10.1.1.100 netmask 255.255.255.255
"packet-tracer input remotesite tcp 10.34.60.15 1025 10.25.102.15 80" shows the following which concerns me.
Phase: 6
Type: NAT
Subtype: host-limits
Result: ALLOW
Config:
static (remotesite,remotesite) 10.34.60.0 10.34.60.0 netmask 255.255.255.0
match ip remoteisite10.34.60.0 255.255.255.0 remotesite any
static translation to 10.34.60.0
translate_hits = 0, untranslate_hits = 0
Additional Information:
Phase: 7
Type: NAT
Subtype:
Result: DROP
Config:
nat (remotesite) 1 0.0.0.0 0.0.0.0
match ip remotesiteany inside any
dynamic translation to pool 1 (No matching global)
translate_hits = 96, untranslate_hits = 0
Additional Information:
Result:
input-interface: remotesite
input-status: up
input-line-status: up
output-interface: inside
output-status: up
output-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule
So, right now the two devices can reach the public internet fine, but they are unable to speak with eachother. I've even included in the config both the
same-security-traffic permit inter-interface
same-security-traffic permit intra-interface
commands enabled so I have no idea what is going on. Either the ASA is trying to make a translation that it shouldn't be, or it is blocking the data in some other way. I have no clue right now why these two lans can't talk to eachother.
01-10-2012 10:33 AM
i've been searching the internet for the past few hours looking for a fix and stumbled upon a few people with similar setups (albeit not exactly the same) and they found an answer by using TCP State Bypass. Is that something worth exploring in this case?
EDIT: I also found this in the log, when i tried to ping the LAN gateway on the ASA 10.25.102.245 from the remotesite router
Jan 10 2012 06:28:35: %ASA-7-609001: Built local-host remotesite:10.34.60.34
Jan 10 2012 06:28:35: %ASA-5-305013: Asymmetric NAT rules matched for forward and reverse flows; Connection for tcp src inside:10.25.102.194/5713 dst remotesite:10.34.60.34/1069 denied due to NAT reverse path failure
Jan 10 2012 06:28:35: %ASA-6-106015: Deny TCP (no connection) from 10.25.102.194/5713 to 10.34.60.34/1069 flags SYN ACK on interface inside
Jan 10 2012 06:28:35: %ASA-7-609002: Teardown local-host remotesite:10.34.60.34 duration 0:00:00
01-10-2012 11:02 AM
Hello,
First change this:
no static (remotesite,remotesite) 10.34.60.0 10.34.60.0 netmask 255.255.255.0
static (remotesite,inside) 10.34.60.0 10.34.60.0 netmask 255.255.255.0
Now lets see the tcp state bypass:
access-list test permit tcp 10.25.102.0 255.255.255.0 10.34.60.0 255.255.255.0
class-map test
match access-list test
policy-map global_policy
class test
set connection advanced-options tcp-state-bypass
Then give it a try
Regards,
01-10-2012 11:23 AM
Still not working, but now I'm seeing these errors
Jan 10 2012 07:04:31: %ASA-7-609001: Built local-host remotesite:10.34.60.49
Jan 10 2012 07:04:31: %ASA-6-106015: Deny TCP (no connection) from 10.25.102.194/5713 to 10.34.60.49/2268 flags FIN ACK on interface inside
Jan 10 2012 07:04:31: %ASA-7-609002: Teardown local-host inside:10.25.102.194 duration 0:00:00
Jan 10 2012 07:04:31: %ASA-7-609002: Teardown local-host remotesite:10.34.60.49 duration 0:00:00
here is the NAT table:
global (inside) 11 interface
global (Internet) 1 interface
nat (inside) 0 access-list inside_nat0_outbound
nat (inside) 1 10.25.102.0 255.255.255.0
nat (inside) 1 0.0.0.0 0.0.0.0
nat (remotesite) 2 10.64.30.0 255.255.255.0
nat (remotesite) 1 0.0.0.0 0.0.0.0
static (remotesite,Internet) x.x.x.194 10.1.1.2 netmask 255.255.255.255
static (inside,remotesite) 10.25.102.100 10.25.102.100 netmask 255.255.255.255
static (remotesite,inside) 10.34.60.0 10.34.60.0 netmask 255.255.255.0
"packet-tracer input remotesite tcp 10.34.60.15 1025 10.25.102.15 80 shows the following which all looks good:
Phase: 10
Type: NAT
Subtype: host-limits
Result: ALLOW
Config:
nat (inside) 1 10.25.102.0 255.255.255.0
match ip inside 10.25.102.0 255.255.255.0 inside any
dynamic translation to pool 1 (No matching global)
translate_hits = 2, untranslate_hits = 0
Additional Information:
Phase: 11
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Phase: 12
Type: FLOW-CREATION
Subtype:
Result: ALLOW
Config:
Additional Information:
New flow created with id 59333, packet dispatched to next module
Result:
input-interface: remotesite
input-status: up
input-line-status: up
output-interface: inside
output-status: up
output-line-status: up
Action: allow
but still, i can't get teh remotesite router to ping 10.25.102.245, which tells me that no address ffrom that site will communicate with a host on the local LAN. sooooo close
01-10-2012 11:36 AM
Hello,
Please add the following and try to ping again:
policy-map global_policy
class inspection_default
inspect icmp
Regards,
01-10-2012 11:46 AM
still not replying, sh log:
Jan 10 2012 07:29:18: %ASA-6-302020: Built inbound ICMP connection for faddr 10.1.1.2/229 gaddr 10.25.102.245/0 laddr 10.25.102.245/0
so its building a connection, but not getting responses back to the remotesite router
01-10-2012 11:52 AM
Hello John,
Lets do a capture
access-list icmp permit icmp host 10.1.1.2 host 10.25.102.245
access-list icmp permit icmp host 10.25.102.245 host 10.1.1.2
capture icmp1 access-list icmp interface remotesite
capture icmp2 access-list icmp interface inside
capture asp type asp-drop all
Please try to ping and provide the following
sh cap icmp1
sh cap icmp2
sh cap asp | include 10.1.1.2
sh cap asp | include 10.25.102.245
01-10-2012 11:56 AM
sh cap icmp1
5 packets captured
1: 07:38:36.328382 802.1Q vlan#789 P0 10.1.1.2 > 10.25.102.245: icmp: echo request
2: 07:38:38.327909 802.1Q vlan#789 P0 10.1.1.2 > 10.25.102.245: icmp: echo request
3: 07:38:40.327390 802.1Q vlan#789 P0 10.1.1.2 > 10.25.102.245: icmp: echo request
4: 07:38:42.326917 802.1Q vlan#789 P0 10.1.1.2 > 10.25.102.245: icmp: echo request
5: 07:38:44.326383 802.1Q vlan#789 P0 10.1.1.2 > 10.25.102.245: icmp: echo request
5 packets shown
sh cap icmp2
0 packet captured
0 packet shown
sh cap asp | include 10.1.1.2
sh cap asp | include 10.25.102.245
these two don't report anything, just jump down to the next line
EDIT: I tried pinging host 10.25.102.201 from the remotesite rouer (since I could ping it from the ASA and get responses), this came up but didn't return any responses back to the remotesite router
sh cap asp | include 10.1.1.2
2799: 08:02:35.773794 802.1Q vlan#789 P0 10.1.1.2 > 10.25.102.201: icmp: echo requ est
2800: 08:02:37.772787 802.1Q vlan#789 P0 10.1.1.2 > 10.25.102.201: icmp: echo requ est
2802: 08:02:39.772176 802.1Q vlan#789 P0 10.1.1.2 > 10.25.102.201: icmp: echo requ est
2804: 08:02:41.771764 802.1Q vlan#789 P0 10.1.1.2 > 10.25.102.201: icmp: echo requ est
2806: 08:02:43.771215 802.1Q vlan#789 P0 10.1.1.2 > 10.25.102.201: icmp: echo requ est
01-10-2012 12:27 PM
Also, don't know if this helps but I ran the following comand from the ASA:
packet-tracer input remotesite tcp 10.1.1.2 1025 10.25.102.15 80
I'm taking it that this attempted to send a packet to 10.1.1.2 (The ip of the router of which I'm pinging from) from the fake internet host of 10.25.102.15. The results are below
Phase: 5
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group Bartonsville in interface BartonsvilleVC
access-list Bartonsville extended permit ip any any
Additional Information:
Phase: 6
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Phase: 7
Type: NAT
Subtype: host-limits
Result: ALLOW
Config:
static (remotesite,Internet) x.x.x.194 10.1.1.2 netmask 255.255.255.255
match ip remotesite host 10.1.1.2 Internet any
static translation to x.x.x.194
translate_hits = 81, untranslate_hits = 49
Additional Information:
Phase: 8
Type: NAT
Subtype:
Result: DROP
Config:
nat (remotesite) 1 0.0.0.0 0.0.0.0
match ip remotesite any inside any
dynamic translation to pool 1 (No matching global)
translate_hits = 171, untranslate_hits = 0
Additional Information:
Result:
input-interface: remotesite
input-status: up
input-line-status: up
output-interface: inside
output-status: up
output-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule
this part, in phase 7
static (remotesite,Internet) x.x.x.194 10.1.1.2 netmask 255.255.255.255
refers to the static that I configured for remote access using a public IP
so this is telling me that data sent from the ASA lan to the remote router ip is failing?
01-10-2012 12:36 PM
Hello John,
Please provide updated config on the ASA, and show ip route of the router on the remote site.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide