10-03-2011 05:53 AM
Hi there,
I am hoping someone here can help. I have been struggling for some time to sort out issues in a VPN we have between our main London office and the Edinburgh branch office. We have an ASA 5510 in London, talking to a Vigor 2820 in Edinburgh.
The London office has a 192.168.0.0/24 subnet, with the default gateway as a Cisco Catalyst at 192.168.0.254, and the Cisco ASA at 192.168.0.254 as the firewall.
The Edinburgh office has the subnet 192.168.2.0/24, with the Vigor running on 192.168.2.1, providing routing, DHCP and firewall services there.
I have the VPN working fine, correctly routing traffic between those two subnets over the IPsec tunnel. However, I have had much trouble adding additional subnets for our VLANs in London.
What I want to happen is traffic from 192.168.2.0/24 to be able to get to and from 192.168.50.0/24 and several similar networks.
Upon tracing it using the Cisco packet tracer, I can see that the packets for the 192.168.50.0/24 subnet are not making it over the tunnel, having being stopped by the VPN: subtype: encrypt rules. Looking at these rules though, I can't spot the problem. Multiple changes of order of the rules, and reloads have not sorted out the problem. When I run a packet trace on the main subnet it works fine. I have attached some of the configuration (below) as well as the output from the packet tracer, and the config of the Vigor router.
I apologise in advance for the length of the post, but I have tried to include all relevant information to see if anyone can help.
Firstly, here's the ASA config that seemed relevant. I tried to remove some since we have quite a few site-to-site tunnels set up, and these are probably not relevant (and are all working correctly).
access-list insideOutboundNonatAcl extended permit ip 192.168.0.0 255.255.255.0 192.168.10.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.0.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.0.0 255.255.255.0 192.168.3.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.0.0 255.255.255.0 192.168.0.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.0.0 255.255.255.0 192.168.20.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.20.0 255.255.255.0 192.168.0.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.50.0 255.255.255.0 192.168.0.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.30.0 255.255.255.0 192.168.0.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.40.0 255.255.255.0 192.168.0.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.20.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.40.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.30.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.50.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip any 192.168.0.192 255.255.255.192
access-list insideOutboundNonatAcl extended permit ip 192.168.0.0 255.255.0.0 192.168.7.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.7.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.2.0 255.255.255.0 192.168.7.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.0.0 255.255.0.0 192.168.2.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.0.0 255.255.0.0 192.168.0.0 255.255.0.0
nat (inside) 0 access-list insideOutboundNonatAcl
nat (inside) 9 access-list vpnNatAcl
nat (inside) 10 192.168.30.5 255.255.255.255
nat (inside) 10 192.168.0.0 255.255.255.0
nat (inside) 10 192.168.20.0 255.255.255.0
nat (inside) 10 192.168.30.0 255.255.255.0
nat (inside) 10 192.168.50.0 255.255.255.0
access-list inside_in extended permit ip 192.168.0.0 255.255.255.0 any
access-list inside_in extended permit tcp host 192.168.5.2 host 192.168.0.2 eq domain
access-list inside_in extended permit ip 192.168.20.0 255.255.255.0 192.168.0.0 255.255.255.0
access-list inside_in extended permit ip 192.168.20.0 255.255.255.0 any
access-list inside_in extended permit ip 192.168.50.0 255.255.255.0 any
access-list inside_in extended permit ip 192.168.30.0 255.255.255.0 any
access-list inside_in extended permit ip 192.168.30.0 255.255.255.0 192.168.0.0 255.255.255.0
access-list inside_in extended permit ip 192.168.40.0 255.255.255.0 192.168.0.0 255.255.255.0
access-list inside_in extended permit ip 192.168.40.0 255.255.255.0 any
access-list inside_in extended permit ip 192.168.10.0 255.255.255.0 any
access-list inside_in extended permit ip host 192.168.2.1 192.168.30.0 255.255.255.0 inactive
access-list inside_in extended permit ip 192.168.2.0 255.255.255.0 192.168.50.0 255.255.255.0
access-list inside_in extended permit ip 192.168.2.0 255.255.255.0 192.168.0.0 255.255.255.0
access-group inside_in in interface inside
access-list outside_2_cryptomap extended permit ip 192.168.20.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list outside_2_cryptomap extended permit ip 192.168.30.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list outside_2_cryptomap extended permit ip 192.168.40.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list outside_2_cryptomap extended permit ip 192.168.50.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list outside_2_cryptomap extended permit ip 192.168.10.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list outside_2_cryptomap extended permit ip 192.168.0.0 255.255.255.0 192.168.2.0 255.255.255.0
route inside 192.168.20.0 255.255.255.0 192.168.0.254 1
route inside 192.168.50.0 255.255.255.0 192.168.0.254 1
route inside 192.168.30.0 255.255.255.0 192.168.0.254 1
route inside 192.168.40.0 255.255.255.0 192.168.0.254 1
crypto ipsec transform-set ESP_DES_MD5 esp-des esp-md5-hmac
crypto ipsec transform-set TRANS_VPN_SET esp-3des esp-md5-hmac
crypto ipsec transform-set TRANS_VPN_SET mode transport
crypto ipsec transform-set TRANS_VPN_SET_2 esp-3des esp-sha-hmac
crypto ipsec transform-set TRANS_VPN_SET_2 mode transport
crypto ipsec transform-set ESP_3DES_SHA esp-3des esp-sha-hmac
crypto ipsec transform-set ESP_3DES_MD5 esp-3des esp-md5-hmac
crypto ipsec transform-set ESP-3DES-MD5 esp-3des esp-md5-hmac
crypto ipsec df-bit clear-df outside
crypto dynamic-map core_vpn_dyn_map 20 set transform-set ESP_3DES_MD5 ESP_DES_MD5 TRANS_VPN_SET TRANS_VPN_SET_2
crypto dynamic-map core_vpn_dyn_map 40 set pfs
crypto dynamic-map core_vpn_dyn_map 40 set transform-set ESP_3DES_SHA ESP_DES_MD5
crypto map outside_map 2 match address outside_2_cryptomap
crypto map outside_map 2 set pfs
crypto map outside_map 2 set peer [branch peer ip]
crypto map outside_map 2 set transform-set ESP_3DES_MD5
crypto isakmp identity address
crypto isakmp identity address
crypto isakmp policy 25
authentication pre-share
encryption 3des
hash md5
group 1
lifetime 28800
crypto isakmp nat-traversal 30
crypto isakmp disconnect-notify
group-policy DfltGrpPolicy attributes
banner none
wins-server none
dns-server none
dhcp-network-scope none
vpn-access-hours none
vpn-simultaneous-logins 100
vpn-idle-timeout none
vpn-session-timeout none
vpn-filter none
vpn-tunnel-protocol IPSec l2tp-ipsec webvpn
password-storage disable
ip-comp disable
re-xauth enable
group-lock none
pfs disable
ipsec-udp disable
ipsec-udp-port 10000
split-tunnel-policy tunnelall
split-tunnel-network-list none
default-domain none
split-dns none
intercept-dhcp 255.255.255.255 disable
secure-unit-authentication disable
user-authentication disable
user-authentication-idle-timeout 30
ip-phone-bypass disable
leap-bypass disable
nem disable
backup-servers keep-client-config
msie-proxy server none
msie-proxy method no-modify
msie-proxy except-list none
msie-proxy local-bypass disable
nac disable
nac-sq-period 300
nac-reval-period 36000
nac-default-acl none
address-pools none
smartcard-removal-disconnect enable
client-firewall none
client-access-rule none
tunnel-group [branch peer ip] type ipsec-l2l
tunnel-group [branch peer ip] ipsec-attributes
pre-shared-key *
Note: [branch peer ip] replaces any instances of the branch office outside IP address
I appreciate there may be some duplicated/redundant rules here - I have been playing with config to try to fix the problem. I'd really appreciate any suggestions on how to track this down.
Here's the vigor config:
So it looks to match ok to me at both ends, unless there is something I missed. The vigor routing table shows:
Key: C - connected, S - static, R - RIP, * - default, ~ - private
* 0.0.0.0/ 0.0.0.0 via [ISP gateway server], WAN1
S [branch peer ip]/ 255.255.255.255 via [branch peer ip], WAN1
S~ 192.168.40.0/ 255.255.255.0 via [London office ip], VPN
S~ 192.168.50.0/ 255.255.255.0 via [London office ip], VPN
S~ 192.168.10.0/ 255.255.255.0 via [London office ip], VPN
S~ 192.168.0.0/ 255.255.255.0 via [London office ip], VPN
C~ 192.168.2.0/ 255.255.255.0 is directly connected, LAN
S~ 192.168.7.0/ 255.255.255.0 via [London office ip], VPN
S~ 192.168.30.0/ 255.255.255.0 via [London office ip], VPN
S~ 192.168.20.0/ 255.255.255.0 via [London office ip], VPN
* [ISP dns server]/ 255.255.255.255 via [ISP gateway server], WAN1
I have replaced IPs here as is shown. You can see the vigor seems to want to route the appropriate traffic over the VPN.
Finally, here is the packet trace output:
ciscoasa# packet-trace input outside tcp 192.168.2.1 echo 192.168.50.10 echo d$
Phase: 1
Type: FLOW-LOOKUP
Subtype:
Result: ALLOW
Config:
Additional Information:
Found no matching flow, creating a new flow
Phase: 2
Type: ROUTE-LOOKUP
Subtype: input
Result: ALLOW
Config:
Additional Information:
in 192.168.50.0 255.255.255.0 inside
Phase: 3
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group outsideInAcl in interface outside
access-list outsideInAcl extended permit ip 192.168.2.0 255.255.255.0 any
Additional Information:
Forward Flow based lookup yields rule:
in id=0x4529e48, priority=12, domain=permit, deny=false
hits=362922, user_data=0x4529e08, cs_id=0x0, flags=0x0, protocol=0
src ip=192.168.2.0, mask=255.255.255.0, port=0
dst ip=0.0.0.0, mask=0.0.0.0, port=0
Phase: 4
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x44057f0, priority=0, domain=permit-ip-option, deny=true
hits=2693939, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip=0.0.0.0, mask=0.0.0.0, port=0
dst ip=0.0.0.0, mask=0.0.0.0, port=0
Phase: 5
Type: NAT-EXEMPT
Subtype: rpf-check
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x44fe9a0, priority=6, domain=nat-exempt-reverse, deny=false
hits=12, user_data=0x44fe800, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
src ip=192.168.2.0, mask=255.255.255.0, port=0
dst ip=192.168.50.0, mask=255.255.255.0, port=0
Phase: 6
Type: NAT
Subtype: rpf-check
Result: ALLOW
Config:
nat (inside) 10 192.168.50.0 255.255.255.0
match ip inside 192.168.50.0 255.255.255.0 outside any
dynamic translation to pool 10 (external [Interface PAT])
translate_hits = 2250, untranslate_hits = 17
Additional Information:
Forward Flow based lookup yields rule:
out id=0x4b80e80, priority=1, domain=nat-reverse, deny=false
hits=32, user_data=0x4b80ce0, cs_id=0x0, flags=0x0, protocol=0
src ip=0.0.0.0, mask=0.0.0.0, port=0
dst ip=192.168.50.0, mask=255.255.255.0, port=0
Phase: 7
Type: NAT
Subtype: host-limits
Result: ALLOW
Config:
nat (inside) 10 192.168.50.0 255.255.255.0
match ip inside 192.168.50.0 255.255.255.0 outside any
dynamic translation to pool 10 (external [Interface PAT])
translate_hits = 2250, untranslate_hits = 17
Additional Information:
Reverse Flow based lookup yields rule:
in id=0x4b80fa0, priority=1, domain=host, deny=false
hits=2811, user_data=0x4b80ce0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip=192.168.50.0, mask=255.255.255.0, port=0
dst ip=0.0.0.0, mask=0.0.0.0, port=0
Phase: 8
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in id=0x4469ef8, priority=0, domain=permit-ip-option, deny=true
hits=2010804, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip=0.0.0.0, mask=0.0.0.0, port=0
dst ip=0.0.0.0, mask=0.0.0.0, port=0
Phase: 9
Type: VPN
Subtype: encrypt
Result: DROP
Config:
Additional Information:
Reverse Flow based lookup yields rule:
out id=0x4887aa8, priority=70, domain=encrypt, deny=false
hits=10, user_data=0x0, cs_id=0x44b18f8, reverse, flags=0x0, protocol=0
src ip=192.168.50.0, mask=255.255.255.0, port=0
dst ip=192.168.2.0, mask=255.255.255.0, port=0
Result:
input-interface: outside
input-status: up
input-line-status: up
output-interface: inside
output-status: up
output-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule
So it seems to find the rule, which it ought to match, but then returns DENY. What's going on here? Perhaps this is misleading and the issue is elsewhere, but it isn't clear from the output here.
For further information, this is output for the WORKING subnet - I have just taken a small part here though:
Phase: 10
Type: VPN
Subtype: encrypt
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
out id=0x4b86418, priority=70, domain=encrypt, deny=false
hits=332214, user_data=0x7da5c, cs_id=0x44b18f8, reverse, flags=0x0, protocol=0
src ip=192.168.0.0, mask=255.255.255.0, port=0
dst ip=192.168.2.0, mask=255.255.255.0, port=0
Thanks very much in advance for any help you can provide - I've been really stuck on this one!
Chris
04-13-2012 07:44 AM
Bumping this, I have the same issue. God I hate it when people write that. No replies on the thread though. Is this resolved or abandoned?
04-14-2012 11:46 AM
Hi,
Can you issue the packet-tracer with the direction beeing your London office -> Remote office?
Also issue the command twice.
Personally I've used packet-tracer with some L2L VPNs to test if the remote end has the configurations correct. Also I've noticed that the first packet-tracer test never goes through. So issue that command twice and show how it goes.
Though I imagine you have tried to connect through the L2L VPN with real host machines and not just the firewalls packet-tracer?
Also I imagine the original info has a typo. You say your ASAs LAN gateway IP and the local L3 switches IP address is the same, 192.168.0.254.
Basically the hardest part regarding L2L VPNs should be the initial setup of the VPN connection. Even though it should be simple people still tend to mess up PSKs or Phase1/2 parameters. But as your L2L VPN is already in working order and you are just adding networks to it, it should be pretty simple.
When you add network and dont require any special NAT configurations, your NAT0 and Encryption domain access-list should look pretty much the same.
And looking at your configurations, it should be like this
access-list outside_2_cryptomap extended permit ip 192.168.20.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list outside_2_cryptomap extended permit ip 192.168.30.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list outside_2_cryptomap extended permit ip 192.168.40.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list outside_2_cryptomap extended permit ip 192.168.50.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list outside_2_cryptomap extended permit ip 192.168.10.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list outside_2_cryptomap extended permit ip 192.168.0.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.20.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.30.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.40.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.50.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.10.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list insideOutboundNonatAcl extended permit ip 192.168.0.0 255.255.255.0 192.168.2.0 255.255.255.0
Btw what is the network 192.168.7.0/24? It seems to have a VPN rule at the remote site but not at the HO site. Though there is a NAT0 rule for that traffic on the HO site.
EDIT: I imagine the VPN network rules should be an exact mirror image of eachother. Though it seems this doesnt stop devices from negotiating the VPN up but who knows if some other device type is picky about that one. Only thing in your situation that I see is the network 192.168.7.0/24 that is not included in the other ends configurations.
EDIT2: Also the reason your test for the already existing rule might be going through without a problem might be because the tunnel is up and working for the networks in question.
EDIT3: Does your Vigor device also have NAT0 rules configured for the new networks?
- Jouni
04-16-2012 12:30 AM
After doing a bit more research it would seem that at least 1 fix to this issue is to set up a separate VPN rule on the Vigor for EACH remote subnet. It seems silly, particularly when there is an option to add multiple subnets to the main crypto map but as soon as I did this my second subnet tunnel came up. Also note, I only had to add my additional subnets to the ASA end's existing crypto map and it did NOT require an additional tunnel.
RE Jouni:
The configuration of VPNs on the Vigor is totally different to the ASA unfortunately. It is very much an SME targeted device and doesn't have the advanced enterprise configuration that you get with the ASA. It does all the NAT0 itself for example and you have little control over it.
The odd thing is that not only does the Vigor suggest that it's working properly (all the relevant subnets appear in the route table) but that packet tracing the ASA shows issues. I changed nothing on my ASA and only added my second VPN rule on the Vigor and it now flows properly. I imagine the Vigor isn't negotiating the crypto maps between devices properly.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide