DMVPN: traffic between two DMVPN's fails when adding ipsec

[cpdev]

Hello Colleagues,

this is our setup (of cause a bit cleaned up):

we configured a Hub and spoke DMVPN with MPLS, OSPF and BGP.

R6-R5(WAN/Tun1) forms a DMVPN and R8-R5(LAN/Tun2) forms a second DMVPN.

DMVPN comes up and we are able to ping each router and resources in the networks behind R3.

Routing appears to be correct.

Now we want to encrypt the WAN traffic via Tun1.

Once we apply IPSEC ikev2 profile on the DMVPN  WAN (Tun1)  traffic flow from DMVPN WAN Spoke R6 to DMVPN LAN Spoke R8 (e.g. a PING) stops and MPLS shows a strange behaviour (e.g. R3 loopback not reachable any more). 

Nonetheless, as far as we tested any other resource is still reachable. 

Spokes - R6 Config

interface Tunnel1
 description DMVPN-WAN
 bandwidth 1000000
 bandwidth inherit 1000000
 ip address 10.253.253.99 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip flow ingress
 ip pim sparse-mode
 ip nhrp authentication GSN-IT
 ip nhrp map multicast dynamic
 ip nhrp map multicast <WANIP>
 ip nhrp map 10.253.253.1 <WANIP>
 ip nhrp network-id 1
 ip nhrp holdtime 10
 ip nhrp nhs 10.253.253.1 priority 1 cluster 1
 ip nhrp nhs fallback 5
 ip nhrp registration timeout 5
 ip tcp adjust-mss 1360
 ip ospf network point-to-multipoint
 ip ospf 1 area 0
 mpls traffic-eng tunnels
 mpls bgp forwarding
 mpls ip
 keepalive 10 3
 cdp enable
 tunnel source Dialer1
 tunnel mode gre multipoint
 tunnel path-mtu-discovery
 tunnel bandwidth transmit 1000000
 tunnel bandwidth receive 1000000
 tunnel protection ipsec profile test1_ipsec_prof
end


crypto ikev2 proposal test1_proposal
 encryption aes-cbc-256 aes-cbc-128 3des
 integrity sha1
 group 2

crypto ikev2 policy test1_ikev2-Policy
 proposal test1_proposal

crypto ikev2 keyring test1_keyr
 peer DMVPN
 address 0.0.0.0 0.0.0.0
 pre-shared-key cisco123
 !

crypto ikev2 profile test1_prof_ikev2
 match fvrf any
 match identity remote any
 authentication local pre-share
 authentication remote pre-share
 keyring local test1_keyr
 dpd 60 2 on-demand

crypto ipsec transform-set AES256 esp-aes 256 esp-sha-hmac
 mode transport

crypto ipsec fragmentation after-encryption

crypto ipsec profile test1_ipsec_prof
 set transform-set AES256
 set ikev2-profile test1_prof_ikev2 

 Hub- R6 Config:

interface Tunnel1
 description DMVPN-WAN
 bandwidth 1000000
 bandwidth inherit 1000000
 ip address 10.253.253.1 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip pim sparse-mode
 ip nhrp authentication <name>
 ip nhrp map multicast dynamic
 ip nhrp network-id 1
 ip nhrp holdtime 600
 ip tcp adjust-mss 1360
 ip ospf network point-to-multipoint
 ip ospf 1 area 0
 mpls ip
 mpls traffic-eng tunnels
 mpls bgp forwarding
 cdp enable
 tunnel source Loopback100
 tunnel mode gre multipoint
 tunnel path-mtu-discovery
 tunnel vrf WAN
 tunnel protection ipsec profile test1_ipsec_prof
 crypto engine slot 2/0 inside 
end

interface Tunnel2
 description DMVPN-LAN
 bandwidth 1000000
 bandwidth inherit 1000000
 ip address 10.253.254.1 255.255.255.0
 no ip redirects
 ip mtu 1440
 ip pim sparse-mode
 ip nhrp authentication <ommit>
 ip nhrp map multicast dynamic
 ip nhrp network-id 100
 ip nhrp holdtime 600
 ip tcp adjust-mss 1360
 ip ospf network point-to-multipoint
 ip ospf 1 area 0
 ip ospf cost 10
 mpls ip
 mpls traffic-eng tunnels
 mpls bgp forwarding
 cdp enable
 tunnel source Loopback2
 tunnel mode gre multipoint
 tunnel path-mtu-discovery
 tunnel vrf LAN
end

crypto ikev2 proposal test1_proposal 
 encryption aes-cbc-256 aes-cbc-128 3des 
 integrity sha1 
 group 2 

crypto ikev2 policy test1_ikev2-Policy 
 match fvrf WAN 
 proposal test1_proposal 

crypto ikev2 keyring test1_keyr 
 peer DMVPN 
  address 0.0.0.0 0.0.0.0 
  pre-shared-key cisco123 
! 
crypto ikev2 profile test1_prof_ikev2 
 match fvrf WAN 
 match identity remote any 
 authentication remote pre-share 
 authentication local pre-share 
 keyring local test1_keyr 
 dpd 60 2 on-demand 

crypto engine mode vrf 
crypto engine gre supervisor 

crypto ipsec transform-set AES256 esp-aes 256 esp-sha-hmac 
 mode transport 

crypto ipsec profile test1_ipsec_prof 
 set transform-set AES256 
 set ikev2-profile test1_prof_ikev2

ip vrf WAN
 description WAN
 rd 65001:10
 route-target export 65001:10
 route-target import 65001:10

ip vrf LAN
 rd 65001:100

We are working on this issue since a few days. So maybe we miss the forest for the trees...

Thanks for your help.

PS: attached a snapshot of the used HW and IOS.

[Georg Pauwen]

Hello,

at first glance it looks like the tunnel on your hub has no ipsec protection configured. Add the line in bold to your tunnel config:

HUB-R6

interface Tunnel1
description DMVPN01
bandwidth 1000000
bandwidth inherit 1000000
ip address 10.253.253.1 255.255.255.0
no ip redirects
ip mtu 1400
ip pim sparse-mode
ip nhrp authentication <name>
ip nhrp map multicast dynamic
ip nhrp network-id 1
ip nhrp holdtime 600
ip tcp adjust-mss 1360
ip ospf network point-to-multipoint
ip ospf 1 area 0
mpls ip
mpls traffic-eng tunnels
mpls bgp forwarding
cdp enable
tunnel source Loopback100
tunnel mode gre multipoint
tunnel path-mtu-discovery
tunnel vrf WAN
crypto engine slot 2/0 inside
tunnel protection ipsec profile test1_ipsec_prof

[cpdev]

Hi George,

you are right, but its "only" a copy and paste error (I correct it in the post).

right now ipsec is off due to the connectivity issues.

[paul driver]

Hello

Just like to ask  - when you mean spoke to spoke - are you saying prior to any IPsec being added you had spoke to spoke phase 2 connectivity- meaning traffic between R6-R8 bypassed the NHS of R5

Now looking at the ospf network type you’ve applied any routes advertised from either of these spokes in ospf would have the point nexthop if the Nhs and not itself?  - So you would not have proper phase two spoke-to spoke reachability 

To rectify this you would need to use either a broadcast or non broadcast ospf  network type which will allow the advertisement of the originating rtr IP address  to be the nexthop and not the NHS

res

Paul 

[cpdev]

Hi Paul,

we will check that and report back.

Thanks.

[cpdev]

I updated the post above to give a more detailed explanantion.

So far its not a routing issue.

[Georg Pauwen]

Hello,

I am in the process of recreating your setup in GNS3. Can you post the full config of your hub and one of the spoke routers ?

[Georg Pauwen]

Hello,

I managed to have full connectivity using your IKEv2/IPSec configs, but with both spokes mapped to the same tunnel on the hub. So I suspect that is where your problem is, the two different tunnels on the hub.

Post the full configs of all (4) routers involved. Without seeing the details it is just guesswork...

[cpdev]

Hi George,

attached the router configs... the 4th is in this case only for showing that there is more behind and not only two spokes and a Hub..

[Georg Pauwen]

Thanks for the configs, I'll adjust my lab. I won't have any results until tomorrow, I am on GMT+1, which means it is almost midnight...:)

[Georg Pauwen]

Hello,

after adjusting my lab, my connections are working using IPSec and IKEv2 exactly as you have configured it. Below is the output of 'sh crypto session'...does yours look similar (actually, post the output of this command from your hub)...

HUB#sh crypto session
Crypto session current status

Interface: Tunnel0
Profile: IKEV2_PROFILE
Session status: UP-ACTIVE
Peer: 192.168.12.2 port 500
Session ID: 3
IKEv2 SA: local 192.168.0.2/500 remote 192.168.12.2/500 Active
IPSEC FLOW: permit 47 host 192.168.0.2 host 192.168.12.2
Active SAs: 2, origin: crypto map

Interface: Tunnel0
Profile: IKEV2_PROFILE
Session status: UP-ACTIVE
Peer: 192.168.11.2 port 500
Session ID: 2
IKEv2 SA: local 192.168.0.2/500 remote 192.168.11.2/500 Active
IPSEC FLOW: permit 47 host 192.168.0.2 host 192.168.11.2
Active SAs: 2, origin: crypto map

Interface: Tunnel0
Profile: IKEV2_PROFILE
Session status: UP-ACTIVE
Peer: 192.168.13.2 port 500
Session ID: 1
IKEv2 SA: local 192.168.0.2/500 remote 192.168.13.2/500 Active
IPSEC FLOW: permit 47 host 192.168.0.2 host 192.168.13.2
Active SAs: 2, origin: crypto map

[cpdev]

Hello Georg,

Sorry for the late answer.

In the meantime we introduced a second WAN Spoke to analyze if the topic is related to IOS or Hardware IOS combination. See below

From the HUB:

Crypto session current status
 
Interface: Tunnel1
Profile: test1_prof_ikev2
Session status: UP-ACTIVE
Peer: WAN-ip-Spoke_1 port 500
 Session ID: 71
 IKEv2 SA: local local-wan-ip/500 remote WAN-ip-Spoke_1/500 Active
 IPSEC FLOW: permit 47 host local-wan-ip host WAN-ip-Spoke_1
 Active SAs: 2, origin: crypto map
 
Interface: Tunnel1
Profile: test1_prof_ikev2
Session status: UP-ACTIVE
Peer: WAN-ip-Spoke_2 port 500
 Session ID: 49
 IKEv2 SA: local local-wan-ip/500 remote WAN-ip-Spoke_2/500 Active
 IPSEC FLOW: permit 47 host local-wan-ip WAN-ip-Spoke_2
 Active SAs: 2, origin: crypto map
 
HUB#sh cry ip sa
 
interface: Tunnel1
 Crypto map tag: Tunnel1-head-0, local addr <local-wan-ip>
 
 protected vrf: (none)
 local ident (addr/mask/prot/port): (local-wan-ip/255.255.255.255/47/0)
 remote ident (addr/mask/prot/port): (WAN-ip-Spoke_2/255.255.255.255/47/0)
 current_peer WAN-ip-Spoke_2 port 500
 PERMIT, flags={origin_is_acl,}
 #pkts encaps: 14393, #pkts encrypt: 14393, #pkts digest: 14393
 #pkts decaps: 19441, #pkts decrypt: 19441, #pkts verify: 19441
 #pkts compressed: 0, #pkts decompressed: 0
 #pkts not compressed: 0, #pkts compr. failed: 0
 #pkts not decompressed: 0, #pkts decompress failed: 0
 #send errors 0, #recv errors 0
 
 local crypto endpt.: local-wan-ip, remote crypto endpt.: WAN-ip-Spoke_2
 plaintext mtu 1458, path mtu 1500, ip mtu 1500, ip mtu idb (none)
 current outbound spi: 0xC89EEDCF(3365858767)
 PFS (Y/N): N, DH group: none
 
 inbound esp sas:
 spi: 0xE2C16B82(3804326786)
 transform: esp-256-aes esp-sha-hmac ,
 in use settings ={Transport, }
 conn id: 2114, flow_id: :114, sibling_flags 80000008, crypto map: Tunnel1-head-0
 sa timing: remaining key lifetime (k/sec): (4605732/1081)
 IV size: 16 bytes
 replay detection support: Y
 Status: ACTIVE(ACTIVE)
 
 inbound ah sas:
 
 inbound pcp sas:
 
 outbound esp sas:
 spi: 0xC89EEDCF(3365858767)
 transform: esp-256-aes esp-sha-hmac ,
 in use settings ={Transport, }
 conn id: 2113, flow_id: :113, sibling_flags 80000008, crypto map: Tunnel1-head-0
 sa timing: remaining key lifetime (k/sec): (4605772/1081)
 IV size: 16 bytes
 replay detection support: Y
 Status: ACTIVE(ACTIVE)