01-18-2022 05:20 AM
I have a head office with Internet over MPLS (MPLS.35). I then have a multi homed branch office that has Line of Site to the head office(MPLS.50) and also BT fibre on a separate VRF (BT.99)
I'm trying to run DMVPN to go over the LoS and the BT link. So if the LoS drops we can use BT as a backup path. I have configured two tunnels with different network keys.
I'm completely at a loss. I've tried to include the config I think is relevant:
Head Office Router ----------------------- interface Port-channel1 description to MPLS ip address MPLS.35 255.255.255.248 ip nat outside zone-member security Internet no negotiation auto interface Tunnel0 description COMPANY-VPN bandwidth 40000 ip address 10.255.250.1 255.255.255.0 no ip redirects ip mtu 1400 ip hello-interval eigrp 10 1 ip hold-time eigrp 10 3 no ip split-horizon eigrp 10 ip nhrp authentication M4 ip nhrp network-id 100000 ip nhrp holdtime 360 ip nhrp nhs 10.255.250.1 zone-member security Trusted ip tcp adjust-mss 1360 cdp enable tunnel source Port-channel1 tunnel mode gre multipoint tunnel key 100000 tunnel protection ipsec profile protect-gre shared interface Tunnel2 description DMVPNwithBT bandwidth 20000 ip address 10.255.2.1 255.255.255.0 no ip redirects ip mtu 1400 ip hello-interval eigrp 10 1 ip hold-time eigrp 10 3 no ip split-horizon eigrp 10 ip nhrp authentication New ip nhrp network-id 100001 ip nhrp holdtime 360 ip nhrp nhs 10.255.2.6 nbma BT.99 multicast zone-member security Trusted ip tcp adjust-mss 1360 tunnel source Port-channel1 tunnel mode gre multipoint tunnel key 100001 tunnel protection ipsec profile protect-gre shared end crypto ipsec profile protect-gre set security-association lifetime seconds 86400 set transform-set COMPANY-Transform Multi Home Branch Office ------------------------- interface GigabitEthernet0/0/0 description Link to BCB-LoS-Rt via LoS ip address 172.16.193.4 255.255.255.248 secondary
!used to manage the LoS equipment) ip address MPLS.50 255.255.255.248 no ip redirects no ip unreachables no ip proxy-arp ip nat outside ip verify unicast source reachable-via rx allow-default 100 zone-member security Zone-Outside negotiation auto no cdp enable crypto map SDM_CMAP_1 ip virtual-reassembly end interface GigabitEthernet0/0/1 description WAN:BT Fibre Broadband (areacode259888) ip vrf forwarding BT ip address dhcp client-id GigabitEthernet0/0/2 ip nat outside zone-member security Zone-BTOutside negotiation auto no cdp enable ip virtual-reassembly end interface Tunnel0 description COMPANY-VPN bandwidth 40000 ip address 10.255.250.6 255.255.255.0 no ip redirects ip mtu 1400 ip nbar protocol-discovery ip hello-interval eigrp 10 1 ip hold-time eigrp 10 3 no ip split-horizon eigrp 10 ip nat inside ip nhrp authentication M4 ip nhrp map 10.255.250.1 MPLS.35 ip nhrp map multicast MPLS.35 ip nhrp network-id 100000 ip nhrp holdtime 360 ip nhrp nhs 10.255.250.1 zone-member security Zone-TunnelToCOMPANY ip tcp adjust-mss 1360 ip ospf network broadcast tunnel source GigabitEthernet0/0/0 tunnel mode gre multipoint tunnel key 100000 tunnel protection ipsec profile protect-gre ip virtual-reassembly end interface Tunnel2 description DMVPNwithBT bandwidth 20000 ip address 10.255.2.6 255.255.255.0 no ip redirects ip mtu 1400 ip hello-interval eigrp 10 1 ip hold-time eigrp 10 3 no ip split-horizon eigrp 10 ip nhrp authentication New ip nhrp network-id 100001 ip nhrp holdtime 360 zone-member security Zone-TunnelToCOMPANY ip tcp adjust-mss 1360 tunnel source GigabitEthernet0/0/1 tunnel mode gre multipoint tunnel key 100001 tunnel vrf BT tunnel protection ipsec profile protect-gre2 end crypto ipsec profile protect-gre set security-association lifetime seconds 86400 set transform-set COMPANY-Transform crypto ipsec profile protect-gre2 set security-association lifetime seconds 86400 set transform-set COMPANY-Transform HeadOffice#sh dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete N - NATed, L - Local, X - No Socket T1 - Route Installed, T2 - Nexthop-override, B - BGP C - CTS Capable, I2 - Temporary # Ent --> Number of NHRP entries with same NBMA peer NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting UpDn Time --> Up or Down Time for a Tunnel ========================================================================== Interface: Tunnel0, IPv4 NHRP Details Type:Hub, NHRP Peers:3, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 MPLS.50 10.255.250.6 UP 1w0d D 1 HomeUser 10.255.250.14 UP 5d22h DN 1 HomeUserB 10.255.250.16 UP 00:29:25 DN Interface: Tunnel2, IPv4 NHRP Details Type:Hub/Spoke, NHRP Peers:2, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 BT.99 10.255.2.6 IKE 3d22h S 1 HomeUserB 10.255.2.16 UP 00:29:25 DN MultiHome#sh dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete N - NATed, L - Local, X - No Socket T1 - Route Installed, T2 - Nexthop-override, B - BGP C - CTS Capable, I2 - Temporary # Ent --> Number of NHRP entries with same NBMA peer NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting UpDn Time --> Up or Down Time for a Tunnel ========================================================================== Interface: Tunnel0, IPv4 NHRP Details Type:Hub/Spoke, NHRP Peers:3, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 MPLS.35 10.255.250.1 UP 1w0d S 1 HomeUSerA 10.255.250.14 UP 6d15h DN 1 HOmeUSerB 10.255.250.16 UP 00:30:49 DN Interface: Tunnel2, IPv4 NHRP Details Type:Hub, NHRP Peers:1, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 HomeUserB 10.255.2.16 UP 00:30:48 DN
01-18-2022 07:15 AM
sh log | s 15230 Jan 18 15:09:40.797: ISAKMP: (15230):processing vendor id payload Jan 18 15:09:40.798: ISAKMP: (15230):vendor ID is Unity Jan 18 15:09:40.798: ISAKMP: (15230):processing vendor id payload Jan 18 15:09:40.798: ISAKMP: (15230):vendor ID is DPD Jan 18 15:09:40.798: ISAKMP: (15230):processing vendor id payload Jan 18 15:09:40.798: ISAKMP: (15230):speaking to another IOS box! Jan 18 15:09:40.798: ISAKMP: (15230):received payload type 20 Jan 18 15:09:40.798: ISAKMP: (15230):His hash no match - this node outside NAT Jan 18 15:09:40.798: ISAKMP: (15230):received payload type 20 Jan 18 15:09:40.798: ISAKMP: (15230):No NAT Found for self or peer Jan 18 15:09:40.798: ISAKMP: (15230):Input = IKE_MESG_INTERNAL, IKE_PROCESS_MAIN_MODE Jan 18 15:09:40.798: ISAKMP: (15230):Old State = IKE_I_MM4 New State = IKE_I_MM4 Jan 18 15:09:40.799: ISAKMP: (15230):Send initial contact Jan 18 15:09:40.799: ISAKMP: (15230):processing CERT_REQ payload. message ID = 0 Jan 18 15:09:40.799: ISAKMP: (15230):peer wants a CT_X509_SIGNATURE cert Jan 18 15:09:40.799: ISAKMP: (15230):peer wants cert issued by cn=sh-net-TH-SVR-DC1-CA,dc=sh-net,dc=local,ou=sh-net.local,o=Company Group Ltd,l=XXX,st=XXX,c=UK Jan 18 15:09:40.799: ISAKMP: (15230):processing CERT_REQ payload. message ID = 0 Jan 18 15:09:40.799: ISAKMP: (15230):peer wants a CT_X509_SIGNATURE cert Jan 18 15:09:40.799: ISAKMP: (15230):peer wants cert issued by cn=Company-HEDGE-CA,dc=Company,dc=local Jan 18 15:09:40.799: ISAKMP: (15230):processing CERT_REQ payload. message ID = 0 Jan 18 15:09:40.799: ISAKMP: (15230):peer wants a CT_X509_SIGNATURE cert Jan 18 15:09:40.800: ISAKMP: (15230):peer wants cert issued by cn=Cisco Licensing Root CA,o=Cisco Jan 18 15:09:40.800: ISAKMP: (15230):IKE->PKI Get self CertificateChain state (I) MM_KEY_EXCH (peer BT.99) Jan 18 15:09:40.800: ISAKMP: (15230):PKI->IKE Got self CertificateChain state (I) MM_KEY_EXCH (peer BT.99) Jan 18 15:09:40.800: ISAKMP: (15230):IKE->PKI Get SubjectName state (I) MM_KEY_EXCH (peer BT.99) Jan 18 15:09:40.802: ISAKMP: (15230):PKI->IKE Got SubjectName state (I) MM_KEY_EXCH (peer BT.99) Jan 18 15:09:40.802: ISAKMP-ERROR: (15230):My ID configured as IPv4 Addr, but Addr not in Cert! Jan 18 15:09:40.802: ISAKMP-ERROR: (15230):Using FQDN as My ID Jan 18 15:09:40.803: ISAKMP: (15230):SA is doing Jan 18 15:09:40.803: ISAKMP: (15230):RSA signature authentication using id type ID_FQDN Jan 18 15:09:40.804: ISAKMP: (15230):ID payload Jan 18 15:09:40.804: ISAKMP: (15230): FQDN name : HeadOffice-Rt1.Company.local Jan 18 15:09:40.804: ISAKMP: (15230): protocol : 17 Jan 18 15:09:40.804: ISAKMP: (15230):Total payload length: 29 Jan 18 15:09:40.804: ISAKMP: (15230):IKE->PKI Get CertificateChain to be sent to peer state (I) MM_KEY_EXCH (peer BT.99) Jan 18 15:09:40.806: ISAKMP: (15230):PKI->IKE Got CertificateChain to be sent to peer state (I) MM_KEY_EXCH (peer BT.99) Jan 18 15:09:40.808: ISAKMP: (15230):constructing CERT payload for cn=HeadOffice-Rt1.sh-net.com,ou=sh-net.local,st=Herefordshire O=Company,c=GB Jan 18 15:09:40.808: ISAKMP: (15230):using the shTrustPoint2023 trustpoint's keypair to sign Jan 18 15:09:40.847: ISAKMP-PAK: (15230):sending packet to BT.99 my_port 500 peer_port 500 (I) MM_KEY_EXCH Jan 18 15:09:40.847: ISAKMP: (15230):Sending an IKE IPv4 Packet. Jan 18 15:09:40.847: ISAKMP: (15230):Input = IKE_MESG_INTERNAL, IKE_PROCESS_COMPLETE Jan 18 15:09:40.847: ISAKMP: (15230):Old State = IKE_I_MM4 New State = IKE_I_MM5 Jan 18 15:09:50.775: ISAKMP-PAK: (15230):received packet from BT.99 dport 500 sport 500 Global (I) MM_KEY_EXCH Jan 18 15:09:50.775: ISAKMP: (15230):phase 1 packet is a duplicate of a previous packet. Jan 18 15:09:50.775: ISAKMP: (15230):retransmitting due to retransmit phase 1 Jan 18 15:09:51.275: ISAKMP: (15230):retransmitting phase 1 MM_KEY_EXCH... Jan 18 15:09:51.275: ISAKMP: (15230):: incrementing error counter on sa, attempt 1 of 5: retransmit phase 1 Jan 18 15:09:51.275: ISAKMP: (15230):retransmitting phase 1 MM_KEY_EXCH Jan 18 15:09:51.275: ISAKMP-PAK: (15230):sending packet to BT.99 my_port 500 peer_port 500 (I) MM_KEY_EXCH Jan 18 15:09:51.275: ISAKMP: (15230):Sending an IKE IPv4 Packet. Jan 18 15:10:00.774: ISAKMP-PAK: (15230):received packet from BT.99 dport 500 sport 500 Global (I) MM_KEY_EXCH Jan 18 15:10:00.774: ISAKMP: (15230):phase 1 packet is a duplicate of a previous packet. Jan 18 15:10:00.774: ISAKMP: (15230):retransmitting due to retransmit phase 1 Jan 18 15:10:01.274: ISAKMP: (15230):retransmitting phase 1 MM_KEY_EXCH... Jan 18 15:10:01.274: ISAKMP: (15230):: incrementing error counter on sa, attempt 2 of 5: retransmit phase 1 Jan 18 15:10:01.274: ISAKMP: (15230):retransmitting phase 1 MM_KEY_EXCH Jan 18 15:10:01.274: ISAKMP-PAK: (15230):sending packet to BT.99 my_port 500 peer_port 500 (I) MM_KEY_EXCH Jan 18 15:10:01.274: ISAKMP: (15230):Sending an IKE IPv4 Packet. Jan 18 15:10:10.773: ISAKMP-PAK: (15230):received packet from BT.99 dport 500 sport 500 Global (I) MM_KEY_EXCH Jan 18 15:10:10.773: ISAKMP: (15230):phase 1 packet is a duplicate of a previous packet. Jan 18 15:10:10.773: ISAKMP: (15230):retransmitting due to retransmit phase 1 Jan 18 15:10:10.816: ISAKMP: (15230):set new node 0 to QM_IDLE Jan 18 15:10:10.816: ISAKMP-ERROR: (15230):SA is still budding. Attached new ipsec request to it. (local MPLS.35, remote BT.99) Jan 18 15:10:11.273: ISAKMP: (15230):retransmitting phase 1 MM_KEY_EXCH... Jan 18 15:10:11.273: ISAKMP: (15230):: incrementing error counter on sa, attempt 3 of 5: retransmit phase 1 Jan 18 15:10:11.273: ISAKMP: (15230):retransmitting phase 1 MM_KEY_EXCH Jan 18 15:10:11.273: ISAKMP-PAK: (15230):sending packet to BT.99 my_port 500 peer_port 500 (I) MM_KEY_EXCH Jan 18 15:10:11.273: ISAKMP: (15230):Sending an IKE IPv4 Packet. Jan 18 15:10:20.774: ISAKMP-PAK: (15230):received packet from BT.99 dport 500 sport 500 Global (I) MM_KEY_EXCH Jan 18 15:10:20.774: ISAKMP: (15230):phase 1 packet is a duplicate of a previous packet. Jan 18 15:10:20.774: ISAKMP: (15230):retransmitting due to retransmit phase 1 Jan 18 15:10:21.274: ISAKMP: (15230):retransmitting phase 1 MM_KEY_EXCH... Jan 18 15:10:21.274: ISAKMP: (15230):: incrementing error counter on sa, attempt 4 of 5: retransmit phase 1 Jan 18 15:10:21.274: ISAKMP: (15230):retransmitting phase 1 MM_KEY_EXCH Jan 18 15:10:21.274: ISAKMP-PAK: (15230):sending packet to BT.99 my_port 500 peer_port 500 (I) MM_KEY_EXCH Jan 18 15:10:21.274: ISAKMP: (15230):Sending an IKE IPv4 Packet. Jan 18 15:10:30.774: ISAKMP-PAK: (15230):received packet from BT.99 dport 500 sport 500 Global (I) MM_KEY_EXCH Jan 18 15:10:30.774: ISAKMP: (15230):phase 1 packet is a duplicate of a previous packet. Jan 18 15:10:30.774: ISAKMP: (15230):retransmitting due to retransmit phase 1 Jan 18 15:10:31.275: ISAKMP: (15230):retransmitting phase 1 MM_KEY_EXCH... Jan 18 15:10:31.275: ISAKMP: (15230):: incrementing error counter on sa, attempt 5 of 5: retransmit phase 1 Jan 18 15:10:31.275: ISAKMP: (15230):retransmitting phase 1 MM_KEY_EXCH Jan 18 15:10:31.275: ISAKMP-PAK: (15230):sending packet to BT.99 my_port 500 peer_port 500 (I) MM_KEY_EXCH Jan 18 15:10:31.275: ISAKMP: (15230):Sending an IKE IPv4 Packet. Jan 18 15:10:40.816: ISAKMP: (15230):set new node 0 to QM_IDLE Jan 18 15:10:40.816: ISAKMP-ERROR: (15230):SA is still budding. Attached new ipsec request to it. (local MPLS.35, remote BT.99) Jan 18 15:10:41.275: ISAKMP: (15230):retransmitting phase 1 MM_KEY_EXCH... Jan 18 15:10:41.275: ISAKMP: (15230):peer does not do paranoid keepalives. Jan 18 15:10:41.275: ISAKMP-ERROR: (15230):deleting SA reason "Death by retransmission P1" state (I) MM_KEY_EXCH (peer BT.99) Jan 18 15:10:41.276: ISAKMP (15230): IPSec has no more SA's with this peer. Won't keepalive phase 1. Jan 18 15:10:41.276: ISAKMP-ERROR: (15230):deleting SA reason "Death by retransmission P1" state (I) MM_KEY_EXCH (peer BT.99) Jan 18 15:10:41.278: ISAKMP: (15230):deleting node 2523801782 error FALSE reason "IKE deleted" Jan 18 15:10:41.278: ISAKMP: (15230):deleting node 3642562032 error FALSE reason "IKE deleted" Jan 18 15:10:41.278: ISAKMP: (15230):deleting node 4100597187 error FALSE reason "IKE deleted" Jan 18 15:10:41.278: ISAKMP: (15230):IKE->PKI End PKI Session state (I) MM_NO_STATE (peer BT.99) Jan 18 15:10:41.279: ISAKMP: (15230):PKI->IKE Ended PKI Session state (I) MM_NO_STATE (peer BT.99) Jan 18 15:10:41.279: ISAKMP: (15230):Input = IKE_MESG_INTERNAL, IKE_PHASE1_DEL Jan 18 15:10:41.279: ISAKMP: (15230):Old State = IKE_I_MM5 New State = IKE_DEST_SA
01-18-2022 09:10 AM
Hi friend,
you have DMVPN VRF aware, this is nice design
Tunnel 0 is up because it is in global VRF and the destination of tunnel is reachable via global.
Tunnel 2 is down because it is in different VRF "BT" and the destination of tunnel is unreachable via this VRF...
so you need static route from VRF BT to global to make Branch VRF BT reach the destination of tunnel.
01-18-2022 09:30 AM
Thanks for the reply. Unfortunately the BT VRF does have a default GW so MPLS.35, which is public, is routable.
#sh ip route vrf BT Routing Table: BT Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2, m - OMP n - NAT, Ni - NAT inside, No - NAT outside, Nd - NAT DIA i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2 ia - IS-IS inter area, * - candidate default, U - per-user static route H - NHRP, G - NHRP registered, g - NHRP registration summary o - ODR, P - periodic downloaded static route, l - LISP a - application route + - replicated route, % - next hop override, p - overrides from PfR & - replicated local route overrides by connected Gateway of last resort is BT.97 to network 0.0.0.0 S* 0.0.0.0/0 [1/0] via BT.97 10.0.0.0/16 is subnetted, 1 subnets S 10.0.0.0 [1/0] via 10.0.1.1 81.0.0.0/8 is variably subnetted, 2 subnets, 2 masks C BT.96/29 is directly connected, GigabitEthernet0/0/1 L BT.99/32 is directly connected, GigabitEthernet0/0/1 172.31.0.0/32 is subnetted, 1 subnets S 172.31.255.254 [254/0] via BT.97, GigabitEthernet0/0/1 192.168.6.0/24 is variably subnetted, 2 subnets, 2 masks C 192.168.6.0/24 is directly connected, GigabitEthernet0/0/2.5 L 192.168.6.1/32 is directly connected, GigabitEthernet0/0/2.5
01-18-2022 10:07 AM - edited 01-18-2022 01:42 PM
OK for mpls.35 you need to config NHRP map in tunnel2.
01-18-2022 11:33 AM - edited 01-18-2022 11:34 AM
Tunnel 2 on the main Head office achieves that doesn't it. It's just coming from the other direction
ip nhrp nhs 10.255.2.6 nbma BT.99 multicast
01-18-2022 01:50 PM
the Tunnel config in both Hub and Spoke,
you config the NHS map which is not need since it it Hub not Spoke.
Hub you need only the map for multicast "dynamic"
for spoke you need two map one for NHS and other for multicast. "after the config the NHS ip address".
I check this missing from the Tunnel2 of spoke
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide