It looks like CSR router has

Dong Ha · ‎10-14-2016

I created an IWAN test bed with CSR routers in the lab following examples in this link: http://www.cisco.com/c/en/us/td/docs/ios-xml/ios/pfrv3/configuration/xe-3s/pfrv3-xe-3s-book/pfrv3.html

Below is my topology

I don't see any channels on the MC router and don't see External WAN interfaces on the Branch border router. Can anyone explain why the 18.18.0.0/16 route go to Null0 on HUB1 and HUB2 and how to fix it?

Below is the output of the commands

MC1#sho domain one master status

*** Domain MC Status ***

Master VRF: Global

Instance Type:    Hub
Instance id:      0
Operational status: Up
Configured status: Up
Loopback IP Address: 18.18.18.18
Global Config Last Publish status: Peering Success
Load Balancing:
   Admin Status: Enabled
   Operational Status: Up
   Enterprise top level prefixes configured: 1
   Max Calculated Utilization Variance: 0%
   Last load balance attempt: never
   Last Reason: Variance less than 20%
   Total unbalanced bandwidth:
         External links: 0 Kbps Internet links: 0 Kbps
Route Control: Enabled
Transit Site Affinity: Enabled
Load Sharing: Enabled
Connection Keepalive: 60 seconds
Mitigation mode Aggressive: Disabled
Policy threshold variance: 20
Minimum Mask Length: 28
Syslog TCA suppress timer: 180 seconds
Traffic-Class Ageout Timer: 5 minutes
Channel Unreachable Threshold Timer: 1 seconds
Minimum Packet Loss Calculation Threshold: 15 packets
Minimum Bytes Loss Calculation Threshold: 1 bytes

Borders:
    IP address: 18.18.19.19
    Version: 2
    Connection status: CONNECTED (Last Updated 03:25:22 ago )
    Interfaces configured:
      Name: Tunnel100 | type: external | Service Provider: MPLS | Status: UP | Zero-SLA: NO | Path of Last Resort: Disabled
          Number of default Channels: 0

Tunnel if: Tunnel0

    IP address: 18.18.20.20
    Version: 2
    Connection status: CONNECTED (Last Updated 03:25:11 ago )
    Interfaces configured:
      Name: Tunnel200 | type: external | Service Provider: INET | Status: UP | Zero-SLA: NO | Path of Last Resort: Disabled
          Number of default Channels: 0

Tunnel if: Tunnel0

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

SPOKE1#sho domain one border status

Sat Oct 15 01:20:08.277
--------------------------------------------------------------------
**** Border Status ****

Instance Status: UP
Present status last updated: 2d05h ago
Loopback: Configured Loopback0 UP (18.18.21.21)
Master: 18.18.21.21
Master version: 2
Connection Status with Master: UP
MC connection info: CONNECTION SUCCESSFUL
Connected for: 2d05h
External Collector: 172.17.101.200 port: 2055
Route-Control: Enabled
Asymmetric Routing: Disabled
Minimum Mask length: 28
Connection Keepalive: 60 seconds
Sampling: off
Channel Unreachable Threshold Timer: 1 seconds
Minimum Packet Loss Calculation Threshold: 15 packets
Minimum Byte Loss Calculation Threshold: 1 bytes
Monitor cache usage: 4000 (20%) Auto allocated
Minimum Requirement: Met
External Wan interfaces:

Auto Tunnel information:

   Name:Tunnel0 if_index: 16
   Virtual Template: Not Configured
   Borders reachable via this tunnel:

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Here is the output of "show ip route" command on the DC BR router

HUB1#   sho ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

      17.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C        17.1.1.0/24 is directly connected, GigabitEthernet3
L        17.1.1.1/32 is directly connected, GigabitEthernet3
    p 18.0.0.0/8 is variably subnetted, 11 subnets, 4 masks
C   p    18.0.100.0/24 is directly connected, Tunnel100
L   p    18.0.100.1/32 is directly connected, Tunnel100
B   p    18.18.0.0/16 [109/0], 03:29:55, Null0
C   p    18.18.1.0/24 is directly connected, GigabitEthernet2
L   p    18.18.1.1/32 is directly connected, GigabitEthernet2
D   p    18.18.2.0/24 [90/3072] via 18.18.1.2, 03:30:25, GigabitEthernet2
D EXp    18.18.10.0/24 [210/3072] via 18.18.1.2, 03:03:15, GigabitEthernet2
D EXp    18.18.11.0/24 [210/3072] via 18.18.1.2, 03:03:15, GigabitEthernet2
D   p    18.18.18.18/32 [90/130816] via 18.18.1.2, 03:30:25, GigabitEthernet2
C   p    18.18.19.19/32 is directly connected, Loopback0
D EXp    18.18.20.20/32 [210/26368] via 18.18.1.2, 03:30:14, GigabitEthernet2
      172.16.0.0/16 is variably subnetted, 4 subnets, 2 masks
C        172.16.1.0/24 is directly connected, GigabitEthernet4
L        172.16.1.1/32 is directly connected, GigabitEthernet4
O        172.16.3.0/24 [110/2] via 172.16.1.2, 00:47:14, GigabitEthernet4
O        172.16.5.0/24 [110/2] via 172.16.1.2, 00:47:14, GigabitEthernet4
      172.17.0.0/16 is variably subnetted, 4 subnets, 2 masks
S        172.17.1.0/24 [1/0] via 172.17.3.1
C        172.17.3.0/24 is directly connected, GigabitEthernet1
L        172.17.3.132/32 is directly connected, GigabitEthernet1
S        172.17.101.0/24 [1/0] via 172.17.3.1

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

HUB2#sho ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

      17.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C        17.1.1.0/24 is directly connected, GigabitEthernet3
L        17.1.1.2/32 is directly connected, GigabitEthernet3
    p 18.0.0.0/8 is variably subnetted, 11 subnets, 4 masks
C   p    18.0.200.0/24 is directly connected, Tunnel200
L   p    18.0.200.1/32 is directly connected, Tunnel200
B   p    18.18.0.0/16 [109/0], 03:30:41, Null0
D   p    18.18.1.0/24 [90/3072] via 18.18.2.2, 03:31:11, GigabitEthernet2
C   p    18.18.2.0/24 is directly connected, GigabitEthernet2
L   p    18.18.2.1/32 is directly connected, GigabitEthernet2
D EXp    18.18.10.0/24 [210/3072] via 18.18.2.2, 03:04:12, GigabitEthernet2
D EXp    18.18.11.0/24 [210/3072] via 18.18.2.2, 03:04:12, GigabitEthernet2
D   p    18.18.18.18/32 [90/130816] via 18.18.2.2, 03:31:11, GigabitEthernet2
D EXp    18.18.19.19/32 [210/26368] via 18.18.2.2, 03:31:11, GigabitEthernet2
C   p    18.18.20.20/32 is directly connected, Loopback0
      172.17.0.0/16 is variably subnetted, 4 subnets, 2 masks
S        172.17.1.0/24 [1/0] via 172.17.3.1
C        172.17.3.0/24 is directly connected, GigabitEthernet1
L        172.17.3.133/32 is directly connected, GigabitEthernet1
S        172.17.101.0/24 [1/0] via 172.17.3.1
HUB2#
HUB2#sho ip route vrf INET2

Routing Table: INET2
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

      172.16.0.0/16 is variably subnetted, 4 subnets, 2 masks
C        172.16.2.0/24 is directly connected, GigabitEthernet4
L        172.16.2.1/32 is directly connected, GigabitEthernet4
O        172.16.4.0/24 [110/2] via 172.16.2.2, 00:55:32, GigabitEthernet4
O        172.16.6.0/24 [110/2] via 172.16.2.2, 00:55:32, GigabitEthernet4

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

SPOKE1#sho ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
a - application route
+ - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

p 18.0.0.0/8 is variably subnetted, 10 subnets, 4 masks
C p 18.0.100.0/24 is directly connected, Tunnel100
L p 18.0.100.10/32 is directly connected, Tunnel100
C p 18.0.200.0/24 is directly connected, Tunnel200
L p 18.0.200.10/32 is directly connected, Tunnel200
B p 18.18.0.0/16 [200/0] via 18.0.100.1, 02:10:54
C p 18.18.21.21/32 is directly connected, Loopback0
C p 18.18.100.0/24 is directly connected, GigabitEthernet4
L p 18.18.100.1/32 is directly connected, GigabitEthernet4
C p 18.18.101.0/24 is directly connected, GigabitEthernet5
L p 18.18.101.1/32 is directly connected, GigabitEthernet5
172.16.0.0/16 is variably subnetted, 4 subnets, 2 masks
O 172.16.1.0/24 [110/2] via 172.16.3.2, 02:18:01, GigabitEthernet2
C 172.16.3.0/24 is directly connected, GigabitEthernet2
L 172.16.3.1/32 is directly connected, GigabitEthernet2
O 172.16.5.0/24 [110/2] via 172.16.3.2, 02:17:51, GigabitEthernet2
172.17.0.0/16 is variably subnetted, 4 subnets, 2 masks
S 172.17.1.0/24 [1/0] via 172.17.3.1
C 172.17.3.0/24 is directly connected, GigabitEthernet1
L 172.17.3.134/32 is directly connected, GigabitEthernet1
S 172.17.101.0/24 [1/0] via 172.17.3.1

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

SPOKE2#sho ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
a - application route
+ - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

p 18.0.0.0/8 is variably subnetted, 10 subnets, 4 masks
C p 18.0.100.0/24 is directly connected, Tunnel100
L p 18.0.100.11/32 is directly connected, Tunnel100
C p 18.0.200.0/24 is directly connected, Tunnel200
L p 18.0.200.11/32 is directly connected, Tunnel200
B p 18.18.0.0/16 [200/0] via 18.0.100.1, 02:12:04
C p 18.18.22.22/32 is directly connected, Loopback0
C p 18.18.110.0/24 is directly connected, GigabitEthernet4
L p 18.18.110.1/32 is directly connected, GigabitEthernet4
C p 18.18.111.0/24 is directly connected, GigabitEthernet5
L p 18.18.111.1/32 is directly connected, GigabitEthernet5
172.16.0.0/16 is variably subnetted, 4 subnets, 2 masks
O 172.16.1.0/24 [110/2] via 172.16.5.2, 02:19:31, GigabitEthernet2
O 172.16.3.0/24 [110/2] via 172.16.5.2, 02:19:31, GigabitEthernet2
C 172.16.5.0/24 is directly connected, GigabitEthernet2
L 172.16.5.1/32 is directly connected, GigabitEthernet2
172.17.0.0/16 is variably subnetted, 4 subnets, 2 masks
S 172.17.1.0/24 [1/0] via 172.17.3.1
C 172.17.3.0/24 is directly connected, GigabitEthernet1
L 172.17.3.135/32 is directly connected, GigabitEthernet1
S 172.17.101.0/24 [1/0] via 172.17.3.1
SPOKE2#sho ip route vr
SPOKE2#sho ip route vrf INET2

Routing Table: INET2
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
a - application route
+ - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

172.16.0.0/16 is variably subnetted, 4 subnets, 2 masks
O 172.16.2.0/24 [110/2] via 172.16.6.2, 02:20:09, GigabitEthernet3
O 172.16.4.0/24 [110/2] via 172.16.6.2, 02:20:09, GigabitEthernet3
C 172.16.6.0/24 is directly connected, GigabitEthernet3
L 172.16.6.1/32 is directly connected, GigabitEthernet3

Georg Pauwen · ‎10-15-2016

Dong,

the 18.18.0.0 overrides are probaly injected into BGP by PfR for loop prevention, ISPs typically inject static null routes for that purpose.

I am looking at the other issues you mentioned...

Dong Ha · ‎10-15-2016

I don't have access to the lab right now so I'll post the configs later. I basically copied the exact configs from this link http://www.cisco.com/c/en/us/td/docs/ios-xml/ios/pfrv3/configuration/xe-3s/pfrv3-xe-3s-book/pfrv3.html

I also noticed that the config in the link above doesn't have the "ip route vrf INET 0.0.0.0 0.0.0.0 x.x.x.x" command as other IWAN guides have. Do I need to set up this "ip route vrf" command also?

Gaurav Mahajan · ‎10-15-2016

Hello Dong,

Check the output of show tcp brief on all router.

if you are using source as loopback 0 each routers in the domain should be reachable via loopback of each other.

check output of "show eigrp service-family ipv4 neighbours.

Like gpauwen mentioned you need to have 'domain domain-name path path-name'. on the tunnel interfaces for Hub Border routers.

IOS-XE version on the Hub master , Hub Border routers should be same.

IOS on the remote routers should also be same.

Take captures on the remote end to see if you are getting udp reply for smart probe with port 19000.

Dong Ha · ‎10-17-2016

Hi Gaurav,

I'm not sure how to take captures like what you described. Below is the output of the 'show tcp brief' command

MC1#show tcp brief
TCB Local Address Foreign Address (state)
7FC5EA6EB3C0 18.18.18.18.17749 18.18.20.20.50718 ESTAB
7FC5936D5338 18.18.18.18.17749 18.18.19.19.49793 ESTAB

HUB1#show tcp brief
TCB Local Address Foreign Address (state)
7F9C7C340850 18.0.100.1.179 18.0.100.10.47553 ESTAB
7F9C7C33EE20 18.0.100.1.179 18.0.100.11.15696 ESTAB
7F9CD3A862D0 18.18.19.19.49793 18.18.18.18.17749 ESTAB

HUB2#show tcp brief
TCB Local Address Foreign Address (state)
7FE51F55DF70 18.18.20.20.50718 18.18.18.18.17749 ESTAB
7FE52BA9A8A8 18.0.200.1.179 18.0.200.11.42317 ESTAB
7FE4D21C3A00 18.0.200.1.179 18.0.200.10.43969 ESTAB

SPOKE1#show tcp brief
TCB Local Address Foreign Address (state)
7F6EACD8A598 18.0.200.10.43969 18.0.200.1.179 ESTAB
7F6EACD27648 18.18.21.21.17749 18.18.21.21.32395 ESTAB
7F6F0771B498 18.18.21.21.32395 18.18.21.21.17749 ESTAB
7F6F06F8C318 18.0.100.10.47553 18.0.100.1.179 ESTAB

SPOKE2#show tcp brief
TCB Local Address Foreign Address (state)
7FF3A39CA720 18.18.22.22.17259 18.18.22.22.17749 ESTAB
7FF3FD7150B8 18.0.100.11.15696 18.0.100.1.179 ESTAB
7FF3FDAFD5C0 18.18.22.22.17749 18.18.22.22.17259 ESTAB
7FF3FD779AA0 18.0.200.11.42317 18.0.200.1.179 ESTAB

Dong Ha · ‎10-20-2016

I don't see the routes for SPOKE1 and SPOKE2 on HUB1's BGP routing table. Does anyone know why?

HUB1(config-router-af)#do sho bg
BGP table version is 24, local router ID is 18.18.19.19
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
     0.0.0.0          0.0.0.0                                0 i
r> 18.18.0.0/16     0.0.0.0                            32768 i
s> 18.18.10.0/24    18.18.1.2             3072         32768 i
s> 18.18.11.0/24    18.18.1.2             3072         32768 i
s> 18.18.18.18/32   18.18.1.2           130816         32768 i
s> 18.18.19.19/32   0.0.0.0                  0         32768 I

SPOKE1#sho bgp
BGP table version is 17, local router ID is 18.18.21.21
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
*>i 18.18.0.0/16     18.0.100.1               0    201      0 i
* i                  18.0.200.1               0    151      0 i
*> 18.18.21.21/32   0.0.0.0                  0         32768 i
*> 18.18.100.0/24   0.0.0.0                  0         32768 i
*> 18.18.101.0/24   0.0.0.0                  0         32768 i

Dong Ha · ‎10-18-2016

It looks like the BGP aggregate route command is the cause and the insertion of BGP route to EIGRP might be another. I'm not familiar with these features. Does anyone know what settings I can change to fix the issue?

Georg Pauwen · ‎10-19-2016

Dong,

are you referring to the BGP summary routes ? Turning off auto-summary will get rid of them. Not sure though what effect that will have on your network...

router bgp 10

no auto-summary

Dong Ha · ‎10-19-2016

It looks like CSR router has the "no auto-summary" as the default setting. I played around with changing the BGP distance value and saw the "18.18.0.0/16" removed from the routing table. However this is not what I want. Do you have any recommendation about what value I should use for the distance command?

Georg Pauwen · ‎10-15-2016

Hello,

hard to tell why you don't see any external interfaces on your borders. The only thing that needs to be configured, in addition of course to the IP address, on the tunnels is 'domain domain-name path path-name'.

Can you post the configs of the MC and one of the borders ?

Dong Ha · ‎10-17-2016

I do have the 'domain domain-name path path-name' commad. All routers are CSR with the same IOS-XE. Below is the config of MC, HUB1 and SPOKE1

MC1#sho run
Building configuration...

!
version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
no platform punt-keepalive disable-kernel-core
platform console virtual
!
hostname MC1
!
boot-start-marker
boot-end-marker
!
!
enable secret 5 $1$6G0E$RfhoOCCkMBhd.ZRtDYjm90
!
no aaa new-model
!
!
!
!
subscriber templating

!
multilink bundle-name authenticated
!
domain one
vrf default
master hub
source-interface Loopback0
site-prefixes prefix-list DATA_CENTER_1
monitor-interval 2 dscp ef
load-balance
enterprise-prefix prefix-list ENTERPRISE
class VOICE sequence 10
match dscp ef policy voice
path-preference MPLS fallback INET
class VIDEO sequence 20
match dscp af41 policy real-time-video
match dscp cs4 policy real-time-video
path-preference INET fallback MPLS
class CRITICAL sequence 30
match dscp af31 policy custom
priority 2 loss threshold 10
priority 1 one-way-delay threshold 600
path-preference MPLS fallback INET
!
!
!
crypto pki trustpoint TP-self-signed-2337190234
enrollment selfsigned
subject-name cn=IOS-Self-Signed-Certificate-2337190234
revocation-check none
rsakeypair TP-self-signed-2337190234
!
!
crypto pki certificate chain TP-self-signed-2337190234

certificate self-signed 01
3082022B 30820194 A0030201 02020101 300D0609 2A864886 F70D0101 05050030
31312F30 2D060355 04031326 494F532D 53656C66 2D536967 6E65642D 43657274
69666963 6174652D 32333337 31393032 3334301E 170D3136 30393238 32323233
33345A17 0D323030 31303130 30303030 305A3031 312F302D 06035504 03132649
4F532D53 656C662D 5369676E 65642D43 65727469 66696361 74652D32 33333731
39303233 3430819F 300D0609 2A864886 F70D0101 01050003 818D0030 81890281
8100B7C9 A143337F 09B24541 3DE4D02A 83C2CCB8 0C3E25DB 13B5F636 D8E7B689
84148910 28DBD2AE 4ABCD2EA 78DB9238 03612E17 31B99C9B 0B001562 911EE015
815413FA 2E2F7AFD F3E4F0F0 C14C87A8 CFD849F1 C8E217BA 36463FF7 7F9137B8
31AF9FD7 4AABEC1B D501114F 9A7C950F F54AEC6C 4846BAC4 ABE3FF89 7AD27880
03D90203 010001A3 53305130 0F060355 1D130101 FF040530 030101FF 301F0603
551D2304 18301680 14DAE35B ECC744FC 222E6CD8 D60368F8 D42EE288 60301D06
03551D0E 04160414 DAE35BEC C744FC22 2E6CD8D6 0368F8D4 2EE28860 300D0609
2A864886 F70D0101 05050003 81810002 1AA10FF5 76775DE9 6098CD0B B83E721F
F2A31D85 0AD4D8AF 073E6F36 159F837A 2CCB399C 4102E49E BEC1878B 005E46D8
BC587072 9C5D1DBC BBE9945D 93AEA967 1BA3B336 489812C4 F406EAE0 CEC61860
99CA27F1 FDACCFD6 6F5EB405 CAF1501C 6DA13384 D8F5AE51 C515365E C96C31D9
4B27E47F 56E4F4C6 0D996C63 943BDD
quit
!
!
!
!
license udi pid CSR1000V sn 92NVV062M36
license boot level ax
!
spanning-tree extend system-id
!
!
redundancy

!
interface Loopback0
ip address 18.18.18.18 255.255.255.255
!
interface GigabitEthernet1
description Mgmt-intf
ip address 172.17.3.130 255.255.255.0
negotiation auto
!
interface GigabitEthernet2
description to-HUB1
ip address 18.18.1.2 255.255.255.0
negotiation auto
!
interface GigabitEthernet3
description to-HUB2
ip address 18.18.2.2 255.255.255.0
negotiation auto
!
interface GigabitEthernet4
description to-ftp-http-server
ip address 18.18.10.1 255.255.255.0
negotiation auto
!
interface GigabitEthernet5
description to-file-share-server
ip address 18.18.11.1 255.255.255.0
negotiation auto
!
!
router eigrp 100
network 18.18.1.0 0.0.0.255
network 18.18.2.0 0.0.0.255
network 18.18.18.18 0.0.0.0
redistribute connected
!
!
virtual-service csr_mgmt
!
ip forward-protocol nd
!
no ip http server
ip http secure-server
ip route 172.17.1.0 255.255.255.0 172.17.3.1
ip route 172.17.101.0 255.255.255.0 172.17.3.1
ip ssh rsa keypair-name ssh-key
ip ssh version 2
ip scp server enable
!
!
ip prefix-list DATA_CENTER_1 seq 5 permit 18.18.0.0/16 le 24
!
ip prefix-list ENTERPRISE seq 5 permit 18.0.0.0/8 le 24
no service-routing capabilities-manager
!
!
!
control-plane

!
!
line con 0
stopbits 1
line vty 0 4
exec-timeout 0 0
login local
transport input ssh
!
!
end

-------------------------------------------------------------------------------------------------------------

HUB1#sho run
Building configuration...

Current configuration : 6302 bytes
!
!
version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
no platform punt-keepalive disable-kernel-core
platform console virtual
!
hostname HUB1
!
boot-start-marker
boot-end-marker
!
!
vrf definition INET1
rd 65512:1
!
address-family ipv4
exit-address-family
!
vrf definition Mgmt-intf
!
address-family ipv4
exit-address-family
!
enable secret 5 $1$YW1X$kMl68hvXPUjHzxxHTyqHF.
!
no aaa new-model
!
!
!
!
!
!
!
!
!
!
subscriber templating

multilink bundle-name authenticated
!
domain one
vrf default
border
source-interface Loopback0
master 18.18.18.18
!
!
!
!
crypto pki trustpoint TP-self-signed-480203191
enrollment selfsigned
subject-name cn=IOS-Self-Signed-Certificate-480203191
revocation-check none
rsakeypair TP-self-signed-480203191
!
!
crypto pki certificate chain TP-self-signed-480203191
certificate self-signed 01
30820229 30820192 A0030201 02020101 300D0609 2A864886 F70D0101 05050030
30312E30 2C060355 04031325 494F532D 53656C66 2D536967 6E65642D 43657274
69666963 6174652D 34383032 30333139 31301E17 0D313630 39323832 32323333
315A170D 32303031 30313030 30303030 5A303031 2E302C06 03550403 1325494F
532D5365 6C662D53 69676E65 642D4365 72746966 69636174 652D3438 30323033
31393130 819F300D 06092A86 4886F70D 01010105 0003818D 00308189 02818100
A6572F5D 83A373C9 974AF0B7 49368CCE D87F69B3 40C35FBE 6C84897C 02657D76
8AB7EC11 8A34150F C12623E8 580888E4 29BAEB30 93545D6C 7BEACB00 DDA614A5
6174504A D56E278C AA478804 FA3B970C E5392CF8 D721F555 A37CD051 5839D991
F5ABDD72 CA498AC4 E2F11A97 C888FE62 62D34BDB 432CEDDB 3C5EDB7A D0F809FB
02030100 01A35330 51300F06 03551D13 0101FF04 05300301 01FF301F 0603551D
23041830 168014AD 11D28F4C 29E47BCB 5C79ACC2 C020B3EB 4788BC30 1D060355
1D0E0416 0414AD11 D28F4C29 E47BCB5C 79ACC2C0 20B3EB47 88BC300D 06092A86
4886F70D 01010505 00038181 008142BB FE0FA042 9B2B00E5 AA34D4BC 7065E1C5
FCB22976 1AFF0BA6 77D9C695 C2DB635D A5E86B31 9DF34352 24E9AD01 3237FF9D
8DDA6CBD CDFD8AD5 7D156DC2 5ECA99CB A59AC501 CF6B5CB5 6913EF95 088F167D
9B71B09D 770A768A 440CCE22 71E485A8 B3FB72BC 662951EB DF7CA4C7 9E3D03EF
14214C32 499C8C7D 7E45B54D 67
quit
!
!
!
!
!
!
!
license udi pid CSR1000V sn 93L8TX50LBK
license boot level ax
!
spanning-tree extend system-id
!
!
redundancy
!
!
!
!
!
!
!
crypto keyring DMVPN-KEYRING1
pre-shared-key address 0.0.0.0 0.0.0.0 key cisco123
!
!
!
!
!
crypto isakmp policy 10
encr aes
authentication pre-share
crypto isakmp performance
crypto isakmp profile ISAKMP-INET1
keyring DMVPN-KEYRING1
match identity address 0.0.0.0
!
crypto ipsec security-association replay disable
crypto ipsec security-association replay window-size 1024
!
crypto ipsec transform-set AES256/SHA/TRANSPORT esp-aes 256 esp-sha-hmac
mode transport
!
crypto ipsec profile DMVPN-PROFILE1
set transform-set AES256/SHA/TRANSPORT
set isakmp-profile ISAKMP-INET1
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface Loopback0
ip address 18.18.19.19 255.255.255.255
!
interface Tunnel100
bandwidth 100000
ip address 18.0.100.1 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication cisco
ip nhrp map multicast dynamic
ip nhrp network-id 1
ip nhrp holdtime 600
ip nhrp redirect
ip tcp adjust-mss 1360
load-interval 30
tunnel source GigabitEthernet4
tunnel mode gre multipoint
tunnel key 100
tunnel protection ipsec profile DMVPN-PROFILE1
domain one path MPLS
!
interface GigabitEthernet1
description Mgmt-intf
ip address 172.17.3.132 255.255.255.0
negotiation auto
!
interface GigabitEthernet2
description to-MC
ip address 18.18.1.1 255.255.255.0
negotiation auto
!
interface GigabitEthernet3
description to-HUB2
ip address 17.1.1.1 255.255.255.0
negotiation auto
!
interface GigabitEthernet4
description to-MPLS
ip address 172.16.1.1 255.255.255.0
negotiation auto
!
!
router eigrp 100
network 18.18.1.0 0.0.0.255
network 18.18.20.20 0.0.0.0
redistribute bgp 10 metric 100000 1 255 255 1500
distance eigrp 90 210
!
router ospf 100
router-id 18.18.19.19
network 172.16.1.1 0.0.0.0 area 0
!
router bgp 10
bgp router-id 18.18.19.19
bgp log-neighbor-changes
bgp listen range 18.0.100.0/24 peer-group MPLS-SPOKES
neighbor MPLS-SPOKES peer-group
neighbor MPLS-SPOKES remote-as 10

neighbor MPLS-SPOKES timers 20 60
!
address-family ipv4
bgp redistribute-internal
network 18.18.10.0 mask 255.255.255.0
network 18.18.11.0 mask 255.255.255.0
network 18.18.18.18 mask 255.255.255.255
network 18.18.19.19 mask 255.255.255.255
aggregate-address 18.18.0.0 255.255.0.0 summary-only
neighbor MPLS-SPOKES activate
neighbor MPLS-SPOKES send-community
neighbor MPLS-SPOKES default-originate
neighbor MPLS-SPOKES route-map MPLS-DC1-IN in
neighbor MPLS-SPOKES route-map MPLS-DC1-OUT out
distance bgp 20 109 109
exit-address-family
!
!
virtual-service csr_mgmt
!
ip forward-protocol nd
!
ip bgp-community new-format
ip community-list standard MPLS-DMVPN permit 10:100
ip community-list standard INET-DMVPN permit 10:200
no ip http server
ip http secure-server
ip route 172.17.1.0 255.255.255.0 172.17.3.1
ip route 172.17.101.0 255.255.255.0 172.17.3.1
ip ssh rsa keypair-name ssh-key
ip ssh version 2
ip scp server enable
!
!
ip prefix-list DC1-LOCAL-ROUTES seq 10 permit 0.0.0.0/0
ip prefix-list DC1-LOCAL-ROUTES seq 20 permit 18.18.0.0/16 le 32
no service-routing capabilities-manager
!
route-map MPLS-DC1-IN deny 10
match ip address prefix-list DC1-LOCAL-ROUTES
!
route-map MPLS-DC1-IN permit 20
set community 10:100
!
route-map TO-PEER permit 10
match ip address prefix-list DC1-LOCAL-ROUTES
set ip next-hop self
set community no-advertise
!
route-map site_prefixes permit 10
match ip address prefix-list site_prefixes
!
route-map MPLS-DC1-OUT permit 10
match ip address prefix-list DC1-LOCAL-ROUTES
set community 10:100
!
route-map MPLS-DC1-OUT permit 20
description readvertise routes learned from MPLS DMVPN cloud
match community MPLS-DMVPN
!
!
!
control-plane
!
!
!
!
!
line con 0
stopbits 1
line vty 0 4
exec-timeout 0 0
login local
transport input ssh
!
!
end

-------------------------------------------------------------------------------------------------------------

SPOKE1#sho run
Building configuration...

Current configuration : 8187 bytes
!
version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
no platform punt-keepalive disable-kernel-core
platform console virtual
!
hostname SPOKE1
!
boot-start-marker
boot-end-marker
!
!
vrf definition INET2
rd 65512:2
!
address-family ipv4
exit-address-family
!
enable secret 5 $1$Tj6G$zDKELmCgLQb/1X.rteZAK0
!
no aaa new-model
!
!

!
!
!
!
!
!
!
!
!
!
subscriber templating
!
multilink bundle-name authenticated
!
domain one
vrf default
border
source-interface Loopback0
master local
master branch
source-interface Loopback0
hub 18.18.18.18
!
!
!
!
crypto pki trustpoint TP-self-signed-1009037223
enrollment selfsigned
subject-name cn=IOS-Self-Signed-Certificate-1009037223
revocation-check none
rsakeypair TP-self-signed-1009037223
!
!
crypto pki certificate chain TP-self-signed-1009037223
certificate self-signed 01
3082022B 30820194 A0030201 02020101 300D0609 2A864886 F70D0101 05050030
31312F30 2D060355 04031326 494F532D 53656C66 2D536967 6E65642D 43657274
69666963 6174652D 31303039 30333732 3233301E 170D3136 30393238 32323233
34345A17 0D323030 31303130 30303030 305A3031 312F302D 06035504 03132649
4F532D53 656C662D 5369676E 65642D43 65727469 66696361 74652D31 30303930
33373232 3330819F 300D0609 2A864886 F70D0101 01050003 818D0030 81890281
810096B1 A5B60807 AB07E882 3F476CD7 5202E073 01649035 4BBDA877 A4034ECC
332BD756 C1DAA72C 32282602 A66EB2F1 F1848A56 705D042C D6FBB91E CA4BFE6D
2599DB67 3664D3D3 52F0F038 DC128391 986B5393 756988D2 329B64B3 18584EF6
339FB993 B225016A BE11F7F4 CBAF5679 F19386C0 9F7811EA A6EDFE23 9BEF0C9A
4A330203 010001A3 53305130 0F060355 1D130101 FF040530 030101FF 301F0603
551D2304 18301680 1419D002 53227DB3 CF62496E 9643C716 E8240501 15301D06
03551D0E 04160414 19D00253 227DB3CF 62496E96 43C716E8 24050115 300D0609
2A864886 F70D0101 05050003 8181003F 4DE70C74 8BC09159 2FED8B57 B179DA0C
DBD7BA1F FCDE924F 4E090B70 8E84B31E 626EED4C 88227295 1A72C64C AA36BD6D
5B19C2B9 06E1B6A4 041D167D 31A24AC6 D664503F BDC749E9 A0938D4D D7D8D161
B2FA6FC5 53A172E4 A2B483DB DBF541A6 59F41D0F 1021589D 35CC1C40 4C58DA78
511A56C9 B85C8A0D 6EE6862D 082227
quit
!
!
!
!
!
!
!
license udi pid CSR1000V sn 9RGKXBZFUXF
license boot level ax
!
spanning-tree extend system-id
!
!
redundancy
!
!
!
!
!
!
!
crypto keyring DMVPN-KEYRING1
pre-shared-key address 0.0.0.0 0.0.0.0 key cisco123
crypto keyring DMVPN-KEYRING2 vrf INET2
pre-shared-key address 0.0.0.0 0.0.0.0 key cisco123
!
!
!
!
!
crypto isakmp policy 10
encr aes
authentication pre-share
crypto isakmp invalid-spi-recovery
crypto isakmp keepalive 40 5
crypto isakmp profile ISAKMP-INET1
keyring DMVPN-KEYRING1
match identity address 0.0.0.0
crypto isakmp profile ISAKMP-INET2
keyring DMVPN-KEYRING2
match identity address 0.0.0.0 INET2
!
crypto ipsec security-association idle-time 60
crypto ipsec security-association replay window-size 512
!
crypto ipsec transform-set AES256/SHA/TRANSPORT esp-aes 256 esp-sha-hmac
mode transport
!
crypto ipsec profile DMVPN-PROFILE1
set transform-set AES256/SHA/TRANSPORT
set isakmp-profile ISAKMP-INET1
!
crypto ipsec profile DMVPN-PROFILE2
set transform-set AES256/SHA/TRANSPORT
set isakmp-profile ISAKMP-INET2
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface Loopback0
ip address 18.18.21.21 255.255.255.255
!
interface Tunnel100
bandwidth 100000
ip address 18.0.100.10 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication cisco
ip nhrp map multicast 172.16.1.1
ip nhrp map 18.0.100.1 172.16.1.1
ip nhrp network-id 1
ip nhrp holdtime 600
ip nhrp nhs 18.0.100.1
ip nhrp registration timeout 60
ip nhrp shortcut
ip tcp adjust-mss 1360
load-interval 30
delay 1000
tunnel source GigabitEthernet2
tunnel mode gre multipoint
tunnel key 100
tunnel protection ipsec profile DMVPN-PROFILE1
!
interface Tunnel200
bandwidth 50000
ip address 18.0.200.10 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication cisco
ip nhrp map multicast 172.16.2.1
ip nhrp map 18.0.200.1 172.16.2.1
ip nhrp network-id 2
ip nhrp holdtime 600
ip nhrp nhs 18.0.200.1
ip nhrp registration timeout 60
ip nhrp shortcut
ip tcp adjust-mss 1360
load-interval 30
delay 1000
tunnel source GigabitEthernet3
tunnel mode gre multipoint
tunnel key 200
tunnel vrf INET2
tunnel protection ipsec profile DMVPN-PROFILE2
!
interface GigabitEthernet1
description Mgmt-intf
ip address 172.17.3.134 255.255.255.0
negotiation auto
!
interface GigabitEthernet2
description to-MPLS
ip address 172.16.3.1 255.255.255.0
negotiation auto
!
interface GigabitEthernet3
description to-INET
vrf forwarding INET2
ip address 172.16.4.1 255.255.255.0
negotiation auto
!
interface GigabitEthernet4
description to-client-1
ip address 18.18.100.1 255.255.255.0
negotiation auto
!
interface GigabitEthernet5
description to-client-2
ip address 18.18.101.1 255.255.255.0
negotiation auto
!
router ospf 200 vrf INET2
network 172.16.4.1 0.0.0.0 area 0
!
router ospf 100
router-id 18.18.21.21
network 172.16.3.1 0.0.0.0 area 0
!
router bgp 10
bgp router-id 18.18.21.21
bgp log-neighbor-changes
neighbor MPLS-HUB peer-group
neighbor MPLS-HUB remote-as 10
neighbor MPLS-HUB timers 20 60
neighbor INET-HUB peer-group
neighbor INET-HUB remote-as 10
neighbor INET-HUB timers 20 60
neighbor 18.0.100.1 peer-group MPLS-HUB
neighbor 18.0.200.1 peer-group INET-HUB
!
address-family ipv4
network 18.18.21.21 mask 255.255.255.255
network 18.18.100.0 mask 255.255.255.0
network 18.18.101.0 mask 255.255.255.0
neighbor MPLS-HUB send-community
neighbor MPLS-HUB route-map MPLS-SPOKE-IN in
neighbor MPLS-HUB route-map MPLS-SPOKE-OUT out
neighbor INET-HUB send-community
neighbor INET-HUB route-map INET-SPOKE-IN in
neighbor INET-HUB route-map INET-SPOKE-OUT out
neighbor 18.0.100.1 activate
neighbor 18.0.100.1 soft-reconfiguration inbound
neighbor 18.0.200.1 activate
neighbor 18.0.200.1 soft-reconfiguration inbound
exit-address-family
!
!
virtual-service csr_mgmt
!
ip forward-protocol nd
!
ip bgp-community new-format
ip community-list standard MPLS-HUB1 permit 10:100
ip community-list standard MPLS-HUB2 permit 10:101
ip community-list standard INET-HUB1 permit 10:200
ip community-list standard INET-HUB2 permit 10:201
no ip http server
ip http secure-server
ip route 172.17.1.0 255.255.255.0 172.17.3.1
ip route 172.17.101.0 255.255.255.0 172.17.3.1
ip ssh rsa keypair-name ssh-key
ip ssh version 2
ip scp server enable
!
ip access-list extended Remote-Client
permit tcp host 18.18.100.2 any
permit tcp host 18.18.101.2 any
ip access-list extended SMP
permit udp any eq 18000 any eq 19000
!
!
ip prefix-list INET-DMVPN seq 5 permit 0.0.0.0/0
ip prefix-list INET-DMVPN seq 10 permit 18.18.0.0/16
!
ip prefix-list MPLS-DMVPN seq 5 permit 0.0.0.0/0
ip prefix-list MPLS-DMVPN seq 10 permit 18.18.0.0/16
no service-routing capabilities-manager
!
route-map MPLS-SPOKE-OUT deny 10
match ip address prefix-list INET-DMVPN
!
route-map MPLS-SPOKE-OUT permit 20
!
route-map INET-SPOKE-OUT deny 10
match ip address prefix-list MPLS-DMVPN
!
route-map INET-SPOKE-OUT permit 20
!
route-map MPLS-SPOKE-IN permit 5
match ip address prefix-list MPLS-DMVPN
match community MPLS-HUB1
set local-preference 201
!
route-map MPLS-SPOKE-IN permit 10
match community MPLS-HUB1
set local-preference 201
!
route-map MPLS-SPOKE-IN permit 20
match community MPLS-HUB2
set local-preference 200
!
route-map site_prefixes permit 10
match ip address prefix-list site_prefixes
!
route-map INET-SPOKE-IN permit 5
match ip address prefix-list MPLS-DMVPN
match community INET-HUB1
set local-preference 151
!
route-map INET-SPOKE-IN permit 30
match community INET-HUB1
set local-preference 151
!
route-map INET-SPOKE-IN permit 40
match community INET-HUB2
set local-preference 150
!
!
!
control-plane
!
!
!
!
!
!
!
!
!
!
line con 0
stopbits 1
line vty 0 4
exec-timeout 0 0
login local
transport input ssh
!
!
end

IWAN route question