SD-WAN Hub and Spoke Lab issue

[JUANNN]

Hello,

I am doing a little lab with a basic SD-WAN setup. I built the VMs for the 3 Control Elements successfully, and got the DTLS tunnels between them up. I put all the control elements on VLAN 214, and because of that, I didn't need a gateway or nothing, just one portgroup on ESXi and that's it. 

Next, I wanted this VLAN 214 to be sitting on a DMZ zone on a HUB router. But I want the hub router to be part of the SD-WAN fabric too. So I onboarded a C1121X-8PLTEPW router and made interface G0/0/0.214 part of VLAN 214 (I used the ip address 214.15.10.254), and sourced Tunnel1 with it.

interface GigabitEthernet0/0/0
no shutdown
ip dhcp client client-id ascii FJC26041LTS
negotiation auto

interface GigabitEthernet0/0/0.214
description ***DMZ_SITE-1 GW1***
no shutdown
encapsulation dot1Q 214
ip address 214.15.10.254 255.255.255.0

sdwan
interface GigabitEthernet0/0/0.214
tunnel-interface
encapsulation ipsec
color private1
allow-service all
no allow-service bgp
allow-service dhcp
allow-service dns
allow-service icmp
no allow-service sshd
no allow-service netconf
no allow-service ntp
no allow-service ospf
no allow-service stun
allow-service https
no allow-service snmp
no allow-service bfd
exit

interface Tunnel1
ip unnumbered GigabitEthernet0/0/0.214
tunnel source GigabitEthernet0/0/0.214
tunnel mode sdwan

So far so good... I used my own PKI server for the certificate portion and I successfully installed both the Root CA certificate and the certificate for the router, and I ended up getting the control connections up:

I configured static default routes on each control element pointing to the router's interface G0/0/0.214. My idea was to have the control elements sit on VPN0 with the HUB router as their Gateway, and then start onboarding the spokes. So after I got the HUB successfully onboarded (and attached to a configuration group that I created for it), I started the process to onboard the first spoke. 

I created a second interface in VPN0:

interface GigabitEthernet0/0/0.700
description ***PUBLIC INET SD-WAN***
no shutdown
encapsulation dot1Q 700
ip address 68.1.1.1 255.255.255.240
exit

sdwan

interface GigabitEthernet0/0/0.700
tunnel-interface
encapsulation ipsec
color public-internet
allow-service all
no allow-service bgp
allow-service dhcp
allow-service dns
allow-service icmp
no allow-service sshd
no allow-service netconf
no allow-service ntp
no allow-service ospf
no allow-service stun
allow-service https
no allow-service snmp
no allow-service bfd
exit

interface Tunnel2
ip unnumbered GigabitEthernet0/0/0.700
tunnel source GigabitEthernet0/0/0.700
tunnel mode sdwan

This interface is facing the SPOKEs side (the public internet, even though the controllers vlan 214 is also public), so I configured the SPOKE to point to it with a default route and a directly connected interface to 68.1.1.0/28:

interface GigabitEthernet0/0/0
description ***PUBLIC INET***
no shutdown
ip address 68.1.1.2 255.255.255.240
ipv6 dhcp client request vendor
ipv6 address dhcp
ipv6 address autoconfig
ipv6 enable
negotiation auto
exit

sdwan
interface GigabitEthernet0/0/0
tunnel-interface
encapsulation ipsec
color public-internet
allow-service all
no allow-service bgp
allow-service dhcp
allow-service dns
allow-service icmp
no allow-service sshd
no allow-service netconf
no allow-service ntp
no allow-service ospf
no allow-service stun
allow-service https
no allow-service snmp
no allow-service bfd
exit

interface Tunnel2
no shutdown
ip unnumbered GigabitEthernet0/0/0
tunnel source GigabitEthernet0/0/0
tunnel mode sdwan

However, I ran into the following issue: for some reason, the spoke router can ping the HUB G0/0/0.700 interface, It can also ping the G0/0/0.214 interface. But for some reason, it cannot ping none of the control elements! This is, I can ping 214.15.10.254 from the spoke, but I cannot ping 214.15.10.1 (vManage) or .2 (vBond) or .3 (vSmart). And they are directly connceted to the HUB. 

I cannot understand this. I was expecting a simple routing on the HUB router, from vlan 700 to vlan 214 and viceversa, all in VPN 0. But I might be missing somehting.

Please, any help is appreciated, I have been stuck many hours with this. Thanks,

Juan

[Jeongjun Park]

Hello, Juan.

These are just my opinion.

1. Try to add VLAN 700 on the Trunk port toward ESXi.

2. In the spoke router, Try to add default static route (next hop hub)

If problem doesnt solve, let me know.

Thank you.

[JUANNN]

Hello everyone,

I got a solution that makes sense (at least as far as my knowledge goes). Here is the link:

Cisco SD-WAN Hub and Spoke Topology - Lessons Discussion - NetworkLessons.com Community Forum

According to it, VPN0 is not meant to be used as a transit layer, resulting in traffic not being routed within it. The recommended approach for design would be placing the control elements on a separate cloud that can be reached from both the HUB and the SPOKES, or on a DMZ zone in the HUB side, sitting behind a firewall for example, but ideally publicly reachable as well. 

[Jeongjun Park]

Hello, Juan

Thank you for your good information.

By the way, Is your LAB not working even you add static route on the Spoke? (Next hop hub)

Can you try to add static route as well ?

Ex) 0.0.0.0 0.0.0.0 68.1.1.1

Thank you.

[JUANNN]

Yeah that's why I had from the beginning (my apologies for not including it on the configs I posted) a static default route on the spoke pointing to G0/0/0.700 on the HUB. Note that if I didn't had such route, I would not been able to ping 214.15.10.254 from spoke to HUB:

"...the spoke router can ping the HUB G0/0/0.700 interface, It can also ping the G0/0/0.214 interface..." (from my first post)

But again, VPN 0 is not meant to be used as a transit VPN. The solution that we have in my organization is:

- HUB as an inner router

- An outer router that faces the public internet and also has the LAN of the SD-WAN control elements attached to it, as well as the inner router (HUB) attached to it on a different subnet. That way, the spokes reach the control elements through the underlay via this outer router, and the HUB as well.