vedge control connection to vbond

[mannycho]

Hello

My vedge routers all have control connections to vbond and in Up state (because of only one vsmart controller) over public internet transport. control connections from vedge routers to vbond state show connect (show control connections) and blank over MPLS transport. As a result all edge routers are in a partial control status state.

Why is the control connection from vedge to vbond over MPLS not Up? I have IP connectivity between vbond and the edge routers.

[MHM Cisco World]

I think this depends on DNS resolve the vbond IP'

Can you check if DNS resolve public IP or private IP?

MHM

[mannycho]

Thanks HM,

it’s an on premise POC, all controllers have private IPs configured 

[MHM Cisco World]

just I want to clear the topolgy here 
Vedge have two transport interface one in internet and other in MPLS 
Vedge can connect to Vbond via internet using it public IP 
vBond send both vSmart (private and public IP) to Vedge 

some of Vedge can connect to Vsmart via MPLS other not ?

if Yes can you check the non-work Vedge if it learn subnet of Vsmart (private IP) via MPLS?
MHM

[balaji.bandi]

when the device onboarding / coming online time only Control connection you see device and vbond,  once the device onboard you only see connect with vsmart and vManage only.

its all depends on how you configured your transport configuration on the device, check the device template configuration.

[mannycho]

I think when you have only one vsmart, the connections from wan edges becomes persistent to vbond, unless you change max connections to 1 on the vpn interface template. Also this is an on premise setup, all controllers have private Ip addresses on the inside of the network. The controllers all have transport on color public internet, but not color mpls. The wan edges have both transport internet and mpls. Internal routing allows the controllers to be able to reach the mpls transport on the wan edges

[Kanan Huseynli]

Hi,

You just have control connection to vSmart over each interface, but to have this you must have control connection to vBond over each transport.

If for example, vBond is in internet and you didnt configure MPLS wan to somehow have connection till vBond, you control connections fails.

Depending on your design there are couple of ways:

1) advertise vBond and other controller public IPs as /32 route in mpls

2) route traffic to internet edge and do nat on somewhere (dc/hq)

3) disable control connection requirement by max-control connection 0 under mpls binded transport interface

In the option3, if you lose control over internet, then you devices will lose overlay control connections and you cant manage untill internet is re-established. However, dataplane still works based on omp graceful restart timer

[mannycho]

Thanks Kanan,

I should also have mentioned this is an on premise POC. The controllers all seat on the inside network , with vbond having a private IP address only, but all controllers have access to the internet. The subnet that the controllers reside on is what I have as color public internet. The vedge routers have established dtls to the controllers over VPN 0 on this internet transport.

vbond currently does not have an interface on the MPLS transport, I will set this up based on your comments and see what happens. Will let the group know the outcome. Thank you

[mannycho]

Kanan,

I wanted to add that vbond can reach the MPLS interfaces on the vedges via internal routing. The vbond does not have a transport at this time on the MPLS network. Is it still a requirement for vbond to have a transport in MPLS for the control sessions to come up in this scenario?

[Kanan Huseynli]

Controllers have only one interface for VPN0. Actually, controller VPN0 are not bound to any transport. But if you need to have control connections from each type of transport of routers towards controllers, you need somehow to make reachibility.

Couple of options exists. If you share details of color/ IP design, more better solutions can be found

[dijix1990]

Do vBond and vSmart have interface placed in vpn0 with color mpls? can you check the connectivity betwen edge and vbond/vsmart via mpls? 

One transient DTLS control connection to the vBond orchestrator over each connected WAN transports only during the onboarding process.
One persistent DTLS control connection to vManage over a single WAN transport.
One persistent DTLS/TLS control connections to vSmart over each connected WAN transports;

[mannycho]

Hello,

vbond and vsmart only have transport in internet and not MPLS. It’s an on premise POC. All controllers are in inside subnet that has access to the internet. That inside subnet is color public internet transport. Internal routing allows the internet transport to reach the MPLS transport.

[dijix1990]

So, you need to have connectivity between mpls and internet via your special router in your DC (where placed controllers)

for example

Branch-2#sh sdwan control connections table
                                             LOCAL                                                                                                                   SHARED  CFG
          PEER     SITE  DOMAIN  LOCAL       PRIVATE             PUBLIC  SYSTEM              LOCAL                PRIVATE    PRIVATE                     CONTROLLER  REGION  SYSTEM  V ORG    BEHIND
INSTANCE  TYPE     ID    ID      PRIVATE IP  PORT     PUBLIC IP  PORT    IP        PROTOCOL  COLOR  REMOTE COLOR  IP         PORT     STATE  UPTIME      GROUP ID    ID SET  IP      NAME     PROXY
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0         vsmart   1     1       10.2.2.204  12386    10.1.1.30  12446   1.1.1.30  dtls      mpls   biz-internet  10.1.1.30  12446    up     0:00:12:13  0           N/A     -       fractal  No
0         vbond    0     0       10.2.2.204  12386    10.1.1.10  12346   0.0.0.0   dtls      mpls   mpls          10.1.1.10  12346    up     0:00:12:14  0           N/A     -       fractal  -
0         vmanage  1     0       10.2.2.204  12386    10.1.1.20  12446   1.1.1.20  dtls      mpls   default       10.1.1.20  12446    up     0:00:11:22  0           -       -       fractal  No


Branch-2#ping 10.1.1.20
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.1.1.20, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/3 ms
Branch-2#trac
Branch-2#traceroute 10.1.1.10
Type escape sequence to abort.
Tracing the route to 10.1.1.10
VRF info: (vrf in name/id, vrf out name/id)
  1 10.2.2.1 5 msec 3 msec 2 msec
  2 10.1.1.10 25 msec 41 msec 41 msec
Branch-2#

• 10.1.1.10 - it's my test vBond with color biz-internet. It has only def route to internet
• 10.2.2.1 it's ,router where I do routing between mpls and internet (there is interface with 10.1.1.1 - it's internet gw and 10.2.2.1 mpls gw)

so in my edge I have some routes in vpn 0

ip route 0.0.0.0/0 10.1.1.1 - public internet
10.2.2.0/24 10.2.2.1 - mpls for connecting between edges
10.1.1.0/24 10.2.2.1 - route to connect my controllers which have only public IP via mpls network

in my controllers

ip route 0.0.0.0/0 10.1.1.1 - def route to internet
ip route 10.2.2.0/24 10.1.1.1 - route to my mpls network via my router for routing between mpls and my internet network (my piblic ip's)

BTW if your mpls is L3vpn you need to your mpls provider do routing between mpls and you block of public ip's (I don't need it because I bought L2VPN for my edges)

[mannycho]

Your default route covers the specific route below

ip route 0.0.0.0/0 10.1.1.1 
ip route 10.2.2.0/24 10.1.1.1

My WAN edge routers mpls interfaces (10.10.x.x) have ip connectivity to the controllers interfaces (190.190.190.x) on biz-internet transport. The WAN edge routers also have an interface on biz-internet transport. So I do not see how this is a connectivity issue between the transports. There is a gateway that has routing between both transports that the WAN edge and Controllers all point to. Again I am using private IPs on all devices

WAN-EDGE-01# show control connections
                                                                                       PEER                                          PEER                                          CONTROLLER
PEER    PEER PEER            SITE       DOMAIN PEER                                    PRIV  PEER                                    PUB                                           GROUP
TYPE    PROT SYSTEM IP       ID         ID     PRIVATE IP                              PORT  PUBLIC IP                               PORT  ORGANIZATION            LOCAL COLOR     PROXY STATE UPTIME      ID      
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
vsmart  dtls 10.10.100.12    255        1      190.190.190.132                         12446 190.190.190.132                         12446 Lindos SD-WAN             biz-internet    No    up     0:13:28:33 0     
vbond   dtls 0.0.0.0         0          0      190.190.190.130                         12346 190.190.190.130                         12346                           mpls            -     connect           0     
vbond   dtls 0.0.0.0         0          0      190.190.190.130                         12346 190.190.190.130                         12346 Lindos SD-WAN             biz-internet    -     up     0:14:25:06 0     
vmanage dtls 10.10.100.10    255        0      190.190.190.131                         12446 190.190.190.131                         12446 Lindos SD-WAN             biz-internet    No    up     0:14:25:06 0 



WAN-EDGE-01# show control connections-history
                                                                       PEER                      PEER
PEER     PEER     PEER             SITE        DOMAIN PEER             PRIVATE  PEER             PUBLIC                                   LOCAL      REMOTE     REPEAT
TYPE     PROTOCOL SYSTEM IP        ID          ID     PRIVATE IP       PORT     PUBLIC IP        PORT    LOCAL COLOR      STATE           ERROR      ERROR      COUNT ORGANIZATION            DOWNTIME
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
vbond    dtls     0.0.0.0          0           0      190.190.190.130  12346    190.190.190.130  12346   mpls             connect         DCONFAIL   NOERR      10    2024-02-13T16:10:00+0000
vbond    dtls     0.0.0.0          0           0      190.190.190.130  12346    190.190.190.130  12346   mpls             connect         DCONFAIL   NOERR      8     2024-02-13T16:07:15+0000
vsmart   dtls     10.10.100.12     255         1      190.190.190.132  12446    190.190.190.132  12446   biz-internet     connect         DCONFAIL   NOERR      3     2024-02-13T02:41:16+0000
vsmart   dtls     10.10.100.12     255         1      190.190.190.132  12446    190.190.190.132  12446   biz-internet     tear_down       VS_TMO     NOERR      1     2024-02-13T02:41:04+0000
vsmart   dtls     10.10.100.12     255         1      190.190.190.132  12446    190.190.190.132  12446   biz-internet     up              RXTRDWN    DISTLOC    0     2024-02-13T02:38:35+0000
vbond    dtls     0.0.0.0          0           0      190.190.190.130  12346    190.190.190.130  12346   mpls             tear_down       DISTLOC    NOERR      0     2024-02-13T02:26:39+0000
vbond    dtls     0.0.0.0          0           0      190.190.190.130  12346    190.190.190.130  12346   mpls             connect         DCONFAIL   NOERR      46    2024-02-13T02:26:35+0000
vbond    dtls     0.0.0.0          0           0      190.190.190.130  12346    190.190.190.130  12346   mpls             connect         DCONFAIL   NOERR      49    2024-02-13T02:22:05+0000
vbond    dtls     0.0.0.0          0           0      190.190.190.130  12346    190.190.190.130  12346   mpls             connect         DCONFAIL   NOERR      41    2024-02-13T02:15:49+0000
vbond    dtls     0.0.0.0          0           0      190.190.190.130  12346    190.190.190.130  12346   mpls             connect         DCONFAIL   NOERR      17    2024-02-13T02:10:33+0000
vbond    dtls     0.0.0.0          0           0      190.190.190.130  12346    190.190.190.130  12346   mpls             connect         DCONFAIL   NOERR      9     2024-02-13T02:08:17+0000

[MHM Cisco World]

vbond   dtls 0.0.0.0         0          0      190.190.190.130                         12346 190.190.190.130                         12346                           mpls            -     connect           0     
vbond   dtls 0.0.0.0         0          0      190.190.190.130                         12346 190.190.190.130                         12346 Lindos SD-WAN             biz-internet    -     up     0:14:25:06 0    

vbond have only one IP 190.190.190.130 it reach via Internet but not reach via MPLS 
I mention in first time you need to add vbond IP that reach via MPLS 
this make vpn 0 mpls transport interface can establish control connection to vbond via MPLS 
MHM