How to force the dtls connection over one interface

[erick.mephon]

Hi,

I'm running some tests in my lab, on one of the site I'm try to implement a design involving both traditional routing and vEdges :

On this pic, right now for the traditional part of the network, only SW3 and R1 are up and running with OSPF (everything is in area 0). My vedges are also running ospf (underlay) and the network is converged. The route to the vManage (1.1.1.1) is elected through SW3. But everytime I unshut ge0/2, although the next hop does not change (SW3), the tcpdump shows me that ge0/2 become the interface via which vEdge2 tries to connect to the vmanage and it no longer works. If I shut ge0/2, the dtls connection revert back to ge0/0 and everything comes back to normal.

I tried to lower the vmanage-connection-preference to 0 on ge0/2 without much success, how can one do to force the dtls connection to the vmanage through one chosen interface ?

[erick.mephon]

By the way, interestingly vedge1 & 2 have the exact same config (same template) and only vedge2 has the issue

[Kanan Huseynli]

Hi,

root cause:

vbond dtls 0.0.0.0 0 0 145.56.17.129 12346 145.56.17.129 12346 private1 - connect 0

You can't create control connection to vBond via private1 (ge0/0) to be able to connect rest controllers. Firstly, vBond should be reachable, but router can not because there is only one path for vBond which is via ge0/2.

show ip route
Codes Proto-sub-type:
IA -> ospf-intra-area, IE -> ospf-inter-area,
E1 -> ospf-external1, E2 -> ospf-external2,
N1 -> ospf-nssa-external1, N2 -> ospf-nssa-external2,
e -> bgp-external, i -> bgp-internal
Codes Status flags:
F -> fib, S -> selected, I -> inactive,
B -> blackhole, R -> recursive, L -> import

PROTOCOL NEXTHOP NEXTHOP NEXTHOP
VPN PREFIX PROTOCOL SUB TYPE IF NAME ADDR VPN TLOC IP COLOR ENCAP STATUS
---------------------------------------------------------------------------------------------------------------------------------------------
0 0.0.0.0/0 ospf E2 ge0/2 172.16.112.11 - - - - F,S
0 1.1.1.0/24 ospf IA ge0/0 172.16.32.3 - - - - F,S
0 10.10.10.3/32 ospf IA ge0/0 172.16.32.3 - - - - F,S
0 10.10.10.102/32 connected - system - - - - - F,S
0 145.56.17.128/30 ospf IA ge0/2 172.16.112.11 - - - - F,S

Check your configurations, OSPF database that why you have one inter-area OSPF route for vBond subnet. It should be ECMP over both ge0/0 and ge0/2. Then vBond's connectivity will be successful and vManage's also.

[erick.mephon]

One question, what do you think about the color ?

I was watching at this and SW3 has ECMP to reach ge0/2 between R1 and vEdge2, but at the same time, ge0/0 and ge 0/2 has different colors, so I believe one interface will never pass the traffic to the other, do you think it could come from that ? I'm going to search for info on how to allow such traffic, since it's impossible for me to configure both with the same color, the OS won't let me do it.

[Kanan Huseynli · Accepted solution]

Hello,

tunnel interfaces should have different color - you cant do something in this case.

One interface traffic can not go via another interface - known behavior.

In your topology, route via ge0/2 has lower metric than route via ge0/0. This is the reason.

To be honest, I don't understand reason for such underlay network design. Normally, you should have multiple transport interfaces with default route (at worst case with also MPLS routes when you have L3 MPLS in CE-PE). Transport network should belong to some SP/ISP channel (MPLS/ LTE/ Internet etc.). If you just for redundant purposes want to have multiple underlay physical interface, you can use loopback interface with tunnel enabled. For this, you should not bind loopback to physical interface and you should remove tunnel config from physical interfaces.

[erick.mephon]

Yeah,

I think the loopback solution should work, let me give it a try and share here if anything

[erick.mephon]

So design issue. I removed all tunnel interfaces from all the phy interfaces, created a loopback interface and added the tunnel there, added it to the OSPF and it started to work properly. I guess by doing so, the color issue has been removed.

I noticed that beside the dtls connection issue, there was actually IP connectivity issue that was probably the result of the color config on the different tunnel interfaces.

[svemulap@cisco.com]

You can try with 'max-control-connections' set to 0 on SW3 (ge0/2)
This will force to establish (DTLS) control via ge0/0

Please check more on this in the SDWAN Design Guide at:
https://www.cisco.com/c/en/us/td/docs/solutions/CVD/SDWAN/cisco-sdwan-design-guide.html?dtid=osscdc000283

HTH

[erick.mephon]

just tried to see if that could have been a workaround but no.

There is a routing issue, it could be related to the color, the segment between R1 and vEdge2 it unreachable from vmanage. Looking to understand why.