VXLAN EVPN + L3 over VPC

[tommy182]

Hello!

Please help me to understand what I`m doing wrong.

I configured VXLAN fabric with BGP EVPN and vPC(everything looks fine)

But I need to connect some L3 device to pair of BorderLeafs(participating in VPC) and establish OSPF

So I configure Vlan,SVI, and vPC PortChannel down to L3 device.

Both peers established OSPF relationships with this device and with each other through Peer-Link

Now I need to redistribute Overlay networks(iBGP) to OSPF for this L3 device.

There is issue with AD, OSPF - 110, iBGP- 200

In the end one of the peers advertise redistributed route(with OSPF ad 110) through peer-link  and other vpc-peer added this route in RIB instead of iBGP(AD 200), this is not normal, I have no idea how solve this.

In conclusion second vpc peer send all traffic destined to VXLAN-Fabric not through Spines but through vPC Peer-Link.

[Rick1776]

It’s because at this time it’s best to have separate L3 links on non VPC line and let the routing protocol create the ECMP links.

Please see the following link...

https://www.cisco.com/c/en/us/support/docs/ip/ip-routing/118997-technote-nexus-00.html

[cetelem_bank]

delete

[tommy182]

So, thank you, I`ve already read this article.
But problem isn`t in routing adjacencies or with route exchange through vPC, problem will arise even if we will use point-to-point links.

I mean we have to configure some east-west routing adjacencies between vpc peers for route table synchronization.
Assume that all routes in OSPF take preference of any iBGP routes because of Administrative Distance.
In the end we will face with strange situations when redistributed from iBGP-to-OSPF route will take precedence from fabric iBGP routes and one of the peers will send traffic towards the fabric not to spine, but towards to another peer through peer-link.

So this problem will arise only in VxLAN BGP EVPN environment because of AD of iBGP routes

[Rick1776]

Hmm I see what you mean, what about configuring different' costs per OSPF interfaces to be higher then the iBGP to steer the traffic to the iBGP.

Also the OSPF is for the underlay in the VxLAN network so you should be able to still route VTEP to VTEP without an issue correct?

[tommy182]

Rick, the issue with AD not with metric.
If I change just metric for interfaces(in this way for routes itself) the routes wouldn`t be less preferred than iBGP routes cause of AD(AD take precedence from metric)
So, I see couple of solutions for this problem.
- First is use dedicated links without East-West link for route replication, in this case Border Leafs wouldn`t advertise to each other redistributed routes through East-West(OSPF through peer-link) adjacencies .For me this solution not so convenient like pure l3-over-vpc solution.
- Second is advertise to Overlay-OSPF just a default route, in this case issue with redistributed from iBGP routes wouldn`t arise. But it`s not applicable for some reasons when I need to doing opposite, like advertising default route from Firewall(l3 device) to Fabric, and two default routes doesn’t co-exist with each other since they in one vrf.
- Third is somehow change the AD for routes received from peer through peer-link East-West adjacencies. I don`t know achieve this, unfortunately. There is mechanism of "table-map" that helps to change AD for routes. But I don`t know how change AD only for routes received from peer.

Maybe You know any possible solutions for change AD just for peer-advertised routes?

[tommy182 · Accepted solution]

Rick, I think I found a solution.

When routes redistribute into Overlay-OSPF from within iBGP they tagged with AS number(65510 in my case)

So with feature Table-Map now we can change AD to 240 for example.

Thus when this redistributed( from iBGP) route will advertise between peers they will change locally AD for this routes(match tag 65510) and this routes wouldn`t take precedence from fabric iBGP routes.

Now I`m analyzing this solution for some weakness, but for now it  looks fine.

router ospf OVERLAY_OSPF

………

  vrf INSIDE_NET

    router-id 2.2.2.4

    redistribute bgp 65510 route-map REDISTR_iBGP_TO_OSPF

    table-map AD_FOR_VPCPEERS_ROUTES_240

route-map AD_FOR_VPCPEERS_ROUTES_240 permit 10

  match tag 65510

  match route-type external

  set distance 240

route-map REDISTR_iBGP_TO_OSPF permit 10

  match route-type internal

router bgp 65510

……..

  vrf INSIDE_NET

    address-family ipv4 unicast

      advertise l2vpn evpn

      redistribute ospf OVERLAY_OSPF route-map REDISTR_OSPF_TO_iBGP

route-map REDISTR_OSPF_TO_iBGP deny 10

  match tag 65510

route-map REDISTR_OSPF_TO_iBGP permit 20

[Rick1776]

That’s awesome. I’ll have to remember that one.

[Rick1776]

That's fantastic. Where did you find that gem in the documentation.

Have a great day.