VXLAN evpn campus with legacy

[mikhailov.ivan]

Hello colleagues! I am a bit lost in the migration process into exlav/evpn fabric mixed with legacy part.

Let me explain. Look at the diagram in attachment. Here is the leaf-spine topology but I simplified it (in real it uses stacking).   There are a c9k switch as a spine and couple of c9k as a leaf (leaf 1 and 2 in the left side) and we have a "standard" vxlan/evpn between them like a base one with ospf + bgp on loopbacks0 with evpn address family between them, it's more or less clear , we have the one stratched vlan (10) in vrf 10 , couple of different subnets in the vrf 20 etc., but the case is that we have to save several more legacy c3750x switches and swap them one by one in a while. And I am not 100% sure how to integrate them.  There are 2 leafs (switch 3 and 4 in the right side) with the ipservice ios software,  they don't support vxlan , but support mp-bgp as far as I see. They terminate several subnet and have link to the spine. And how should I integrate routes from c3750 switches (in different VRFs) with evpn routes? In my perspective the most elegant way is configuring the spine as a participant of routing process for each VRF address family and on 3750 I can just push the default gateway. Each c9k should see legacy routes as a type-5 and put them into the rib for each vrf.  The spine becomes a "central" route exchange point. So the leaf 4 (3750) will see just a default route per each vrf toward the spine, and c9k leafs will see the route for instance 10.1.5.0/24 in the rib per each vrf and as route type5 in the bgp table, right?

Is it correct way or I should install bgp session with each evpn switch ? Should I establish bgp sessions between c9k leafs and spine in both evpn and VRF address family? 

Sorry if I formalized my thoughts confused,  I can rewrite if anything.

Thanks in advance!

[vishalbhandari]

@mikhailov.ivan 

Your approach sounds mostly correct. The spine can act as the central route exchange point, and you can use EVPN Type 5 routes to inject the legacy 3750X subnets into the EVPN fabric. This way, the C9K leaf switches will receive the routes from the 3750X switches as Type 5 routes and forward them accordingly. The spine will indeed participate in the routing process for each VRF address family and handle the exchange of routes between VRFs.

You don’t necessarily need to establish separate BGP sessions between the C9K leaf switches and the spine in both EVPN and VRF address families, as long as the spine is routing for both address families. The BGP sessions should be established between the C9K spine and leaf switches with the EVPN address family, which will include the Type 5 routes, and the VRF address family for the individual VRFs. This way, both EVPN and legacy routes are exchanged properly.

In summary, using the spine as the central point for route exchange is efficient, and BGP sessions can be simplified as long as the spine is handling both EVPN and VRF routes.

[MHM Cisco World]

https://www.cisco.com/c/en/us/support/docs/switches/catalyst-9300-series-switches/221574-implement-and-verify-bgp-only-vxlan-evpn.html

no need to run address-family ipv4 vrf x between leafs and spine
MHM

[mikhailov.ivan]

Thanks mates! If I correctly understood , I should run "redistribute connected" on each leaf switch per vrf  and on the spine run "advertise l2vpn evpn" for each vrf, correct ?  By this way evpn routes will be injected into VRF's address families, and on the spine I do the "route-target stiching"  for the opposite process right ?  Between the spine and c9k leafs we keep only evpn address family.

[MHM Cisco World]

not only evpn but address family ipv4 unicast also 

MHM

[mikhailov.ivan]

Hmm, but in the previous message you said that no need to run address-family between leafs and spine, so one more time please , between spine and leafs (all c9k) we need both address-family , for evpn + per each vrf , right ? Could we avoid it and keep only evpn between c9k leafs and spine and do all exchanges only on the spine by advertising l2vpn evpn in a vrf address-family and by RT stiching correspondingly?  If I correctly understand it the spine will have the evpn bgp table and standard rib for each vrf , looks like it can do some exchange 

[MHM Cisco World]

I edit my reply to make it more clear.

MHM

[mikhailov.ivan]

Got it, thanks.  About other actions on the spine , are my thoughts correct? 

[mikhailov.ivan]

Guys, I got a bit confused. Within the vxlan\evpn fabric everything works fine, but with legacy C3750 as a leaf I can establish bgp session in a particular VRF , but can't get routes advertised\recieved. From my perspective activating the neighbor in the ipv4 vrf should be enough. Could someone please show the point ?  So, I would like to establish mp-bgp session between the spine c9k and the c3750 and exchange routes within VRF (not vrf-lite).

here is the part of the spine config

#########SPINE##############
vrf definition 10
description test
rd 1:10
route-target export 1:10
route-target import 1:10
!
address-family ipv4
route-target export 1:10 stitching
route-target import 1:10 stitching
exit-address-family

router bgp 65555
template peer-policy iBGP_Policy
route-reflector-client
send-community both
exit-peer-policy
!
template peer-session iBGP
remote-as 65555
update-source Loopback0
fall-over bfd
exit-peer-session
!
bgp log-neighbor-changes
bgp update-delay 1
bgp graceful-restart
no bgp default ipv4-unicast
neighbor 10.100.0.73 inherit peer-session iBGP #C3750
neighbor 10.100.0.134 inherit peer-session iBGP #C9k
neighbor 10.100.0.135 inherit peer-session iBGP #C9k
!
address-family ipv4 #For c3750
neighbor 10.100.0.73 activate
neighbor 10.100.0.73 send-community both
neighbor 10.100.0.73 next-hop-self
maximum-paths 2
exit-address-family
!
address-family l2vpn evpn #for C9K
neighbor 10.100.0.134 activate
neighbor 10.100.0.134 send-community both
neighbor 10.100.0.134 inherit peer-policy iBGP_Policy
neighbor 10.100.0.135 activate
neighbor 10.100.0.135 send-community both
neighbor 10.100.0.135 inherit peer-policy iBGP_Policy
exit-address-family
!
address-family ipv4 vrf 10
advertise l2vpn evpn
network 10.10.10.10 mask 255.255.255.255 #test Lo int
redistribute connected
neighbor 10.100.0.73 remote-as 65555
neighbor 10.100.0.73 activate
neighbor 10.100.0.73 send-community both
exit-address-family

and the C3750

########C3750#############
vrf definition 10
description test
rd 1:10
!
address-family ipv4
route-target export 1:10
route-target import 1:10
exit-address-family
!

router bgp 65555
bgp router-id 10.100.0.73
bgp log-neighbor-changes
bgp update-delay 1
bgp graceful-restart restart-time 120
bgp graceful-restart stalepath-time 360
bgp graceful-restart
no bgp default ipv4-unicast
neighbor iBGP peer-group
neighbor iBGP remote-as 65555
neighbor iBGP update-source Loopback0
neighbor iBGP fall-over bfd
neighbor 10.100.0.133 peer-group iBGP
!
address-family ipv4
neighbor iBGP send-community both
neighbor 10.100.0.133 activate
exit-address-family
!
address-family ipv4 vrf 101
redistribute connected #has an Lo as well for tests
neighbor 10.100.0.133 remote-as 65555 #Spine
neighbor 10.100.0.133 activate
neighbor 10.100.0.133 send-community both
exit-address-family
!

[mikhailov.ivan]

My concern is that c3750 doesn't support the bgp address-family vpnv4 ,  and maybe somebody knows how to pass it more elegant (without vrf lite)

[MHM Cisco World]

I don't think all SW support vxlan,

It not only l2vpn addresses family missing it many evpn commands need to make vxlan work. 

MHM

[mikhailov.ivan]

Er, of course it's not the whole config , just the bgp part. And the question was how to make a friendship between the spine and legacy 3750 because c3750 doesn't support the address-family vpnv4. Btw within c9k vxlan/evpn works well.

[MHM Cisco World]

Friend we talking about l2 not l3 so vpnv4 is not use here and not need config vrf' we need vrf for vxlan and since c3k not support vxlan we dont need it.

What you can do I think is interconnect c3k with c9k leaf SW.

Sorry for this bad news 

MHM

[mikhailov.ivan]

We are talking about L2 if  the vrf-lite is only one way. In this case we will need to get routes on the spine from evpn into rib for each vrf and then establish bgp sessions using sub.interfaces and address-family ipv4 vrf X. Correct me if I am wrong , for avoiding this L2 interaction we could use the single L3 link and the address-family vpnv4 for exchanging routes with RT between vrfs , right? But the c3750 doesn't support vpnv4 , that is why I raised the topic 

[mikhailov.ivan]

well, after a long time of attempting, I didn't get success with L3 single links between nodes. I can reach the situation when control plane routing information is delivered into the VRF's RIB , but with the next hop from the GRT, so the data plabe ttaffic can't be delivered correctly. The legacy leaf (3750) is connected to the spine via a Trunk with sub.int and VRF-lite schema, it's ugly.  I expected the spine acts as a routing process participant like EVPN + VRF-lite at the same time, but didn't get it. Seems the only way is the VRF-lite for all leaves including VXLAN\EVPN switches and legacy. In the Trunk one of sub will be used in GRT for VXLAN tunneling across these 2 c9K swithces and other subs for each VRF, thus from C9k we will use address-family l2vpn evpn  and address-family ipv4 vrf X at the same time. And in this scenario the spine will act just as an RR. That is sad, despite that according to this doc https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst9300/software/release/17-10/configuration_guide/vxlan/b_1710_bgp_evpn_vxlan_9300_cg/configuring_evpn_vxlan_external_connectivity.html   border spine design is supported, but there are no config examples.