VXLAN EVPN Multisite Prefixes not advertising between Border Gateways

Adam Swindell · ‎10-27-2023

Hello,

I am working on a PoC design within EVE-NG and virtual Nexus 9k's. Everything I've read about virtual 9k's is that vxlan evpn is supported, and multisite is supported as long as the border gateways are anycast and not vPC. Mine are not vPC.

I am using NDFC to deploy the individual fabrics and the multisite fabric. Even though I am using NDFC, I am familiar with BGP and the CLI to manually configure VXLAN EVPN.

The scenario that is not working for me is as follows.

1. I've created a VRF 'datacenter-default' and attached it to both fabrics

2. I've created L3OUT interfaces off the BGWs to the "WAN" for the VRF datacenter-default

3. I've created the DCI underlay interfaces of the BGWs to the "WAN" for the default VRF

4. I've created the DCI overlay network between the BGWs. The peers are up and I can ping between loopback 0 of all the border gateways.

5. I've created the multisite fabric, which sets up the Anycast loopback address (100) on the BGWs and the other msite configuration.

6. I've created a network (vlan 2002) and attached it to both fabrics and the border gateways.

The issue I am seeing is that EVPN prefixes are not being sent between the border gateways over the eBGP multihop peers.

I am seeing these debugs. But I don't know what they mean. Google has not been helpful. What do these debugs mean?

2023 Oct 27 13:18:28.994802 bgp: [9727] (default) UPD: [L2VPN EVPN] 10.80.0.2 10.241.0.4:3:[5]:[0]:[0]:[30]:[10.243.2.4]/224 path-id 1 not sent to peer due to: EVPN Border Gateway: no crossing via remote RD
2023 Oct 27 13:18:28.995160 bgp: [9727] (default) UPD: [L2VPN EVPN] 10.80.0.2 10.241.0.4:34768:[2]:[0]:[0]:[48]:[5004.0000.1b08]:[0]:[0.0.0.0]/216 path-id 1 not sent to peer due to: EVPN Border Gateway: no crossing via remote RD
2023 Oct 27 13:18:28.995224 bgp: [9727] (default) UPD: [L2VPN EVPN] 10.80.0.2 10.241.0.4:33567:[2]:[0]:[0]:[48]:[5004.0000.1b08]:[0]:[0.0.0.0]/216 path-id 1 not sent to peer due to: EVPN Border Gateway: no crossing via remote RD
2023 Oct 27 13:18:28.995241 bgp: [9727] (default) UPD: [L2VPN EVPN] 10.80.0.2 10.241.0.4:3:[5]:[0]:[0]:[30]:[10.190.1.0]/224 path-id 1 not sent to peer due to: EVPN Border Gateway: no crossing via remote RD
2023 Oct 27 13:18:28.995255 bgp: [9727] (default) UPD: [L2VPN EVPN] 10.80.0.2 10.241.0.4:3:[5]:[0]:[0]:[30]:[10.190.1.4]/224 path-id 1 not sent to peer due to: EVPN Border Gateway: no crossing via remote RD
2023 Oct 27 13:18:28.995267 bgp: [9727] (default) UPD: [L2VPN EVPN] 10.80.0.2 10.241.0.4:3:[5]:[0]:[0]:[30]:[10.190.2.0]/224 path-id 1 not sent to peer due to: EVPN Border Gateway: no crossing via remote RD
2023 Oct 27 13:18:28.995280 bgp: [9727] (default) UPD: [L2VPN EVPN] 10.80.0.2 10.241.0.4:3:[5]:[0]:[0]:[30]:[10.190.2.4]/224 path-id 1 not sent to peer due to: EVPN Border Gateway: no crossing via remote RD
2023 Oct 27 13:18:28.995293 bgp: [9727] (default) UPD: [L2VPN EVPN] 10.80.0.2 10.241.0.4:3:[5]:[0]:[0]:[32]:[10.80.0.2]/224 path-id 1 not sent to peer due to: EVPN Border Gateway: no crossing via remote RD
2023 Oct 27 13:18:28.995346 bgp: [9727] (default) UPD: [L2VPN EVPN] 10.80.0.2 10.241.0.4:3:[5]:[0]:[0]:[32]:[10.80.0.5]/224 path-id 1 not sent to peer due to: EVPN Border Gateway: no crossing via remote RD
2023 Oct 27 13:18:28.995377 bgp: [9727] (default) UPD: [L2VPN EVPN] 10.80.0.2 10.241.0.4:3:[5]:[0]:[0]:[32]:[10.80.1.2]/224 path-id 1 not sent to peer due to: EVPN Border Gateway: no crossing via remote RD
2023 Oct 27 13:18:28.995390 bgp: [9727] (default) UPD: [L2VPN EVPN] 10.80.0.2 10.241.0.4:3:[5]:[0]:[0]:[32]:[10.80.1.4]/224 path-id 1 not sent to peer due to: EVPN Border Gateway: no crossing via remote RD
2023 Oct 27 13:18:28.995403 bgp: [9727] (default) UPD: [L2VPN EVPN] 10.80.0.2 10.241.0.4:3:[5]:[0]:[0]:[32]:[10.99.99.1]/224 path-id 1 not sent to peer due to: EVPN Border Gateway: no crossing via remote RD
2023 Oct 27 13:18:28.995415 bgp: [9727] (default) UPD: [L2VPN EVPN] 10.80.0.2 10.241.0.5:3:[5]:[0]:[0]:[24]:[10.180.70.0]/224 path-id 1 not sent to peer due to: EVPN Border Gateway: no crossing via remote RD
2023 Oct 27 13:18:28.995428 bgp: [9727] (default) UPD: [L2VPN EVPN] 10.80.0.2 10.241.0.5:3:[5]:[0]:[0]:[24]:[10.245.201.0]/224 path-id 1 not sent to peer due to: EVPN Border Gateway: no crossing via remote RD

M02@rt37 · ‎10-27-2023

Hello @Adam Swindell,

The debug messages you're seeing in the context of your VXLAN EVPN setup suggest that the prefixes are not being sent between the border gateways due to an issue related to the "remote RD". Each prefix in an EVPN should have a unique RD, and this error is indicating that there might be a problem with the RD configuration or the uniqueness of RD values in your setup.

Ensure that the RD values are unique for each VRF on both sides of the border gateways. The RD is used to differentiate prefixes within the same VPN.

Best regards
.ı|ı.ı|ı. If This Helps, Please Rate .ı|ı.ı|ı.

Adam Swindell · ‎10-31-2023

Hello,

I'm not sure I understand how the RD's could be overlapping. I am using RD AUTO, which uses the router id and then some random value (apparently).... so it looks like this on BGW1-Site1

"Route Distinguisher: 10.80.0.2:34769 (L2VNI 30000)"

and on BGW1-Site2

"Route Distinguisher: 10.241.0.2:34769 (L2VNI 30000)"

So, the RD's are unique... right?