VRF route leak

networkadmin AQ · ‎06-24-2024

I have the following network;

A ring backbone consisting of Nokia Routers running multiprotocol BGP, MPLS, LDP, SDP, VPRN service. The Sites would run a Cisco Switch that communicates with the Noka Router through BGP.
The Cisco switches can communicate with the Nokia routers fine.
I am running VRF on the cisco switches and I am able to ping vrf to the other side and vice versa. However I am not able to run a successful ping from the clients at each side of the Cisco switches.
I have read a lot about VRF route leaking however I am unable to get it to work.
I need some help with this.

MHM Cisco World · ‎06-24-2024

Can you draw topolgy

Thanks

MHM

networkadmin AQ · ‎06-24-2024

Just some more info:
The nodes CE_02(10.9.3.1) and CE_01(106.3.1) are both Cisco C9300 switches.
I am able to run a 'ping vrf VRF1000 10.X.3.1' from both sides and be able to communicate.
I have a client at each side; 10.X.3.15, however I am not able to do a normal ping from client to client.

I see all the routes in the vrf routing table though.
I need the clients to be able to communicate with each other.

MHM Cisco World · ‎06-24-2024

CE connect to PE via BGP

point make traffic drop
1- each site not learn prefix of other sites this can due to
A- you dont config VRF Route target correctly
B-you use same BGP AS in each site and that need as ""allowas in""
2- the VLAN you config in site is not config with ""ip vrf forwarding""

MHM

networkadmin AQ · ‎06-25-2024

Can you be more specific on your points ?

MHM Cisco World · ‎06-25-2024

can I see how you config BGP in cisco SW
and
show ip route vrf VRF <<- in CE cisco SW

MHM

networkadmin AQ · ‎06-25-2024

BN_03_1030301#show running-config | sec bgp
router bgp 65001
bgp router-id 10.255.255.206
bgp log-neighbor-changes
no bgp default ipv4-unicast
neighbor 10.255.255.205 remote-as 65000
neighbor 10.255.255.205 update-source Vlan800
!
address-family ipv4
redistribute connected
neighbor 10.255.255.205 activate
exit-address-family
!
address-family ipv4 vrf VRF1000
network 10.6.3.0 mask 255.255.255.0
redistribute connected
neighbor 10.255.255.205 remote-as 65000
neighbor 10.255.255.205 activate
neighbor 10.255.255.205 send-community both
neighbor 10.255.255.205 soft-reconfiguration inbound
exit-address-family

IS_03_1090301#show running-config | sec bgp
router bgp 65002
bgp router-id 10.255.255.210
bgp log-neighbor-changes
no bgp default ipv4-unicast
neighbor 10.255.255.209 remote-as 65000
neighbor 10.255.255.209 update-source Vlan800
!
address-family ipv4
redistribute connected
neighbor 10.255.255.209 activate
exit-address-family
!
address-family ipv4 vrf VRF1000
network 10.9.3.0 mask 255.255.255.0
redistribute connected
neighbor 10.255.255.209 remote-as 65000
neighbor 10.255.255.209 activate
neighbor 10.255.255.209 send-community both
neighbor 10.255.255.209 soft-reconfiguration inbound
exit-address-family

BN_03_1030301#show ip route vrf VRF1000

Routing Table: VRF1000
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2, m - OMP
n - NAT, Ni - NAT inside, No - NAT outside, Nd - NAT DIA
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
H - NHRP, G - NHRP registered, g - NHRP registration summary
o - ODR, P - periodic downloaded static route, l - LISP
a - application route
+ - replicated route, % - next hop override, p - overrides from PfR
& - replicated local route overrides by connected

Gateway of last resort is not set

10.0.0.0/8 is variably subnetted, 8 subnets, 3 masks
B 10.1.3.0/24 [20/0] via 10.255.255.205, 00:25:05
C 10.6.3.0/24 is directly connected, Vlan1603
L 10.6.3.1/32 is directly connected, Vlan1603
B 10.9.3.0/24 [20/0] via 10.255.255.205, 00:25:05
C 10.255.255.204/30 is directly connected, Vlan800
L 10.255.255.206/32 is directly connected, Vlan800
B 10.255.255.208/30 [20/0] via 10.255.255.205, 00:25:05
B 10.255.255.224/30 [20/0] via 10.255.255.205, 00:25:05
172.16.0.0/32 is subnetted, 1 subnets
C 172.16.0.6 is directly connected, Loopback0

IS_03_1090301#show ip route vrf VRF1000

Routing Table: VRF1000
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2, m - OMP
n - NAT, Ni - NAT inside, No - NAT outside, Nd - NAT DIA
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
H - NHRP, G - NHRP registered, g - NHRP registration summary
o - ODR, P - periodic downloaded static route, l - LISP
a - application route
+ - replicated route, % - next hop override, p - overrides from PfR
& - replicated local route overrides by connected

Gateway of last resort is not set

10.0.0.0/8 is variably subnetted, 8 subnets, 3 masks
B 10.1.3.0/24 [20/0] via 10.255.255.209, 00:25:21
B 10.6.3.0/24 [20/0] via 10.255.255.209, 00:25:21
C 10.9.3.0/24 is directly connected, Vlan1903
L 10.9.3.1/32 is directly connected, Vlan1903
B 10.255.255.204/30 [20/0] via 10.255.255.209, 00:25:21
C 10.255.255.208/30 is directly connected, Vlan800
L 10.255.255.210/32 is directly connected, Vlan800
B 10.255.255.224/30 [20/0] via 10.255.255.209, 00:25:21
172.16.0.0/32 is subnetted, 1 subnets
C 172.16.0.9 is directly connected, Loopback0

Harold Ritter · ‎06-27-2024

Hi @networkadmin AQ ,

The same neighbor cannot be configured in the global and in the VRF. You need to remove it from global config.

router bgp 65001
no neighbor 10.255.255.205

You need to do the same on all CEs.

As for the PCs not being able to ping from one site to another, it could be due to local FW on the PC not allowing it. Disable the FW for testing.

Regards,

Harold Ritter
Sr Technical Leader
CCIE 4168 (R&S, SP)
harold@cisco.com
México móvil: +52 1 55 8312 4915
Cisco México
Paseo de la Reforma 222
Piso 19
Cuauhtémoc, Juárez
Ciudad de México, 06600
México

networkadmin AQ · ‎06-27-2024

I removed it, .. the configs on both switches is as below:
BN_03:
router bgp 65001
bgp router-id 10.255.255.206
bgp log-neighbor-changes
no bgp default ipv4-unicast
!
address-family ipv4
redistribute connected
exit-address-family
!
address-family ipv4 vrf VRF1000
network 10.6.3.0 mask 255.255.255.0
redistribute connected
neighbor 10.255.255.205 remote-as 65000
neighbor 10.255.255.205 activate
neighbor 10.255.255.205 send-community both
neighbor 10.255.255.205 soft-reconfiguration inbound
exit-address-family

IS_03:
router bgp 65002
bgp router-id 10.255.255.210
bgp log-neighbor-changes
no bgp default ipv4-unicast
!
address-family ipv4
redistribute connected
exit-address-family
!
address-family ipv4 vrf VRF1000
network 10.9.3.0 mask 255.255.255.0
redistribute connected
neighbor 10.255.255.209 remote-as 65000
neighbor 10.255.255.209 activate
neighbor 10.255.255.209 send-community both
neighbor 10.255.255.209 soft-reconfiguration inbound
exit-address-family

On BN_03: I get these in the logs ...

Jun 27 10:54:37.904: BGP: topo global:MVPNv6 Unicast:base Scanning routing tables
Jun 27 10:55:37.913: BGP: topo global:IPv4 Unicast:base Scanning routing tables
Jun 27 10:55:37.913: BGP: topo global:IPv6 Unicast:base Scanning routing tables
Jun 27 10:55:37.913: BGP: topo global:VPNv4 Unicast:base Scanning routing tables
Jun 27 10:55:37.913: BGP: topo VRF1000:VPNv4 Unicast:base Scanning routing tables
Jun 27 10:55:37.913: BGP: topo global:IPv4 Multicast:base Scanning routing tables
Jun 27 10:55:37.913: BGP: topo global:L2VPN E-VPN:base Scanning routing tables
Jun 27 10:55:37.913: BGP: topo global:VPNv4 Multicast:base Scanning routing tables
Jun 27 10:55:37.913: BGP: topo VRF1000:VPNv4 Multicast:base Scanning routing tables
Jun 27 10:55:37.913: BGP: topo global:MVPNv4 Unicast:base Scanning routing tables
Jun 27 10:55:37.913: BGP: topo VRF1000:MVPNv4 Unicast:base Scanning routing tables
Jun 27 10:55:37.913: BGP: topo global:MVPNv6 Unicast:base Scanning routing tables
Jun 27 10:56:37.922: BGP: topo global:IPv4 Unicast:base Scanning routing tables
Jun 27 10:56:37.922: BGP: topo global:IPv6 Unicast:base Scanning routing tables
Jun 27 10:56:37.922: BGP: topo global:VPNv4 Unicast:base Scanning routing tables
Jun 27 10:56:37.922: BGP: topo VRF1000:VPNv4 Unicast:base Scanning routing tables
Jun 27 10:56:37.922: BGP: topo global:IPv4 Multicast:base Scanning routing tables
Jun 27 10:56:37.922: BGP: topo global:L2VPN E-VPN:base Scanning routing tables
Jun 27 10:56:37.922: BGP: topo global:VPNv4 Multicast:base Scanning routing tables
Jun 27 10:56:37.922: BGP: topo VRF1000:VPNv4 Multicast:base Scanning routing tables

* I do not have Firewall running on the pc's.
*Even when I try to ping the Router from the pc at the other side, .. it doesn't work, .. if I leave a continous ping ... somehow it works for a while .. then it drops.

MHM Cisco World · ‎06-25-2024

the BGP and vrf prefix is excellent,

10.0.0.0/8 is variably subnetted, 8 subnets, 3 masks
B 10.1.3.0/24 [20/0] via 10.255.255.205, 00:25:05
C 10.6.3.0/24 is directly connected, Vlan1603
L 10.6.3.1/32 is directly connected, Vlan1603
B 10.9.3.0/24 [20/0] via 10.255.255.205, 00:25:05
C 10.255.255.204/30 is directly connected, Vlan800
L 10.255.255.206/32 is directly connected, Vlan800
B 10.255.255.208/30 [20/0] via 10.255.255.205, 00:25:05
B 10.255.255.224/30 [20/0] via 10.255.255.205, 00:25:05
172.16.0.0/32 is subnetted, 1 subnets
C 172.16.0.6 is directly connected, Loopback0

10.0.0.0/8 is variably subnetted, 8 subnets, 3 masks
B 10.1.3.0/24 [20/0] via 10.255.255.209, 00:25:21
B 10.6.3.0/24 [20/0] via 10.255.255.209, 00:25:21
C 10.9.3.0/24 is directly connected, Vlan1903
L 10.9.3.1/32 is directly connected, Vlan1903
B 10.255.255.204/30 [20/0] via 10.255.255.209, 00:25:21
C 10.255.255.208/30 is directly connected, Vlan800
L 10.255.255.210/32 is directly connected, Vlan800
B 10.255.255.224/30 [20/0] via 10.255.255.209, 00:25:21
172.16.0.0/32 is subnetted, 1 subnets
C 172.16.0.9 is directly connected, Loopback0

do traceroute from client see if the IP of CE is appear as first hop

check the FW in client PC it can turn ON and drop the packet

MHM

networkadmin AQ · ‎06-25-2024

Yesterday I was pinging, and I was getting successful pings for a while. But then it stopped.
I checked the logs on the switches, and I see the output below on both switches.

When I do a traceroute on the client, I get :
1 10.128.128.128 reports: Destination net unreachable

See the logs on the switches below:

Switch 1:

Jun 25 07:24:17.479: BGP: 10.255.255.205 Active open failed - update-source Vlan800 is not available, open active delayed 9216ms (35000ms max, 60% jitter)
Jun 25 07:24:26.695: BGP: 10.255.255.205 Active open failed - update-source Vlan800 is not available, open active delayed 12288ms (35000ms max, 60% jitter)
Jun 25 07:24:38.983: BGP: 10.255.255.205 Active open failed - update-source Vlan800 is not available, open active delayed 11264ms (35000ms max, 60% jitter)
Jun 25 07:24:50.247: BGP: 10.255.255.205 Active open failed - update-source Vlan800 is not available, open active delayed 14336ms (35000ms max, 60% jitter)
Jun 25 07:25:04.583: BGP: 10.255.255.205 Active open failed - update-source Vlan800 is not available, open active delayed 11264ms (35000ms max, 60% jitter)
Jun 25 07:25:10.348: BGP: topo global:IPv4 Unicast:base Scanning routing tables
Jun 25 07:25:10.348: BGP: topo global:IPv6 Unicast:base Scanning routing tables
Jun 25 07:25:10.348: BGP: topo global:VPNv4 Unicast:base Scanning routing tables
Jun 25 07:25:10.348: BGP: topo VRF1000:VPNv4 Unicast:base Scanning routing tables
Jun 25 07:25:10.348: BGP: topo global:IPv4 Multicast:base Scanning routing tables
Jun 25 07:25:10.348: BGP: topo global:L2VPN E-VPN:base Scanning routing tables
Jun 25 07:25:10.348: BGP: topo global:VPNv4 Multicast:base Scanning routing tables
Jun 25 07:25:10.348: BGP: topo VRF1000:VPNv4 Multicast:base Scanning routing tables
Jun 25 07:25:10.348: BGP: topo global:MVPNv4 Unicast:base Scanning routing tables
Jun 25 07:25:10.348: BGP: topo VRF1000:MVPNv4 Unicast:base Scanning routing tables
Jun 25 07:25:10.348: BGP: topo global:MVPNv6 Unicast:base Scanning routing tables
Jun 25 07:25:15.847: BGP: 10.255.255.205 Active open failed - update-source Vlan800 is not available, open active delayed 7168ms (35000ms max, 60% jitter)
Jun 25 07:25:23.017: BGP: 10.255.255.205 Active open failed - update-source Vlan800 is not available, open active delayed 8192ms (35000ms max, 60% jitter)
Jun 25 07:25:31.209: BGP: 10.255.255.205 Active open failed - update-source Vlan800 is not available, open active delayed 7168ms (35000ms max, 60% jitter)
Jun 25 07:25:38.378: BGP: 10.255.255.205 Active open failed - update-source Vlan800 is not available, open active delayed 14336ms (35000ms max, 60% jitter)
Jun 25 07:25:47.873: BGP: Applying map to find origin for 10.6.3.0/24
Jun 25 07:25:47.874: BGP: Applying map to find origin for 10.6.3.0/24
Jun 25 07:25:47.874: BGP: Applying map to find origin for 10.6.3.0/24
Jun 25 07:25:52.716: BGP: 10.255.255.205 Active open failed - update-source Vlan800 is not available, open active delayed 12288ms (35000ms max, 60% jitter)
Jun 25 07:26:00.960: %SYS-5-CONFIG_I: Configured from console by admin on console
Jun 25 07:26:01.002: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/24, changed state to down
Jun 25 07:26:03.680: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/24, changed state to up
Jun 25 07:26:05.006: BGP: 10.255.255.205 Active open failed - update-source Vlan800 is not available, open active delayed 11264ms (35000ms max, 60% jitter)
Jun 25 07:26:05.685: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/24, changed state to up
Jun 25 07:26:10.357: BGP: topo global:IPv4 Unicast:base Scanning routing tables
Jun 25 07:26:10.357: BGP: topo global:IPv6 Unicast:base Scanning routing tables
Jun 25 07:26:10.357: BGP: topo global:VPNv4 Unicast:base Scanning routing tables
Jun 25 07:26:10.357: BGP: topo VRF1000:VPNv4 Unicast:base Scanning routing tables
Jun 25 07:26:10.357: BGP: topo global:IPv4 Multicast:base Scanning routing tables
Jun 25 07:26:10.357: BGP: topo global:L2VPN E-VPN:base Scanning routing tables
Jun 25 07:26:10.357: BGP: topo global:VPNv4 Multicast:base Scanning routing tables
Jun 25 07:26:10.357: BGP: topo VRF1000:VPNv4 Multicast:base Scanning routing tables
Jun 25 07:26:10.357: BGP: topo global:MVPNv4 Unicast:base Scanning routing tables
Jun 25 07:26:10.357: BGP: topo VRF1000:MVPNv4 Unicast:base Scanning routing tables
Jun 25 07:26:10.357: BGP: topo global:MVPNv6 Unicast:base Scanning routing tables
Jun 25 07:26:16.272: BGP: 10.255.255.205 Active open failed - update-source Vlan800 is not available, open active delayed 8192ms (35000ms max, 60% jitter)

Switch 2:

Jun 25 07:24:17.959: %PKI-6-TRUSTPOINT_CREATE: Trustpoint: TP-self-signed-721357120 created succesfully
Jun 25 07:25:06.100: %SYS-5-CONFIG_I: Configured from console by admin on console
Jun 25 07:25:24.438: %BGP-5-NBR_RESET: Neighbor 10.255.255.209 reset (Interface flap)
Jun 25 07:25:24.440: %BGP-5-ADJCHANGE: neighbor 10.255.255.209 vpn vrf VRF1000 Down Interface flap
Jun 25 07:25:24.440: %BGP_SESSION-5-ADJCHANGE: neighbor 10.255.255.209 IPv4 Unicast vpn vrf VRF1000 topology base removed from session Interface flap
Jun 25 07:25:30.121: %BGP-5-ADJCHANGE: neighbor 10.255.255.209 vpn vrf VRF1000 Up
Jun 25 07:25:41.075: %SYS-5-CONFIG_I: Configured from console by admin on console
Jun 25 07:26:11.697: %LINK-5-CHANGED: Interface GigabitEthernet1/0/48, changed state to administratively down
Jun 25 07:26:11.700: %BGP-5-NBR_RESET: Neighbor 10.255.255.209 reset (Interface flap)
Jun 25 07:26:11.703: %BGP-5-ADJCHANGE: neighbor 10.255.255.209 vpn vrf VRF1000 Down Interface flap
Jun 25 07:26:11.703: %BGP_SESSION-5-ADJCHANGE: neighbor 10.255.255.209 IPv4 Unicast vpn vrf VRF1000 topology base removed from session Interface flap
Jun 25 07:26:12.698: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/48, changed state to down
Jun 25 07:26:12.698: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan800, changed state to down
Jun 25 07:26:14.801: %SYS-5-CONFIG_I: Configured from console by admin on console
Jun 25 07:26:14.844: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/48, changed state to down

networkadmin AQ · ‎06-27-2024

Any luck finding a solution to my problem ?

MHM Cisco World · ‎06-27-2024

sorry I make you waiting
you run L3VPN not L2 VPN ?
if So
the client first hop must be the VLAN SVI
I see IP 10.128.128.128 ??
also in BGP dont redistribute connect, we use redistribute connect in some case and with route-map to filter which prefix is redistribute, that can explain the BGP flapping you see

MHM

networkadmin AQ · ‎06-27-2024

When I run 'show ip bgp neighbors', I get ' Address tracking is enabled, the RIB does not have a route to 10.255.255.205'

networkadmin AQ · ‎06-27-2024

See debugging logs below on one side:

Jun 27 09:35:37.206: BGP: topo global:IPv4 Unicast:base Scanning routing tables
Jun 27 09:35:37.206: BGP: topo global:IPv6 Unicast:base Scanning routing tables
Jun 27 09:35:37.206: BGP: topo global:VPNv4 Unicast:base Scanning routing tables
Jun 27 09:35:37.206: BGP: topo VRF1000:VPNv4 Unicast:base Scanning routing tables
Jun 27 09:35:37.206: BGP: topo global:IPv4 Multicast:base Scanning routing tables
Jun 27 09:35:37.206: BGP: topo global:L2VPN E-VPN:base Scanning routing tables
Jun 27 09:35:37.206: BGP: topo global:VPNv4 Multicast:base Scanning routing tables
Jun 27 09:35:37.206: BGP: topo VRF1000:VPNv4 Multicast:base Scanning routing tables
Jun 27 09:35:37.206: BGP: topo global:MVPNv4 Unicast:base Scanning routing tables
Jun 27 09:35:37.206: BGP: topo VRF1000:MVPNv4 Unicast:base Scanning routing tables
Jun 27 09:35:37.206: BGP: topo global:MVPNv6 Unicast:base Scanning routing tables
Jun 27 09:35:41.376: BGP: 10.255.255.205 Active open failed - no route to peer, open active delayed 13312ms (35000ms max, 60% jitter)
Jun 27 09:35:54.689: BGP: 10.255.255.205 Active open failed - no route to peer, open active delayed 13312ms (35000ms max, 60% jitter)
Jun 27 09:36:08.004: BGP: 10.255.255.205 Active open failed - no route to peer, open active delayed 8192ms (35000ms max, 60% jitter)
Jun 27 09:36:16.198: BGP: 10.255.255.205 Active open failed - no route to peer, open active delayed 8192ms (35000ms max, 60% jitter)
Jun 27 09:36:24.391: BGP: 10.255.255.205 Active open failed - no route to peer, open active delayed 10240ms (35000ms max, 60% jitter)