VRF between Nexus 93360 and ASR 920 (MPLS BGP with LDP)

d.palermo76 · ‎11-25-2023

Hi,

I'm trying to add a couple of Nexus 93360YC-FX2 to an existing network built with some ASR 920 routers.

Physically, I've got a Catalyst 9500 in the middle temporarily: two edge ASR 920 routers are connected with Port-channels to the 9500 and so are the Nexus 93360. Logically, I've got p2p connections from the ASRs to the Nexuses (Nexi?) on vlans 1600 to 1603.

On the ASRs I had multiple VRFs, two routing protocols processes (OSPF 1 and BGP 65010) and LDP configuration. OSPF 1 is distributing paths to the nodes loopbacks of the local and remote ASRs.

I added OSPF 2 both on the edge ASRs and the Nexuses and activated it on the p2p networks. Then redistributed, on the edge ASR nodes, OSPF 2 to OSPF 1 and viceversa (filtering only the loopback ip addresses, there were some other routes on the ASR side I didn't want to get).

In added the VRF and the BGP process (same AS) on the Nexuses and I'm receiving routes on each VRF. However, I cannot reach from the Nexuses any of the BGP received networks in any of the VRF. I think that's must be related to the lack of LDP support on them.

If I add another network on the trunk connecting one ASR to one Nexus and I assign it to a VRF, I can ping from that Nexus an interface that I have on the ASR on the same VRF, even if packets are flowing (debug mpls packet) on the main p2p link and not the secondary one I created.

Honestly I don't even know if my approach is correct but lacking some piece of configuration or if I have to rethink entirely the solution and so I need help, thanks.

Bye, Dario

MHM Cisco World · ‎11-25-2023

Prefix is show in bgp and rib or only in bgp table ?

d.palermo76 · ‎11-26-2023

I assume you mean vrf route prefixes, right? I got them in the rib.

I'm also adding some additional info: I "splitted" completely the routing protocols, on the ASR I have now OSPF 1 to do what it was doing up to today and OSPF 2 to talk to the Nexuses and I also change the AS on the Nexuses switching to eBGP (had to set the multihop to compensate the 9500 being in the middle).

Now things are starting to work, from the nexuses I can reach the remote networks on the VRFs but I'm failing to inject the local VRF network from the Nexuses to the ASRs.

Maybe I need to post some configs and a diagram, right?

bye, Dario

MHM Cisco World · ‎11-26-2023

It be so helpful if you share topolgy and config.

MHM

d.palermo76 · ‎11-26-2023

I'm attaching some diagrams and pieces of config from both one ASR and one Nexus (the two I was able to make talk to each other on VRF Internet-Diretta).

Please let me know if I missed something.physicallogical

Nodo5A_export

ip vrf Internet-Diretta
rd 701:701
route-target export 701:701
route-target import 701:701
!
ip vrf Intranet
rd 101:101
route-target export 101:101
route-target import 101:101
!
ip vrf LoRaWan
rd 801:801
route-target export 801:801
route-target import 801:801
!
ip vrf Mng
description Management
rd 301:301
route-target export 301:301
route-target import 301:301
!
ip vrf Semaforica
rd 260:260
route-target export 260:260
route-target import 260:260
!
ip vrf Shelter
description Sicurezza Shelter
rd 501:501
route-target export 501:501
route-target import 501:501
!
ip vrf Video-Sorv
description Videosorveglianza
rd 201:201
route-target export 201:201
route-target import 201:201
!
ip vrf Voip
description Voip
rd 401:401
route-target export 401:401
route-target import 401:401
!
ip vrf WIFI
rd 601:601
route-target export 601:601
route-target import 601:601
!
ip vrf WIFI-Management
rd 901:901
route-target export 901:901
route-target import 901:901
!

interface Port-channel1
description -> Catalyst 9500
mtu 9216
no ip address
load-interval 30
carrier-delay msec 10
service instance 1 ethernet
encapsulation untagged
l2protocol peer cdp stp lldp lacp udld loam esmc elmi ptppd
bridge-domain 1
!
service instance 1600 ethernet
description -> Nexus-01
encapsulation dot1q 1600
rewrite ingress tag pop 1 symmetric
l2protocol peer cdp stp lldp lacp udld loam esmc elmi ptppd
bridge-domain 1600
!
service instance 1601 ethernet
encapsulation dot1q 1601
rewrite ingress tag pop 1 symmetric
l2protocol peer cdp stp lldp lacp udld loam esmc elmi ptppd
bridge-domain 1601
!
service instance 1604 ethernet
encapsulation dot1q 1604
rewrite ingress tag pop 1 symmetric
l2protocol peer cdp stp lldp lacp udld loam esmc elmi ptppd
bridge-domain 1604
!
service instance 2510 ethernet
encapsulation dot1q 2510
rewrite ingress tag pop 1 symmetric
l2protocol peer cdp stp lldp lacp udld loam esmc elmi ptppd
bridge-domain 2510
!
!

router ospf 1
router-id 10.30.0.5
auto-cost reference-bandwidth 100000
fast-reroute per-prefix enable prefix-priority low
fast-reroute per-prefix remote-lfa tunnel mpls-ldp
timers throttle spf 50 50 5000
timers throttle lsa 10 20 5000
timers lsa arrival 10
timers pacing flood 5
redistribute ospf 2 subnets route-map CentroStella
network 10.22.0.0 0.0.0.3 area 0
network 10.22.0.20 0.0.0.3 area 0
network 10.24.0.0 0.0.0.3 area 0
network 10.30.0.5 0.0.0.0 area 0
bfd all-interfaces
mpls ldp sync
!
router ospf 2
router-id 1.1.1.1
redistribute connected subnets
redistribute ospf 1 subnets route-map AnelloTelematico
network 10.22.0.24 0.0.0.3 area 0
network 10.22.0.28 0.0.0.3 area 0
network 10.30.0.5 0.0.0.0 area 0
!
router bgp 65010
bgp router-id 10.30.0.5
bgp cluster-id 1
bgp log-neighbor-changes
no bgp default ipv4-unicast
neighbor NODI peer-group
neighbor NODI remote-as 65010
neighbor NODI update-source Loopback0
neighbor 10.21.2.45 remote-as 65003
neighbor 10.30.0.1 peer-group NODI
neighbor 10.30.0.2 remote-as 65010
neighbor 10.30.0.2 update-source Loopback0
neighbor 10.30.0.3 peer-group NODI
neighbor 10.30.0.4 peer-group NODI
neighbor 10.30.0.6 peer-group NODI
neighbor 10.30.1.2 peer-group NODI
neighbor 10.30.1.3 peer-group NODI
neighbor 10.30.2.3 peer-group NODI
neighbor 10.30.3.1 peer-group NODI
neighbor 10.30.3.2 peer-group NODI
neighbor 10.30.4.5 peer-group NODI
neighbor 10.30.4.6 peer-group NODI
neighbor 10.30.4.8 peer-group NODI
neighbor 10.30.4.9 peer-group NODI
neighbor 10.30.5.1 remote-as 65020
neighbor 10.30.5.1 ebgp-multihop 2
neighbor 10.30.5.1 update-source Loopback0
neighbor 10.31.0.1 peer-group NODI
neighbor 10.31.0.2 peer-group NODI
neighbor 10.31.0.3 peer-group NODI
neighbor 10.31.0.4 peer-group NODI
neighbor 10.31.0.5 remote-as 65010
neighbor 10.31.0.5 update-source Loopback0
neighbor 10.31.0.6 peer-group NODI
neighbor 10.31.1.2 peer-group NODI
neighbor 10.31.1.3 peer-group NODI
neighbor 10.31.2.3 peer-group NODI
neighbor 10.31.3.1 peer-group NODI
neighbor 10.31.3.2 peer-group NODI
neighbor 10.31.4.5 peer-group NODI
neighbor 10.31.4.6 peer-group NODI
neighbor 10.31.5.1 remote-as 65020
neighbor 10.31.5.1 ebgp-multihop 2
neighbor 10.31.5.1 update-source Loopback0
!
address-family ipv4
exit-address-family
!
address-family vpnv4
bgp additional-paths install
neighbor NODI send-community both
neighbor NODI route-reflector-client
neighbor 10.30.0.1 activate
neighbor 10.30.0.2 activate
neighbor 10.30.0.2 send-community both
neighbor 10.30.0.3 activate
neighbor 10.30.0.4 activate
neighbor 10.30.0.6 activate
neighbor 10.30.1.2 activate
neighbor 10.30.1.3 activate
neighbor 10.30.2.3 activate
neighbor 10.30.3.1 activate
neighbor 10.30.3.2 activate
neighbor 10.30.4.5 activate
neighbor 10.30.4.6 activate
neighbor 10.30.4.8 activate
neighbor 10.30.4.9 activate
neighbor 10.30.5.1 activate
neighbor 10.30.5.1 send-community both
neighbor 10.31.0.1 activate
neighbor 10.31.0.2 activate
neighbor 10.31.0.3 activate
neighbor 10.31.0.4 activate
neighbor 10.31.0.5 activate
neighbor 10.31.0.5 send-community both
neighbor 10.31.0.6 activate
neighbor 10.31.1.2 activate
neighbor 10.31.1.3 activate
neighbor 10.31.2.3 activate
neighbor 10.31.3.1 activate
neighbor 10.31.3.2 activate
neighbor 10.31.4.5 activate
neighbor 10.31.4.6 activate
neighbor 10.31.5.1 activate
neighbor 10.31.5.1 send-community both
exit-address-family
!
address-family ipv4 vrf Internet-Diretta
redistribute connected
redistribute static
default-information originate
exit-address-family
!
address-family ipv4 vrf Intranet
redistribute connected
redistribute static
neighbor 10.20.5.1 remote-as 65003
neighbor 10.20.5.1 description eBGP Switch C9500
neighbor 10.20.5.1 activate
exit-address-family
!
address-family ipv4 vrf LoRaWan
redistribute connected
redistribute static
neighbor 10.50.0.30 remote-as 65003
neighbor 10.50.0.30 activate
default-information originate
exit-address-family
!
address-family ipv4 vrf Mng
redistribute connected
redistribute static
neighbor 10.21.2.21 remote-as 65003
neighbor 10.21.2.21 activate
default-information originate
exit-address-family
!
address-family ipv4 vrf Semaforica
redistribute connected
redistribute static
neighbor 10.60.5.4 remote-as 65003
neighbor 10.60.5.4 activate
default-information originate
exit-address-family
!
address-family ipv4 vrf Shelter
redistribute connected
redistribute static
exit-address-family
!
address-family ipv4 vrf Video-Sorv
redistribute connected
redistribute static
neighbor 10.30.203.228 remote-as 65003
neighbor 10.30.203.228 activate
neighbor 10.30.203.228 route-map Video-Sorv in
default-information originate
exit-address-family
!
address-family ipv4 vrf Voip
redistribute connected
redistribute static
neighbor 10.21.2.29 remote-as 65003
neighbor 10.21.2.29 description eBGP Switch C9500
neighbor 10.21.2.29 activate
exit-address-family
!
address-family ipv4 vrf WIFI
redistribute connected
redistribute static
default-information originate
exit-address-family
!
address-family ipv4 vrf WIFI-Management
redistribute connected
redistribute static
neighbor 10.21.2.13 remote-as 65003
neighbor 10.21.2.13 activate
default-information originate
exit-address-family
!
ip forward-protocol nd
ip forward-protocol udp 5246

ip prefix-list AnelloTelematico seq 5 permit 10.30.0.1/32
ip prefix-list AnelloTelematico seq 10 permit 10.30.0.2/32
ip prefix-list AnelloTelematico seq 15 permit 10.30.0.3/32
ip prefix-list AnelloTelematico seq 20 permit 10.30.0.4/32
ip prefix-list AnelloTelematico seq 25 permit 10.30.0.5/32
ip prefix-list AnelloTelematico seq 30 permit 10.30.0.6/32
ip prefix-list AnelloTelematico seq 35 permit 10.30.1.2/32
ip prefix-list AnelloTelematico seq 40 permit 10.30.1.3/32
ip prefix-list AnelloTelematico seq 45 permit 10.30.2.3/32
ip prefix-list AnelloTelematico seq 50 permit 10.30.3.1/32
ip prefix-list AnelloTelematico seq 55 permit 10.30.3.2/32
ip prefix-list AnelloTelematico seq 60 permit 10.30.4.5/32
ip prefix-list AnelloTelematico seq 65 permit 10.30.4.6/32
ip prefix-list AnelloTelematico seq 70 permit 10.30.4.8/32
ip prefix-list AnelloTelematico seq 75 permit 10.30.4.9/32
ip prefix-list AnelloTelematico seq 80 permit 10.31.0.1/32
ip prefix-list AnelloTelematico seq 85 permit 10.31.0.2/32
ip prefix-list AnelloTelematico seq 90 permit 10.31.0.3/32
ip prefix-list AnelloTelematico seq 95 permit 10.31.0.4/32
ip prefix-list AnelloTelematico seq 100 permit 10.31.0.5/32
ip prefix-list AnelloTelematico seq 105 permit 10.31.0.6/32
ip prefix-list AnelloTelematico seq 110 permit 10.31.1.2/32
ip prefix-list AnelloTelematico seq 115 permit 10.31.1.3/32
ip prefix-list AnelloTelematico seq 120 permit 10.31.2.3/32
ip prefix-list AnelloTelematico seq 125 permit 10.31.3.1/32
ip prefix-list AnelloTelematico seq 130 permit 10.31.3.2/32
ip prefix-list AnelloTelematico seq 135 permit 10.31.4.5/32
ip prefix-list AnelloTelematico seq 140 permit 10.31.4.6/32
!
ip prefix-list CentroStella seq 5 permit 10.30.5.1/32
ip prefix-list CentroStella seq 10 permit 10.22.0.24/30
ip prefix-list CentroStella seq 15 permit 10.22.0.32/30
ip prefix-list CentroStella seq 20 permit 10.31.5.1/32
ip prefix-list CentroStella seq 25 permit 10.22.0.28/30
ip prefix-list CentroStella seq 30 permit 10.22.0.36/30

route-map AnelloTelematico permit 10
match ip address prefix-list AnelloTelematico
!
route-map CentroStella permit 10
match ip address prefix-list CentroStella

Centro_Stella-01_export

Centro_Stella-O1

vrf context Internet-Diretta
rd 701:701
address-family ipv4 unicast
route-target import 701:701
route-target export 701:701
vrf context Mng
rd 301:301
address-family ipv4 unicast
route-target import 301:301
route-target export 301:301
vrf context management
vrf context vpc
vpc domain 1
peer-switch
role priority 100
system-priority 1024
peer-keepalive destination 10.40.255.2 source 10.40.255.1 vrf vpc
delay restore 150
peer-gateway
layer3 peer-router
auto-recovery
ip arp synchronize

interface loopback0
ip address 10.30.5.1/32
ip router ospf 2 area 0.0.0.0

interface Ethernet1/43
switchport
switchport mode trunk
switchport trunk allowed vlan 100
no shutdown

interface Ethernet1/47
switchport
switchport mode trunk
switchport trunk allowed vlan 1600-1604
channel-group 4096 mode active
no shutdown

interface Ethernet1/48
switchport
switchport mode trunk
switchport trunk allowed vlan 1600-1604
channel-group 4096 mode active
no shutdown

interface Ethernet1/49
mtu 9216
channel-group 4095 mode active
no shutdown

interface Ethernet1/93
description -> Catalyst 9500X
switchport
switchport mode trunk
switchport trunk allowed vlan 1600-1604
mtu 9216
channel-group 4000 mode active
no shutdown

interface Ethernet1/94
description vpc monitor
vrf member vpc
ip address 10.40.255.1/30
no shutdown

interface Ethernet1/95
switchport
switchport mode trunk
switchport trunk allowed vlan 1600-1604
channel-group 4096 mode active
no shutdown

interface Ethernet1/96
switchport
switchport mode trunk
switchport trunk allowed vlan 1600-1604
channel-group 4096 mode active
no shutdown

interface port-channel4000
description -> Catalyst 9500X
switchport
switchport mode trunk
switchport trunk allowed vlan 1600-1604
mtu 9216
vpc 4000

interface port-channel4095
description -> Centro_Stella-02 (L3)
mtu 9216
bfd interval 50 min_rx 50 multiplier 5
no bfd echo
no ip redirects
ip address 10.24.0.25/30
no ipv6 redirects
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf 2 area 0.0.0.0
ip ospf bfd
mpls ip forwarding

interface port-channel4096
description vpc peer link
switchport
switchport mode trunk
switchport trunk allowed vlan 1600-1604
spanning-tree port type network
vpc peer-link

interface Vlan100
no shutdown
no ip redirects
ip address 10.1.100.4/29
no ipv6 redirects
no ip ospf passive-interface
ip router ospf 1 area 0.0.0.0

interface Vlan1600
description -> Nodo 5A
no shutdown
carrier-delay msec 10
bfd interval 50 min_rx 50 multiplier 5
no bfd echo
no ip redirects
ip address 10.22.0.26/30
no ipv6 redirects
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf 2 area 0.0.0.0
ip ospf bfd
mpls ip forwarding

interface Vlan1602
description -> Nodo 5B
no shutdown
carrier-delay msec 10
bfd interval 50 min_rx 50 multiplier 5
no bfd echo
no ip redirects
ip address 10.22.0.34/30
no ipv6 redirects
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf 2 area 0.0.0.0
ip ospf bfd
mpls ip forwarding

interface Vlan1604
description -> Nodo 5A VRF Internet-Diretta
no shutdown
carrier-delay msec 10
vrf member Internet-Diretta
no ip redirects
ip address 10.22.0.42/30
no ipv6 redirects
ip ospf passive-interface

interface Vlan2510
no shutdown
no autostate
vrf member Internet-Diretta
bfd interval 50 min_rx 50 multiplier 5
no bfd echo
ip address 10.100.108.1/24
no ip ospf passive-interface
ip router ospf 2 area 0.0.0.0
ip ospf bfd
mpls ip forwarding

router ospf 1
router-id 10.30.5.1
default-information originate
redistribute ospf 2 route-map any
table-map Test filter
passive-interface default
router ospf 2
router-id 10.30.5.15
default-information originate
redistribute static route-map any
passive-interface default
vrf Internet-Diretta
vrf Mng
router bgp 65020
router-id 10.30.5.1
log-neighbor-changes
address-family vpnv4 unicast
additional-paths send
additional-paths receive
neighbor 10.30.0.5
remote-as 65010
log-neighbor-changes
description Nodo5A
update-source loopback0
ebgp-multihop 2
address-family vpnv4 unicast
send-community
send-community extended
neighbor 10.31.0.5
remote-as 65010
log-neighbor-changes
description Nodo5A
update-source loopback0
ebgp-multihop 2
address-family vpnv4 unicast
send-community
send-community extended
vrf Internet-Diretta
router-id 10.30.5.1
address-family ipv4 unicast
network 0.0.0.0/0
redistribute direct route-map any
vrf Mng
router-id 10.30.5.1
address-family ipv4 unicast
network 0.0.0.0/0
redistribute direct route-map any

bye, Dario