Solved: Re: Sharing eBGP Routes with other VTEP Peers - VXLAN

PingWhisperer · ‎01-11-2024

I am having an issue where I have an eBGP route coming into my VXLAN vrf frfom FW1 but that route is not being advertised to the other VTEP (NXOS2). Thus making any layer 2 devices that move to the other VTEP unable to reach the DMZ on FW1. I've tried about everything I can think of, from import/export with VRF-LITE, iBGP, and eBGP. I was able to share it by creating a static route, but I really want to use BGP.

FW1#show running-config
Building configuration...

Current configuration : 3187 bytes
!
! Last configuration change at 15:12:03 UTC Tue Jan 9 2024
!
version 15.2
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
service compress-config
!
hostname FW1
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
!
!
!
!
!
!
!
!
ip cef
no ipv6 cef
!
!
!
spanning-tree mode pvst
spanning-tree extend system-id
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface GigabitEthernet0/0
 no switchport
 ip address 208.67.222.2 255.255.255.252
 negotiation auto
!
interface GigabitEthernet0/1
 no switchport
 ip address 172.16.1.1 255.255.255.0
 negotiation auto
!
interface GigabitEthernet0/2
 negotiation auto
!
interface GigabitEthernet0/3
 negotiation auto
!
interface GigabitEthernet1/0
 negotiation auto
!
interface GigabitEthernet1/1
 negotiation auto
!
interface GigabitEthernet1/2
 negotiation auto
!
interface GigabitEthernet1/3
 negotiation auto
!
router bgp 65534
 bgp router-id 208.67.222.2
 bgp log-neighbor-changes
 network 172.16.1.0 mask 255.255.255.0
 neighbor 208.67.222.1 remote-as 65535
 neighbor 208.67.222.1 update-source GigabitEthernet0/0
 neighbor 208.67.222.1 default-originate
!
ip forward-protocol nd
!
ip http server
ip http secure-server
!
ip ssh server algorithm encryption aes128-ctr aes192-ctr aes256-ctr
ip ssh client algorithm encryption aes128-ctr aes192-ctr aes256-ctr
!
!
!
!
!
!
control-plane
!
!
line con 0
 logging synchronous
line aux 0
line vty 0 4
!
!
end

FW2#show running-config
Building configuration...

Current configuration : 3146 bytes
!
! Last configuration change at 15:38:46 UTC Tue Jan 9 2024
!
version 15.2
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
service compress-config
!
hostname FW2
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
!
!
!
!
!
!
!
!
ip cef
no ipv6 cef
!
!
!
spanning-tree mode pvst
spanning-tree extend system-id
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface GigabitEthernet0/0
 no switchport
 ip address 108.67.222.2 255.255.255.252
 negotiation auto
!
interface GigabitEthernet0/1
 no switchport
 ip address 172.16.2.1 255.255.255.0
 negotiation auto
!
interface GigabitEthernet0/2
 negotiation auto
!
interface GigabitEthernet0/3
 negotiation auto
!
interface GigabitEthernet1/0
 negotiation auto
!
interface GigabitEthernet1/1
 negotiation auto
!
interface GigabitEthernet1/2
 negotiation auto
!
interface GigabitEthernet1/3
 negotiation auto
!
router bgp 65534
 bgp router-id 108.67.222.2
 bgp log-neighbor-changes
 network 172.16.2.0 mask 255.255.255.0
 neighbor 108.67.222.1 remote-as 65535
 neighbor 108.67.222.1 update-source GigabitEthernet0/0
!
ip forward-protocol nd
!
ip http server
ip http secure-server
!
ip ssh server algorithm encryption aes128-ctr aes192-ctr aes256-ctr
ip ssh client algorithm encryption aes128-ctr aes192-ctr aes256-ctr
!
!
!
!
!
!
control-plane
!
!
line con 0
 logging synchronous
line aux 0
line vty 0 4
!
!
end

NXOS1# show running-config

!Command: show running-config
!Running configuration last done at: Tue Jan  9 15:38:41 2024
!Time: Tue Jan  9 15:49:48 2024

version 10.3(4a) Bios:version
hostname NXOS1
vdc NXOS1 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4097
  limit-resource port-channel minimum 0 maximum 511
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

nv overlay evpn
feature ospf
feature bgp
feature interface-vlan
feature vn-segment-vlan-based
feature nv overlay

no password strength-check
username admin password 5 $5$GFPMEJ$ZT97vgSalc27Wq9r0DhBXB/i0yuI3kd4dVsjhe.Fuw1  role network-admin
ip domain-lookup
copp profile strict
hardware access-list tcam region racl 512
hardware access-list tcam region e-racl 512
hardware access-list tcam region arp-ether 256 double-wide
snmp-server user admin network-admin auth md5 0142EC31804EDFAF7F2C60B8B0A5CC6BDDC2 priv aes-128 364BEC328312DDB3586475BAAFAFFA3598C7 localizedV2key
rmon event 1 log trap public description FATAL(1) owner PMON@FATAL
rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO

fabric forwarding anycast-gateway-mac 0000.2222.3333
vlan 1,101,1000-1001
vlan 101
  vn-segment 900001
vlan 1000
  vn-segment 5000
vlan 1001
  vn-segment 5005

vrf context VXLAN
  vni 900001
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context management

interface Vlan1

interface Vlan101
  no shutdown
  vrf member VXLAN
  ip forward

interface Vlan1000
  no shutdown
  vrf member VXLAN
  ip address 192.168.10.1/24
  fabric forwarding mode anycast-gateway

interface Vlan1001
  no shutdown
  vrf member VXLAN
  ip address 192.168.11.1/24
  fabric forwarding mode anycast-gateway

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  member vni 5000
    suppress-arp
    ingress-replication protocol bgp
  member vni 5005
    suppress-arp
    ingress-replication protocol bgp
  member vni 900001 associate-vrf

interface Ethernet1/1
  no switchport
  vrf member VXLAN
  ip address 208.67.222.1/30
  no shutdown

interface Ethernet1/2

interface Ethernet1/3

interface Ethernet1/4
  description Host Computer
  switchport access vlan 1000

interface Ethernet1/5

interface Ethernet1/6

interface Ethernet1/7
  no switchport
  ip address 192.168.255.1/30
  ip router ospf 150 area 0.0.0.0
  no shutdown

interface Ethernet1/8

interface mgmt0
  vrf member management

interface loopback0
  ip address 1.1.1.1/32
  ip router ospf 150 area 0.0.0.0
icam monitor scale

line console
line vty
boot nxos bootflash:/nxos64-cs.10.3.4a.M.bin
router ospf 150
  router-id 1.1.1.1
router bgp 65535
  router-id 1.1.1.1
  neighbor 2.2.2.2
    remote-as 65535
    update-source loopback0
    address-family l2vpn evpn
      send-community extended
  vrf VXLAN
    address-family ipv4 unicast
      network 192.168.10.0/24
      advertise l2vpn evpn
    neighbor 208.67.222.2
      remote-as 65534
      local-as 65535
      update-source Ethernet1/1
      address-family ipv4 unicast
evpn
  vni 5000 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 5005 l2
    rd auto
    route-target import auto
    route-target export auto

NXOS2# show running-config

!Command: show running-config
!Running configuration last done at: Tue Jan  9 15:39:08 2024
!Time: Tue Jan  9 15:50:30 2024

version 10.3(4a) Bios:version
hostname NXOS2
vdc NXOS2 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4097
  limit-resource port-channel minimum 0 maximum 511
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

nv overlay evpn
feature ospf
feature bgp
feature interface-vlan
feature vn-segment-vlan-based
feature nv overlay

no password strength-check
username admin password 5 $5$NOKIAD$dmSy1Fds8ExPMvgZ/g0jI0O0eGOo9.Z.6d5FmmapvXC  role network-admin
ip domain-lookup
copp profile strict
hardware access-list tcam region racl 512
hardware access-list tcam region e-racl 512
hardware access-list tcam region arp-ether 256 double-wide
snmp-server user admin network-admin auth md5 53119D5D6FD9E2D1D4820B7A6B32F06A3114 priv aes-128 49539601EDF6D5FFE3EC226F6305BF256E54 localizedV2key
rmon event 1 log trap public description FATAL(1) owner PMON@FATAL
rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO

fabric forwarding anycast-gateway-mac 0000.2222.3333
vlan 1,101,1000-1001
vlan 101
  vn-segment 900001
vlan 1000
  vn-segment 5000
vlan 1001
  vn-segment 5005

vrf context VXLAN
  vni 900001
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context management

interface Vlan1

interface Vlan101
  no shutdown
  vrf member VXLAN
  ip forward

interface Vlan1000
  no shutdown
  vrf member VXLAN
  ip address 192.168.10.1/24
  fabric forwarding mode anycast-gateway

interface Vlan1001
  no shutdown
  vrf member VXLAN
  ip address 192.168.11.1/24
  fabric forwarding mode anycast-gateway

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  member vni 5000
    suppress-arp
    ingress-replication protocol bgp
  member vni 5005
    suppress-arp
    ingress-replication protocol bgp
  member vni 900001 associate-vrf

interface Ethernet1/1
  no switchport
  vrf member VXLAN
  ip address 108.67.222.1/30
  no shutdown

interface Ethernet1/2

interface Ethernet1/3

interface Ethernet1/4
  description Host Computer
  switchport access vlan 1001

interface Ethernet1/5

interface Ethernet1/6

interface Ethernet1/7
  no switchport
  ip address 192.168.255.2/30
  ip router ospf 150 area 0.0.0.0
  no shutdown

interface Ethernet1/8

interface mgmt0
  vrf member management

interface loopback0
  ip address 2.2.2.2/32
  ip router ospf 150 area 0.0.0.0
icam monitor scale

line console
line vty
boot nxos bootflash:/nxos64-cs.10.3.4a.M.bin
router ospf 150
  router-id 2.2.2.2
router bgp 65535
  router-id 2.2.2.2
  neighbor 1.1.1.1
    remote-as 65535
    update-source loopback0
    address-family l2vpn evpn
      send-community extended
  vrf VXLAN
    address-family ipv4 unicast
      network 192.168.11.0/24
      advertise l2vpn evpn
    neighbor 108.67.222.2
      remote-as 65534
      local-as 65535
      update-source Ethernet1/1
      address-family ipv4 unicast
evpn
  vni 5000 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 5005 l2
    rd auto
    route-target import auto
    route-target export auto

Harold Ritter · ‎01-11-2024

Hi @PingWhisperer ,

You need to change the BGP configuration on both NXOS devices as follow:

NXOS1:

router bgp 65535

  vrf VXLAN

    neighbor 208.67.222.2

      no local-as (This is for the local-as feature and breaks things in this context)

      address-family ipv4 unicast

        as-override (Required because both FWs use the same same ASN)




NXOS2:

router bgp 65535

  vrf VXLAN

    neighbor 108.67.222.2

      no local-as

      address-family ipv4 unicast

        as-override

That should fix it.

Regards,

Harold Ritter
Sr Technical Leader
CCIE 4168 (R&S, SP)
harold@cisco.com
México móvil: +52 1 55 8312 4915
Cisco México
Paseo de la Reforma 222
Piso 19
Cuauhtémoc, Juárez
Ciudad de México, 06600
México

View solution in original post

Harold Ritter · ‎01-11-2024

Hi @PingWhisperer ,

You need to change the BGP configuration on both NXOS devices as follow:

NXOS1:

router bgp 65535

  vrf VXLAN

    neighbor 208.67.222.2

      no local-as (This is for the local-as feature and breaks things in this context)

      address-family ipv4 unicast

        as-override (Required because both FWs use the same same ASN)




NXOS2:

router bgp 65535

  vrf VXLAN

    neighbor 108.67.222.2

      no local-as

      address-family ipv4 unicast

        as-override

That should fix it.

Regards,

Harold Ritter
Sr Technical Leader
CCIE 4168 (R&S, SP)
harold@cisco.com
México móvil: +52 1 55 8312 4915
Cisco México
Paseo de la Reforma 222
Piso 19
Cuauhtémoc, Juárez
Ciudad de México, 06600
México

PingWhisperer · ‎01-11-2024

Ah I see, that did fix it. I also tried with local-as 65535 no-prepend and that fixed it as well. I didn't realize when I used local-as the tag 65535 was added, and that AS was also added to the end.

Harold Ritter · ‎01-11-2024

Hi @PingWhisperer ,

Glad the issue is fixed. The local-as is only required when you want the peer to see you as a different ASN as the one configured locally (router bgp xxx). It does not add any value if the locally configured ASN and the one used on the local-as "as-number" are the same.

The other important change is the as-override, as the two FWs use the same ASN, the update from FW1 would be rejected by FW2 and vice versa.

Regards,

Harold Ritter
Sr Technical Leader
CCIE 4168 (R&S, SP)
harold@cisco.com
México móvil: +52 1 55 8312 4915
Cisco México
Paseo de la Reforma 222
Piso 19
Cuauhtémoc, Juárez
Ciudad de México, 06600
México