Re: VXLAN with EBGP Peers - Routes Missing in VXLAN

PingWhisperer · ‎01-08-2024

Hello all!

I have a test VXLAN environment, that is currently hosting 4 networks between 4 9k's and everything with VXLAN is working great. I added 2 firewalls, that my 9k's connect too and get their default routes from. With that, I also added 2 DMZ's to those new firewalls and created eBGP neighbors back to my VXLAN VRF. I am getting my DMZ routes from my eBGP relationship, but what I notice is my VTEP peers dont seem to get any VXLAN routes for those new eBGP routes.

Basically, I want to be able to from any VTEP reach my new eBGP relationships and essentialy distribute those new eBGP routes into VXLAN.

I've noticed if I use OSPF and redistribute it, I can do effectively that by using redistribute in the BGP process. Yet the BGP routes dont seem to show up in VXLAN at all.

Any suggestions?

balaji.bandi · ‎01-08-2024

how is your topology looks like and post show sample config and output to understand the issue.

also refer below configuration guide :

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/92x/vxlan-92x/configuration/guide/b-cisco-nexus-9000-series-nx-os-vxlan-configuration-guide-92x/b_Cisco_Nexus_9000_Series_NX-OS_VXLAN_Configuration_Guide_9x_chapter_0100.html

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

M02@rt37 · ‎01-08-2024

Hello @PingWhisperer

Please provide us a topology of your network.

Thanks a lot.

Best regards
.ı|ı.ı|ı. If This Helps, Please Rate .ı|ı.ı|ı.

Georg Pauwen · ‎01-08-2024

Hello,

also add the full running configs of all devices involved, so we can (possible) lab this up...

PingWhisperer · ‎01-09-2024

In this network, the VXLAN networks, 192.168.11.10/24 and 192.168.10.10/24 are working fine with their VXLAN topology.

The 2 DMZ networks, DMZ-1 (172.16.1.0/24) is only reachable via NXOS1, and DMZ-2 (172.16.2.0/24) is only reachable via NXOS2.

What I want/need is VPC6, to be able to reach DMZ-1. When you check the VRF on NXOS1 it has a BGP route to DMZ-1, but it does NOT share that route via VXLAN to NXOS2.

When I create OSPF neighbors instead with FW1, and I redistribute that into the BGP topolgy, it then shares the routes via VXLAN. I don't want to use OSPF, I want to use either iBGP or eBGP but need to share my routes via VXLAN.

Hopefully that makes more sense.

FW1#show running-config
Building configuration...

Current configuration : 3187 bytes
!
! Last configuration change at 15:12:03 UTC Tue Jan 9 2024
!
version 15.2
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
service compress-config
!
hostname FW1
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
!
!
!
!
!
!
!
!
ip cef
no ipv6 cef
!
!
!
spanning-tree mode pvst
spanning-tree extend system-id
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface GigabitEthernet0/0
 no switchport
 ip address 208.67.222.2 255.255.255.252
 negotiation auto
!
interface GigabitEthernet0/1
 no switchport
 ip address 172.16.1.1 255.255.255.0
 negotiation auto
!
interface GigabitEthernet0/2
 negotiation auto
!
interface GigabitEthernet0/3
 negotiation auto
!
interface GigabitEthernet1/0
 negotiation auto
!
interface GigabitEthernet1/1
 negotiation auto
!
interface GigabitEthernet1/2
 negotiation auto
!
interface GigabitEthernet1/3
 negotiation auto
!
router bgp 65534
 bgp router-id 208.67.222.2
 bgp log-neighbor-changes
 network 172.16.1.0 mask 255.255.255.0
 neighbor 208.67.222.1 remote-as 65535
 neighbor 208.67.222.1 update-source GigabitEthernet0/0
 neighbor 208.67.222.1 default-originate
!
ip forward-protocol nd
!
ip http server
ip http secure-server
!
ip ssh server algorithm encryption aes128-ctr aes192-ctr aes256-ctr
ip ssh client algorithm encryption aes128-ctr aes192-ctr aes256-ctr
!
!
!
!
!
!
control-plane
!
banner exec ^C
IOSv - Cisco Systems Confidential -

Supplemental End User License Restrictions

This IOSv software is provided AS-IS without warranty of any kind. Under no circumstances may this software be used separate from the Cisco Modeling Labs Software that this software was provided with, or deployed or used as part of a production environment.

By using the software, you agree to abide by the terms and conditions of the Cisco End User License Agreement at http://www.cisco.com/go/eula. Unauthorized use or distribution of this software is expressly prohibited.
^C
banner incoming ^C
IOSv - Cisco Systems Confidential -

Supplemental End User License Restrictions

This IOSv software is provided AS-IS without warranty of any kind. Under no circumstances may this software be used separate from the Cisco Modeling Labs Software that this software was provided with, or deployed or used as part of a production environment.

By using the software, you agree to abide by the terms and conditions of the Cisco End User License Agreement at http://www.cisco.com/go/eula. Unauthorized use or distribution of this software is expressly prohibited.
^C
banner login ^C
IOSv - Cisco Systems Confidential -

Supplemental End User License Restrictions

This IOSv software is provided AS-IS without warranty of any kind. Under no circumstances may this software be used separate from the Cisco Modeling Labs Software that this software was provided with, or deployed or used as part of a production environment.

By using the software, you agree to abide by the terms and conditions of the Cisco End User License Agreement at http://www.cisco.com/go/eula. Unauthorized use or distribution of this software is expressly prohibited.
^C
!
line con 0
 logging synchronous
line aux 0
line vty 0 4
!
!
end

FW2#show running-config
Building configuration...

Current configuration : 3146 bytes
!
! Last configuration change at 15:38:46 UTC Tue Jan 9 2024
!
version 15.2
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
service compress-config
!
hostname FW2
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
!
!
!
!
!
!
!
!
ip cef
no ipv6 cef
!
!
!
spanning-tree mode pvst
spanning-tree extend system-id
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface GigabitEthernet0/0
 no switchport
 ip address 108.67.222.2 255.255.255.252
 negotiation auto
!
interface GigabitEthernet0/1
 no switchport
 ip address 172.16.2.1 255.255.255.0
 negotiation auto
!
interface GigabitEthernet0/2
 negotiation auto
!
interface GigabitEthernet0/3
 negotiation auto
!
interface GigabitEthernet1/0
 negotiation auto
!
interface GigabitEthernet1/1
 negotiation auto
!
interface GigabitEthernet1/2
 negotiation auto
!
interface GigabitEthernet1/3
 negotiation auto
!
router bgp 65534
 bgp router-id 108.67.222.2
 bgp log-neighbor-changes
 network 172.16.2.0 mask 255.255.255.0
 neighbor 108.67.222.1 remote-as 65535
 neighbor 108.67.222.1 update-source GigabitEthernet0/0
!
ip forward-protocol nd
!
ip http server
ip http secure-server
!
ip ssh server algorithm encryption aes128-ctr aes192-ctr aes256-ctr
ip ssh client algorithm encryption aes128-ctr aes192-ctr aes256-ctr
!
!
!
!
!
!
control-plane
!
banner exec ^C
IOSv - Cisco Systems Confidential -

Supplemental End User License Restrictions

This IOSv software is provided AS-IS without warranty of any kind. Under no circumstances may this software be used separate from the Cisco Modeling Labs Software that this software was provided with, or deployed or used as part of a production environment.

By using the software, you agree to abide by the terms and conditions of the Cisco End User License Agreement at http://www.cisco.com/go/eula. Unauthorized use or distribution of this software is expressly prohibited.
^C
banner incoming ^C
IOSv - Cisco Systems Confidential -

Supplemental End User License Restrictions

This IOSv software is provided AS-IS without warranty of any kind. Under no circumstances may this software be used separate from the Cisco Modeling Labs Software that this software was provided with, or deployed or used as part of a production environment.

By using the software, you agree to abide by the terms and conditions of the Cisco End User License Agreement at http://www.cisco.com/go/eula. Unauthorized use or distribution of this software is expressly prohibited.
^C
banner login ^C
IOSv - Cisco Systems Confidential -

Supplemental End User License Restrictions

This IOSv software is provided AS-IS without warranty of any kind. Under no circumstances may this software be used separate from the Cisco Modeling Labs Software that this software was provided with, or deployed or used as part of a production environment.

By using the software, you agree to abide by the terms and conditions of the Cisco End User License Agreement at http://www.cisco.com/go/eula. Unauthorized use or distribution of this software is expressly prohibited.
^C
!
line con 0
 logging synchronous
line aux 0
line vty 0 4
!
!
end

NXOS1# show running-config

!Command: show running-config
!Running configuration last done at: Tue Jan  9 15:38:41 2024
!Time: Tue Jan  9 15:49:48 2024

version 10.3(4a) Bios:version
hostname NXOS1
vdc NXOS1 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4097
  limit-resource port-channel minimum 0 maximum 511
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

nv overlay evpn
feature ospf
feature bgp
feature interface-vlan
feature vn-segment-vlan-based
feature nv overlay

no password strength-check
username admin password 5 $5$GFPMEJ$ZT97vgSalc27Wq9r0DhBXB/i0yuI3kd4dVsjhe.Fuw1  role network-admin
ip domain-lookup
copp profile strict
hardware access-list tcam region racl 512
hardware access-list tcam region e-racl 512
hardware access-list tcam region arp-ether 256 double-wide
snmp-server user admin network-admin auth md5 0142EC31804EDFAF7F2C60B8B0A5CC6BDDC2 priv aes-128 364BEC328312DDB3586475BAAFAFFA3598C7 localizedV2key
rmon event 1 log trap public description FATAL(1) owner PMON@FATAL
rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO

fabric forwarding anycast-gateway-mac 0000.2222.3333
vlan 1,101,1000-1001
vlan 101
  vn-segment 900001
vlan 1000
  vn-segment 5000
vlan 1001
  vn-segment 5005

vrf context VXLAN
  vni 900001
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context management

interface Vlan1

interface Vlan101
  no shutdown
  vrf member VXLAN
  ip forward

interface Vlan1000
  no shutdown
  vrf member VXLAN
  ip address 192.168.10.1/24
  fabric forwarding mode anycast-gateway

interface Vlan1001
  no shutdown
  vrf member VXLAN
  ip address 192.168.11.1/24
  fabric forwarding mode anycast-gateway

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  member vni 5000
    suppress-arp
    ingress-replication protocol bgp
  member vni 5005
    suppress-arp
    ingress-replication protocol bgp
  member vni 900001 associate-vrf

interface Ethernet1/1
  no switchport
  vrf member VXLAN
  ip address 208.67.222.1/30
  no shutdown

interface Ethernet1/2

interface Ethernet1/3

interface Ethernet1/4
  description Host Computer
  switchport access vlan 1000

interface Ethernet1/5

interface Ethernet1/6

interface Ethernet1/7
  no switchport
  ip address 192.168.255.1/30
  ip router ospf 150 area 0.0.0.0
  no shutdown

interface Ethernet1/8

interface Ethernet1/9

interface Ethernet1/10

interface Ethernet1/11

interface Ethernet1/12

interface Ethernet1/13

interface Ethernet1/14

interface Ethernet1/15

interface Ethernet1/16

interface Ethernet1/17

interface Ethernet1/18

interface Ethernet1/19

interface Ethernet1/20

interface Ethernet1/21

interface Ethernet1/22

interface Ethernet1/23

interface Ethernet1/24

interface Ethernet1/25

interface Ethernet1/26

interface Ethernet1/27

interface Ethernet1/28

interface Ethernet1/29

interface Ethernet1/30

interface Ethernet1/31

interface Ethernet1/32

interface Ethernet1/33

interface Ethernet1/34

interface Ethernet1/35

interface Ethernet1/36

interface Ethernet1/37

interface Ethernet1/38

interface Ethernet1/39

interface Ethernet1/40

interface Ethernet1/41

interface Ethernet1/42

interface Ethernet1/43

interface Ethernet1/44

interface Ethernet1/45

interface Ethernet1/46

interface Ethernet1/47

interface Ethernet1/48

interface Ethernet1/49

interface Ethernet1/50

interface Ethernet1/51

interface Ethernet1/52

interface Ethernet1/53

interface Ethernet1/54

interface Ethernet1/55

interface Ethernet1/56

interface Ethernet1/57

interface Ethernet1/58

interface Ethernet1/59

interface Ethernet1/60

interface Ethernet1/61

interface Ethernet1/62

interface Ethernet1/63

interface Ethernet1/64

interface mgmt0
  vrf member management

interface loopback0
  ip address 1.1.1.1/32
  ip router ospf 150 area 0.0.0.0
icam monitor scale

line console
line vty
boot nxos bootflash:/nxos64-cs.10.3.4a.M.bin
router ospf 150
  router-id 1.1.1.1
router bgp 65535
  router-id 1.1.1.1
  neighbor 2.2.2.2
    remote-as 65535
    update-source loopback0
    address-family l2vpn evpn
      send-community extended
  vrf VXLAN
    address-family ipv4 unicast
      network 192.168.10.0/24
      advertise l2vpn evpn
    neighbor 208.67.222.2
      remote-as 65534
      local-as 65535
      update-source Ethernet1/1
      address-family ipv4 unicast
evpn
  vni 5000 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 5005 l2
    rd auto
    route-target import auto
    route-target export auto

NXOS2# show running-config

!Command: show running-config
!Running configuration last done at: Tue Jan  9 15:39:08 2024
!Time: Tue Jan  9 15:50:30 2024

version 10.3(4a) Bios:version
hostname NXOS2
vdc NXOS2 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4097
  limit-resource port-channel minimum 0 maximum 511
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

nv overlay evpn
feature ospf
feature bgp
feature interface-vlan
feature vn-segment-vlan-based
feature nv overlay

no password strength-check
username admin password 5 $5$NOKIAD$dmSy1Fds8ExPMvgZ/g0jI0O0eGOo9.Z.6d5FmmapvXC  role network-admin
ip domain-lookup
copp profile strict
hardware access-list tcam region racl 512
hardware access-list tcam region e-racl 512
hardware access-list tcam region arp-ether 256 double-wide
snmp-server user admin network-admin auth md5 53119D5D6FD9E2D1D4820B7A6B32F06A3114 priv aes-128 49539601EDF6D5FFE3EC226F6305BF256E54 localizedV2key
rmon event 1 log trap public description FATAL(1) owner PMON@FATAL
rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO

fabric forwarding anycast-gateway-mac 0000.2222.3333
vlan 1,101,1000-1001
vlan 101
  vn-segment 900001
vlan 1000
  vn-segment 5000
vlan 1001
  vn-segment 5005

vrf context VXLAN
  vni 900001
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context management

interface Vlan1

interface Vlan101
  no shutdown
  vrf member VXLAN
  ip forward

interface Vlan1000
  no shutdown
  vrf member VXLAN
  ip address 192.168.10.1/24
  fabric forwarding mode anycast-gateway

interface Vlan1001
  no shutdown
  vrf member VXLAN
  ip address 192.168.11.1/24
  fabric forwarding mode anycast-gateway

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  member vni 5000
    suppress-arp
    ingress-replication protocol bgp
  member vni 5005
    suppress-arp
    ingress-replication protocol bgp
  member vni 900001 associate-vrf

interface Ethernet1/1
  no switchport
  vrf member VXLAN
  ip address 108.67.222.1/30
  no shutdown

interface Ethernet1/2

interface Ethernet1/3

interface Ethernet1/4
  description Host Computer
  switchport access vlan 1001

interface Ethernet1/5

interface Ethernet1/6

interface Ethernet1/7
  no switchport
  ip address 192.168.255.2/30
  ip router ospf 150 area 0.0.0.0
  no shutdown

interface Ethernet1/8

interface Ethernet1/9

interface Ethernet1/10

interface Ethernet1/11

interface Ethernet1/12

interface Ethernet1/13

interface Ethernet1/14

interface Ethernet1/15

interface Ethernet1/16

interface Ethernet1/17

interface Ethernet1/18

interface Ethernet1/19

interface Ethernet1/20

interface Ethernet1/21

interface Ethernet1/22

interface Ethernet1/23

interface Ethernet1/24

interface Ethernet1/25

interface Ethernet1/26

interface Ethernet1/27

interface Ethernet1/28

interface Ethernet1/29

interface Ethernet1/30

interface Ethernet1/31

interface Ethernet1/32

interface Ethernet1/33

interface Ethernet1/34

interface Ethernet1/35

interface Ethernet1/36

interface Ethernet1/37

interface Ethernet1/38

interface Ethernet1/39

interface Ethernet1/40

interface Ethernet1/41

interface Ethernet1/42

interface Ethernet1/43

interface Ethernet1/44

interface Ethernet1/45

interface Ethernet1/46

interface Ethernet1/47

interface Ethernet1/48

interface Ethernet1/49

interface Ethernet1/50

interface Ethernet1/51

interface Ethernet1/52

interface Ethernet1/53

interface Ethernet1/54

interface Ethernet1/55

interface Ethernet1/56

interface Ethernet1/57

interface Ethernet1/58

interface Ethernet1/59

interface Ethernet1/60

interface Ethernet1/61

interface Ethernet1/62

interface Ethernet1/63

interface Ethernet1/64

interface mgmt0
  vrf member management

interface loopback0
  ip address 2.2.2.2/32
  ip router ospf 150 area 0.0.0.0
icam monitor scale

line console
line vty
boot nxos bootflash:/nxos64-cs.10.3.4a.M.bin
router ospf 150
  router-id 2.2.2.2
router bgp 65535
  router-id 2.2.2.2
  neighbor 1.1.1.1
    remote-as 65535
    update-source loopback0
    address-family l2vpn evpn
      send-community extended
  vrf VXLAN
    address-family ipv4 unicast
      network 192.168.11.0/24
      advertise l2vpn evpn
    neighbor 108.67.222.2
      remote-as 65534
      local-as 65535
      update-source Ethernet1/1
      address-family ipv4 unicast
evpn
  vni 5000 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 5005 l2
    rd auto
    route-target import auto
    route-target export auto

paul driver · ‎07-07-2024

Hello
at first glance it looks like you have a two AS design, meaning your leaf as will see source/destination as the same thus negate route advertisement.
try:
router bgp 65535
address-family l2vpn evpn
allowas-in
vrf VXLAN
address-family ipv4 unicast
allowas-in

Please rate and mark as an accepted solution if you have found any of the information provided useful.
This then could assist others on these forums to find a valuable answer and broadens the community’s global network.

Kind Regards
Paul

Harold Ritter · ‎07-07-2024

Hi @paul driver ,

Just a small precision. allows-in would need to be applied to AS65534 (FWs) rather than to AS65535.

Regards,

Harold Ritter
Sr Technical Leader
CCIE 4168 (R&S, SP)
harold@cisco.com
México móvil: +52 1 55 8312 4915
Cisco México
Paseo de la Reforma 222
Piso 19
Cuauhtémoc, Juárez
Ciudad de México, 06600
México

paul driver · ‎07-08-2024

Hello @Harold Ritter
thanks for the clarification mate- tbh i only glanced over the topology as to which ASN was the “transit” so i got it wrong.
@PingWhisperer apologies

Please rate and mark as an accepted solution if you have found any of the information provided useful.
This then could assist others on these forums to find a valuable answer and broadens the community’s global network.

Kind Regards
Paul

Harold Ritter · ‎07-07-2024

Hi @PingWhisperer ,

I would suggest you remove the command "local-as 65535", as it is not required in this scenario and can sometimes cause issues when used in the improper scenario.

Regards,

Harold Ritter
Sr Technical Leader
CCIE 4168 (R&S, SP)
harold@cisco.com
México móvil: +52 1 55 8312 4915
Cisco México
Paseo de la Reforma 222
Piso 19
Cuauhtémoc, Juárez
Ciudad de México, 06600
México