Solved: problem with arp NX-OS

kirill-pototskiy98 · ‎08-21-2025

Hello! I have a problem.

I have 2 leaf, which have connections to client 1,2. When client 1 try ping, it sends arp request, but it doesn't have reply. When I see dump, I see that leaf 1 replace arp-request on vxlan, but it doesn't send reply to client 1. I don't understand this situation.

My config, and debug information

hostname leaf3

cfs eth distribute

nv overlay evpn

feature ospf

feature bgp

feature interface-vlan

feature vn-segment-vlan-based

feature lacp

feature lldp

feature nv overlay

no password strength-check

role network-admin

ip domain-lookup

copp profile strict

rmon event 1 log trap public description FATAL(1) owner PMON@FATAL

rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL

rmon event 3 log trap public description ERROR(3) owner PMON@ERROR

rmon event 4 log trap public description WARNING(4) owner PMON@WARNING

rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO

fabric forwarding anycast-gateway-mac 0000.2222.3333

vlan 1,201-202,999

vlan 201

name cli1_tanant1

vn-segment 50201

vlan 202

name cli2_tanant1

vn-segment 50202

vlan 999

name L3_VNI

vn-segment 50999

route-map permitall permit 10

vrf context Tenant-1

vni 50999

rd auto

address-family ipv4 unicast

route-target both auto

route-target both auto evpn

vrf context management

interface Vlan1

interface Vlan201

no shutdown

vrf member Tenant-1

no ip redirects

ip address 10.0.201.254/24

fabric forwarding mode anycast-gateway

interface Vlan202

no shutdown

vrf member Tenant-1

no ip redirects

ip address 10.0.202.254/24

fabric forwarding mode anycast-gateway

interface Vlan999

no shutdown

vrf member Tenant-1

ip forward

interface nve1

no shutdown

host-reachability protocol bgp

source-interface loopback1

member vni 201

member vni 50201

ingress-replication protocol bgp

member vni 50202

ingress-replication protocol bgp

member vni 50999 associate-vrf

interface Ethernet1/1

no switchport

mtu 8000

port-type fabric

medium p2p

no ip redirects

ip unnumbered loopback0

no ipv6 redirects

ip ospf network point-to-point

ip router ospf 1 area 0.0.0.0

no shutdown

interface Ethernet1/2

no switchport

mtu 8000

port-type fabric

medium p2p

no ip redirects

ip unnumbered loopback0

no ipv6 redirects

ip ospf network point-to-point

ip router ospf 1 area 0.0.0.0

no shutdown

interface Ethernet1/3

switchport access vlan 201

interface Ethernet1/4

switchport access vlan 201

interface Ethernet1/5

interface Ethernet1/6

interface Ethernet1/7

interface Ethernet1/8

interface loopback0

description routerID underlay

ip address 10.10.10.3/32

ip router ospf 1 area 0.0.0.0

interface loopback1

description nve

ip address 10.200.200.3/32

ip router ospf 1 area 0.0.0.0

icam monitor scale

line console

line vty

router ospf 1

router-id 10.10.10.3

router bgp 65400

router-id 10.10.10.3

address-family l2vpn evpn

retain route-target all

neighbor 10.10.100.6

remote-as 65400

update-source loopback0

address-family ipv4 unicast

address-family l2vpn evpn

send-community

send-community extended

neighbor 10.10.100.7

remote-as 65400

update-source loopback0

address-family ipv4 unicast

address-family l2vpn evpn

send-community

send-community extended

vrf Tenant-1

address-family ipv4 unicast

redistribute direct route-map permitall

evpn

vni 50201 l2

rd auto

route-target import auto

route-target export auto

vni 50202 l2

rd auto

route-target import auto

route-target export auto

leaf3# show bgp l2vpn evpn

Network Next Hop Metric LocPrf Weight Path

Route Distinguisher: 10.10.10.1:3

*>i[5]:[0]:[0]:[24]:[10.0.201.0]/224

10.200.200.254 0 100 0 ?

*>i[5]:[0]:[0]:[24]:[10.0.202.0]/224

10.200.200.254 0 100 0 ?

Route Distinguisher: 10.10.10.1:32968

*>i[2]:[0]:[0]:[48]:[0050.7966.6800]:[0]:[0.0.0.0]/216

10.200.200.1 100 0 i

*>i[2]:[0]:[0]:[48]:[0c5a.0000.1b08]:[0]:[0.0.0.0]/216

10.200.200.254 100 0 i

*>i[2]:[0]:[0]:[48]:[0050.7966.6800]:[32]:[10.0.201.1]/272

10.200.200.1 100 0 i

*>i[3]:[0]:[32]:[10.200.200.254]/88

10.200.200.254 100 0 i

Route Distinguisher: 10.10.10.1:32969

*>i[2]:[0]:[0]:[48]:[0c5a.0000.1b08]:[0]:[0.0.0.0]/216

10.200.200.254 100 0 i

*>i[3]:[0]:[32]:[10.200.200.254]/88

10.200.200.254 100 0 i

Route Distinguisher: 10.10.10.3:32968 (L2VNI 50201)

*>i[2]:[0]:[0]:[48]:[0050.7966.6800]:[0]:[0.0.0.0]/216

10.200.200.1 100 0 i

*>l[2]:[0]:[0]:[48]:[0050.7966.6801]:[0]:[0.0.0.0]/216

10.200.200.3 100 32768 i

*>i[2]:[0]:[0]:[48]:[0c5a.0000.1b08]:[0]:[0.0.0.0]/216

10.200.200.254 100 0 i

*>i[2]:[0]:[0]:[48]:[0050.7966.6800]:[32]:[10.0.201.1]/272

10.200.200.1 100 0 i

*>l[2]:[0]:[0]:[48]:[0050.7966.6801]:[32]:[10.0.201.2]/272

10.200.200.3 100 32768 i

*>l[3]:[0]:[32]:[10.200.200.3]/88

10.200.200.3 100 32768 i

*>i[3]:[0]:[32]:[10.200.200.254]/88

10.200.200.254 100 0 i

Route Distinguisher: 10.10.10.3:32969 (L2VNI 50202)

*>i[2]:[0]:[0]:[48]:[0c5a.0000.1b08]:[0]:[0.0.0.0]/216

10.200.200.254 100 0 i

*>l[3]:[0]:[32]:[10.200.200.3]/88

10.200.200.3 100 32768 i

*>i[3]:[0]:[32]:[10.200.200.254]/88

10.200.200.254 100 0 i

Route Distinguisher: 10.10.10.3:3 (L3VNI 50999)

*>i[2]:[0]:[0]:[48]:[0050.7966.6800]:[32]:[10.0.201.1]/272

10.200.200.1 100 0 i

* i[5]:[0]:[0]:[24]:[10.0.201.0]/224

10.200.200.254 0 100 0 ?

*>l 10.200.200.3 0 100 32768 ?

* i[5]:[0]:[0]:[24]:[10.0.202.0]/224

10.200.200.254 0 100 0 ?

*>l 10.200.200.3 0 100 32768 ?

leaf3# show l2route evpn mac all

201 0050.7966.6800 BGP SplRcv 0 10.200.200.1 (Label:

50201)

201 0050.7966.6801 Local L, 0 Eth1/4

999 0200.0ac8.c8fe VXLAN Rmac 0 10.200.200.254

999 0c5a.0000.1b08 VXLAN Rmac 0 10.200.200.1

MHM Cisco World · ‎08-21-2025

vni 50201 l2

rd auto

route-target import auto

route-target export auto

suppress-arp <<- add this to both leaf and check

MHM

View solution in original post

Enes Simnica · ‎08-21-2025

gDay @kirill-pototskiy98. The problem here is that ur NVE configuration has both member vni 201 and member vni 50201. Now, since VNI 201 is not actually mapped to VLAN 201, that extra line causes confusion in the control-plane. What u should have is only the 50201/50202 VNIs with ingress replication, mapped consistently to VLANs 201 and 202 across both leaves. Also make sure ur anycast gateway MAC and SVI IPs are identical on both leaves and that ARP suppression is enabled if needed. So my Cisco friend, in short, remove the stray VNI 201 from the NVE, double-check that VLAN-to-VNI mappings match on both leaves, and confirm your NVE peers and VNIs are up, once those are corrected, the ARP replies will start working.../

hope it helps and PEACE!

-Enes

more Cisco?!
more Gym?!

If this post solved your problem, kindly mark it as Accepted Solution. Much appreciated!

kirill-pototskiy98 · ‎08-21-2025

I send debug information message below

MHM Cisco World · ‎08-21-2025

why ARP need here
BGP advertise the MAC-IP so no need for ARP
ping from host to host and check

MHM

kirill-pototskiy98 · ‎08-21-2025

ping don't work, because cli1 doesn't have mac addr cli2

MHM Cisco World · ‎08-21-2025

check below my comment please share info I need

MHM

kirill-pototskiy98 · ‎08-21-2025

leaf3# show run | sec vni
vni 50999
member vni 50201
ingress-replication protocol bgp
member vni 50202
ingress-replication protocol bgp
member vni 50999 associate-vrf
vni 50201 l2
rd auto
route-target import auto
route-target export auto
vni 50202 l2
rd auto
route-target import auto
route-target export auto
leaf3# show run | sec vlan
limit-resource vlan minimum 16 maximum 4094
feature interface-vlan
feature vn-segment-vlan-based
vlan 1,201-202,999
vlan 201
name cli1_tanant1
vn-segment 50201
vlan 202
name cli2_tanant1
vn-segment 50202
vlan 999
name L3_VNI
vn-segment 50999
switchport access vlan 201
switchport access vlan 201
leaf3# show nve peers
Interface Peer-IP State LearnType Uptime Route
r-Mac
--------- -------------------------------------- ----- --------- -------- -----
------------
nve1 10.200.200.1 Up CP 02:45:07 0c5a.
0000.1b08
nve1 10.200.200.254 Up CP 05:05:56 0200.
0ac8.c8fe

leaf3# show nve vni
Codes: CP - Control Plane DP - Data Plane
UC - Unconfigured SA - Suppress ARP
S-ND - Suppress ND
SU - Suppress Unknown Unicast
Xconn - Crossconnect
MS-IR - Multisite Ingress Replication
HYB - Hybrid IRB mode

Interface VNI Multicast-group State Mode Type [BD/VRF] Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1 50201 UnicastBGP Up CP L2 [201]
nve1 50202 UnicastBGP Up CP L2 [202]
nve1 50999 n/a Up CP L3 [Tenant-1]

leaf3# show ip arp suppression topo-info
ARP L2RIB Topology information
Topo-id ARP-suppression mode(HMM SDB value)
201 ARP Suppression Disabled (ARP Suppression Disabled)
202 ARP Suppression Disabled (ARP Suppression Disabled)

!Command: show running-config interface Vlan201
!Running configuration last done at: Thu Aug 21 06:40:50 2025
!Time: Thu Aug 21 11:21:28 2025

version 10.3(1) Bios:version

interface Vlan201
no shutdown
vrf member Tenant-1
no ip redirects
ip address 10.0.201.254/24
no ipv6 redirects
fabric forwarding mode anycast-gateway

leaf1#
leaf1# show run | sec evpn
nv overlay evpn
route-target both auto evpn
address-family l2vpn evpn
retain route-target all
address-family l2vpn evpn
send-community
send-community extended
address-family l2vpn evpn
send-community
send-community extended
evpn
vni 50201 l2
rd auto
route-target import auto
route-target export auto
vni 50202 l2
rd auto
route-target import auto
route-target export auto
leaf1# show run | sec vni
name L3-vni-tenant1
vni 50999
member vni 50201
ingress-replication protocol bgp
member vni 50202
ingress-replication protocol bgp
member vni 50999 associate-vrf
vni 50201 l2
rd auto
route-target import auto
route-target export auto
vni 50202 l2
rd auto
route-target import auto
route-target export auto
leaf1# show run | sec vlan
limit-resource vlan minimum 16 maximum 4094
feature interface-vlan
feature vn-segment-vlan-based
vlan 1,99,201-202,999
vlan 99
name svi
vlan 201
name cli1
vn-segment 50201
vlan 202
name cli2_tenant-1
vn-segment 50202
vlan 999
name L3-vni-tenant1
vn-segment 50999
switchport access vlan 201
leaf1# show nve peers
Interface Peer-IP State LearnType Uptime Route
r-Mac
--------- -------------------------------------- ----- --------- -------- -----
------------
nve1 10.200.200.3 Up CP 05:06:02 0c91.
0000.1b08

leaf1# show nve vni
Codes: CP - Control Plane DP - Data Plane
UC - Unconfigured SA - Suppress ARP
S-ND - Suppress ND
SU - Suppress Unknown Unicast
Xconn - Crossconnect
MS-IR - Multisite Ingress Replication
HYB - Hybrid IRB mode

Interface VNI Multicast-group State Mode Type [BD/VRF] Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1 50201 UnicastBGP Up CP L2 [201]
nve1 50202 UnicastBGP Up CP L2 [202]
nve1 50999 n/a Up CP L3 [Tenant-1]

leaf1#

MHM Cisco World · ‎08-21-2025

issue is solved or not ?

MHM

kirill-pototskiy98 · ‎08-21-2025

no, sorry

MHM Cisco World · ‎08-21-2025

201 0050.7966.6800 BGP SplRcv 0 10.200.200.1 (Label:

50201)

201 0050.7966.6801 Local L, 0 Eth1/4

above for leaf3 what about leaf1 what you see ?

MHM

kirill-pototskiy98 · ‎08-21-2025

Network Next Hop Metric LocPrf Weight PathRoute Distinguisher: 10.10.10.1:32968 (L2VNI 50201)
*>l[2]:[0]:[0]:[48]:[0050.7966.6800]:[0]:[0.0.0.0]/216 10.200.200.1 100 32768 i
*>i[2]:[0]:[0]:[48]:[0050.7966.6801]:[0]:[0.0.0.0]/216 10.200.200.3 100 0 i
*>l[2]:[0]:[0]:[48]:[0c5a.0000.1b08]:[0]:[0.0.0.0]/216 10.200.200.254 100 32768 i
*>l[2]:[0]:[0]:[48]:[0050.7966.6800]:[32]:[10.0.201.1]/272 10.200.200.1 100 32768 i
*>i[2]:[0]:[0]:[48]:[0050.7966.6801]:[32]:[10.0.201.2]/272 10.200.200.3 100 0 i
*>i[3]:[0]:[32]:[10.200.200.3]/88 10.200.200.3 100 0 i
*>l[3]:[0]:[32]:[10.200.200.254]/88 10.200.200.254 100 32768 i
Route Distinguisher: 10.10.10.1:32969 (L2VNI 50202)
*>l[2]:[0]:[0]:[48]:[0c5a.0000.1b08]:[0]:[0.0.0.0]/216 10.200.200.254 100 32768 i
*>i[3]:[0]:[32]:[10.200.200.3]/88 10.200.200.3 100 0 i
*>l[3]:[0]:[32]:[10.200.200.254]/88 10.200.200.254 100 32768 i
Route Distinguisher: 10.10.10.3:3
*>i[5]:[0]:[0]:[24]:[10.0.201.0]/224 10.200.200.3 0 100 0 ?
*>i[5]:[0]:[0]:[24]:[10.0.202.0]/224 10.200.200.3 0 100 0 ?
Route Distinguisher: 10.10.10.3:32968
*>i[2]:[0]:[0]:[48]:[0050.7966.6801]:[0]:[0.0.0.0]/216 10.200.200.3 100 0 i
*>i[2]:[0]:[0]:[48]:[0050.7966.6801]:[32]:[10.0.201.2]/272 10.200.200.3 100 0 i
*>i[3]:[0]:[32]:[10.200.200.3]/88 10.200.200.3 100 0 i
Route Distinguisher: 10.10.10.3:32969
*>i[3]:[0]:[32]:[10.200.200.3]/88 10.200.200.3 100 0 i
Route Distinguisher: 10.10.10.1:3 (L3VNI 50999)
*>i[2]:[0]:[0]:[48]:[0050.7966.6801]:[32]:[10.0.201.2]/272 10.200.200.3 100 0 i
*>l[5]:[0]:[0]:[24]:[10.0.201.0]/224 10.200.200.254 0 100 32768 ?
* i 10.200.200.3 0 100 0 ?*>l[5]:[0]:[0]:[24]:[10.0.202.0]/224
10.200.200.254 0 100 32768 ?* i 10.200.200.3 0 100 0 ?

MHM Cisco World · ‎08-21-2025

show l2route evpn mac all <<- this please

MHM

kirill-pototskiy98 · ‎08-21-2025

message below

kirill-pototskiy98 · ‎08-21-2025

leaf-1
----------- -------------- ------ ------------- ---------- ---------------------------------------------------------
201 0050.7966.6800 Local L,Orp 0 Eth1/3
201 0050.7966.6801 BGP SplRcv 0 10.200.200.3 (Label:50201)
201 0c5a.0000.1b08 VXLAN Stt,Nho, 0 10.200.200.254
202 0c5a.0000.1b08 VXLAN Stt,Nho, 0 10.200.200.254
999 0c5a.0000.1b08 VXLAN Stt,Nho, 0 10.200.200.254
999 0c91.0000.1b08 VXLAN Rmac 0 10.200.200.3

leaf3
----------- -------------- ------ ------------- ---------- ---------------------
------------------------------------
201 0050.7966.6800 BGP SplRcv 0 10.200.200.1 (Label:
50201)
201 0050.7966.6801 Local L, 0 Eth1/4

999 0200.0ac8.c8fe VXLAN Rmac 0 10.200.200.254

999 0c5a.0000.1b08 VXLAN Rmac 0 10.200.200.1

MHM Cisco World · ‎08-21-2025

Leaf1
201 0050.7966.6800 Local L,Orp 0 Eth1/3 <<- this LOCAL i.e. host direct connect to this port
201 0050.7966.6801 BGP SplRcv 0 10.200.200.3 (Label:50201) <<- this learn via BGP

Leaf3

201 0050.7966.6800 BGP SplRcv 0 10.200.200.1 (Label:50201)
201 0050.7966.6801 Local L, 0 Eth1/4

So control plane is OK and it advertise MAC-IP