VXLAN Static Ingress Replication: NVE Peers Up, No Remote MAC Learning

[marjanwasti]

Dear @
I am  experiencing an issue with VXLAN static ingress replication on a pair of Nexus switches running NX-OS 7.0(3)I7(2). The setup consists of two leaf switches (NX-6 and NX-7), where VLAN 100 is mapped to VNI 5000 on both switches. Each switch uses a loopback interface as the NVE source (NX-6: 1.1.1.1, NX-7: 2.2.2.2), and OSPF is successfully advertising the loopbacks across the underlay. OSPF adjacency between the switches is established over a VLAN configured on a trunk port connecting the two Nexus devices. NVE peers are configured with static ingress replication, and the peer state is up on both sides. The VLAN-to-VNI mapping is verified, and the VNI is shown as up. Local MAC addresses on access ports are learned correctly. However, no remote MAC addresses are learned through the VXLAN tunnel, and no encapsulated or decapsulated packet counts are observed on the NVE interface. ARP and broadcast traffic generated from hosts or SVIs does not appear to trigger MAC learning over VXLAN. I am seeking guidance on what could cause this behavior despite what appears to be correct configuration and control-plane establishment, and whether this might be linked to a known issue or a missing configuration detail on NX-OS 7.0(3)I7(2).

[MHM Cisco World]

If bgp not use then ingress replication will not work' are yoh sure the config is correct 

Can i see it 

MHM

[marjanwasti]

Hi, please find the details of both switches below. 

NX6 Details:

sh ip ospf neighbors
OSPF Process ID 1 VRF default
Total number of neighbors: 1
Neighbor ID Pri State Up Time Address Interface
2.2.2.2 1 FULL/ - 01:23:31 10.10.10.2 Vlan175

sh nve peers
Interface Peer-IP State LearnType Uptime Router-Mac
--------- --------------- ----- --------- -------- -----------------
nve1 2.2.2.2 Up DP 01:24:49 n/a

sh nve vni
Codes: CP - Control Plane DP - Data Plane
UC - Unconfigured SA - Suppress ARP
SU - Suppress Unknown Unicast

Interface VNI Multicast-group State Mode Type [BD/VRF] Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1 5000 UnicastStatic Up DP L2 [100]

interface nve1
no shutdown
source-interface loopback0
member vni 5000
ingress-replication protocol static
peer-ip 2.2.2.2

sh system internal l2fwder mac
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False, C - ControlPlane MAC
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
* 175 5000.0007.0007 dynamic 00:00:18 F F Eth1/1
* 100 0050.7966.6801 dynamic 00:02:29 F F Eth1/2
G 100 5000.0006.0007 static - F F sup-eth1(R)
G 175 5000.0006.0007 static - F F sup-eth1(R)

NX-6# show run vlan 100

!Command: show running-config vlan 100
!Time: Tue Jul 1 14:44:15 2025

version 7.0(3)I7(2)
vlan 100
vlan 100
vn-segment 5000

NX7:

NX-7# sh ip ospf neighbors
OSPF Process ID 1 VRF default
Total number of neighbors: 1
Neighbor ID Pri State Up Time Address Interface
1.1.1.1 1 FULL/ - 01:27:13 10.10.10.1 Vlan175
NX-7# sh run int nve 1

!Command: show running-config interface nve1
!Time: Tue Jul 1 14:42:01 2025

version 7.0(3)I7(2)

interface nve1
no shutdown
source-interface loopback0
member vni 5000
ingress-replication protocol static
peer-ip 1.1.1.1

NX-7# sh nve peers
Interface Peer-IP State LearnType Uptime Router-Mac
--------- --------------- ----- --------- -------- -----------------
nve1 1.1.1.1 Up DP 01:27:18 n/a

NX-7# sh nve vni
Codes: CP - Control Plane DP - Data Plane
UC - Unconfigured SA - Suppress ARP
SU - Suppress Unknown Unicast

Interface VNI Multicast-group State Mode Type [BD/VRF] Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1 5000 UnicastStatic Up DP L2 [100]

NX-7# sh vxlan
Vlan VN-Segment
==== ==========
100 5000
NX-7# sh run vlan 100

!Command: show running-config vlan 100
!Time: Tue Jul 1 14:42:24 2025

version 7.0(3)I7(2)
vlan 100
vlan 100
vn-segment 5000

NX-7# sh l2
l2 l2fwder l2protocol l2rib l2route
NX-7# sh system internal l2f
l2fm l2fwder
NX-7# sh system internal l2fwder mac
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False, C - ControlPlane MAC
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
G 175 5000.0007.0007 static - F F sup-eth1(R)
* 100 0050.7966.6802 dynamic 00:04:47 F F Eth1/2
* 175 5000.0006.0007 dynamic 00:02:28 F F Eth1/1
G 100 5000.0007.0007 static - F F sup-eth1(R)

[MHM Cisco World]

ingress-replication protocol static <<- you use static 

So i think you need to add 

Ingress replication peer-list <peer ip that reachable >

[marjanwasti]

After static command , peer ip is added (loopback address)

[MHM Cisco World]

This nexus platform 

If yes 

Did you  check command I mention above?

MHM

[MHM Cisco World]

No need to check' I already check your config is correct'

Peer ip is add correctly

Let me make second review to see issue

MHM

[marjanwasti]

Thank you for your time and consideration in reviewing this matter. I look
forward to your feedback.

Additionally, I would like to highlight that I am unable to determine why
the configuration is not functioning as expected when the ports between the
Nexus switches are configured as trunks and OSPF is established using
VLANs. VXLAN operates correctly when routed ports are used. I have also
increased the MTU to 9216, but the issue persists with trunk connectivity.

[M02@rt37]

Hello @marjanwasti 

I let @MHM Cisco World following its investigation but I have a question about this:

Nexus switches are configured as trunks and OSPF is established using
VLANs. VXLAN operates correctly when routed ports are used.

You mean you have Trunks between each Leaf and their Spine ? If yes, regarding this architecture you should have L3 P2P link between each Leaf and their Spine. All routed links with an IGP (ospf in your case) should form your Underlay Network. With this architecture again, no more L2 trouble needed ! So no Trunks !

[marjanwasti]

I have two leaf switches in my setup (no spine). When I connect the leafs using Layer 3 point-to-point (P2P) routed links, VXLAN works fine. However, when I change the connectivity between the leafs to trunk links and run OSPF over VLANs, VXLAN stops working. The NVE peers are up, but traffic does not pass through the fabric.

I want to understand why VXLAN fails when I move the Layer 3 P2P connectivity from routed links to VLAN-based links, even though the configuration appears correct. Is there something I might be missing?

I have attached the configurations for both scenarios: one where VXLAN works over routed links, and another where it fails over VLAN-based Layer 3 P2P.

NX-6:
interface Vlan175
no shutdown
mtu 9216
ip address 10.10.10.1/30
ip ospf network point-to-point
ip router ospf 1 area 0.0.0.0

NX-6# sh run int ethernet 1/1

!Command: show running-config interface Ethernet1/1
!Time: Wed Jul 2 06:39:58 2025

version 7.0(3)I7(2)

interface Ethernet1/1
switchport mode trunk
switchport trunk allowed vlan 175

NX-6# sh nve peer
Interface Peer-IP State LearnType Uptime Router-Mac
--------- --------------- ----- --------- -------- -----------------
nve1 2.2.2.2 Up DP 00:02:08 n/a

NX-6# sh nve vni
Codes: CP - Control Plane DP - Data Plane
UC - Unconfigured SA - Suppress ARP
SU - Suppress Unknown Unicast

Interface VNI Multicast-group State Mode Type [BD/VRF] Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1 5000 UnicastStatic Up DP L2 [100]

NX-6# sh system internal l2fwder mac
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False, C - ControlPlane MAC
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
* 175 5000.0007.0007 dynamic 00:03:03 F F Eth1/1
* 100 0050.7966.6801 dynamic 00:00:30 F F Eth1/2
G 100 5000.0006.0007 static - F F sup-eth1(R)
G 175 5000.0006.0007 static - F F sup-eth1(R)

VPCS> ping 10.99.99.2

host (10.99.99.2) not reachable


NX-7:
interface Vlan175
no shutdown
mtu 9216
ip address 10.10.10.2/30
ip ospf network point-to-point
ip router ospf 1 area 0.0.0.0

NX-7# sh run int ethernet 1/1

!Command: show running-config interface Ethernet1/1
!Time: Wed Jul 2 06:41:31 2025

version 7.0(3)I7(2)

interface Ethernet1/1
switchport mode trunk
switchport trunk allowed vlan 175

NX-7# sh nve peer
Interface Peer-IP State LearnType Uptime Router-Mac
--------- --------------- ----- --------- -------- -----------------
nve1 1.1.1.1 Up DP 00:03:43 n/a

NX-7# sh nve vni
Codes: CP - Control Plane DP - Data Plane
UC - Unconfigured SA - Suppress ARP
SU - Suppress Unknown Unicast

Interface VNI Multicast-group State Mode Type [BD/VRF] Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1 5000 UnicastStatic Up DP L2 [100]

NX-7# sh system internal l2f
l2fm l2fwder
NX-7# sh system internal l2fwder mac
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False, C - ControlPlane MAC
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
G 175 5000.0007.0007 static - F F sup-eth1(R)
* 100 0050.7966.6802 dynamic 00:01:20 F F Eth1/2
* 175 5000.0006.0007 dynamic 00:04:15 F F Eth1/1
G 100 5000.0007.0007 static - F F sup-eth1(R)


VPCS> ping 10.99.99.1

host (10.99.99.1) not reachable

NX-6:
interface Ethernet1/1
no switchport
mtu 9216
ip address 10.10.10.1/30
ip ospf network point-to-point
ip router ospf 1 area 0.0.0.0
no shutdown

NX-6# sh nve peers
Interface Peer-IP State LearnType Uptime Router-Mac
--------- --------------- ----- --------- -------- -----------------
nve1 2.2.2.2 Up DP 00:00:32 n/a

NX-6# sh nve vni
Codes: CP - Control Plane DP - Data Plane
UC - Unconfigured SA - Suppress ARP
SU - Suppress Unknown Unicast

Interface VNI Multicast-group State Mode Type [BD/VRF] Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1 5000 UnicastStatic Up DP L2 [100]

NX-6# sh run int nve 1

!Command: show running-config interface nve1
!Time: Wed Jul 2 06:25:16 2025

version 7.0(3)I7(2)

interface nve1
no shutdown
source-interface loopback0
member vni 5000
ingress-replication protocol static
peer-ip 2.2.2.2

NX-6# sh system internal l2f
l2fm l2fwder
NX-6# sh system internal l2fwder mac
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False, C - ControlPlane MAC
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
* 100 0050.7966.6801 dynamic 00:02:30 F F Eth1/2
G 100 5000.0006.0007 static - F F sup-eth1(R)
* 100 0050.7966.6802 dynamic 00:00:50 F F (0x47000001) nve-peer1
2.2.2.2

VPCS> ping 10.99.99.2

84 bytes from 10.99.99.2 icmp_seq=1 ttl=64 time=23.894 ms
84 bytes from 10.99.99.2 icmp_seq=2 ttl=64 time=19.865 ms
84 bytes from 10.99.99.2 icmp_seq=3 ttl=64 time=17.456 ms
84 bytes from 10.99.99.2 icmp_seq=4 ttl=64 time=11.484 ms
84 bytes from 10.99.99.2 icmp_seq=5 ttl=64 time=10.424 ms

NX-7:
interface Ethernet1/1
no switchport
mtu 9216
ip address 10.10.10.2/30
ip ospf network point-to-point
ip router ospf 1 area 0.0.0.0
no shutdown

NX-7# sh nve peer
Interface Peer-IP State LearnType Uptime Router-Mac
--------- --------------- ----- --------- -------- -----------------
nve1 1.1.1.1 Up DP 00:02:15 n/a

NX-7# sh nve vni
Codes: CP - Control Plane DP - Data Plane
UC - Unconfigured SA - Suppress ARP
SU - Suppress Unknown Unicast

Interface VNI Multicast-group State Mode Type [BD/VRF] Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1 5000 UnicastStatic Up DP L2 [100]

NX-7# sh run int nve 1

!Command: show running-config interface nve1
!Time: Wed Jul 2 06:26:53 2025

version 7.0(3)I7(2)

interface nve1
no shutdown
source-interface loopback0
member vni 5000
ingress-replication protocol static
peer-ip 1.1.1.1

NX-7# sh system internal l2fwder mac
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False, C - ControlPlane MAC
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
* 100 0050.7966.6801 dynamic 00:02:22 F F (0x47000001) nve-peer1
1.1.1.1
* 1 5000.0006.0007 dynamic 00:03:17 F F Eth1/1
* 100 0050.7966.6802 dynamic 00:04:11 F F Eth1/2
G 100 5000.0007.0007 static - F F sup-eth1(R)

VPCS> ping 10.99.99.1

84 bytes from 10.99.99.1 icmp_seq=1 ttl=64 time=9.402 ms
84 bytes from 10.99.99.1 icmp_seq=2 ttl=64 time=11.039 ms
84 bytes from 10.99.99.1 icmp_seq=3 ttl=64 time=18.058 ms
84 bytes from 10.99.99.1 icmp_seq=4 ttl=64 time=15.449 ms
84 bytes from 10.99.99.1 icmp_seq=5 ttl=64 time=13.334 ms

[MHM Cisco World]

Thanks for more details 

Only make sure the vlan 100 is not allowed in trunk between two nexus SW.

Let vxlan do bridge job not extended vlan over l2 trunk.

MHM

[marjanwasti]

vlan 100 is not allowed over trunk. can be seen in the configuration. how vxlan will do bridge job, because still not working with the correct configuration.

[MHM Cisco World]

Make sure vlan not allow in trunk between two vtep

Run below 

ethanalyzer local interface inband capture-filter "udp port 4789"

Share output 

MHM

[marjanwasti]

i ran this command, does not give any output. stuck on this

NX-7# ethanalyzer local interface inband capture-filter "udp port 4789"

Capturing on inband


when i terminate the command using Ctrl + C, it says 0 packets captured. 

[MHM Cisco World]

ethanalyzer local interface inband detail

Run this and share result 

MHM