VXLAN over Nexus9000 C9396PX Chassis

[erastusangula]

Hi all

I am trying to build a VXLAN tunnel between C9396PX and C93180YC-FX. when i run a "show nve peer" on the C93180YC-FX, it shows that the peer (which is  a C93180YC-FX) is up and i am learning mac address over the nve interface. When i run the same command on the C9396PX, it  doesn't show any peer and i am not learning any mac over the nve, i ran the below command too but shows no peer, can it be that there are hardware limitations on the C9396PX? do i  need hardware store and forward then a reboot on the C9396PX?

NXOS image file is: bootflash:///n9000-dk9.6.1.2.I3.1.bin image on the C9396PX

NAM-HO-N9K1-VPC-P# show nve internal platform interface detail
Printing details of all NVE Interfaces
|======|=========================|===============|===============|=====|=====|
|Intf |State |PriIP |SecIP |Vnis |Peers|
|======|=========================|===============|===============|=====|=====|
|nve1 |UP |1.1.1.58 |1.1.1.52 |1 | |
|======|=========================|===============|===============|=====|=====|

SW_BD/VNIs of interface nve1:
================================================
|======|======|=========================|======|
|Sw BD |Vni |State |Intf |
|======|======|=========================|======|
|3 |10001 |UP |nve1 |
|======|======|=========================|======|

Peers of interface nve1:
============================================
no peers

[Sergiu.Daniluk · Accepted solution]

Hi @erastusangula 

Can you share the "show run int nve1" from both switches?

Which flavor of vxlan you have? Flood & Learn or BGP-EVPN?

If it's F&L it could be possible that the peer is not listed simply because there is no user traffic.

Cheers,

Sergiu

[erastusangula]

Hi Sergiu I am running flood and learn. i have shared the diagram and the nve configs below, on one VTEP, the nve peers shows up and i am learning MACs from the remote VTEP, but on that remote VTEP which is a C9396PX, there is no peer showing up and i am not learning MAC from the NVE interface. interface nve1 source-interface loopback0 member vni 10001 mcast-group 226.6.6.6 no shutdown NAM-HO-N9K1-VPC-P# show run int lo0 !Command: show running-config interface loopback0 !Time: Mon May 30 20:22:41 2022 version 6.1(2)I3(1) interface loopback0 no ip redirects ip address 1.1.1.58/32 ip address 1.1.1.52/32 secondary ip router ospf 2022 area 2.2.2.2 ip pim sparse-mode interface nve1 no shutdown source-interface loopback0 member vni 10001 mcast-group 226.6.6.6 NAM-HO-CAMPUS-N9K1-VPC-P# show run int lo0 !Command: show running-config interface loopback0 !Running configuration last done at: Sat May 28 20:17:33 2022 !Time: Mon May 30 20:25:46 2022 version 9.2(2) Bios:version 05.33 interface loopback0 ip address 1.1.1.55/32 ip address 1.1.1.51/32 secondary ip router ospf 2022 area 2.2.2.2 ip pim sparse-mode NAM-HO-CAMPUS-N9K1-VPC-P# show nve peers Interface Peer-IP State LearnType Uptime Router-Mac --------- --------------- ----- --------- -------- ----------------- nve1 1.1.1.52 Up DP 6d04h n/a NAM-HO-CAMPUS-N9K1-VPC-P# show mac address-table dynamic vlan 3 Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vPC Peer-Link, (T) - True, (F) - False, C - ControlPlane MAC, ~ - vsan VLAN MAC Address Type age Secure NTFY Ports ---------+-----------------+--------+---------+------+----+------------------ * 3 0004.f303.cd1f dynamic 0 F F nve1(1.1.1.52) * 3 000a.f7c8.3210 dynamic 0 F F nve1(1.1.1.52) * 3 0014.5e77.c3aa dynamic 0 F F nve1(1.1.1.52) * 3 001e.6780.513b dynamic 0 F F nve1(1.1.1.52) * 3 0050.5629.650e dynamic 0 F F nve1(1.1.1.52)

[erastusangula]

Hi Sergiu

I am running flood and learn. i have shared the diagram and the nve configs below. On one VTEP, the nve peer shows up and i am learning MACs from the remote VTEP.  But on that remote VTEP which is a C9396PX, there is no peer showing up and i am not learning MAC from the NVE interface. I have a laptop connected on that VTEP in the VLAN

NAM-HO-N9K1-VPC-P

interface loopback0
no ip redirects
ip address 1.1.1.58/32
ip address 1.1.1.52/32 secondary
ip router ospf 2022 area 2.2.2.2
ip pim sparse-mode

interface nve1
no shutdown
source-interface loopback0
member vni 10001 mcast-group 226.6.6.6

NAM-HO-CAMPUS-N9K1-VPC-P

interface nve1
no shutdown
source-interface loopback0
member vni 10001 mcast-group 226.6.6.6

!

interface loopback0
ip address 1.1.1.55/32
ip address 1.1.1.51/32 secondary
ip router ospf 2022 area 2.2.2.2
ip pim sparse-mode

!

NAM-HO-CAMPUS-N9K1-VPC-P# show nve peers
Interface Peer-IP State LearnType Uptime Router-Mac
--------- --------------- ----- --------- -------- -----------------
nve1 1.1.1.52 Up DP 6d04h n/a

NAM-HO-CAMPUS-N9K1-VPC-P# show mac address-table dynamic vlan 3
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False, C - ControlPlane MAC, ~ - vsan
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
* 3 0004.f303.cd1f dynamic 0 F F nve1(1.1.1.52)
* 3 000a.f7c8.3210 dynamic 0 F F nve1(1.1.1.52)
* 3 0014.5e77.c3aa dynamic 0 F F nve1(1.1.1.52)
* 3 001e.6780.513b dynamic 0 F F nve1(1.1.1.52)
* 3 0050.5629.650e dynamic 0 F F nve1(1.1.1.52)

[Sergiu.Daniluk]

Again, for a VTEP in VXLAN F&L to see it's neighbors, you need user traffic to be received (most of the times, it start with ARPs).

Looking at the topology, it looks like you have two vPC domains, but only one peer is connected to underlay. Have you tried to connect the second peer to the underlay? I am quite positive that this type of topology is not supported.

Cheers,

Sergiu

[erastusangula]

Hi Sergiu

No i have not tried that yet, i though it was not necessary since both VTEP peers share the same secondary loopback IP. Do you mean having them connected like in the attached diagram (green L3 links)? I will connect all of them to the underlay network and test, i will get back to you on this. Thanks for your input.

[Sergiu.Daniluk]

Correct. Exactly like in your attached diagram. Looking forward for the results.