Nexus 9K(v) Learning MAC but Can't Ping VM or Other Way Around

[hguenfaf]

Hello Guys,

I hope y'all are doing great. So basically, I'm trying to ping the SVI for VLAN 10 IP or 7.7.1.94 from the Linux VM or the other way around but it's timing out even though the 9K is able to dynamically learn the MAC address of 7.7.1.65 just fine even after I `clear mac address-table`.

The hypervisor is a VMWare ESXi.

Here is a high level overview of the topology,

VM ([Linux] 7.7.1.65/27) <---> VM ([Nexus 9K] 7.7.1.94/27) <---> Physical NIC

Nexus 9K configuration below

```

!Command: show running-config
!Running configuration last done at: Tue Feb 20 10:22:23 2024
!Time: Tue Feb 20 10:22:31 2024

version 10.1(1) Bios:version
vdc switch id 1
limit-resource vlan minimum 16 maximum 4094
limit-resource vrf minimum 2 maximum 4096
limit-resource port-channel minimum 0 maximum 511
limit-resource m4route-mem minimum 58 maximum 58
limit-resource m6route-mem minimum 8 maximum 8
feature interface-vlan

mac address-table aging-time 120

username admin password 5 $5$DAPIFN$qskJ3acZMmRtMc0nDywE/EnvoE9lHXlDHoamYhvZfX8
role network-admin
ip domain-lookup
copp profile strict
snmp-server user admin network-admin auth md5 0xc8f962dbe98d83204064426fc062ce5b
priv aes-128 0xc8f962dbe98d83204064426fc062ce5b localizedkey
rmon event 1 log trap public description FATAL(1) owner PMON@FATAL
rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO

vlan 1,10-11
vlan 10
name PRE-Admin
vlan 11
name PRE-Aux

vrf context management

interface Vlan1

interface Vlan10
no shutdown
no ip redirects
ip address 7.7.1.94/27

interface Vlan11
no shutdown
no ip redirects
ip address 7.7.2.94/27

interface Ethernet1/1
switchport
switchport access vlan 10
no shutdown

interface Ethernet1/2
switchport
switchport access vlan 11
no shutdown

interface Ethernet1/3

interface Ethernet1/4

interface Ethernet1/5

interface Ethernet1/6

interface Ethernet1/7

interface Ethernet1/8

interface Ethernet1/9

interface Ethernet1/10

interface Ethernet1/11

interface Ethernet1/12

interface Ethernet1/13

interface Ethernet1/14

interface Ethernet1/15

interface Ethernet1/16

interface Ethernet1/17

interface Ethernet1/18

interface Ethernet1/19

interface Ethernet1/20

interface Ethernet1/21

interface Ethernet1/22

interface Ethernet1/23

interface Ethernet1/24

interface Ethernet1/25

interface Ethernet1/26

interface Ethernet1/27

interface Ethernet1/28

interface Ethernet1/29

interface Ethernet1/30

interface Ethernet1/31

interface Ethernet1/32

interface Ethernet1/33

interface Ethernet1/34

interface Ethernet1/35

interface Ethernet1/36

interface Ethernet1/37

interface Ethernet1/38

interface Ethernet1/39

interface Ethernet1/40

interface Ethernet1/41

interface Ethernet1/42

interface Ethernet1/43

interface Ethernet1/44

interface Ethernet1/45

interface Ethernet1/46

interface Ethernet1/47

interface Ethernet1/48

interface Ethernet1/49

interface Ethernet1/50

interface Ethernet1/51

interface Ethernet1/52

interface Ethernet1/53

interface Ethernet1/54

interface Ethernet1/55

interface Ethernet1/56

interface Ethernet1/57

interface Ethernet1/58

interface Ethernet1/59

interface Ethernet1/60

interface Ethernet1/61

interface Ethernet1/62

interface Ethernet1/63

interface Ethernet1/64

interface mgmt0
vrf member management
icam monitor scale

line console
line vty

```

Any help would be truly appreciated,

Thanks,

[Richard Burts]

Can you tell us where the VM is connected?

Would you post the output of these commands:

show interface status

show arp

show mac

[hguenfaf]

Hello Richard,

Please find attched the output for the requested commands,

I also did a tcpdump through guestshell on "Vlan10" interface while pinging 7.7.1.94 from the Linux VM  which is showing that the Linux VM is actually sending arp requests and 9k router is sending replays, but the issue I noticed is that the MAC address the router is sending is configured on all interfaces (you can see the "ip a" commad output) which could be the main issue,

Find `ip address` and `tcpdump -i Vlan10` outputs attached as well,

Thank you,

```

[Reza Sharifi]

Hi,

What happens if you make interface Ethernet1/1 a trunk port?

HTH

[hguenfaf]

Hi Reza,

I still can't ping the SVI, but this time no arp packets are captured over at the "Vlan10" interface. Also he ESXi virtual switch is on VLAN 0, so it's not tagging any traffic,

[Reza Sharifi]

Hi,

Isn't vlan 0 in VM equal to vlan1 (native vlan) in the Cisco world? If yes, maybe add a new VLAN (vlan 10) to the ESXi host and try again.

Or make vlan 10 on the switch a native vlan.

HTH 

[hguenfaf]

Hi,

Actually in VMWare VLAN 0 configuration in a switch means no VLAN tagging at all, please refer to the below link for more info,

https://communities.vmware.com/t5/Networking-Members/VLAN-0-Concept/td-p/2918292

[Reza Sharifi]

Thanks for that info! So, in this case, the switch config is correct. It should not make a difference but when pinging from the switch, can you add a source address or a source interface?

HTH

[hguenfaf]

In the 9K, yes, I can specify a source. but not in the Linux VM,

Edit, I actually can specify a source interface in the VM which should do the same thing. Giving the admin interface (7.7.1.65) as the source still isn't solving the issue unfortunately 

[hguenfaf]

I also wanted to note that the VMWare vswitch could be completely ignored here since it's sole role here is to link the two VMs, so for all intenents and purposes it's an Ethernet cable between the two guests

And just for the record, I have just quickly deployed a Vyos as a test router to make sure the issue isn't VMWare related and the ping worked as expected, so there must be something else going on with the K9

What do you think?

Thanks for the help,

[MHM Cisco World]

As @Reza Sharifi mention 

Change port to trunk 

Interface x

Switchport mode trunk 

Switchport trunk allow vlan 10

Also make sure that SW use vlan1 as native vlan.

The vm dont understand un-tag frame so we need to make frame tag with trunk port 

MHM

[hguenfaf]

Hi MHM,

I'm not sure I understood your suggestion here, so please correct me if I'm wrong.

So the traffic is coming from/to the Linux VM which does not tag it in any way, through the vmware vswich (which also does not tag the traffic), through the 9K Eth1/1 port which as far as I know should be an "access" port for vlan 10 since only untagged vlan 10 traffic will be reaching it anyways. How is making Eth1/1 a trunk port going to help here ?

So I did it any ways, please find the the applied config below, 

interface Ethernet1/1
switchport
switchport mode trunk
switchport access vlan 10
switchport trunk allowed vlan 10
no shutdown

The interface is still not reachable through L3, I also lost ARP which makes sense since the interface is now passing tagged traffic to the vswich / Linux VM which will naturally be dropped,

Thanks,

[MHM Cisco World]

With access port and with trunk port 

Show mac address' check which vlan SW add for vm mac address.

This can give us hint which vlan vm use 

MHM

[hguenfaf]

Hi MHM,

Please find the output for both scenarios attached,

Edit: So when the interface is in trunk mode, the MAC address of the remote interface does not show in the mac address-table since the switch is expecting tagged traffic through Eth1/1. Whereas, in access mode it dynamically leans the mac address but for some odd reason it fails to communicate in L3.

Edit2: The only thing I'm suspecting right now is ARP, but I'm not sure what could be happening

[MHM Cisco World]

*   10     000c.29be.5c55   dynamic  0         F      F    Eth1/1

this meaning that the VM server send untag traffic and it add successfully to  MAC with vlan 10 

there is something else preventing the ping 
let me more check