Server MAC Address being replaced to a weird entry

[fassilva]

Hi,

I'm facing a weird situation in a LAN which Catalyst 9410 is the core of the network, I have a few ports in a Ether-Channel and trunk to two VMware ESX Hosts.

Also, the servers vlan have 2 IPs (primary and secondary) due a legacy devices. What I'm facing is which randomly a server can reach other servers in the same vlan (also in the different ESX hosts) but not the gateway of this vlan IP, which is c9410. For example, some servers can't reach the primary IP, while other can't reach the secondary IP. I can solve this changing the gateway's IP of the servers to the primary or secondary IP and sometimes creating a static ARP entry into c9410.

In a few times during this issue, I was able to see which few servers had their ARP changed from the VMware MAC 00:50:56 to CC:20:D9:F1:11:11, which seems to be the same MAC for different servers during this issue. But this MAC address isn't found in mac-vendors.com, IANA or any other MAC list.

So, did someone faced the same kind of situation? I have no idea what to search for anymore.

Thanks

Fernando

[QuinnBowman]

Could that be the MAC address belonging to one of the Virtual Machines running on that host?

I've seen this happen before when all traffic (Host management, VM Data, etc) is coming out the same NIC (or Etherchannel)
The Server (host) and the VM end up haphazardly sort of taking turns being the device the switch thinks is on the port.

[fassilva]

Hi,

I already checked all VMs MAC address configured into vmx files, which are all correct. But I have all traffic using this Etherchannel, when you saw this there is a MAC address change in the arp table also?

Thanks!

[fassilva]

Hi,

New information, I faced this today in a Physical server too.. I'm out of ideas to Troubleshoot that..

Any Guidance will be really appreciate.

Thanks you so much folks!

Regards,
Fernando