Cisco ucs-Esxi nic not working

[varokyan]

Hi All,

Iam haveing connectivity issue.We have esxi server 5.1 running on cisco ucs 6120xp firmware 2.28a

Esxi server 5.1 is haveing 2 nics .nic0 and nic1 .both are configured to use managment network,vmotion and also for vm data.But if i enable both nics0 &nic1 on esxi  server is not pinging .If i enable nic0 server is pinging .

Please advice what could be issue.All the vlan is allowed and there is no port channael configured.

Thanks,

victor

[Walter Dey]

Hi Victor

Can you ping the mgmt interface of your ESXi host from outside the UCS system ?

On the ESXi system: did you specify a Vlan or none; if none, the Vlan in which Mgt interface is, must be native.

2 vnics: is one attached to fabric A resp. B ?

How is the North bound connectivity ? 2 Ethernet switches ? are the trunk between the 2 configured for the all the necessary vlans (I assume you use the same vlans for fabric A and B).

[varokyan]

Hi Walter,

the esxi server has 2 nics .nic0 and nic1 .I can ping if i enable  only nic0 on esxi server from outside ucs.

nic0 is attached F-A and nic1 is attached to F-B. I tried doing failover to F-B as lead when  I do it nic1 will start to ping but not nic0.

yes all the vlan are alowed on both F-A & F-B

Thanks,

victor

[Walter Dey]

Hi Victor

The vnics are Ethernet interfaces, which I assume are attached to a vswitch, or are you using a DVS. Your IP address is assigned to a kernel port ? Can you post a screenshot how this looks like ?

Walter.

PS. You can ping the ESXi mgt port from the outside ? and between the 2 ESXi ?

[varokyan]

we are useing standard vswtich and ip is assinged to kernal port.yes i can ping esxi mgt port out side when only nic0 is enabled and nic1 is disabled.We have only one vswitch and nic0 and nic1 are teamed with this swtich..

thinking something wrong at UCS..If there is port block if i change lead to F-b why nic1 is pinging and why nic0 is not pinging......were as in F-A nic0 ping and nic1 not..

Not sure how to fix.

[Walter Dey]

Change lead to F-b doesn't have any impact; it doesn't matter if FI-A or -B is the lead switch.

Which Bonding / Teaming are you using on vswitch ?

This list is applicable for both the native operating system and for a hypervisor environment with virtual machines; see e.g. https://www.cisco.com/c/en/us/support/docs/servers-unified-computing/ucs-b-series-blade-servers/200519-UCS-B-series-Teaming-Bonding-Options-wi.html

Operating system	Supported	Not supported
VMWare ESXi	Route Based on Originating Port ID Route Based on Source MAC Hash	Route Based on IP Hash Route Based on Physical NIC Load

Does your ESXi see 2 path which are up and working, if not

I would disconnect the upstream cables from FI-A and then start to troubleshoot; e.g. does your ESXi see now 1 path ?

[varokyan]

Hi walter,

Fould that duplicate mac address on physical nic and vmknics.I deleted mac address on vmknics .After readding it vmknics took new system mac address.

But after reboot of esxi server it again mac address is confliect .the physiclal nic mac address is automaticaly assinged to vmknics.

Please let me know how to fix this.

Thanks,

Victor

[Wes Austin]

You can follow the instructions here.

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1031111

Duplicate MAC addresses

Confirm by checking the physical MAC address of all physical NICs and their link status, the existing vSwitch configuration, and the current vmkernel interfaces configuration. For ESXi, use these commands: 

# esxcfg-nics -l

# esxcfg-vswitch -l
# esxcfg-vmknic -l

If a duplicate MAC address is confirmed, the MAC address must be changed by deleting and recreating the vmkernel interface.

To delete a vmknic from a port group, use this command:

# esxcfg-vmknic -d -p pgName

or

# esxcfg-vmknic -d pgName

To add a vmknic to a port group, run the command:

# esxcfg-vmknic -a -i DHCP -p pgName

or

# esxcfg-vmknic -a -i x.x.x.x -n 255.255.255.0 pgName

Workaround 

To work around the issue, manually configure the MAC address on the ESXi host:

1. In the troubleshooting console, run the command:
   
   esxcfg-advcfg -s 1 /Net/FollowHardwareMac
   
   
2. Restart the ESXi server.

[varokyan]

i used  same steps mac address change .But after reboot again mac address conflicts

[Wes Austin]

If you delete the vmkernel mac address and it comes back with a 00:50 mac address from vmware, then upon reboot, the changes are reverted, you need to engage VMware and understand why the hypervisor is not holding onto this information.

I have never seen this behavior before.

[Walter Dey]

Can you check the relevant duplicate mac error on v-center and provide more information:

http://www.virtuallyghetto.com/2015/02/detecting-duplicate-vm-mac-address-using-vcenter-server-alarm.html

Chris Wahl has a in depth analysis of the problem:

http://wahlnetwork.com/2014/12/29/duplicate-mac-address/