VMs on Nexus 1000v not able to communicate?

PeterLB01 · ‎12-07-2013

Hello people. :-)

We've had problems with VMs on the Nexus 1000v not being able to communicate at times, when we move them to the standard switch, the problem disappears. We've stumbled upon some lines in the vmkernel.log on the ESXi host with the problem that might give us a hint, but we can't understand what they mean:

2013-12-03T06:45:27.675Z cpu62:8254)<3>nx_nic[vmnic3]: Bad Rcv descriptor ring

2013-12-03T06:45:27.763Z cpu62:8254)<3>nx_nic[vmnic3]: Bad Rcv descriptor ring

2013-12-03T06:45:27.783Z cpu62:8254)<3>nx_nic[vmnic3]: Bad Rcv descriptor ring

2013-12-03T06:45:27.951Z cpu63:8255)<3>nx_nic[vmnic3]: Bad Rcv descriptor ring

2013-12-03T06:45:27.988Z cpu58:8250)<3>nx_nic[vmnic3]: Got a buffer index:11f for Jumbo desc type. Max is 80

2013-12-03T06:45:27.991Z cpu58:8250)<3>nx_nic[vmnic3]: Got a buffer index:117 for Jumbo desc type. Max is 80

2013-12-04T12:29:56.114Z cpu44:8287)sf_netif_port_unreserve: DEBUG port 33554454-2000016 clientName is SRV7WWWMID002 ethernet0, unlicensed/headless VEM

2013-12-05T08:43:44.725Z cpu26:5272602)sf_netif_port_connect: Cannot set uplink tree capabilities, error returned was Not found

2013-12-06T08:59:58.554Z cpu5:8285)sf_netif_port_unreserve: DEBUG port 33554449-2000011 clientName is AV01_Prod.eth0, unlicensed/headless VEM

2013-12-06T09:02:10.196Z cpu56:8287)sf_netif_port_unreserve: DEBUG port 33554458-200001a clientName is SRV9DMZAPP002 ethernet0, unlicensed/headless VEM

Can you help us interpret it?

ESXi 5.0 Update 2.

VEM/VSM version 4.2(1)SV1(5.1)

/Pete

esettu · ‎12-08-2013

Hi Peter,

From the logs it seems VEM is either unlicensed or headless mode. Please check for the same using below command.

* show module ==> This should have your modules up and it should show status as "ok". You should not find any status as unlicensed.

Thanks & Regards,

Elan

sprasath · ‎12-09-2013

Hi Pete,

- You've mentioned that you move the VM to the standard switch to recover from this

- Do you move the VM back to the Nexus 1000v again?

- If yes, does it work this time?

- Please get the following in the problem state:

- vemcmd show card

- vemcmd show port

- vemcmd show port vlans

Thanks,

Shankar

PeterLB01 · ‎12-12-2013

Thanks for the input guys.

The log errors didn't relate to the problem we had. We had a Cisco technician look through our setup and logs and he didn't find anything. He suspects that we're running a too old version of VSM/VEM for the ESXi build we're running. Either that or a hardware driver failure. We're updating everything and then we'll see how it goes.

He recommended getting these logs if we experience the error again:

ESXi: vem-support all

Nexus 1000v: show tech-support svs

Take care

/Pete

sprasath · ‎12-12-2013

The 'unlicensed/headless VEM' message is actually a bit misleading. We corrected it in 4.2(1)SV2(1.1) through

CSCue41379.

Before the fix for this bug, the message would come up for any Veth interface programming error.

The conditions/workaround mentioned in the defect may not suit your scenario. The idea of the bug was to correct the misleading error message.

Thanks,

Shankar