UCS C210 M2 and Intel Quad-Port ethernet adapter

[nick.fritsch]

When trying to connect a Intel ET Quad-Port ethernet adapter (located in a C210 M2) to the network, I continue to lose connectivity to the network from the server. This is happening on 2 different servers with the same exact hardware specs. It almost seems like a firmware / driver incompatibility. Any help is appreciated.

Sent from Cisco Technical Support iPhone App

[padramas]

Nick,

You can check the driver / firmware / OS / CIMC compatibility matrix here

http://www.cisco.com/en/US/products/ps10477/prod_technical_reference_list.html

If it is in supported matrix and the problem exists, please provide us your setup information to further assist you.

HTH

Padma

[nick.fritsch]

I used the Cisco host udpate utility to update the driver / firmware on the Intel ET quad-port ethernet adapter, the LOM and the BIOS; none of which resolved the problem.  I used version 1.4(2).

The problem occurs after connecting 2 of the ports on the ethernet adapter to a Cisco 3560.  I have connectivity up to the point that I assign the 2 ports on the switch to a port-channel.  I've tried using etherchannel, passive and active mode channeling, none of which resolved the problem.  If I remove the ports on the switch from the channel-group, I do not regain connectivity at the switch level until I restart the server.

The OS running on the C210 M2 is VMware ESXi 5.0.  I have checked the VMware HCL and the Intel ET series quad-port card is supported for ESXi 5.0.

Here is the setup:

Inside of ESXi, I created a standard vSwitch that contains network adapters.  Both adapters are active on the vSwitch and I am using 'route based on IP hash' as the load-balancing method (since I am etherchanneling).

On the Cisco switch, I am using the following commands setup for each port connected to this vSwitch:

switchport access vlan 200 (vlan specified for ESXi 5.0 management network)

switchport mode acess

channel-group 2 mode on (also tried passive and active)

On the port-channel I have the same config:

switchport access vlan 200

switchport mode acess

Any help is appreciated.  Thank you.

[padramas]

Hello Nick,

How do you verify that server loses network connectivity ?

Output of following commands would help here

vmware -v

esxcfg-nics -l

ethool -i vmincX

Padma

[nick.fritsch]

Padramas,

I am pinging the IP address I assigned to the NFS port group from the switch. I receive a response all the way up to the point that I place the 2 ports I have designated for NFS in a etherchannel or LACP.

Also, the NFS datastores go inactive on the host.

Sent from Cisco Technical Support iPhone App

[padramas]

Nick,

Please open a TAC service request as we need more information.

Padma

[nick.fritsch]

Padma,

I have a case open with TAC however they only want to replace the Quad-Port ethernet adapter.  I don't believe that's the issue considering I'm having the same probelm on 2 differenct C210 M2 servers with the same Quad-Port NIC.  Therefore I tured to the communities to see if anyone else was having similar issues.

Thanks.

[Robert Burns]

Nick,

Since you're seeing this on another system (same behavior), the only other commonalities are the upstream switch, and your configuration within VMware. Have you tried connecting to another upstream switch (assuming you have one available)?

Since ESX doesn't support LACP, the only channelling method you should be using is "mode on".  LACP will fail as it will not negociate an etherchannel.  IP Hash is the correct swtich teaming mode on the ESX vSwitch side.

Please post the following outputs:

esxcfg-nics -l

ethtool -i vmnicX  (where X is one of the affected quad NIC ports)

I would agree with you this does not seem hardware related, but more driver or upstream switch related..

Regards,

Robert

[padramas]

Nick,

I have shared this post with the engineer. Since this is tracked via TAC SR, we will assist you in resolving this issue.

Padma

[Live2 Bicycle]

I had the same issue with all 4 of my C210 boxes.  It turned out to be a VMware bug.  Once I made these changes all the network issues went away.

Try disabling LRO:

http://docwiki.cisco.com/wiki/Disable_LRO

This has caused a lot of problems with TCP (which would explain the large amount of out of order packets and resends)

Give it a shot and let me know how it goes.

[nick.fritsch]

Hopewell,

This sounds more along the lines of what I was thinking, a bug somewhere either on the C210 on in VMware that is causing my host to drop network connectivity.  Quick question though, the disabling of LRO seems to be more of a virtual machine setting than a host setting.  Just to make sure we're on the same page, changing this setting on your ESXi host helped resolved connectivity issues between your host and the network?

Thanks.

[Live2 Bicycle]

Yes, you are correct that is more of a VMware setting.  I guess my issue was a little differenet then yours.  I am using the 4 port ethernet adapter for my virtual host running on the C210.  The two embeded nics is what I am using to connect via vSphere.

My scenrio:

I have 3 Communication Managers running on a C210.  When I went to upgrade them from 8.5 to 8.6 I had to install a small cop.sng file that was 80 megs.  It took over two hours and it still was not finished.  I took a packet capture and saw a lot of packets out of sequence and retransmits.  TAC found the bug and sent me the info I posted earlier.  Once I made the changes and restarted the 80 meg file took 3 seconds to upload.