Nexus 1000v VEM keeps dropping off

minghui.qi · ‎04-01-2011

Hi all,

I am having problem with N1k in the below topology:

Nexus 1010 VSA (running VSM) --- Cat 3750 --- Cat 6509 --- Nexus 5548 --- Nexus 2232 --- HP VC Flex-10 --- HP BL 490c G7 ESXi (running N1k VEM)

VSA: version 4.2(1)SP1(2)

VSM/VEM: version 4.2(1)SV1 (4)

VMware ESXi 4.1 U1 build-348481

blade adapter: NC553i CNA

HP VC Flex-10 version: 3.17

Issue:

When adding hosts into N1k vDS, from N1k, I can see new VEM module connected, but after about 15-20 seconds VEM module will drop off because of losing heartbeat to VEM. Host then becomes disconnected from network.

more info:

Flex-10 is configured in VLAN tunnelling mode, 2 ethernet networks are configured for each NIC onboard, supposed to pass all VLAN traffic untouched
Tried swapping Flex-10 with HP 10G Pass-thru module, N1k works perfectly, adding host without any issues.
The control vlan traffic seems being interupted somehow by Flex-10
VEM does not see unifcast traffic from VSM, hence drops off.
Also tried configuring VLAN mapping mode in Flex-10 using multiple networks, same issue.
for testing control vlan, tried temporarily assign an IP under interface control 0 in VSM, and creating a vmk interface in control vlan in ESXi. ping successful continously

is there anyone using same hardware? any issues?

Much appreciated

thanks

ming

Robert Burns · ‎04-03-2011

Ming,

At least you've isolated the issue to the Flex-10 module.

Please provide the following:

- Ethernet uplink port profile config from your VSM

- "ethtool -i vmnic[x]" for one of the CNA adapters. Ensure you're using driver versoin 2.102.525.9 for the NC533i

I'm also attaching a 1000v & Virtual Connect whitepaper so you can confirm your Virtual Connect settings are correct.

Beyond the basic, driver, firmware & configuration being validated, you'd want to chase up HP support next as PassThrough modules are unaffected.

Regards,

Robert

minghui.qi · ‎04-03-2011

hi Robert,

thanks a lot for your reply.

here is the uplink port profile config:

port-profile type ethernet HostUplink

vmware port-group

switchport mode trunk

switchport trunk native vlan 1

switchport trunk allowed vlan 1-3967,4048-4093

channel-group auto mode on mac-pinning

no shutdown

system vlan 803,832,900-901

state enabled

803 is mgmt for both esx and vsm, 900 is control, 901 is packet.

my NC553i driver version is different from what you suggested, it is 2.102.518.0, any reason why you suggest to use the version you specified?

I have been trying to contact HP support, to be honest, not saying overall, but from my experience, they are much less helpful then Cisco support, I will keep trying.

thanks

ming

Hongjun Ma · ‎04-09-2011

Hi Ming,

Did you get this working? If not, I have a couple of questions

1) How many flexNICs do you have per Flex-10 module going to host? what's the speed of each of them?

2) Did you try to shut down the uplink from Flex-10 in Interconnect Bay2 to N2K-2 to see if you can get VEM added just by one path?

Thanks

Hongjun

minghui.qi · ‎04-10-2011

Hi Hongjun,

Thanks for your reply,

single Flex-NIC, so 2 x 10G FlexNIC are presented to the host
havne't tried single path.

I manage to find a G6 blade and put it in the same chassis, and it worked without any issues, so it means there is no config problem on N1k and Flex-10. It is now with HP support to determine what is the difference between the G7 and G6 that is causing this issue.

Thanks

ming