VEM problem ? Dropping received frames - Page 2

born.jason · ‎03-30-2010

Hi,

since december we have the nexus 1000v in our company. Everything working fine and looks good. We have a ESX cluster with two hosts and 7 NICs uplink. Today after a long time i have a look in the logging and saw this:

Dropping received frames from duplicate VSM - kernel

How could i get rid of this? This message flooding the logging.

Here is the configuration.....

sh ver:

Software
loader:    version 1.2(2) [last: image booted through mgmt0]
kickstart: version 4.0(4)SV1(1)
system:    version 4.0(4)SV1(1)
kickstart image file is:
kickstart compile time: 4/2/2009 23:00:00
system image file is:    bootflash:/nexus-1000v-mz.4.0.4.SV1.1.bin
system compile time:     4/2/2009 23:00:00 [04/26/2009 13:27:23]

and here the uplink profiles and other profiles for service console and so on:

port-profile system-uplink
capability uplink
vmware port-group
switchport mode trunk
switchport trunk allowed vlan 2,51-52 (VLAN 2 is not in use at the moment)
channel-group auto mode on sub-group cdp
no shutdown
system vlan 51-52
state enabled

port-profile vm-uplink
capability uplink
vmware port-group
switchport mode trunk
switchport trunk allowed vlan 1,10,101-200
channel-group auto mode on sub-group cdp
no shutdown
state enabled

port-profile nexus-control
vmware port-group
switchport mode access
switchport access vlan 51
no shutdown
system vlan 51
state enabled
port-profile nexus-packet
vmware port-group
switchport mode access
switchport access vlan 52
no shutdown
system vlan 52
state enabled

port-profile ServiceConsole
vmware port-group
switchport mode trunk
switchport trunk allowed vlan 1-2 (maybe here ist the problem, this is configured from a guy before)
no shutdown
state enabled
port-profile VMotion
vmware port-group
switchport mode access
switchport access vlan 10
no shutdown
state enabled

We have vlan 1 as the defaultvlan for office network, service console belongs to vlan 1 (Yes i know this is not good but at this time there is no other choice :-( )

On the uplink switch a port-channel is created with five ports for vm-uplink and another port-channel with two nics for system-uplink, which connected with the vmnic`s.

If i have to reconfigure some points please let me know if i can make this changes while the system is running.....

born.jason · ‎04-04-2010

Yes ok, you`re right cdusio.

Have setup a new VSM as secondary after reboot of this machine and the machine is up i recieved on the other nexus 1000v (which is at the moment the only working VSM) the "Dropping received frames" message. But everything working fine.

On the new VSM there is no config or modules on it. So it not worked.

What do now?

@Robert, do you have also any suggestions?

cdusio · ‎04-04-2010

See last post... Just get the domain id info in there, save it, reboot...

born.jason · ‎04-04-2010

Ok, have now setup the new machine as secondary and it shows how you said only "switch" at the prompt.

Now the steps to go:

1. Make the working secondary to a primary with "system redundancy role primary"

Are the any other changes if i then reboot the switch? It important that the system is running fine after that.

2. On the new nexus 1000v "system redundancy role secondary"

3. On the new nexus 1000v make domain settings? Why, i thougt that it replicated from the new primary?

cdusio · ‎04-04-2010

Follow the steps I sent you. The domain id has to be in there for the new secondary to know it's part of a DVS with a domain id. This is how when it reboots, it knows to pull the config from VC and also important to make sure it's secondary before or you'll get those duplicate frame errors.

Sounds like you are almost there.

cdusio · ‎04-04-2010

And no there are no other steps on the sup you are making as the new primary, it's just a role change.

Should be fine.

born.jason · ‎04-04-2010

ok now i have a big problem, no connection to the service console and no connection to the nexus 1000v. Don`t know what do do.....

cdusio · ‎04-04-2010

ok hold on. I'll set up a meetingplace.

Version:1.0 StartHTML:0000000149 EndHTML:0000000952 StartFragment:0000000199 EndFragment:0000000918 StartSelection:0000000199 EndSelection:0000000918 To join the online meeting (Now from iPhones too!)
-------------------------------------------------------
1. Go to https://inxi-mp.webex.com/inxi-mp/e.php?AT=MI&EventID=133038152&UID=0&RT=MiMxMQ%3D%3D
2. Enter your name and email address.
3. Enter the meeting password: (This meeting does not require a password.)
4. Click "Join Now".
5. Follow the instructions that appear on your screen.

call my cell. 7745261699

born.jason · ‎04-04-2010

ok lets start a second webex i have send you an email and try to call you.

born.jason · ‎04-07-2010

Ok, VSMs are up and it looks good.

Mod Ports Module-Type                      Model              Status
--- ----- -------------------------------- ------------------ ------------
1    0      Virtual Supervisor Module        Nexus1000V         active *
2    0      Virtual Supervisor Module        Nexus1000V         ha-standby
3    248    Virtual Ethernet Module          NA                 ok
4    248    Virtual Ethernet Module          NA                 ok

Mod Sw               Hw
--- --------------- ------
1    4.0(4)SV1(2)     0.0
2    4.0(4)SV1(2)     0.0
3    4.0(4)SV1(1)     1.11
4    4.0(4)SV1(1)     1.11

In the vSphere client i see under networking

"Configuration Issues: An upgrade for the Distributed Virtual Switch in datacenter is in progress"

This is because i have initiated the command from the nexus.... but than i fall in trouble as you can see the messages above...

How can i now upgrade the VEMs one host by one host. First step is to set one esx host in maintance mode. What are the next steps?

@Robert

I have two esx hosts, in vSphere client both servers shows build 236512 but if i perform a vem version on both hosts see here

esx1:

Running esx version -208167 x86_64
VEM Version: 4.0.4.1.1.30-1.9.16
VSM Version: 4.0(4)SV1(2)
System Version: VMware ESX 4.0.0 Releasebuild-208167

esx2:

Running esx version -208167 x86_64
VEM Version: 4.0.4.1.1.31-1.11.11
VSM Version: 4.0(4)SV1(2)
System Version: VMware ESX 4.0.0 Releasebuild-236512

cdusio · ‎04-07-2010

Are you using VUM? If not, you have to manually install the software on the servers then once that is done, you will need to accept the upgrade in vcenter.

http://www.ciscosystems.com/en/US/docs/switches/datacenter/nexus1000/sw/4_0_4_s_v_1_2/upgrade/software/guide/n1000v_upgrade_software.html#wp57718

This is a step by step guide but first you will need to upgrade the software on the hosts.

http://www.cisco.com/en/US/docs/switches/datacenter/nexus1000/sw/4_0_4_s_v_1_2/install/vem/guide/n1000v_vem_install.html

This shows you how to upgrade the VEM's.

Robert Burns · ‎04-07-2010

Jason,

You're on the right track. Next grab the .vib that corresponds to your ESX build (you can obtain this by opening a browser to your VSM IP's.) The .vib you want should be "cross_cisco-vem-v110-4.0.4.1.2.0.80-1.9.179-release.vib". You see from the table in browser which .vib matches your build.

Transfer the .vib to your each ESX hosts one at a time, and install it:

esxupdate -b cross_cisco-vem-v110-4.0.4.1.2.0.80-1.9.179-release.vib update --maintenancemode

Once ALL VEM hosts have been upgraded they should show the new version from your VSM "show module".

When all your VEMs are done, you can "Accept" the upgrade in vCenter, where VUM will take no action as all your VEM hosts will already be updated. After this, go to your VSM and issue "vmware vem upgrade complete" the upgrade as documented in the Upgrade guide.

Last step - Raise the VEM feature level:

system update vem feature level 1

Let me know if you have any issues.

Regards,

Rob

born.jason · ‎07-06-2010

Trying to update the VEM with esxupdate -b cross_cisco-vem-v110-4.0.4.1.2.0.80-1.9.179-release.vib update --maintenancemode

Following error appears:

The following problems were encountered trying to resolve dependencies:
   No VIB provides 'vmknexus1kvapi-1-9' (required by cross_cisco-
   vem-v110-esx_4.0.4.1.2.0.80-1.9.179)
   Requested VIB cross_cisco-vem-v110-esx_4.0.4.1.2.0.80-1.9.179 conflicts with
   the host
   Requested VIB cross_cisco-vem-v110-esx_4.0.4.1.2.0.80-1.9.179 is obsoleted by
   the host

How could i fix this?

abbharga · ‎07-06-2010

Hi Jason,

What is the version of the ESX host, you can get that by using the command 'vmware -v' and for the VSM are you still on 1.2?

Abhinav

born.jason · ‎07-06-2010

Yes, VSM version is 1.2. See the above forum posts.

[root@]# vmware -v
VMware ESX 4.0.0 build-208167
[root@]# rpm -qa | grep vmkernel | awk -F. '{print $5}'
236512
[root@]#

born.jason · ‎07-06-2010

Have also some issues in VUM...