ā04-18-2012 03:38 AM
Hello,
We are installing a N1K 4.2(1)SV1(5.1) and for this purpose installed VSM on a Seperate DataCenter Vsphere 5 Cluster onto a blank ESX host. Our problem is that VSM and VEM cannot see each other. What step may we missing?
Thanks in Advance,
Best Regards,
We followed the Installation and Getting started guides:
Cisco Nexus 1000V Installation and Upgrade Guide, Release 4.2(1)SV1(5.1)
from other Discussions we understand that something must be missing with respect to "Control Vlan" which will provide L2 communication between VSM and VEM.
show module vem mapping
Mod Status UUID License Status
--- ----------- ------------------------------------ --------------
3 absent 34343335-3237-4742-3838-346337417236 unlicensed
ā04-18-2012 05:39 AM
Can you provide the output of "show run" from the VSM.
Can you clarify the exact VC toplogy. Provide a screen shot if need be.
It's very likely this is a system vlan or other configuration related issue.
Thanks,
Robert
ā04-20-2012 06:21 AM
It seems pretty obvious that the control vlan is not set as a system vlan on your vethernet port profile as well as your uplink port profile.
You will need to purchase licenses for your modules as well as even if you able to get the module online you will not be able to pass traffic on Veth ports.
Please upload the "show run" as Rob requested and we will look at your configuration.
Joey
ā04-24-2012 02:48 PM
Hello,
I am having the exact same problem as Canero with the same software version. My setup is as follows:
1. L3 mode with VSM and VEM on different subnets/hosts
2. With the same set of port profiles and UCS service profile cloned, I have one host showing up in VSM and another not showing up. In my view, this rule out mis-configuration in
a. system vlan on veth port profile
b. vlan configuration in upstream switch and UCS fabric interconnect
3. Sometimes, the VEM did managed to show up but can drop without any reason
4. Sniffer trace indicates that there are bi-directional traffic between VSM and VEM
5. show logging logfile has the following error:
2012 Apr 24 15:21:47 VSM-PROD %VMS-5-DVS_HOSTMEMBER_INFO: A host with name=[172.16.105.131] and uuid=[a7f30c6a-6eb0-11e1-2000-000000000006] is added to the dvs. The host is not found as a module in the VSM configuration.
6. Opaque Data in VC shows the following:
"data-version 1.0
switch-domain 200
switch-name VSM-PROD
cp-version 4.2(1)SV1(5.1)
control-vlan 1
system-primary-mac 00:50:56:ab:44:01
active-vsm packet mac 00:50:56:ab:44:03
active-vsm mgmt mac 00:50:56:ab:44:02
standby-vsm ctrl mac 0050-56ab-4404
inband-vlan 1
svs-mode L3
l3control-ipaddr 192.168.1.200
upgrade state 0 mac 0050-56ab-4404 l3control-ipv4 null
profile dvportgroup-149 trunk 105-106,109,111
profile dvportgroup-149 mtu 1500
profile dvportgroup-151 access 105
profile dvportgroup-151 mtu 1500
profile dvportgroup-152 access 106
profile dvportgroup-152 mtu 1500
profile dvportgroup-155 access 111
profile dvportgroup-155 mtu 1500
profile dvportgroup-155 capability l3control
profile dvportgroup-161 access 109
profile dvportgroup-161 mtu 1500
end-version 1.0
"
In VSM:
VSM-PROD# sh svs domain
SVS domain config:
Domain id: 200
Control vlan: 1
Packet vlan: 1
L2/L3 Control mode: L3
L3 control interface: mgmt0
Status: Config push to VC successful.
VSM-PROD# sh mod vem missing
Mod Server-IP Server-UUID Server-Name
--- --------------- ------------------------------------ --------------------
4 172.16.105.31 a7f30c6a-6eb0-11e1-2000-000000000007 NA
5 172.16.105.131 a7f30c6a-6eb0-11e1-2000-000000000006 NA
VSM-PROD# sh mod vem map
Mod Status UUID License Status
--- ----------- ------------------------------------ --------------
3 powered-up a7f30c6a-6eb0-11e1-2000-000000000008 licensed
4 absent a7f30c6a-6eb0-11e1-2000-000000000007 n/a
5 absent a7f30c6a-6eb0-11e1-2000-000000000006 n/a
In host:
Below is "show run"
VSM-PROD# sh run
!Command: show running-config
!Time: Tue Apr 24 16:34:38 2012
version 4.2(1)SV1(5.1)
feature telnet
username admin password 5 $1$oCIIPBun$l0pIWvJ/OeQ3mNSY38OTG0 role network-admin
banner motd #Nexus 1000v Switch#
ssh key rsa 2048
ip domain-lookup
hostname VSM-PROD
vem 3
host vmware id a7f30c6a-6eb0-11e1-2000-000000000008
vem 4
host vmware id a7f30c6a-6eb0-11e1-2000-000000000007
vem 5
host vmware id a7f30c6a-6eb0-11e1-2000-000000000006
snmp-server user admin network-admin auth md5 0xd86407e6438a2a6179f6581e7d378d4b priv 0xd86407e6438a2a6179f6581
e7d378d4b localizedkey
vrf context management
ip route 0.0.0.0/0 192.168.1.1
vlan 1,100-103,105-106,109,111-112,200
vlan 1,100-103,105-106,109,111-112,200
port-channel load-balance ethernet source-mac
port-profile default max-ports 32
port-profile type ethernet Unused_Or_Quarantine_Uplink
vmware port-group
shutdown
description Port-group created for Nexus1000V internal usage. Do not use.
state enabled
port-profile type vethernet Unused_Or_Quarantine_Veth
vmware port-group
shutdown
description Port-group created for Nexus1000V internal usage. Do not use.
state enabled
port-profile type vethernet storage
vmware port-group
switchport mode access
switchport access vlan 109
no shutdown
system vlan 109
state enabled
port-profile type ethernet n1kv-uplink
vmware port-group
switchport mode trunk
switchport trunk allowed vlan 105-106,109,111-112
channel-group auto mode on mac-pinning
no shutdown
system vlan 105-106,109,111
state enabled
port-profile type vethernet vm
vmware port-group
switchport mode access
switchport access vlan 112
no shutdown
state enabled
port-profile type vethernet esx_mgmt
vmware port-group
switchport mode access
switchport access vlan 105
no shutdown
system vlan 105
state enabled
port-profile type vethernet esx_vmotion
vmware port-group
switchport mode access
switchport access vlan 106
no shutdown
system vlan 106
state enabled
port-profile type vethernet n1kv-L3-dmz
capability l3control
vmware port-group
switchport mode access
switchport access vlan 111
no shutdown
system vlan 111
state enabled
vdc VSM-PROD id 1
limit-resource vlan minimum 16 maximum 2049
limit-resource monitor-session minimum 0 maximum 2
limit-resource vrf minimum 16 maximum 8192
limit-resource port-channel minimum 0 maximum 768
limit-resource u4route-mem minimum 1 maximum 1
limit-resource u6route-mem minimum 1 maximum 1
limit-resource m4route-mem minimum 58 maximum 58
limit-resource m6route-mem minimum 8 maximum 8
interface port-channel1
inherit port-profile n1kv-uplink
vem 3
interface mgmt0
ip address 192.168.1.200/24
interface Vethernet1
inherit port-profile esx_mgmt
description VMware VMkernel, vmk0
vmware dvport 320 dvswitch uuid "a5 5d 2b 50 90 f4 f8 5a-7e 21 8b 8e f9 62 6e d2"
vmware vm mac 0025.B500.A008
interface Vethernet2
inherit port-profile n1kv-L3-dmz
description VMware VMkernel, vmk2
vmware dvport 480 dvswitch uuid "a5 5d 2b 50 90 f4 f8 5a-7e 21 8b 8e f9 62 6e d2"
vmware vm mac 0050.5672.4908
interface Vethernet4
inherit port-profile esx_vmotion
description VMware VMkernel, vmk3
vmware dvport 352 dvswitch uuid "a5 5d 2b 50 90 f4 f8 5a-7e 21 8b 8e f9 62 6e d2"
vmware vm mac 0050.567B.E4B6
interface Vethernet7
inherit port-profile storage
description VMware VMkernel, vmk1
vmware dvport 577 dvswitch uuid "a5 5d 2b 50 90 f4 f8 5a-7e 21 8b 8e f9 62 6e d2"
vmware vm mac 0050.567D.5E12
interface Ethernet3/3
inherit port-profile n1kv-uplink
interface Ethernet3/4
inherit port-profile n1kv-uplink
interface control0
line console
boot kickstart bootflash:/nexus-1000v-kickstart-mz.4.2.1.SV1.5.1.bin sup-1
boot system bootflash:/nexus-1000v-mz.4.2.1.SV1.5.1.bin sup-1
boot kickstart bootflash:/nexus-1000v-kickstart-mz.4.2.1.SV1.5.1.bin sup-2
boot system bootflash:/nexus-1000v-mz.4.2.1.SV1.5.1.bin sup-2
svs-domain
domain id 200
control vlan 1
packet vlan 1
svs mode L3 interface mgmt0
svs connection vcenter
protocol vmware-vim
remote ip address 192.168.1.252 port 80
vmware dvs uuid "a5 5d 2b 50 90 f4 f8 5a-7e 21 8b 8e f9 62 6e d2" datacenter-name Test-Servers
max-ports 8192
connect
vsn type vsg global
tcp state-checks
vnm-policy-agent
registration-ip 0.0.0.0
shared-secret **********
log-level
Also the DVS screen shot for this host:
ā04-28-2012 04:25 PM
At firt glance this all looks fine as far as configuration. Can you provide the following output from the problem host:
vemcmd show port
vemcmd show port vlans
vem version
vem status
I'm assuming the host is pingable the entire time?
Thanks,
Joey
ā04-30-2012 06:23 AM
Hi Joey,
Thanks for your response. The VSM eventually saw VEM after 3 hours, from the overnight log, there were a few add/remove events for the VEM. The host was pingable the entire time.
Unfortunately, we have to tear down the testbed for a totally different setup. I will update the thread if we see the same issue on this version of software.
Cheers,
Biao
ā05-04-2012 11:50 AM
Hello All,
The problem turned out to be related with the VSM-VEM Communication (which is broadcast by default in L2 mode) when the SYSTEM-UPLINK port profile traffic passed through the HP Flex-10 switches. This is how we did the testing:
So we used a SYSTEM-UPLINK port profile for only the control vlan, and a VM-UPLINK port profile for all VM Vlan
traffic, here are some useful commands that may be helpful
vemcmd show trunk
show port-profile virtual usage
show port-channel usage
show port-profile usage
show module vem counters
show module vem license-usage
show license usage Nexus1000v_PKG
Best Regards,
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide