06-25-2013 11:39 AM
Hi
We installed 1000v VSMs as VMs on a HP blade server with virtual connect modules.
It all worked fine and we migrated most of the VMs to 1000v. Even HA between the primary and backup VSMs worked.
Now we find that if you live migrate the active VSM is looses all contact with the VEMs but the backup VSM doesnt take over unless you power off the active one, The same thing happened when the customers Trend Deep Security Manager made soem update to the VSM VM.
We have the 1000v deployed in L3 mode using the control interface and the VEMs have their own vmkernel IP address which as it happens is on teh same IP subnet as the VSM control interface.
We have both the 1000v VSM control interface and managment interface configured as system VLANs
1000v version 4.2(1)sv2(2.1)ON VMware 5.1
Any ideas ? This kind of thing is usually a system VLAN issue.
Config info below.
Thanks
Pat
vlan 60
name mgmt <--- ssh to VSM etc
vlan 66
name VEM-mgmt <-- VSM to VEM
port-profile type vethernet n1kv_NEXUS_L3_CONTROL
capability l3control
vmware port-group
switchport mode access
switchport access vlan 66
no shutdown
system vlan 66
state enabled
port-profile type vethernet n1kv_VLAN60
vmware port-group
switchport mode access
switchport access vlan 60
no shutdown
system vlan 60
state enabled
port-profile type ethernet n1kv_UPLINK
vmware port-group
switchport mode trunk
switchport trunk allowed vlan 4,7,9,18,54,57-66,68,71,94-95
service-policy type queuing output uplink-policy-map
channel-group auto mode on mac-pinning relative
no shutdown
system vlan 60,66
max-ports 32
state enabled
svs-domain
domain id 10
control vlan 1
packet vlan 1
svs mode L3 interface control0
svs connection vcenter
protocol vmware-vim
remote ip address 172.30.60.10 port 80
vmware dvs uuid "f7 29 21 50 82 15 50 7f-52 20 2d 6d 89 f1 6c a7" datacenter-name Gatwick
admin user n1kUser
max-ports 30000
connect
The following are from when its working:
show svs dom
SVS domain config:
Domain id: 10
Control vlan: NA
Packet vlan: NA
L2/L3 Control mode: L3
L3 control interface: control0
Status: Config push to VC successful.
Control type multicast: No
show svs con
connection vcenter:
ip address: 172.30.60.10
remote port: 80
protocol: vmware-vim https
certificate: default
datacenter name: Gatwick
admin: n1kUser(user)
max-ports: 30000
DVS uuid: f7 29 21 50 82 15 50 7f-52 20 2d 6d 89 f1 6c a7
config status: Enabled
operational status: Connected
sync status: Complete
version: VMware vCenter Server 5.1.0 build-1064983
vc-uuid: 8C96DBA4-C1AA-490E-86E4-50B76DFA31AC
Solved! Go to Solution.
07-02-2013 10:13 AM
Yeah the problem is that any port-profile with "capability l3control" is meant to only work with vmk interfaces.
You never want to put a VM interface on port-profile with l3control. We used to allow this but for some reason it was pulled.
Create another port-profile just for control 0 and see if that fixes the problem. Again make it identical except for the "capability l3control"
Your version is good. I'll have to play in my lab. I'm kind of confused why your VSM2 gets hung at the loading prompt when you vmotion VSM1. That should not happen now.
louis
06-26-2013 06:33 AM
Patrick,
A couple of things to check.
When you vmotion the VSM can you run "show system internal redundancy trace" and see what mode you are in for HA. I believe you are in "degraded mode" which means that your VSMs are still talking to each other but over the mgmt interface. This is why the VEMs modules are not switching to the second VSM. Your control interface is dropping and not coming back. We need to figure out what is going on with your control 0 interface.
Which leads me to the second thing to check. Where is your control 0 interface connected to? is it connected to the VEM? or is it connected to a vSwitch. If its connected to the VEM which port-profile is it connected to?
louis
07-02-2013 02:37 AM
Hi
This is the profile the control interface should be using:
port-profile type vethernet n1kv_NEXUS_L3_CONTROL
capability l3control
vmware port-group
switchport mode access
switchport access vlan 66
no shutdown
system vlan 66
state enabled
I will get the customer to get teh other information (I am not at that site anymore).
Thanks
07-02-2013 04:53 AM
Here you go with the "show system internal redundancy trace" and some "show mod" sorry its a word doc but the customer pasted in screen caps rather than logging the terminal output.
The server is a HP C7000 blade server with HP Virtual Connect modules (hence the mac pinning) and the switches above that are Nexus 5548.
Thanks
Pat
07-02-2013 09:51 AM
Your control 0 interface on the VSM is using the n1k_NEXUS_L3_CONTROL port-profile?
If thats true, then it's a problem.
Any port-profile with "capability l3control" can only have vmk interfaces assigned to it. You'll need to create a new port-profile exactly like n1kv_NEXUS_L3_CONTROL but without "capability l3control"
What version are you running? The document shows behavior that I would expect with either 1.5 code.
louis
07-02-2013 09:59 AM
Louis
The only thing on VLAN 66 is the vmk interfaces for the VEMs and VSM control interface.
Are you saying that the VSM control interface will need its own port profile without " capabilty L3 control" ?
If so thats a good one. I would normally have had the VEM vmk interafces on a different subnet to the VSM interface (either control or management) so have never run into this one before as they would have had different port profiles.
Regards
Pat
NB It shoudl be 4.2(1)SV2(1.1a) but I will get them to check. It was supposed to be the new freemium 1000v as they only need basic features. I will be a bit cross if the engineer installed the old version that needs licensing.
07-02-2013 10:05 AM
Confirmed software version:
!Command: show running-config
!Time: Tue Jul 2 11:57:06 2013
version 4.2(1)SV2(1.1a)
svs switch edition essential
07-02-2013 10:13 AM
Yeah the problem is that any port-profile with "capability l3control" is meant to only work with vmk interfaces.
You never want to put a VM interface on port-profile with l3control. We used to allow this but for some reason it was pulled.
Create another port-profile just for control 0 and see if that fixes the problem. Again make it identical except for the "capability l3control"
Your version is good. I'll have to play in my lab. I'm kind of confused why your VSM2 gets hung at the loading prompt when you vmotion VSM1. That should not happen now.
louis
07-04-2013 12:53 AM
Louis
That did the trick. Both vMotion and Trend work fine now without causing loss of conectivity.
Thanks
Pat
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: