Nexus 1000v VEM module bouncing between hosts

netdood · ‎06-17-2013

I'm receiving these error messages on my N1KV and don't know how to fix it. I've tried removing, rebooting, reinstalling host B's VEM but that did not fix the issue. How do I debug this?

My setup,

Two physical hosts running esxi 5.1, vcenter appliance, n1kv with two system uplinks and two uplinks for iscsi for each host. Let me know if you need more output from logs or commands, thanks.

N1KV# 2013 Jun 17 18:18:07 N1KV %VEM_MGR-2-VEM_MGR_DETECTED: Host 192.168.52.100 detected as module 3

2013 Jun 17 18:18:07 N1KV %VEM_MGR-2-MOD_ONLINE: Module 3 is online

2013 Jun 17 18:18:08 N1KV %VEM_MGR-2-VEM_MGR_REMOVE_UNEXP_NODEID_REQ: Removing VEM 3 (Unexpected Node Id Request)

2013 Jun 17 18:18:09 N1KV %VEM_MGR-2-MOD_OFFLINE: Module 3 is offline

2013 Jun 17 18:18:13 N1KV %VEM_MGR-2-VEM_MGR_DETECTED: Host 192.168.51.100 detected as module 3

2013 Jun 17 18:18:13 N1KV %VEM_MGR-2-MOD_ONLINE: Module 3 is online

2013 Jun 17 18:18:16 N1KV %VEM_MGR-2-VEM_MGR_REMOVE_UNEXP_NODEID_REQ: Removing VEM 3 (Unexpected Node Id Request)

2013 Jun 17 18:18:17 N1KV %VEM_MGR-2-MOD_OFFLINE: Module 3 is offline

2013 Jun 17 18:18:21 N1KV %VEM_MGR-2-VEM_MGR_DETECTED: Host 192.168.52.100 detected as module 3

2013 Jun 17 18:18:21 N1KV %VEM_MGR-2-MOD_ONLINE: Module 3 is online

2013 Jun 17 18:18:22 N1KV %VEM_MGR-2-VEM_MGR_REMOVE_UNEXP_NODEID_REQ: Removing VEM 3 (Unexpected Node Id Request)

2013 Jun 17 18:18:23 N1KV %VEM_MGR-2-MOD_OFFLINE: Module 3 is offline

2013 Jun 17 18:18:28 N1KV %VEM_MGR-2-VEM_MGR_DETECTED: Host 192.168.51.100 detected as module 3

2013 Jun 17 18:18:29 N1KV %VEM_MGR-2-MOD_ONLINE: Module 3 is online

2013 Jun 17 18:18:44 N1KV %PLATFORM-2-MOD_DETECT: Module 2 detected (Serial number :unavailable) Module-Type Virtual Supervisor Module Model :unavailable

N1KV# sh module

Mod Ports Module-Type Model Status

--- ----- -------------------------------- ------------------ ------------

1 0 Virtual Supervisor Module Nexus1000V ha-standby

2 0 Virtual Supervisor Module Nexus1000V active *

3 248 Virtual Ethernet Module NA ok

Mod Sw Hw

--- ------------------ ------------------------------------------------

1 4.2(1)SV2(1.1a) 0.0

2 4.2(1)SV2(1.1a) 0.0

3 4.2(1)SV2(1.1a) VMware ESXi 5.1.0 Releasebuild-838463 (3.1)

Mod MAC-Address(es) Serial-Num

--- -------------------------------------- ----------

1 00-19-07-6c-5a-a8 to 00-19-07-6c-62-a8 NA

2 00-19-07-6c-5a-a8 to 00-19-07-6c-62-a8 NA

3 02-00-0c-00-03-00 to 02-00-0c-00-03-80 NA

Mod Server-IP Server-UUID Server-Name

--- --------------- ------------------------------------ --------------------

1 192.168.54.2 NA NA

2 192.168.54.2 NA NA

3 192.168.51.100 03000200-0400-0500-0006-000700080009 NA

* this terminal session

~ # vemcmd show card

Card UUID type 2: 03000200-0400-0500-0006-000700080009

Card name:

Switch name: N1KV

Switch alias: DvsPortset-1

Switch uuid: e6 dc 36 50 c0 a9 d9 a5-0b 98 fb 90 e1 fc 99 af

Card domain: 2

Card slot: 3

VEM Tunnel Mode: L3 Mode

L3 Ctrl Index: 49

L3 Ctrl VLAN: 51

VEM Control (AIPC) MAC: 00:02:3d:10:02:02

VEM Packet (Inband) MAC: 00:02:3d:20:02:02

VEM Control Agent (DPA) MAC: 00:02:3d:40:02:02

VEM SPAN MAC: 00:02:3d:30:02:02

Primary VSM MAC : 00:50:56:b6:0c:b2

Primary VSM PKT MAC : 00:50:56:b6:35:3f

Primary VSM MGMT MAC : 00:50:56:b6:d5:12

Standby VSM CTRL MAC : 00:50:56:b6:96:f2

Management IPv4 address: 192.168.51.100

Management IPv6 address: 0000:0000:0000:0000:0000:0000:0000:0000

Primary L3 Control IPv4 address: 192.168.54.2

Secondary VSM MAC : 00:00:00:00:00:00

Secondary L3 Control IPv4 address: 0.0.0.0

Upgrade : Default

Max physical ports: 32

Max virtual ports: 216

Card control VLAN: 1

Card packet VLAN: 1

Control type multicast: No

Card Headless Mode : No

Processors: 4

Processor Cores: 4

Processor Sockets: 1

Kernel Memory: 16669760

Port link-up delay: 5s

Global UUFB: DISABLED

Heartbeat Set: True

PC LB Algo: source-mac

Datapath portset event in progress : no

Licensed: Yes

~ # vemcmd show card

Card UUID type 2: 03000200-0400-0500-0006-000700080009

Card name:

Switch name: N1KV

Switch alias: DvsPortset-0

Switch uuid: e6 dc 36 50 c0 a9 d9 a5-0b 98 fb 90 e1 fc 99 af

Card domain: 2

Card slot: 3

VEM Tunnel Mode: L3 Mode

L3 Ctrl Index: 49

L3 Ctrl VLAN: 52

VEM Control (AIPC) MAC: 00:02:3d:10:02:02

VEM Packet (Inband) MAC: 00:02:3d:20:02:02

VEM Control Agent (DPA) MAC: 00:02:3d:40:02:02

VEM SPAN MAC: 00:02:3d:30:02:02

Primary VSM MAC : 00:50:56:b6:0c:b2

Primary VSM PKT MAC : 00:50:56:b6:35:3f

Primary VSM MGMT MAC : 00:50:56:b6:d5:12

Standby VSM CTRL MAC : 00:50:56:b6:96:f2

Management IPv4 address: 192.168.52.100

Management IPv6 address: 0000:0000:0000:0000:0000:0000:0000:0000

Primary L3 Control IPv4 address: 192.168.54.2

Secondary VSM MAC : 00:00:00:00:00:00

Secondary L3 Control IPv4 address: 0.0.0.0

Upgrade : Default

Max physical ports: 32

Max virtual ports: 216

Card control VLAN: 1

Card packet VLAN: 1

Control type multicast: No

Card Headless Mode : Yes

Processors: 4

Processor Cores: 4

Processor Sockets: 1

Kernel Memory: 16669764

Port link-up delay: 5s

Global UUFB: DISABLED

Heartbeat Set: False

PC LB Algo: source-mac

Datapath portset event in progress : no

Licensed: Yes

! ports 1-6 connected to physical host A

interface GigabitEthernet1/0/1

description VMWARE ESXi Trunk

switchport trunk encapsulation dot1q

switchport mode trunk

switchport nonegotiate

spanning-tree portfast trunk

spanning-tree bpdufilter enable

spanning-tree bpduguard enable

channel-group 1 mode active

!

! ports 7-12 connected to phys host B

!

interface GigabitEthernet1/0/7

description VMWARE ESXi Trunk

switchport trunk encapsulation dot1q

switchport mode trunk

switchport nonegotiate

spanning-tree portfast trunk

spanning-tree bpdufilter enable

spanning-tree bpduguard enable

channel-group 2 mode active

sprasath · ‎06-17-2013

Hi Tim,

- How/when did you land in this situation?

- You've mentioned two hosts running ESX 5.1, but there is only one present here.

- Is the other added to the N1k?

- Get the following from the VSM:

- show vms internal info dvs

- show svs neighbors

Thanks,

Shankar

netdood · ‎06-18-2013

Hi Shankar,

The two vemcmds are from two different hosts, you can see the switch alias and L3 control vlan are slightly different. A lot of the other information is the same.

I deleted both N1KV vm's and uninstalled the VEMs. I'm trying to reinstall again. I'll reply here if I run into it again.

sprasath · ‎06-18-2013

Hi Tim,

The L3 control VLAN depends on what you chose in the port-profile that you assign to your L3-capable vmknic. I didn't realise that they're from two different hosts. The UUID is the same and that is the reason for the error in your case.

You need to make sure that the UUID is different.

Thanks,

Shankar

netdood · ‎06-18-2013

ok after deleteing the n1kv vms and vcenter and then reinstalling all I got the error again,

N1KV# 2013 Jun 18 17:48:12 N1KV %VEM_MGR-2-VEM_MGR_REMOVE_STATE_CONFLICT: Removing VEM 3 due to state conflict VSM(NodeId Processed), VEM(ModIns End Rcvd)

2013 Jun 18 17:48:13 N1KV %VEM_MGR-2-MOD_OFFLINE: Module 3 is offline

2013 Jun 18 17:48:16 N1KV %VEM_MGR-2-VEM_MGR_DETECTED: Host 192.168.52.100 detected as module 3

2013 Jun 18 17:48:16 N1KV %VEM_MGR-2-MOD_ONLINE: Module 3 is online

2013 Jun 18 17:48:22 N1KV %VEM_MGR-2-VEM_MGR_REMOVE_STATE_CONFLICT: Removing VEM 3 due to state conflict VSM(NodeId Processed), VEM(ModIns End Rcvd)

2013 Jun 18 17:48:23 N1KV %VEM_MGR-2-MOD_OFFLINE: Module 3 is offline

2013 Jun 18 17:48:34 N1KV %VEM_MGR-2-VEM_MGR_DETECTED: Host 192.168.52.100 detected as module 3

2013 Jun 18 17:48:34 N1KV %VEM_MGR-2-MOD_ONLINE: Module 3 is online

2013 Jun 18 17:48:41 N1KV %VEM_MGR-2-VEM_MGR_REMOVE_STATE_CONFLICT: Removing VEM 3 due to state conflict VSM(NodeId Processed), VEM(ModIns End Rcvd)

2013 Jun 18 17:48:42 N1KV %VEM_MGR-2-MOD_OFFLINE: Module 3 is offline

2013 Jun 18 17:49:03 N1KV %VEM_MGR-2-VEM_MGR_DETECTED: Host 192.168.52.100 detected as module 3

2013 Jun 18 17:49:03 N1KV %VEM_MGR-2-MOD_ONLINE: Module 3 is online

2013 Jun 18 17:49:10 N1KV %VEM_MGR-2-VEM_MGR_REMOVE_STATE_CONFLICT: Removing VEM 3 due to state conflict VSM(NodeId Processed), VEM(ModIns End Rcvd)

2013 Jun 18 17:49:11 N1KV %VEM_MGR-2-MOD_OFFLINE: Module 3 is offline

2013 Jun 18 17:49:29 N1KV %VEM_MGR-2-VEM_MGR_DETECTED: Host 192.168.51.100 detected as module 3

2013 Jun 18 17:49:29 N1KV %VEM_MGR-2-MOD_ONLINE: Module 3 is online

2013 Jun 18 17:49:35 N1KV %VEM_MGR-2-VEM_MGR_REMOVE_STATE_CONFLICT: Removing VEM 3 due to state conflict VSM(NodeId Processed), VEM(ModIns End Rcvd)

2013 Jun 18 17:49:36 N1KV %VEM_MGR-2-MOD_OFFLINE: Module 3 is offline

2013 Jun 18 17:49:53 N1KV %VEM_MGR-2-VEM_MGR_DETECTED: Host 192.168.51.100 detected as module 3

2013 Jun 18 17:49:53 N1KV %VEM_MGR-2-MOD_ONLINE: Module 3 is online

2013 Jun 18 17:49:59 N1KV %VEM_MGR-2-VEM_MGR_REMOVE_STATE_CONFLICT: Removing VEM 3 due to state conflict VSM(NodeId Processed), VEM(ModIns End Rcvd)

2013 Jun 18 17:50:00 N1KV %VEM_MGR-2-MOD_OFFLINE: Module 3 is offline

2013 Jun 18 17:50:05 N1KV %VEM_MGR-2-VEM_MGR_DETECTED: Host 192.168.52.100 detected as module 3

2013 Jun 18 17:50:05 N1KV %VEM_MGR-2-MOD_ONLINE: Module 3 is online

Host A

~ # vemcmd show card

Card UUID type 2: 03000200-0400-0500-0006-000700080009

Card name:

Switch name: N1KV

Switch alias: DvsPortset-0

Switch uuid: e6 dc 36 50 c0 a9 d9 a5-0b 98 fb 90 e1 fc 99 af

Card domain: 2

Card slot: 1

VEM Tunnel Mode: L3 Mode

L3 Ctrl Index: 49

L3 Ctrl VLAN: 52

VEM Control (AIPC) MAC: 00:02:3d:10:02:00

VEM Packet (Inband) MAC: 00:02:3d:20:02:00

VEM Control Agent (DPA) MAC: 00:02:3d:40:02:00

VEM SPAN MAC: 00:02:3d:30:02:00

Primary VSM MAC : 00:50:56:b6:96:f2

Primary VSM PKT MAC : 00:50:56:b6:11:b6

Primary VSM MGMT MAC : 00:50:56:b6:48:c6

Standby VSM CTRL MAC : ff:ff:ff:ff:ff:ff

Management IPv4 address: 192.168.52.100

Management IPv6 address: 0000:0000:0000:0000:0000:0000:0000:0000

Primary L3 Control IPv4 address: 192.168.54.2

Secondary VSM MAC : 00:00:00:00:00:00

Secondary L3 Control IPv4 address: 0.0.0.0

Upgrade : Default

Max physical ports: 32

Max virtual ports: 216

Card control VLAN: 1

Card packet VLAN: 1

Control type multicast: No

Card Headless Mode : Yes

Processors: 4

Processor Cores: 4

Processor Sockets: 1

Kernel Memory: 16669764

Port link-up delay: 5s

Global UUFB: DISABLED

Heartbeat Set: False

PC LB Algo: source-mac

Datapath portset event in progress : no

Licensed: No

Host B

~ # vemcmd show card

Card UUID type 2: 03000200-0400-0500-0006-000700080009

Card name:

Switch name: N1KV

Switch alias: DvsPortset-0

Switch uuid: bf fb 28 50 1b 26 dd ae-05 bd 4e 48 2e 37 56 f3

Card domain: 2

Card slot: 3

VEM Tunnel Mode: L3 Mode

L3 Ctrl Index: 49

L3 Ctrl VLAN: 51

VEM Control (AIPC) MAC: 00:02:3d:10:02:02

VEM Packet (Inband) MAC: 00:02:3d:20:02:02

VEM Control Agent (DPA) MAC: 00:02:3d:40:02:02

VEM SPAN MAC: 00:02:3d:30:02:02

Primary VSM MAC : 00:50:56:a8:f5:f0

Primary VSM PKT MAC : 00:50:56:a8:3c:62

Primary VSM MGMT MAC : 00:50:56:a8:b4:a4

Standby VSM CTRL MAC : 00:50:56:a8:30:d5

Management IPv4 address: 192.168.51.100

Management IPv6 address: 0000:0000:0000:0000:0000:0000:0000:0000

Primary L3 Control IPv4 address: 192.168.54.2

Secondary VSM MAC : 00:00:00:00:00:00

Secondary L3 Control IPv4 address: 0.0.0.0

Upgrade : Default

Max physical ports: 32

Max virtual ports: 216

Card control VLAN: 1

Card packet VLAN: 1

Control type multicast: No

Card Headless Mode : No

Processors: 4

Processor Cores: 4

Processor Sockets: 1

Kernel Memory: 16669760

Port link-up delay: 5s

Global UUFB: DISABLED

Heartbeat Set: True

PC LB Algo: source-mac

Datapath portset event in progress : no

Licensed: Yes

I used the nexus 1000v java installer so I don't know what it keeps assigning the same UUID nor do I know how to change it.

Here is the other output you requested,

N1KV# show vms internal info dvs

DVS INFO:

-------------

DVS name: [N1KV]

UUID: [bf fb 28 50 1b 26 dd ae-05 bd 4e 48 2e 37 56 f3]

Description: [(null)]

Config version: [1]

Max ports: [8192]

DC name: [Galaxy]

OPQ data: size [1121], data: [data-version 1.0

switch-domain 2

switch-name N1KV

cp-version 4.2(1)SV2(1.1a)

control-vlan 1

system-primary-mac 00:50:56:a8:f5:f0

active-vsm packet mac 00:50:56:a8:3c:62

active-vsm mgmt mac 00:50:56:a8:b4:a4

standby-vsm ctrl mac 0050-56a8-30d5

inband-vlan 1

svs-mode L3

l3control-ipaddr 192.168.54.2

upgrade state 0 mac 0050-56a8-30d5 l3control-ipv4 null

cntl-type-mcast 0

profile dvportgroup-26 trunk 1,51-57,110

profile dvportgroup-26 mtu 9000

profile dvportgroup-27 access 51

profile dvportgroup-27 mtu 1500

profile dvportgroup-27 capability l3control

profile dvportgroup-28 access 52

profile dvportgroup-28 mtu 1500

profile dvportgroup-28 capability l3control

profile dvportgroup-29 access 53

profile dvportgroup-29 mtu 1500

profile dvportgroup-30 access 54

profile dvportgroup-30 mtu 1500

profile dvportgroup-31 access 55

profile dvportgroup-31 mtu 1500

profile dvportgroup-32 access 56

profile dvportgroup-32 mtu 1500

profile dvportgroup-34 trunk 220

profile dvportgroup-34 mtu 9000

profile dvportgroup-35 access 220

profile dvportgroup-35 mtu 1500

profile dvportgroup-35 capability iscsi-multipath

end-version 1.0

]

push_opq_data flag: [1]

show svs neighbors

Active Domain ID: 2

AIPC Interface MAC: 0050-56a8-f5f0

Inband Interface MAC: 0050-56a8-3c62

Src MAC Type Domain-id Node-id Last learnt (Sec. ago)

------------------------------------------------------------------------

0050-56a8-30d5 VSM 2 0201 1020.45

0002-3d40-0202 VEM 2 0302 1.33

I cannot add Host A to the N1KV it errors out with,

vDS operation failed on host 192.168.52.100, An error occurred during host configuration. got (vim.fault.PlatformConfigFault) exception

Host B (192.168.51.100) was added fine, then I moved a vmkernel to the N1KV which brought up the VEM and got the VEM flapping errors.

sprasath · ‎06-19-2013

Tim,

N1k just picks up the UUID from the server hardware. You can check it from the ESX CLI:

esxcfg-info | grep "BIOS UUID"

What server are you using? Unless this problem is cleared, you'll not be able to add both the hosts at the same time.

Thanks,

Shankar

netdood · ‎06-19-2013

Two shuttle boxes with i5 cpu.

192.168.51.100 shuttle

~ # esxcfg-info | grep "BIOS UUID"

|----BIOS UUID................................................0x3 0x0 0x2 0x0 0x4 0x0 0x5 0x0 0x0 0x6 0x0 0x7 0x0 0x8 0x0 0x9

|----World Command Line.................................grep BIOS UUID

192.168.52.100 shuttle

~ # esxcfg-info | grep "BIOS UUID"

|----BIOS UUID................................................0x3 0x0 0x2 0x0 0x4 0x0 0x5 0x0 0x0 0x6 0x0 0x7 0x0 0x8 0x0 0x9

|----World Command Line.................................grep BIOS UUID

The UUID looks exactly the same. Definitely two different physical boxes.

netdood · ‎06-19-2013

I've found a manual fix, though I think it's an installation issue/bug. If the hardware is not vmware approved it seems to use a generic uuid of 03000200-0400-0500-0006-000700080009.

Here is the vmware doc about generic uuid problems,

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006250

UUID from unrecognized hardware vendor approved list for esxi

https://communities.cisco.com/message/10271

So I manually changed the uuid on one of my esxi hosts

Check the uuid of the esxi host by going to https://YOUR_ESXI_HOST_IP/mob/?moid=ha-host&doPath=hardware%2esystemInfo

Or this command,

~ # vemcmd show card

Card UUID type 2: 03000200-0400-0500-0006-000700080009

Card name:

Switch name: N1KV

Switch alias: DvsPortset-0

Switch uuid: 12 93 1d 50 54 e8 53 1d-65 20 91 82 12 5f 53 79

Card domain: 2

Card slot: 3

VEM Tunnel Mode: L3 Mode

L3 Ctrl Index: 49

L3 Ctrl VLAN: 52

VEM Control (AIPC) MAC: 00:02:3d:10:02:02

VEM Packet (Inband) MAC: 00:02:3d:20:02:02

VEM Control Agent (DPA) MAC: 00:02:3d:40:02:02

VEM SPAN MAC: 00:02:3d:30:02:02

Primary VSM MAC : 00:50:56:9d:ee:3f

Primary VSM PKT MAC : 00:50:56:9d:d8:08

Primary VSM MGMT MAC : 00:50:56:9d:cd:cf

Standby VSM CTRL MAC : 00:50:56:9d:18:18

Management IPv4 address: 192.168.52.100

Management IPv6 address: 0000:0000:0000:0000:0000:0000:0000:0000

Primary L3 Control IPv4 address: 192.168.54.2

Secondary VSM MAC : 00:00:00:00:00:00

Secondary L3 Control IPv4 address: 0.0.0.0

Upgrade : Default

Max physical ports: 32

Max virtual ports: 216

Card control VLAN: 1

Card packet VLAN: 1

Control type multicast: No

Card Headless Mode : No

Processors: 4

Processor Cores: 4

Processor Sockets: 1

Kernel Memory: 16669764

Port link-up delay: 5s

Global UUFB: DISABLED

Heartbeat Set: True

PC LB Algo: source-mac

Datapath portset event in progress : no

Licensed: Yes

ENTER THIS COMMAND ON PHYSICAL ESXi server

~ # vemcmd card uuid vmware 03000200-0400-0500-0006-000700080099

~ #

~ # vemcmd show card

Card UUID type 2: 03000200-0400-0500-0006-000700080099

Card name:

Switch name: N1KV

Switch alias: DvsPortset-0

Switch uuid: 12 93 1d 50 54 e8 53 1d-65 20 91 82 12 5f 53 79

Card domain: 2

Card slot: 4

VEM Tunnel Mode: L3 Mode

L3 Ctrl Index: 49

L3 Ctrl VLAN: 52

VEM Control (AIPC) MAC: 00:02:3d:10:02:03

VEM Packet (Inband) MAC: 00:02:3d:20:02:03

VEM Control Agent (DPA) MAC: 00:02:3d:40:02:03

VEM SPAN MAC: 00:02:3d:30:02:03

Primary VSM MAC : 00:50:56:9d:ee:3f

Primary VSM PKT MAC : 00:50:56:9d:d8:08

Primary VSM MGMT MAC : 00:50:56:9d:cd:cf

Standby VSM CTRL MAC : 00:50:56:9d:18:18

Management IPv4 address: 192.168.52.100

Management IPv6 address: 0000:0000:0000:0000:0000:0000:0000:0000

Primary L3 Control IPv4 address: 192.168.54.2

Secondary VSM MAC : 00:00:00:00:00:00

Secondary L3 Control IPv4 address: 0.0.0.0

Upgrade : Default

Max physical ports: 32

Max virtual ports: 216

Card control VLAN: 1

Card packet VLAN: 1

Control type multicast: No

Card Headless Mode : No

Processors: 4

Processor Cores: 4

Processor Sockets: 1

Kernel Memory: 16669764

Port link-up delay: 5s

Global UUFB: DISABLED

Heartbeat Set: True

PC LB Algo: source-mac

Datapath portset event in progress : no

Licensed: Yes

confirm by show run,

hostname N1KV

errdisable recovery cause failed-port-state

vem 3

host vmware id 03000200-0400-0500-0006-000700080009

vem 4

host vmware id 03000200-0400-0500-0006-000700080099

and show module,

N1KV(config)# sh module

Mod Ports Module-Type Model Status

--- ----- -------------------------------- ------------------ ------------

1 0 Virtual Supervisor Module Nexus1000V active *

2 0 Virtual Supervisor Module Nexus1000V ha-standby

3 248 Virtual Ethernet Module NA ok

4 248 Virtual Ethernet Module NA ok

Mod Sw Hw

--- ------------------ ------------------------------------------------

1 4.2(1)SV2(1.1a) 0.0

2 4.2(1)SV2(1.1a) 0.0

3 4.2(1)SV2(1.1a) VMware ESXi 5.1.0 Releasebuild-838463 (3.1)

4 4.2(1)SV2(1.1a) VMware ESXi 5.1.0 Releasebuild-838463 (3.1)

Mod MAC-Address(es) Serial-Num

--- -------------------------------------- ----------

1 00-19-07-6c-5a-a8 to 00-19-07-6c-62-a8 NA

2 00-19-07-6c-5a-a8 to 00-19-07-6c-62-a8 NA

3 02-00-0c-00-03-00 to 02-00-0c-00-03-80 NA

4 02-00-0c-00-04-00 to 02-00-0c-00-04-80 NA

Mod Server-IP Server-UUID Server-Name

--- --------------- ------------------------------------ --------------------

1 192.168.54.2 NA NA

2 192.168.54.2 NA NA

3 192.168.51.100 03000200-0400-0500-0006-000700080009 192.168.51.100

4 192.168.52.100 03000200-0400-0500-0006-000700080099 NA

* this terminal session

sprasath · ‎06-19-2013

Ok. N1k and ESX take the UUID from the BIOS. I don't think it is a bug. It is similar to the 169.. address that you get when a NIC in Windows does not get an IP address from DHCP.

Either way, did you have to reload the server? And what is the UUID reported at the ESX level (

esxcfg-info | grep "BIOS UUID")

Thanks,

Shankar

netdood · ‎06-20-2013

No I did not have to reload the server to get n1kv to recognize the different uuid.

If you are referring to after the manual UUID change, the UUID is still the same at the ESXi level,

Host A

~ # esxcfg-info | grep "BIOS UUID"

|----BIOS UUID................................................0x3 0x0 0x2 0x0 0x4 0x0 0x5 0x0 0x0 0x6 0x0 0x7 0x0 0x8 0x0 0x9

|----World Command Line.................................grep BIOS UUID

Host B

~ # esxcfg-info | grep "BIOS UUID"

|----BIOS UUID................................................0x3 0x0 0x2 0x0 0x4 0x0 0x5 0x0 0x0 0x6 0x0 0x7 0x0 0x8 0x0 0x9

|----World Command Line.................................grep BIOS UUID

netdood · ‎06-20-2013

Well the UUID did not survive a reboot. I tried modify vssnet-functions but there is some weird permissions plus overwriting or restoring that I don't understand with that whole directory.

In vssnet-functions you can try to manually have it set the UUID,

https://supportforums.cisco.com/thread/2166139

#Usage: setBiosUuid

#

# Gets the hosts's BIOS UUID and sets the card UUID

#

setBiosUuid()

{

local UUID

# UUID=$(esxcfg-info -u | awk '{print tolower($1)}')

### vemcmd card uuid vmware 03000200-0400-0500-0006-000700080099

UUID="03000200-0400-0500-0006-000700080099"

if [ "${UUID}" != "" ] ; then

doCommand ${VEMCMD} card uuid vmware ${UUID}

fi

}

#Usage: setServerUuid

#

# Set the card UUID to the Host's Server UUID

#

setServerUuid()

{

local UUID

UUID=$(grep '^/system/uuid' ${VMWARECONF} | \

sed -e 's/^.* = "//' -e 's/".*$//')

if [ "${UUID}" != "" ] ; then

doCommand ${VEMCMD} card uuid vmware ${UUID}

fi

}

It would not let me write changes to vssnet-functions even if I put 777 permissions on the file. I was able to copy that file to another, make my edits and then copy it back to the original file but those changes didn't survive a reboot and the system mysteriously reverts the changes, deletes my copy of the file, and sets the permissions back to the original.

/opt/cisco/v152/nexus/vem-v152/shell # ls

vem-install-functions vem-preuninstall vssnet-functions

vem-postinstall vsi-functions vssnet-functions.orig

/opt/cisco/v152/nexus/vem-v152/shell # ll

-sh: ll: not found

/opt/cisco/v152/nexus/vem-v152/shell # ls -all

drwxr-xr-x 1 root root 512 Jun 20 18:20 .

drwxr-xr-x 1 root root 512 Jun 20 17:59 ..

-r-xr--r-- 1 root root 3428 Jan 21 10:26 vem-install-functions

-r-xr--r-- 1 root root 718 Jan 21 10:26 vem-postinstall

-r-xr--r-- 1 root root 1001 Jan 21 10:26 vem-preuninstall

-r-xr--r-- 1 root root 36517 Jan 21 10:26 vsi-functions

-rwxrwxrwx 1 root root 40529 Jan 21 10:26 vssnet-functions

-rwxr-xr-x 1 root root 40575 Jun 20 18:28 vssnet-functions.orig

/opt/cisco/v152/nexus/vem-v152/shell # cp vssnet-functions.orig vssnet-functions

/opt/cisco/v152/nexus/vem-v152/shell #

/opt/cisco/v152/nexus/vem-v152/shell # chmod 644 vssnet-functions

/opt/cisco/v152/nexus/vem-v152/shell # chmod 644 vssnet-functions.orig

/opt/cisco/v152/nexus/vem-v152/shell # reboot

Now directory is cleaned out

/opt/cisco/v152/nexus/vem-v152/shell # ls -all

drwxr-xr-x 1 root root 512 Jun 20 18:39 .

drwxr-xr-x 1 root root 512 Jun 20 18:39 ..

-r-xr--r-- 1 root root 3428 Jan 21 10:26 vem-install-functions

-r-xr--r-- 1 root root 718 Jan 21 10:26 vem-postinstall

-r-xr--r-- 1 root root 1001 Jan 21 10:26 vem-preuninstall

-r-xr--r-- 1 root root 36517 Jan 21 10:26 vsi-functions

-r-xr--r-- 1 root root 40529 Jan 21 10:26 vssnet-functions

sprasath · ‎06-20-2013

Hi Tim,

What you're doing is not recommended/supported. You should look at changing the UUID using a proper procedure, if any, from the hardware perspective.

Changing it from the Nexus 1000v level (using vemcmd, vssnet) are only temporary workarounds, as you've seen.

Thanks,

Shankar