error adding host into 1000v

[Kassim Ismail]

Hi All,

I am currently running version Nexus 1000V Version 4.2.1.SV1(4) on our site & had 4 hosts connected correctly & licensed. for some reason yesterday one of the hosts "dropped" off from the nexus 1000v. when we check the networking screen on the host, the dvs entry was blank?  i ran vem status on the host & it said vem not found. when i pressed alt F11 on the host, it said that the host had been rolled back to a previous image (not sure how that happened, as i certainly did not do that)

anyway, when trying to add the host back into the nexus, i get the following error:

"vDS operation failed on host hostnamexxxx, An error occurred during host configuration. got (vim.fault.PlatformConfigFault) exception"

part of adding the host works in the sense that the vem is installed via update manager & when i run vem status it say that the vem agent is running.

I would appreciate some pointers on why its failing.

Thanks

[Robert Burns]

Hey Kassim,

First & foremost, you should investigate your VMkernel logs as to "why" your host rolled back.  It must have rolled back your VEM agent out of the host also. I can't say I've ever seen this happen by itself.  The esxupdate DB might also be out of sync.

Also make sure the correct version of VEM is indeed running.  Sounds like the host & vCenter might be out of sync.  If I were you, I would completely re-install the VEM on this hist, and remove the DVS config from vCenter and the VSM. and then re-add it back "cleanly".

1. From VC remove the host from the DVS, if it will not allow you to do this, you might need to "Disconnect" your host and then remove the Host from the "Networking" view of your 1000v - "Hosts" tab.

2. Uninstall the VEM software from the host using "vem-remove -d".  If this doesn't complete successfully it will advise you to use the hotswap.sh script.  The syntax is "hotswap.sh -u".

3.  Once the VC and Host are clean, issue a "no vem x" on the VSM where X= the VEM # for this host.  Save your config.

4. Reboot your host to flush any cached static data.  Then re-install the VEM software either manually or via VUM.  Confirm the ESX Build with the VEM version installed "vem version" and then look this up in your VSM Webpage http://<vsm-ip>

5. Once the host is re-added to the vCenter, check the VSM for module insertion.  "show mod".

That should sort you out.

Regards,

Robert

[Kassim Ismail]

Hi Robert,

Thanks for the prompt reply & the pointers provided. I will go through them & let you know how i get on.

Kassim

[Kassim Ismail]

Hi Robert,

I have gone through the steps below & unfortunately still does not work as i still get the error i mentioned earlier.  the VEM is being installed by VUM, but errors out during the host config. i checked the vem status on the host & it matches the version shown on VSM webpage.

i may now look at rebuilding the host & see if it helps.

Thanks

[Kassim Ismail]

Hi Robert,

bit of an update. I rebuilt the problem host & it is now successfully added to the Nexus 1000v.

Thanks for your help earlier.

Kassim

[Alex Knox]

All,

I had this exact issue.

It turned out to be a missing directory on the ESX host.

When I try the manual  install as per the documentation, it still fails however with a message:

Encountered error FileIOError:

The error data is:

   Filename - /var/tmp

   Message - I/O Error (17) on file /var/tmp: [Errno 17] file exisits: '/var/tmp'

   Errno  - 10

   Description - Unable to create, write or read a file as expected.

When I examine /var  there is a link for /var/tmp to /scratch/var/tmp.

When I look for  /scratch/var/tmp there is another link to  /tmp/scratch.

In /tmp there is no  scratch.

We created the directory in /tmp and retried the installation and everything worked as it should have done in the first place.

I have been informed that this is a known VMWare issue.

Kind regards,

Alex Knox

[Alex Knox]

All,

A minor change to my previous post:

We issued the following command:

mkdir -p /tmp/scratch/var/tmp

Then it worked properly.

Alex Knox

[Robert Burns]

Thanks for the update Alex.  I have seen this before also. Recreating the folder structure is the fix.  I'll see if I can find the VMware Bug #.

http://communities.vmware.com/thread/307808

Regards,

Robert