vCenter server lost

[Aki1]

Hi!

I have been testing Nexus 1000V in various lab environments and have come up with several interesting notes. I will post all of them starting with this one;

What happens, if you lose your vCenter server? This happened to me and I am wondering, how to get Nexus 1000V up and running in another vCenter server. Attachement is not a problem - install plug-in and you are done. The difficult part is changing the host attachements inside Nexus 1000V. Is this possible at all or does losing your vCenter server mean that you have to re-install everything (VSM and all the VEMs)?

My setup had two ESX hosts. One is running VSM and VEM (everything is behind VEM) and another is running VEM (everything is behind this). vCenter was installed on another machine. I lost vCenter machine and now VSM/VEM are still running, but I cannot make any adjustments to VMs or other networking properties.

BR;

Aki Anttila

[aroshan]

Hi Aki,

There are many ways to protect the VI Infrastructure. These are some of the tips on top of my head.

1. Deploy the database component in another server. Take perioidic backup. Incase of downtime, one can bringup a new database server and use the backed up database.
2. You can point to the database server from the vCenter Server. If the vCenter Server needs to be replaced, you can install a new vCenter and point to the database server.
3. Also take a look at VMware vCenter Server Heartbeat http://www.vmware.com/products/vcenter-server-heartbeat/

In the case where you mentioned, if one doesn't have any backup then the following steps can be used to connect N1KV VSM to new vCenter:

• Install new vCenter Server
• Connect the VSM to the new vCenter Server. Register extension/plug-in and issue connect command.
  
• From the vSphere Client:
  • Add the hosts to the vCenter Inventory.
  • Go to the Host tab Networking->Configuration->Distributed Virtual Switch
  • Click Remove. This will remove the stale dvs from the host.
• Add the host to the DVS
• Edit the Virtual Machines Settings to use the port-profiles/port-groups.

Hope this helps.

regards,
Roshan

[Aki1]

Hi!

Thanks for the reply. I know that there are ways to protect vCenter environment and in production environment you should use them. However, I am 100% sure that especially in smaller deployments, these measurements are not taken and therefore there is vulnerability in the vCenter environment.

I tried to recover from the situation by doing the first steps. I.e.:

* Installed new vCenter.

* Registered and connected VSM to the new vCenter.

Via these steps, everything seemed to be fine. However, when I tried to address individual hosts (Networking -> DVS), there was nothing seen. And therefore I was not able to release the PNICs from the "old" Nexus1000V. And if you are not able to remove PNICs, then you are not able to remove DVS from the hosts. All I got was an error message (unfortunately I did not record that one).

BR,

Aki

[aroshan]

Hi Aki,

If you remember the error that's shown on the vSphere Client, it would be helpful. Without the error it will be hard to comment. My guess is that the error would be if there are any ports in use. In such case, you will need to move those portgroups to use some temporary portgroups on the vswitch and once the host is added successfully to the dvs, you can add them back to use the N1K portgroups.

regards,

Roshan

[Aki1]

Hi!

Thanks for the reply. If I remember correctly, the error message was something like "Could not remove, the object is in use". My guess is that this is related to the fact that host's components (PNICs) were connected to the DVS. But the actual problem is that these cannot be removed, since the DVS configuration part is empty (showing no information about the DVS except the name). The question is; from where should I remove the PNICs? Directly from ESX host console or from VSM? If I remove them from the VSM, I end up in a situation, where SC is not usable, since that is behind VEM. Removing them from ESX host console directly is an option, but how does Nexus react to this? Should I completely remove VEM from the console, create new vSwitch, port-group and SC to get the host back alive? What if I had 64 of these - all in production use?

BR,

Aki

[aroshan]

Since the new vCenter doesn't know the previous DVS configurations, they will not be shown on the vSphere Client. You can remove the ports in use directly from the ESX console. No need to remove anything from VSM.

From the ESX console:

1. First list the DV ports in use:

esxcfg-vswitch -l | more

2.  Find the vmnic & associated DVPort ID for that vmnic.

3. You can then remove it by (assuming you want to remove "vmnic1" with DVPort ID "100"):

esxcfg-vswitch -Q vmnic1 -V 100 <DVS Switch Name>

4. Once all the DV ports in use are removed/moved to vswitch then you can goto vSphere Client and Remove the DVS from the Host Networking Page.

Also do remember the suggestions to protect the vCenter Server and backing up the database etc in case of production environments. vCenter is central to VMWare VI Infrastructure.

thanks,

Roshan