One of our Vcenter's has crashed and per the recommendation of Vmware we are planning a fresh install of the appliance. They also said the distributed switch (1000v) would have to be rebuilt. I am concerned with adding the hosts back to the new Vcenter without the 1000V being present and I am also concerned that adding the host with the switch added that the host will stop traffic.
All hosts are up and working but other going directly to the esxi host there is no management.
Anyone had to experience with this or performed anything similar?
I am assuming you want to retain your N1k configuration as far as the defined port groups, and configs go.
Your new vcenter will not have the N1k extension key registered yet, so even if you bring the VSM online, and it has connectivity to vcenter, no config should get passed.
To confirm, you can go into your VSM console, edit the config, SVS connection, and do a 'no connect' which should keep the VSM from even trying to sync/connect with vcenter.
Any of your guestVM nics that still reference the N1k DVS port group configs of the original vcenter will likely throw an error about 'no backing' due to the N1k DVS object that will not exist when you rejoin your esxi hosts to the rebuilt vcenter.
Re-adding your hosts back to a vcenter that no longer has the N1k port-groups/uplinks will have unpredictable results, and you will probably want to define a generic dVS and port groups to migrate your host & guestVMs to until you get your N1k extension key registered with the rebuilt vcenter, and the SVS connection re-established.
If you were rebuilding a VCA, and then importing a database backup, then you would likely not have to worry about these additional steps, but that does not sound like the current plan.
At this point it sounds like you probably have working hosts and guests, that are now all on their own islands, that won't let you vmotion, HA, etc. I would expect to have guestVM outage on each of your esxi hosts as you rejoin the new vcenter, until you can reassign the port groups to guestVM nics.
Having the N1k re-integrated to new vcenter before adding hosts back to vcenter, would save you some steps, as you could migrate guestVM networking back into the correct N1k DVS port groups as you are rejoining the hosts to the new vcenter.
You should be able to retain/leave the VSM as is, for configuration (don't redeploy VSM OVA), unless you really want to start from scratch, and re-import the running configs.
Just to clarify:
"Having the N1k re-integrated to new vcenter before adding hosts back to vcenter, would save you some steps, as you could migrate guestVM networking back into the correct N1k DVS port groups as you are rejoining the hosts to the new vcenter."
Are you meaning simply go ahead and build the vcenter and then add the vsm extension to it and allow it register with no hosts added. Once the config has been pushed, add each host into the vcenter and migrate the existing vm's to the 1000v port group.
You really need to open a TAC case, have the details of your environment reviewed.
I was discussing your scenario with some other engineers and there have been a few cases where the vems didn't seem to like the programming from the new VSM that was using the same IP, domain ID, etc. It's possible you may want also rebuild your VSM (maybe use new domain ID), and just reapply your your port-profiles to it from your current config.
Since the hosts will likely default your guests over to local vswitch initially, when readded to new vcenter, you might want to reconfig a local vswitch on a few of your critical hosts/guests so you have the right local port groups/vlan tagging going on.