I would open a TAC case. Would really be interested to see what caused this. You should be able to boot off cdrom and get to the config on the boxes. From there TAC should be able to help you get back up and running.
For customers that have VSM on ESXi take advantage of the ability to clone your VSM before you make any changes. We support clone to template with a copy of the most current running-config as a way to backup and restore your VSM. This is a great way to prevent issues and to help in troubleshooting issues like this.
Unfortunately they cant find the contract details so we cant log a TAC case at the moment and need to get it up and running again.
I booted of a 1000v ISO and it reisntalled the VSM (I chose the only install if disk is unformatted option).
I have had a word with the storage admin and it seems the HP 3PAR SAN had a wobble when they turned on replication a while ago and corrupted some stuff. I am guessing that the VSM storage volumes got corrupted but they have been quite happily continuing on in RAM until I rebooted them.
Due to time constraints I am just going to reinstall from scratch as I have a copy of the config and it wont be too painfull.
In the end I botted teh VMs from teh ISO and chose the first option:
Install 1000v and install new image
This installed fine and I copied teh config back. Remarkably even though it has installed a new image and the config was blank the license files were there ! All I had to do was appply them.
So much powering on and off to test and both the primary and secondary VSM VMs are fine now and survive power off
It would be instructive to have some detailed information on exactly what the various options on the ISO actually do when you use them, perhaps with a discription of the disc structure of the VSM VMs and what gets overwritten by each of the four options on the ISO.