Factory restore from recovery dvd

[Danny Mainprize]

I have been having some massive issues with my video storage server, a msp 4U.  Working with support we have replaced 3 bad hard drives and now the entire chassis.  I am using the recovery DVD for 6.3.1 and doing "factory" when it boots to the dvd.  The factory recovery loads 5-600 rpms, but it seems awfully quick for loading the total Suse 10 SP1.  After that the DVD ejects, the server reboots, and just sits at a blinking cursor while the drives blink like mad doing their background initialization.

Is there something else I am supposed to be doing to get the OS loaded?

[Jason Rossi]

That scenario sounds very familiar. I have wrestled with 4RU hard drives and then the entire chassis myself recently. All working fine now though.

After doing the rebuild a few times now and reading your description below, I believe there is a step missing. When using 6.3.1 recovery it does indeed load rather quickly and then reboot as you have noted. But, it always then ran a second time, after the reboot, and finished the install (and took longer than the first time). I would type factory again during that second round, and about 30 minutes later it would finish (after a second reboot).

My CD did not eject after the first boot, but it did after the second boot. Perhaps you need to stick the CD back in the moment it ejects, so it boots from CD for round 2? Just a guess, but it definitely seems like the install is only half way done as it took two rounds every time I did it.

The only other thing I can think of is the RAID array. Did you rebuild the span using mega-raid? Might be worth hitting Ctrl-H while the bios is coming up to make sure your VD's are present and configured properly.

Sent from Cisco Technical Support iPhone App

[Danny Mainprize]

Sorry, I didn't explain myself fully.  The first time I do factory it deletes all the virtual drives and reboots.  This takes a whole couple of minutes.  The second time I do factory the virtual drives are created and a bunch of SUSE yast stuff runs installing the 600 odd RPMs.  Once that finishes, the DVD ejects, the configuration saves, the server reboots again and does the background initialization for the next 16-20 hours.  All the while just sitting at the blinking cursor.

[Jason Rossi · Accepted solution]

16-20 hours??? That has not been my experience at all when recovering 6.x. It's been more like 1 hour max start to finish every time I've done it. I'm not sure what the problem is. Below is a procedure I compiled for recovery situations (lifted from a few cisco docs as well as a very helpful RAID configuration doc). It steps thrpough everything starting from RAID config, to 6.3.1 factory install, upgrade to 6.3.2, and finally to restore the most recent VSOM backup.

1. After the motherboard BIOS loads the 'MegaRAID SAS-MFG BIOS' will load and display a long list of the controller configuration - controller, list of disks, list of volumes; at the bottom of this display the message to press CTRL-H to enter the MegaRAID configuration.
2. When the configuration program starts you will first be presented with a brief menu to select the MegaRAID controller - only one controller is possible so press SPACEBAR to continue or click the 'Start' button.
3. To configure the RAID for VSMS server select ‘Configuration Wizard’ -> ‘New Configuration’ -> ‘Custom Configuration’
4. Create a Disk Group (DG) by selecting each of the Physical Disks and clicking the 'AddtoArray' button. You can select each disk one at a time, or select multiple disks with the SHIFT key depressed. Each of the disks will appear in the righthand DiskGroups window. Once all of the disks are in the DiskGroup , click Accept DG to create DG0. Click Next to continue.
5. Configure a Span by selecting DG0 from the Array With Free Space window and clicking the Add to SPAN button. Click Next to continue.
6. Create two Virtual Disks, starting with VD0. In the VD Definition form, set the following values:
   RAID Level RAID 5
   Strip Size 64 KB
   Access Policy RW
   Read Policy Normal
   Write Policy WThru
   Wrthru for BAD BBU Checked
   IO Policy Direct
   Disk Cache NoChange?
   Disable BGI No
   Select Size 12 GB
7. Click Accept to create VD0.
8. Click Back
9. If the unit has more than (12) hard drives:
   1. Select DG0 from the Array With Free Space window again and click the Add to SPAN button.
   2. Click Next to continue.
   3. Create VD1. In the VD Definition form, set the following values:
      RAID Level RAID 5
      Strip Size 64 KB
      Access Policy RW
      Read Policy Normal
      Write Policy Wthru
      Wrthru for BAD BBU Checked
      IO Policy Direct
      Disk Cache NoChange
      Disable BGI No
   4. In the Select Size box enter a value one half the maxium size that was filled in. The virtual disk must be less than 15TB, and it is recommended that storage larger than 15TB be divided evenly into two virtual disks.
   5. Click Accept to create VD1
   6. Click Back
   7. Select DG0 from the Array With Free Space window again and click the Add to SPAN button.
   8. Click Next to continue.
   9. Create VD2. In the VD Definition form, set the following values:
      RAID Level RAID 5
      Strip Size 64 KB
      Access Policy RW
      Read Policy Normal
      Write Policy Wthru
      Wrthru for BAD BBU Checked
      IO Policy Direct
      Disk Cache NoChange
      Disable BGI No
      1. The Select Size box will auto-fill with the correct value.
      2. Click Accept to create VD2
      3. Click Next to continue.
      4. The Disk Group and Virtual Disk configuration will be displayed as follows:
         -+ DG 0
         |- VD 0
         |- VD 1
         |- VD 2
10. Click Accept to accept the configuration.
11. Click Yes to save the configuration.
12. Click Yes to initialize the created Virtual Disks.
13. The Virtual Disk details page will appear and display the new Virtual Disks with progress as they are initialized. It should only take a minute or less to initialize a 10TB VD.
14. Once the process is complete, press Home.
15. To exit the WebBIOS click the Exit option from the main menu. When prompted, reboot the server by pressing CTRL-ALT-DEL.

16) when the server comes back up open YAST from the desktop. Go to network devices and configure the NIC for , SNM /24, gateway (under routing) . Make sure you can ping it from the network.

17) Insert the 6.3.1 recovery CD into the CD Drive on top of the machine. Reboot the server. It will book from the CD. Type FACTORY when prompted. It will reboot again and you type FACTORY again.

18) Now it has 6.3.1. Copy the files from to any temporary directory.

19) Make the upgrade  script executable   (chmod +x vsmSoftware632.sh). Run the script (./vsmSoftware632.sh). Answer simple questions when prompted.

20) Now it is at 6.3.2. Copy the most recent backup to a temp directory (there is a copy of one in archer along with the other files). Type this command:

shell> /usr/BWhttpd/bin/vsom_backup_restore -f filename

For example:

shell> /usr/BWhttpd/bin/vsom_backup_restore -f

VSOM_psbu-dev03_backup_20100128164352.tar.gz

[Danny Mainprize]

Talk about nailing it.  Just did that.  I now finally have a server with an OS on it again.  Been working on this for over a dang month now.

Thanks a million.