Can I force UCS to change Fabric Interconnect *hardware* role as A/B when restoring a fullstate backup?

[jmunk]

At my customer, who has a large UCS installation, we need to switch hardware “roles” with respect to who is Fabric Interconnect A and and who is B for our two 6296UP v.2.2(2c) Fabric Interconnects (Note: it is *not* with respect to UCSM primary and subordinate roles) (the UCS domain in question is up running with applications and very close to being put into production, and we came to configure the hardware  FI-A / FI-B roles wrongly at installation). We really want to do this based on restore of Fullstate backup files, and *not* by physically swapping the two FIs.

We have tested this: we follow the “by the book” restore: we have taken full state backup, erase config on both FI’s on CLI. Then on the FI, which should rightly be FI-A, we connect to console, choose “restore” and restore the fullstate backup. Next we go to the rightful FI-B console and choose “part of existing cluster”. However the inclusion of the rightful FI-B in the cluster by connecting to the rightful FI-A fails and synchronize (no error messages). When we connect to rightful FI-A through GUI, we get rejected with “GUI not available on secondary FI”.

So everything points at, that the fullstate backup file contains hardware bound identification (e.g. serial number) of which FI is A and which is B (at the time of taking the fullstate backup), and UCS rejects switching hardware roles as FI-A respectively FI-B, when restoring the fullstate backup.

1. 1.      Is this correct?
2. 2.      Is there any way to circumwent this, such that I get my FabricInterconnects up with swapped hardware A/B roles when restoring the fullstate backup?

[Jeffrey Silberman · Accepted solution]

Hello,

"primary/subordinate" with respect to UCSM are the only aspects which can be considered "roles".   If you're talking about "A/B" those are just "names" --- their is no "role" associated there.    So you could not have configured the "FI-A/FI-B roles wrongly" --- because there is no "wrong".

The only issue I can think of is physical/box location :   You wanted one FI and one (or more) rack to be the "A" side and ther other to be the "B" side, but the FI's got "mislabeled" during the serial-console setup.   Is that the problem?

You are correct in that a "full state" backup is really a binary-image backup.    So if you restore that, everything will come back (quickly) in exactly the same way.   If you're trying to change the names of the FI's --- then that method won't work.

What I'd suggest is doing an "All Configuration" backup, which is a logical backup.    The "erase config" will force you to reinitialize at the serial console.   This will allow you to set "A" and "B" whichever way you want.  Once both sides are active, then you can "Import" your "All Configuration" backup, and you should be good to go.  

(And if you have a sandbox, then I'd suggest doing a dry-run test to make sure everything comes back the way you want, before trying this in production).

Thanks,

    -Jeff

[jmunk]

Jeff

-      Great with quick response

o  I have taken note, that the FI-A / FI-B “hardware roles” are just names, however we need to in compliance with our standard rack design, and hence must fix the problem at hand

-      We had actually already before I wrote my question to the Community, done exactly what You propose

-      However we used restore of the all-config backup from GUI (on a “clean” UCS with rightfull FI-A and FI-B, but with no configuration at all except the IP config on FI) with option “replace”, and that restore failed

-      After Your response, we tried this again with option “merge” and that worked.

-      So lesson learned:

o  You can switch FI-A / FI-B “hardware roies” by erasing all config on both FIs, configure the FI’s with IP and hostname and nothing else, and then from GUI restore an all-config backup with option “merge”, but not with option “replace”

-      Do You have any bid upon why that is so?

BR

Jesper

-

[Jeffrey Silberman]

Jesper,

            o  You can switch FI-A / FI-B “hardware roies” by erasing all config on both FIs, configure the FI’s with IP and hostname and         

               nothing else, and then from GUI restore an all-config backup with option “merge”, but not with option “replace”

              -      Do You have any bid upon why that is so?

System Management - Backing Up and Restoring the Configuration  [Cisco UCS Manager] - Cisco Systems

Import Methods

You can use one of the following methods to import and update a system configuration through Cisco UCS Manager:

• Merge—The information in the imported configuration file is compared with the existing configuration information. If there are conflicts, the import operation overwrites the information on the Cisco UCS instance with the information in the import configuration file.
• Replace—The current configuration information is replaced with the information in the imported configuration file one object at a time.

"Merge" works because if you've done an "erase configuration", then there are no conflicts, whereas "replace" will reinstantiate the previous "roles".

Glad you got it to work.

-Jeff