Standby APIC Controller

[Curtis Parish]

We have four APIC controllers for our ACI stretched fabric.  Two are active in data center 1 and one is active in data center 2.  The fourth is in data center 2 and is powered off.  I have read and understand the below  document about enabling a standby controller. The only thing that is not made clear, is what state the standby controller needs to be kept in.  Do we have to power it up and manually  install the latest software when the rest of the controllers are updated?   Or can it be upgraded from the number three controller at the time it is activated in a disaster situation?    

The procedure for restoring the fabric in data center 1 involves resetting all the nodes and controllers  to factory default.  That seems to be drastic and time-consuming.   Can anyone provide commentary on that?

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/kb/b_kb-aci-stretched-fabric.html#concept_4B9644FE992A43D9A261F1531DBC9588

thanks

curtis 

[Tomas de Leon · Accepted solution]

Curtis,

Thank you for using the Cisco Support Community for ACI!

You have brought up a good question.  With the instructions and recommendations listed in the mentioned link, there is not easy answer.  The problem that you will run into is the following.

For example:

• you have a total of 4 APICs. APIC 1, 2a, 3, & 2b are running ACI-version1
• you put APIC 2b out of service
• you then upgrade APICs 1, 2a, and 3 to ACI-version2.  APIC 2b is not upgraded since powered down.
• connectivity is lost to site1 or APICs 1 & 2a are lost
• Now it is time to bring APIC 2b online but it is running ACI-version1
• You decommission APIC 2a from cluster and then power APIC 2b
• APIC 2b will not be able to joing the cluster since it is running  ACI-version1 and the Cluster is running  ACI-version2.

I had filed an feature enhancement to allow this:

CSCuz40545 [apic cluster] Enh: Allow different version to join Cluster for Upgrades 

This feature has not been added yet.

So now you will need to manually upgrade APIC-2b to ACI-version2 before it will be able to join the fabric cluster. 

• You will need to download the ACI-version2.iso from www.cisco.com
• Configure CIMC configuration on APIC-2b (all apics should have CIMC configured for remote access)
• Mount the ACI-version2.iso using the CIMC KVM
• power on APIC-2b to upgrade to ACI-version2
• After the ACI-version2 has been installed on APIC-2b, it will reboot to the setup script.  You will need to configure node-Id 2 and the fabric name & TEP Pool the same as APIC3.
• APIC-2b will be configured now and should join the cluster with the assumption that all the fabric parameters match the existing Cluster configuration.

I would highly recommend scheduling a maintenance window to test this disaster recovery method.  Also, if you run into issues during the test, please open a TAC case so a CSE can assist you with any questions or issues during the test.

Cheers!

T.

[abahaa]

Thank you Tomas ,
I found this very helpful, I just have one question....
I think that I don't need to reset the APIC's in case the standby APIC is upgraded to the same version.
So all what I need to assign the standby APIC with the node ID (ex: 2) and decommission the primary POD controller.
Correct me if I am wrong

[Curtis Parish]

quite a bit has changed since my original post. The standby controller does not have to be kept shut down and it will be upgraded at the same time the others are upgraded.