Accepted Solutions
Hi,
I believe the reason why you are confused, might be because you are thinking at the redundancy from APIC perspective as a unit and not from shards (database), which is the critical component when it comes to configuration.
The APIC cluster uses a technology from large databases called sharding - very similar concept as horizontal database partitioning, but better. Better in the sense that it increase redundancy and performance because the db tables are split across servers, and smaller tables are replicated as complete units. Cisco APIC uses a replication factor of 3, meaning each shard has 3 instances/replicas (one active and two backup) across the cluster, regardless of the number of APIC nodes present in the cluster.
The read-only state actually refers to shards. When 2 out of 3 copies are down, the remaining shard goes into read-only state, to prevent loss of data in case the other two are comes backup.
In case of a 2 node failure, this is how a 3 node cluster vs 5 node cluster will look like from shards perspective:
So as you can see here, in case of a 5 node cluster there might be a chance that some shards to be in read-only and other in read-write.
if we change to a multi-pod, you need to consider carefully the distribution of the APICs, to avoid total loss of a shard. Golden rule is: do not keep 3 active APICs in the same pod, because this can happen:
The green shard is lost in case of hardware failure of all 3 APICs from pod1. To recover from this state you will need to contact TAC and BU. The procedure is called ‘ID Recovery’ to restore the whole fabric state to the latest taken configuration snapshot (if you have one).
I hope this was informative and is more clear now.
Note: I took all these details and images from the following ciscolive presentation:
I recommend you have a look on the presentation (video). You can do that on ciscolive.com and search for BRKACI-2003 in the on-demand library.
Cheers,
Sergiu
Hi,
I believe the reason why you are confused, might be because you are thinking at the redundancy from APIC perspective as a unit and not from shards (database), which is the critical component when it comes to configuration.
The APIC cluster uses a technology from large databases called sharding - very similar concept as horizontal database partitioning, but better. Better in the sense that it increase redundancy and performance because the db tables are split across servers, and smaller tables are replicated as complete units. Cisco APIC uses a replication factor of 3, meaning each shard has 3 instances/replicas (one active and two backup) across the cluster, regardless of the number of APIC nodes present in the cluster.
The read-only state actually refers to shards. When 2 out of 3 copies are down, the remaining shard goes into read-only state, to prevent loss of data in case the other two are comes backup.
In case of a 2 node failure, this is how a 3 node cluster vs 5 node cluster will look like from shards perspective:
So as you can see here, in case of a 5 node cluster there might be a chance that some shards to be in read-only and other in read-write.
if we change to a multi-pod, you need to consider carefully the distribution of the APICs, to avoid total loss of a shard. Golden rule is: do not keep 3 active APICs in the same pod, because this can happen:
The green shard is lost in case of hardware failure of all 3 APICs from pod1. To recover from this state you will need to contact TAC and BU. The procedure is called ‘ID Recovery’ to restore the whole fabric state to the latest taken configuration snapshot (if you have one).
I hope this was informative and is more clear now.
Note: I took all these details and images from the following ciscolive presentation:
I recommend you have a look on the presentation (video). You can do that on ciscolive.com and search for BRKACI-2003 in the on-demand library.
Cheers,
Sergiu
