APIC unable to join the cluster after the replacement

[DanDan]

Hello ACI Folks,

I am working on a interesting case and I am here to ask for an opinion.

We had a cluster of three APICs, M2s running 5.2(5e).

One of the APICs got replaced for a new APIC L3, unfortunately an engineer did not notice

the SW version difference, and the new APIC was on the version 5.2(3e). After initial APIC setup,

APIC did not join the cluster and you can see the fault being generated about a SW version mismatch.

APIC was decommissioned, upgraded to the correct matching SW version and then once again reconnected and reconfigured.

In the GUI the new Controller Health Status shows as : "Data Layer Partially Diverged".

When I run on the APIC : show controller, I am unable to see the new APIC.

What is the most concerning is when I run the : acidiag avread,

it shows me that the Fabric domain etc is now set to the old version that was on the new APIC

trying to join the cluster, the 5.2(3e) instead of 5.2(5e) as it was before the replacement.

I am curious if anyone have already come across a similar issue and how to resolve this issue.

Thank you in advance.

Dan.

[DanDan · Accepted solution]

My problem has been just resolved,

Just an update if anyone is interested.

As mentioned in the previous post, after the new APIC was connected to the Fabric with mismatching SW version, it was decommissioned and upgraded to the correct SW version of the Fabric. All the APICs were showing the right version, but when command avread was issued, the Fabric Domain SW version was showing the incorrect SW version of the new APIC(SW version when it was connected first time).

We have tried from the APIC the following commands :

Config>

Firmware>

Controller-group>

Firmware-version>

and selecting the correct SW version.

However that did not help and avread was still showing the incorrect SW version for the Fabric Domain.

The new APIC was decommissioned and wiped clean again. After initial setup, we repeated the commands in order to change the SW version. This time it worked, avread showed the right SW version and the new APIC changed to "fully fit".

I hope this help to someone.

Cheers.