Reset/Wipe only a single APIC in the cluster

[vv0bbLeS]

Hello all,

We recently upgraded the CIMC version on one of our APIC's (APIC1) to 4.1(2k), and afterwards APIC1 has fallen out of the cluster. I do have local admin access to APIC1, and running avread on APIC1 I can see that APIC1 only sees itself as active and APIC2 and APIC3 as inactive , but running avread on APIC2 and APIC3 shows that APIC2 and 3 are both active and APIC1 is the inactive one, which matches reality.

APIC1 is able to ping APIC2 and APIC3.

After the CIMC upgrade on APIC1, the CIMC had a different MAC address than before - thought that was interesting.

APIC1 didn't get a new ChassisID when it rebooted after the CIMC upgrade, so I know it's not that. I would like to avoid having to rebuild APIC1 via .iso file on USB or IIS over SoL as that takes a while.

Is there a set of commands I can run on APIC1 that will wipe only APIC1's config? I only want to clear/reset APIC1 so I can rejoin it to the otherwise healthy cluster of APIC2 and APIC3.

Thanks!

[Robert Burns]

Log into the IMC of APIC and issue the following commands:

APIC1-IMC# scope chassis
APIC1-IMC /chassis # scope adapter 1
APIC1-IMC /chassis/adapter # show detail | inc LLDP

Paste the output here.

Robert

[vv0bbLeS · Accepted solution]

@Robert Burns Thankfully I was able to get this one resolved, but to answer your question here is the output  you requested. Should LLDP be enabled, or no?

apic1-cimc /chassis/adapter # show detail | grep LLDP
    LLDP: Disabled

So this one actually ended up resolving itself as the cluster finally became Fully Fit but it took a while (3 hours). I'm not used to an APIC disconnecting from the cluster after a CIMC upgrade. : )

And in terms of my original question in this post, I was actually able to solve this one on my own. I ended up reading through @RedNectar 's reply to a forum post about how to wipe an entire fabric, and I inferred from Steps 2 and 3 that for my scenario I can just run the acidiag touch clean and acidiag touch setup commands on the single APIC in question (APIC1), which would only wipe the local APIC config.

So I used our LAB instance of ACI to test this, and I logged in to LAB-APIC2 and from there I decommissioned LAB-APIC1 from the cluster, and then I ran those 2 commands locally on LAB-APIC1 to wipe its config and then used acidiag reboot to reload it. After it came up I went through the standard APIC Setup and entered the initial config parameters as taken from LAB-APIC2 using this awesome guide I had saved that was posted by @Tomas de Leon . Then all I had to do was use LAB-APIC2 to Commission LAB-APIC1 back into the cluster and everything was just as happy as ever. : )

But thankfully I didn't end up needing to do this in production as the cluster finally became Fully Fit after 3 hours.

[Robert Burns · Accepted solution]

Yes, LLDP should show disabled.  This is so the APIC's OS can consume the LLDP packets.  The reason I had you check this is because you mentioned an IMC upgrade, and I have seen in cases where some of the IMC settings get reset during upgrades.  Didn't want you to have to run through a wipe/erase but since it was lab not a big deal.  Glad you got it resolved!

Robert

[vv0bbLeS]

Great! I'll add that info about LLDP to my notes as well. Thanks again!