topic Re: Cisco ISE Primary Admin failover and failback issue. in Network Access Control

Cisco ISE Primary Admin failover and failback issue.

adamscottmaster2013 — Mon, 19 Jan 2026 21:56:29 GMT

I need advice from experts on this forum regarding a very strange issue that occurred in March 2025 at my current workplace, one month before I joined the company.

We have a Cisco ISE 3.1 Patch 10 cluster deployed across two data centers as follows:

ise01: Primary Admin / Secondary MnT (Data Center A)
ise02: Secondary Admin / Primary MnT (Data Center B)
ise03: PSN (Data Center A)
ise04: PSN (Data Center A)
ise05: PSN (Data Center B)

In March 2025, we performed a Disaster Recovery (DR) exercise that required completely cutting off connectivity to Data Center A. Prior to disconnecting Data Center A, we promoted ise02 to Primary Admin / Primary MnT, resulting in the following state:

ise01: Secondary Admin / Secondary MnT (Data Center A)
ise02: Primary Admin / Primary MnT (Data Center B)
ise03: PSN (Data Center A)
ise04: PSN (Data Center A)
ise05: PSN (Data Center B)

After the promotion, everything looked good, and we cut off connectivity to and from Data Center A. During the outage, we were able to successfully authenticate 802.1X on both wired and wireless networks between ISE and Cisco switches and Cisco wireless controllers. MAC Address Bypass (MAB) also worked as expected.

After completing the DR exercise, we restored network connectivity to and from Data Center A. Once connectivity was restored, the ISE cluster appeared healthy, with all nodes in green status. The following day, we promoted ise01 back to Primary Admin.

At that point, everything appeared normal; however, we discovered that all MAC addresses in the MAB database were missing, which caused a widespread outage affecting printers and Cisco IP phones. To recover, I restored a previous ISE backup to a lab evaluation instance, extracted the MAB database, and restored it to the production environment.

We opened a support case with Cisco, but the root cause was inconclusive. Since then, we have upgraded the ISE environment to ISE 3.3 Patch 7.

We are planning another DR exercise in approximately five weeks, and I am very nervous about failover and failback with Cisco ISE. What is the likelihood that this issue could occur again?

Thoughts?

Re: Cisco ISE Primary Admin failover and failback issue.

Arne Bier — Tue, 20 Jan 2026 00:59:26 GMT

Nothing surprises me anymore with ISE. I don't think you have done anything wrong to cause this issue. It's just the usual software quality that bites when you least expect it. I find that gremlins creep into the system after upgrades and patching, especially the more upgrades that have been done. The best run I have had has been after rebuilding ISE from scratch (painfully) instead of upgrading. But since 3.3 to 3.4 the gremlins have returned. But I can't afford to rebuild the entire thing again.

But an operational activity such as promoting the PAN should never result in such an a catastrophic outcome.

Re: Cisco ISE Primary Admin failover and failback issue.

Marcelo Morais — Fri, 23 Jan 2026 01:24:27 GMT

@adamscottmaster2013 ,

before promoting the SPAN to a new PPAN, I recommend a Syncup (at Administration > System > Deployment) with the PPAN.

Hope this helps !

Re: Cisco ISE Primary Admin failover and failback issue.

adamscottmaster2013 — Tue, 27 Jan 2026 20:31:33 GMT

@Marcelo Morais wrote:

@adamscottmaster2013 ,

before promoting the SPAN to a new PPAN, I recommend a Syncup (at Administration > System > Deployment) with the PPAN.

Hope this helps !

Everything is already Sync'ed as showing "green" status in the UI.