- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-28-2025 09:57 AM
Let say you have a 4 nodes ISE environment:
node1: Primary PAN/Primary MnT in AWS USEast-1,
node2: Secondary SAN/Secondary MnT in AWS USWest-1,
node3: PSN in AWS USEast-1,
node3: PSN in AWS USWest-1,
Let say node1 goes down unexpectedly and you promote node2 to be the PAN and PMnT. Two hours later, node1 comes back online. What is going to happen to your cluster because both node1 and node2 are now PAN and Primary MnT? Is this going to cause an issue? How are you going to fix this?
Solved! Go to Solution.
- Labels:
-
Identity Services Engine (ISE)
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-26-2025 08:53 AM - edited 06-26-2025 08:54 AM
Hi @adamscottmaster2013 ,
when the PPAN is not online during the SPAN promotion to primary:
- the other Nodes identify and accept the SPAN as their New PPAN.
- the SPAN identity that the PPAN is not online
- after SPAN promotion to primary with PPAN down, the PPAN Nodes Status in the SPAN Deployment page is red
- after PPAN is back online, all Nodes "understand" that this PPAN is not the "real PPAN" and in the SPAN Deployment the Yellow icon - Replication Stopped appear:
- during this time, the PPAN back online become a SPAN:
- at this point, going back to the SPAN and execute a Sync with the (old) PPAN is the right thing to do:
- sync in progress ...
- the (old) PPAN is now connected to the SPAN (the new PPAN) :
At this point you are able to promote the old PPAN back to PPAN.
Hope this helps !!!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-28-2025 02:13 PM
Nothing. You need to manually fail back. node1 will come in as secondary PAN.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-28-2025 02:19 PM
Are you sure about "Nothing. You need to manually fail back. node1 will come in as secondary PAN"? Because that is not what I experienced, and I was running ISE 3.1 patch-9.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-28-2025 02:24 PM
That assumes that the node has operational communication still with the other PAN. ymmv if you are having WAN transport issues.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-28-2025 02:36 PM
@ahollifield: There was no WAN issue because I purposely null route between AWS USEast-1 and AWS USWest-1 VPCs where those ISEs resided. When I removed null route ten hours later, I had issues with ISEs. The latency between USEast-1 and USWest-1 is around 60ms, well within the limits of ISE (I think). Node1 could ping node2 and vice versa, and Security Group is wide open to allow 0.0.0.0/0 on all tcp and udp ports.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-29-2025 08:37 AM
What do you mean? You removing the route literally was causing a WAN issue...
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-29-2025 09:29 AM
@ahollifield: Yes, I removed the VPC peering to cause WAN outage in order to simulate a DR scenario. When I restored the VPC peering ten hours later, it should NOT have caused any issues, according to what you said, but it did.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-29-2025 09:45 AM
What issues exactly?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-29-2025 10:19 AM
Both node1 and node2 were showing up as PAN/PMnT. That's the issue.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-29-2025 10:42 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-29-2025 11:08 AM
In node1 UI, it shows node2 as "red". In node2 UI, it shows node1 as "red". Node1 said it is PAN/PMnT. Node2 said it is PAN/PMnT.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-29-2025 11:26 AM
Did this actually cause any operational issues?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-29-2025 11:34 AM
WAN has been restored for the past 36 hours after being broken for about 28 hours. It is causing any operational issues because I do not have a need to make any configuration change at this time. This is not a good situation.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-29-2025 12:04 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-29-2025 03:44 AM
What issues did you experience? ISE does not support preemption. As @ahollifield mentioned, if the primary PAN goes down and you promote the secondary PAN to become the primary, then when the original primary comes back online it will become the new secondary PAN and will stay like that until you repromote it to become the primary again. Same thing when you use auto-failover, when the original primary comes back online it will become the new secondary node until you manually repromote it to become the primary.
