topic Re: Cisco ISE question in Network Access Control

Cisco ISE question

adamscottmaster2013 — Wed, 28 May 2025 16:57:14 GMT

Let say you have a 4 nodes ISE environment:

node1: Primary PAN/Primary MnT in AWS USEast-1,

node2: Secondary SAN/Secondary MnT in AWS USWest-1,

node3: PSN in AWS USEast-1,

node3: PSN in AWS USWest-1,

Let say node1 goes down unexpectedly and you promote node2 to be the PAN and PMnT. Two hours later, node1 comes back online. What is going to happen to your cluster because both node1 and node2 are now PAN and Primary MnT? Is this going to cause an issue? How are you going to fix this?

Re: Cisco ISE question

ahollifield — Wed, 28 May 2025 21:13:11 GMT

Nothing. You need to manually fail back. node1 will come in as secondary PAN.

Re: Cisco ISE question

adamscottmaster2013 — Wed, 28 May 2025 21:19:33 GMT

Are you sure about "Nothing. You need to manually fail back. node1 will come in as secondary PAN"? Because that is not what I experienced, and I was running ISE 3.1 patch-9.

Re: Cisco ISE question

ahollifield — Wed, 28 May 2025 21:24:37 GMT

That assumes that the node has operational communication still with the other PAN. ymmv if you are having WAN transport issues.

https://www.cisco.com/c/en/us/products/collateral/security/identity-services-engine/identity-service-engine-software-3-1-3-2.html

Re: Cisco ISE question

adamscottmaster2013 — Wed, 28 May 2025 21:36:52 GMT

@ahollifield: There was no WAN issue because I purposely null route between AWS USEast-1 and AWS USWest-1 VPCs where those ISEs resided. When I removed null route ten hours later, I had issues with ISEs. The latency between USEast-1 and USWest-1 is around 60ms, well within the limits of ISE (I think). Node1 could ping node2 and vice versa, and Security Group is wide open to allow 0.0.0.0/0 on all tcp and udp ports.

Re: Cisco ISE question

Aref Alsouqi — Thu, 29 May 2025 10:44:33 GMT

What issues did you experience? ISE does not support preemption. As @ahollifield mentioned, if the primary PAN goes down and you promote the secondary PAN to become the primary, then when the original primary comes back online it will become the new secondary PAN and will stay like that until you repromote it to become the primary again. Same thing when you use auto-failover, when the original primary comes back online it will become the new secondary node until you manually repromote it to become the primary.

Re: Cisco ISE question

adamscottmaster2013 — Thu, 29 May 2025 11:07:12 GMT

@Aref Alsouqi: Here is what happened. Everything was working fine. Node1 was PAN/PMnT and node2 was SAN/SMnT. I removed VPC peering between USEast-1 and USWest-1, so that node1 & node3 could NOT communicate with node2 and node4. I also performed went into AWS console and power OFF node 1. After that, I promoted node2 to PAN/PMnT. Ten hours later, I restored the VPC peering between USEast-1 and USWest-1 and powered up node1 shortly after that. This is where node1 and node2 were both showed up as PAN/PMnT.

Re: Cisco ISE question

Aref Alsouqi — Thu, 29 May 2025 14:09:21 GMT

Would the whole time that node1 was down exceeded 12 hours?

Re: Cisco ISE question

ahollifield — Thu, 29 May 2025 15:37:27 GMT

What do you mean? You removing the route literally was causing a WAN issue...

Re: Cisco ISE question

adamscottmaster2013 — Thu, 29 May 2025 16:29:43 GMT

@ahollifield: Yes, I removed the VPC peering to cause WAN outage in order to simulate a DR scenario. When I restored the VPC peering ten hours later, it should NOT have caused any issues, according to what you said, but it did.

Re: Cisco ISE question

adamscottmaster2013 — Thu, 29 May 2025 16:35:13 GMT

@Aref Alsouqi: it is possible that node1 was down for more than 28 hours, now that I remember. Cisco documentation stated that:

Actions must be taken to bring the PAN back into deployment within 12 hours.

https://www.cisco.com/c/en/us/td/docs/security/ise/3-0/admin_guide/b_ISE_admin_3_0/b_ISE_admin_30_deployment.html

What happened if the PAN node is down for more than 12 hours?

Re: Cisco ISE question

ahollifield — Thu, 29 May 2025 16:45:15 GMT

You will need to perform a manual sync on the node.

Re: Cisco ISE question

ahollifield — Thu, 29 May 2025 16:45:34 GMT

What issues exactly?

Re: Cisco ISE question

adamscottmaster2013 — Thu, 29 May 2025 17:19:45 GMT

Both node1 and node2 were showing up as PAN/PMnT. That's the issue.

Re: Cisco ISE question

ahollifield — Thu, 29 May 2025 17:42:13 GMT

did one have a yellow triangle next to it? Do you have screenshot you can share? Did this actually cause any operational issue?

Re: Cisco ISE question

adamscottmaster2013 — Thu, 29 May 2025 18:08:10 GMT

In node1 UI, it shows node2 as "red". In node2 UI, it shows node1 as "red". Node1 said it is PAN/PMnT. Node2 said it is PAN/PMnT.

Re: Cisco ISE question

ahollifield — Thu, 29 May 2025 18:26:08 GMT

was the WAN still broken at this time? The icon should have been yellow, not red.

Did this actually cause any operational issues?

Re: Cisco ISE question

adamscottmaster2013 — Thu, 29 May 2025 18:34:11 GMT

WAN has been restored for the past 36 hours after being broken for about 28 hours. It is causing any operational issues because I do not have a need to make any configuration change at this time. This is not a good situation.

Re: Cisco ISE question

adamscottmaster2013 — Thu, 29 May 2025 18:38:36 GMT

Can anyone explain what this mean? According to Cisco documentation:

Actions must be taken to bring the PAN back into deployment within 12 hours.

What happen if the PAN is down for more than 12 hours? What will happen then?

Re: Cisco ISE question

ahollifield — Thu, 29 May 2025 19:04:08 GMT

Open a TAC case.