ISE design between two data-centers

[john5]

Hi all,

my customer has two data-centers to host ISE nodes as the following :

- PAN node in primary DC and another PAN in Secondary DC.

- MnT node in primary DC and another MnT in Secondary DC.

- some PSNs behind primary load balancer in the primary DC and another PSNs behind secondary load balancer in the secondary DC.

- in each DC, there is one PSN node act as health check point to monitor the PAN node.

- each load balancer will distribute traffic on all PSNs in the two DCs and they also in the same node group.

- the two DCs are connected via backdoor links and communication between ISE nodes go through them and not through the WAN links "as they are in the same AS number".

my question here what if the backdoor links fail and not the WAN links so no communication between the nodes in the two datacenters, Is PAN in the secondary DC will be active ? what is the mechanism used to let the secondary PAN node failover ? 

also what will happen if the two PAN nodes become active as all nodes in the two DCs can't reach each other ? and how to prevent that ?

I know that if the backdoor links fail I will lose connectivity between load balancer to the nodes in the second DC but what else could happen in that scenario ?

Thanks

[Nadav · Accepted solution]

Hey there,

1) What if the backdoor links fail and not the WAN links so no communication between the nodes in the two datacenters, Is PAN in the secondary DC will be active ? what is the mechanism used to let the secondary PAN node failover ? 

You configured PAN failover, so once the health node from the secondary DC doesn't see the primary PAN for a certain number heartbeats, the seconday PAN takes over. By default this is 10 minutes, and it takes another 10 minutes for the secondary PAN to become active. The mechanism is described in detail here:

https://www.cisco.com/c/en/us/td/docs/security/ise/2-4/admin_guide/b_ise_admin_guide_24/b_ise_admin_guide_24_new_chapter_011.html?bookSearch=true#reference_58F40B0E4D354B4DBB9940E4DB8DC8ED

2) what will happen if the two PAN nodes become active as all nodes in the two DCs can't reach each other ? and how to prevent that ?

That is called a split-brain scenario. It's a known architectural risk for any of these distributed hot-standby scenarios. The thing is that you need to manually re-sync the other nodes with the secondary PAN in order for them to be synced. It says so in the link I provided. So as long as you don't perform this sync, they will continue to work with the Primary PAN even during a split-brain scenario. If this is desireable for you, awesome. If not, resync the nodes with the Secondary PAN. Just make sure that when connectivity returns that you resync all the nodes with your Primary PAN so that your distribution continues to work as is designed.

3) I know that if the backdoor links fail I will lose connectivity between load balancer to the nodes in the second DC but what else could happen in that scenario ?

Complex question which depends on your architecture. You stated that all ISE traffic goes through this inter-dc link. If the load-balancers only balance PSN traffic between the two DC's, then if the link goes down then each site will only serve from the local PSN's. No big deal assuming you performed your sizing correctly according to your SLA. I for one would not assume that I need an inter-dc link to be up in order to server my clients, otherwise the moment you need to perform maintenance between the two DC's you may negatively impact your entire deployment.

As for what else is impacted by this inter-dc link falling: 

1. You have PAN failover configured, so there is the split-brain scenario mentioned earlier
2. The MNT's won't see eachother, so each PAN will be server by his local MNT since each will think the other is down and therefore also enter a split-brain scenario. For MNT's this is automatic, no sync or promotion necessary.
3. PSN's are independant, so they'll be fine for serving RADIUS/T+ requests. Keep in mind that only nodes in the same node group which can see eachother will share session state information as described in the same link I provided earlier.

By the way, just because subnets are in the same AS doesn't mean that they aren't redistributed at the ASBR. You should make sure that the routing protocol database doesn't redistribute these networks across the AS boundary if you want to understand how your deployment will behave during an inter-dc link failure.

[boonsue_pat]

Does this deployment require 2 set of license (base,apex,plus and tacacs+).

[Jason Kunst]

I'd recommend looking at the ordering guide. 1 ISE deployment requires only 1 set of licenses. https://www.cisco.com/c/dam/en/us/products/collateral/security/identity-services-engine/guide_c07-656177.pdf