ISE VM Large used as PAN, PSN, PxGrid

[fellai]

In the ISE installation guide, it states that ISE Large VM cannot be used as PAN, PSN or PxGrid. Is it a hard restriction (system will check and prevent such configuration?) or just a suggestion? I am planning to have a 6-node design where all 6 nodes are VM Large based on SNS3695. PAN and PSN will be designated on Large VM. Is it a supported design?

"The Large memory size is only for use as a performance-enhanced MnT node. You cannot use the Large VM as a PAN, PSN, or pxGrid node."

[Damien Miller · Accepted solution]

It sounds like you are reading the ISE 2.4 guide, or there is a typo in the 2.6 guide, link what you were reading and we can clear it up. Using a large VM for all roles is supported in 2.6 and in standalone (1-2 nodes), or hybrid (PAN/MNT same nodes, 7 nodes max), with scale up to 50,000 active endpoints.

It would probably be wise to use 6 or 8x3655 appliances. This for example would have peak support for 200k active, and 100k if half the PSN's failed. 50k active per 3655 PSN in a dedicated deployment.
2xPAN
2x MNT
4x PSN - 200k total, 100k HA

optional, use 2 3655 PSN
2x PSN - 100k total, 50k HA

[Jason Kunst]

that's referring to the Super MNT as something like the 3695 size introduced in 2.6. in 2.4 it will get your a faster reporting, in , it can certainly be used but just overkill. in 2.6 will give you 2M active endpoints (dot1x/mab only)

3655 will give you 500k active endpoints with separate PAN/MNT

Or a distributed deployment of 2 PAN/MNT on same box and up to 5 separate PSNs

https://community.cisco.com/t5/security-documents/ise-performance-amp-scale/ta-p/3642148

I would recommend looking at the cisco live slides for Designing ISE for Scale & High Availability as well
https://community.cisco.com/t5/security-documents/ise-training/ta-p/3619944#toc-hId-1281981443

[Arne Bier]

Hi @Jason Kunst  and @Damien Miller 

Apologies for digging this subject up again. 

If I understand Damien's point, running LARGE VM on any persona is technically OK because the software will run and not complain about it.  And Cisco TAC should hopefully have no issues supporting a non-MnT node deployed as a VM LARGE?

It was always my understanding that throwing more RAM and CPU at the MnT made sense because this persona could benefit from it.  My question is, what benefit would a PAN node get if it had 256GB RAM?  It doesn't change any of the hard limits of 2 million endpoints, right?  In my opinion, this RAM would be wasted (or at best, used by Linux as a cache of some sort).  And what significance does more RAM and CPU have for a PSN?  

I would never want to stop anyone buying LARGE VM licenses if they can afford it :-) I am more interested in what effect it has on the different Personas.

thanks in advance

[Surendra]

Adding my 2 cents here.

IMHO, adding more resources to PAN helps when there is large deployment of say 10 + nodes. PAN does a lot of functions and it just is not an interface for configuration. There are so many micro services that run in PAN that bring coordination between different features/components. For example, Endpoint persistence from various PSN when any change is done to an endpoint, maintaining the configuration in sync across all the nodes in the deployment (replication and JGroups are no joke, they can seriously consume a lot of resources in a big deployment), maintenance work is done predominantly done on the PAN and most importantly, the number of calls it needs to make to each and every node in the deployment to display whatever you see in the ISE GUI. All of these are not easily done and one of the very reason where we push for a separate node for PAN in a deployment. Improving resources will improve the experience of the admin using ISE and also greatly reduces the risk of any sort of issues in the deployment. For the PSN, it is just not one server if you look at it in a much deeper split. PSN hosts two apache servers one for regular ISE operations and one for CA services. Practically every persona on a PSN is a service in its own and they put their share of load on all of it. With the increasing number of endpoints that hit a PSN, the load increases in a way exponentially. Having said this, all things considered, the recommendation is given by our PMs and TMEs in the performance and scaling guide. However, increasing resources to a node is not a bad thing at all and having large VMs for each node is a safe haven approach for ISE.

[Damien Miller]

I would have a concern that not running a standard vm template could result in the wrong platform properties being picked up at boot.

The platform properties file sets various components such as java memory allocation, tomcat threads, oracle settings, profiler settings, etc. So if a VM gets picked up as something other than a SNS35x5 or SNS 36x5 template, it could use a UCS or some default and cause all sorts of issues.

Now that 2.6 supports 3695 VM's for any persona, and 2.4 p9 includes platform properties for 36x5 templates, I suspect there is little risk if you were running 2.4 P9 on a 3695 template. I wouldn't push endpoint counts beyond what 2.4 was tested for.

[Nidhi]

Large VM which essentially had more resources, was introduced to improve the MnT performance. We clearly mentioned that this can be used as an MnT node only since we did not qualify the other personas on the large VM. Hence not recommended. 

but having said that, you can still allocate more resources to your PAN and PSN nodes other than the standard available. 

the equivalent of large VM is the new 3695 appliances which can run any of the personas today. 

with 2.4 patch 9, on 3695 as PAN and MnT , the max concurrent sessions supported is 500k sessions. there were some code changes done to achieve 2 M in 2.6 hence we dont recommend scaling beyond 2.4 scaling numbers.

Nidhi