How to decide on the right PAN size (small or medium)?

[Arne Bier]

Hello

I have gone over the scaling guide and the install guide but it's never been clear to me why the fully distributed deployment PAN needs to have so much CPU and memory. The job of the PAN is to keep the database synchronised with all the other nodes - not too CPU intensive I would think.

When I think of performance bottlenecks, I think mainly about the MNT node and logging - hence, for a 50,000 concurrent session deployment of RADIUS (MAB mostly) I would consider that the two MNT nodes should be a Medium VM. But does that also imply that the PAN MUST be medium too? It's quite a jump from 32GB to 96GB RAM and I don't see any evidence in the documentation to suggest that this is mandatory.

Has anyone tried running 64GB RAM and 8 cores ? Is there any benefit in allocating more memory to an ISE VM without causing the VM to be detected as a next level up (e.g. small VM spec but with more RAM than usual)? There are cases where customers dedicate an entire server chassis but there is spare RAM and CPU left over - it seems like a shame not to use any spare resources lying around.  

[Marcelo Morais]

Hi @Arne Bier ,

 I had 8x dedicated Hosts (with spare CPU and Memory) for 8x Nodes in one Site.

 We upgraded the CPU and Memory from: 16vCPU and 64GB RAM to: 24vCPU and 96GB RAM.

 I didn't see any benefit to add more CPU and Memory, probably because ISE was using less than (on average) 50% of CPU and Memory before the "hardware upgrade".

 Even though we added more CPU and Memory, I still notice some Load Average on the tech top command.

 At the end of the day I realize that it was a good move to use all the CPU and Memory available because it was there without a use.

Note: ISE 2.7 P3, 16xPSNs, 2xPANs and 2xMnT.

Regards

[Arne Bier]

thanks @Marcelo Morais  -

Do you remember what VM License you needed when at 16vCPU and 64GB RAM ? The memory and CPU is not enough for Medium - I assume this is still 'small' VM?

I guess you also went from Small VM to Medium VM once you bumped the RAM to 96 GB.

With so many resources the system should be flying to the moon and back! Storage also plays a part - if you're waiting on IO then all the CPU in the world doesn't help.

I have seen in many cases that we allocate the recommended RAM and CPU to ISE nodes but in the steady state, the VM resources consumed are just a fraction. It makes sense why customer want to oversubscribe - it seems like a royal waste.

It should be up to customers to decide their own fate - oversubscribe if you also spend the time to manage your infrastructure properly. But if you just don't care and have enough money to spare, then just reserve what is documented. 

[Marcelo Morais]

Hi @Arne Bier ,

 on ISE Installation Guide, 2.1 and BRKSEC-3432, a VM simulating a SNS 3595 with 16vCPU and 64GB RAM was considered Large.

 on ISE Installation Guide, 2.4, the same reservation was considered Medium (to be Large the Memory should be equal to 256GB RAM).

 on ISE Installation Guide, 2.7, there is no reference for SNS 3595 ... a SNS 3655 with 24vCPU and 96GB RAM, is considered Medium.

 Please check the pg.79 of BKSEC-3432, if I add more Memory (96GB RAM) but my CPU remains the same (16vCPU), the hardware remains the same (SNS 3595), for ex: if it is considered Small, then it would still be Small.

[Damien Miller · Accepted solution]

I wouldn't recommend 64 GB of ram and 8 vcpu, that will pick you up a resource template from the platform properties that is undesirable vs the tested scaling numbers. The platform properties file would expect 64 GB and 16 vcpu to land on a SNS 3595 template. 

64 GB and 8 vcpu = UCS_Large template. That means near nothing unless you look in the resource tables of the platform properties so I've done just that. It will allocate half the Java memory, the same connection template for tomcat, ers, and admin as a 3515, about half to 2/3 the expected oracle resources, half the memory for the session management vs a 3595, half the max allowable sessions vs a 3595, and half the prrt sessions to that of a 3595. 

Why deviate from the tried and tested deployment templates certified by the BU and happily TAC supported? If VM resources are an issue the recommendation will always be to go with physical SNS. It's stated in the admin guide that you don't have to use reservations and you can monitor the deployment closely, but in the cases where I have had performance issues, it's usually been because the node didn't have the reservations and the customer had host contention. 

In short, just give it at least 64 GB of memory and 16 vcpu or buy a 3595/3655. 

[Arne Bier]

thanks Damien - that's the sort of analysis I was looking for. I have a situation where the customer is somewhat resource constrained and I was looking for a happy medium.

Makes sense then to go for the Medium (16 vCPU) and assign 64GB instead of the 96 (not enough RAM in the servers) - and then watch the stats over a month or more - if the VMs are suffering then we'd need to move them. But I find that hard to believe, since their SNS-3495's are doing just fine with 32 GB RAM. That's the dilemma. If you compare SNS-3495 (customer happy) and then move to the new world where the expectation is 96 GB RAM to achieve the same thing, it involves some head scratching. And it's not because Linux suddenly got larger - or that ISE trippled in size. It makes very little sense.

I have Aruba Clearpass customers happily cruising on 8GB of RAM without any complaints. Of course they don't have pxGrid etc but nor do most of my ISE customers. In the SMB space I see a lot of customers who need rock solid 802.1X and MAB and TACACS+ - basically, they need ACS functionality - 96GB of RAM for that seems like a crying shame to me.