ISE Logs too slow

[manvik]

ISE logs tooooo slow, it keeps showing "fetching records" for tacacs, radius, alrm logs etc

deployment is a simple one with DC & DR nodes

MNT, ADM, monito are all in one node

[ahollifield]

Version?  Are you within the scale limits for a standalone deployment?  VM or appliance?

https://www.cisco.com/c/en/us/td/docs/security/ise/performance_and_scalability/b_ise_perf_and_scale.html

[manvik]

ISE 3.1 patch 3

VM

new installation

2 node standalone

concurrent tacacs auth 200 devices. Total 400 devices

[ahollifield]

Do you have the proper VM resources allocated? Also, are you seeing any alarms on the dashboard for disk read/write speeds?

https://www.cisco.com/c/en/us/td/docs/security/ise/3-1/install_guide/b_ise_InstallationGuide31/b_ise_InstallationGuide31_chapter_2.html#hardware-virtual-appliance-req

[manvik]

yes 32 GB RAM and 16 CPU

There was a "queue link error timeout" between DC and DR in the Alarma

[ahollifield]

That is related to the ISE messaging service (how logs are transferred between nodes) and is likely the cause of your issue.  Re-generate the ISE Root CA and then the ISE messaging certificate for all nodes.  

[Marcelo Morais]

Hi @manvik ,

 if my understanding is correct, at Home > Alarms dashboard, you have a Queue Link Error, am I correct?

 If the answer is yes, could you please:

1. click the Queue Link Error link

2. take a PrtScr of the new window (generated by the Queue Link Error link)

Regards

[thomas]

Please see How to Ask The Community for Help to provide the community with enough details to help you in the future.

[manvik]

Alarm says; Queue Link Error: Message=From DR ISE to DC ISE; cause=Timeout

I removed DR ISE from the cluster, now there no delay in logs.

[ahollifield]

Timeouts typically indicate the ISE messaging port TCP/8671 is not allowed through the firewall.  

[Marcelo Morais]

Hi @manvik ,

 please take a look at the following document: ISE - Queue Link Error.

Hope this helps !!!

[manvik]

thank you @ahollifield @Marcelo Morais 

the ports are open between both ISE nodes. I removed DR ISE from cluster slowness disappeared.

Again added DR ISE to cluster, now the logs are slow again.

I get the attached error now.

[ahollifield]

Unknown CA.  You need to re-generate the ISE root CA and then the ISE Messaging certificates on all nodes.

[adamscottmaster2013]

I run into this issue before.  To resolve it, I did:

1- Administration --> Logging --> Log settings

Uncheck the box "Use "ISE Messaging Service" for UDP Syslogs delivery to MnT" 

I think that will resolve your issue.