topic Re: ISE authentication latency. in Network Access Control

ISE authentication latency.

Dustin Anderson — Tue, 11 Apr 2017 15:42:09 GMT

OK, I have a TAC case open since Feb 21st on this, but have the rep that never responds, so want to see if anyone here has had this same issue and can give me some direction.

So, we are running on 2.1 unpatched. all works fine and we see standard 10-20ms auth latency. The problem started when we applied update 3. The average auth latency went to ~5000ms with some as high as 16000ms.This was causing items to give up connecting due to the delay.

This is when I opened the TAC case. We did not hear anything for a week and ended up rolling back since Cisco didn't respond.

We ended up spinning up a test ISE and was able to reproduce the issue. I also tried to upgrade to 2.2 to see if it was patch specific and problem still persists. We need to move to ISE 2.2 for Passive ID for server 2016 since they are not updating the CDA's for 2016 and forcing us to move it to ISE.

Anyway, anyone run into this in your deployments? The latency wouldn't affect us as much if it din't cause disruption and disconnects to the clients.

Re: ISE authentication latency.

Jason Kunst — Tue, 11 Apr 2017 15:44:31 GMT

What is the TAC Case?

I suggest you request a requeue and escalation.

Re: ISE authentication latency.

Dustin Anderson — Tue, 11 Apr 2017 15:49:30 GMT

I did, escalated to a lvl 2, but the new tech resigned it back to the old tech.

Re: ISE authentication latency.

Jason Kunst — Tue, 11 Apr 2017 15:50:32 GMT

thanks, please ask for requeue I will see how i can help from my side

Re: ISE authentication latency.

Jason Kunst — Tue, 11 Apr 2017 19:21:52 GMT

I escalated myself, someone is going to reach out

Re: ISE authentication latency.

Joakim Backlund — Wed, 10 May 2017 06:41:55 GMT

Did you get any response on this?

I have the same issue. Running a deployment at 2.1p1 working just fine but after I have applied patch 2 and 3 I get latencies well over 10 seconds without any issues on the AD backend.

I've planned to do an upgrade to 2.2 tomorrow but perhaps I have to schedule for a downgrade instead.

Re: ISE authentication latency.

Dustin Anderson — Wed, 10 May 2017 12:55:19 GMT

We have a ticket open with MobileIron. It doesn't show, but we have the latency in 2.1 also. In 2.1, seems to be 10-12 seconds. Patched or 2.2 went to 14-18 seconds, and it reports it now.

I didn't notice the latency in 2.1 until I did a TCPdump and we use Omnipeek that flagged the latency, so was easier to find. And, I think until the latency got over 14 seconds, phones just waited. Now, it will drop the attempt.

Anyway, I'll post what MobileIron comes back with.

Are you seeing the latency with a MDM, or another authc step?

Re: ISE authentication latency.

Joakim Backlund — Wed, 10 May 2017 13:30:24 GMT

We don't use any MDM instead we see it with AD as auth backend. Still investigating if the problem is within ISE or actually an AD issue.

Re: ISE authentication latency.

Dustin Anderson — Wed, 10 May 2017 14:24:41 GMT

I haven't seen AD latency myself, but there are some bugs from upgrades causing latency. I did not see anything specific to 2.1 patch 2 though. I would open a TAC to see if there is a bug I'm not aware of. If you have a test system, or can spin up a test VM, see if you can recreate the issue, then update it to 2.2 and see if it disappears. This would at least give you an idea if you can upgrade to 2.2 to alleviate the issue.

Re: ISE authentication latency.

Ping Zhou — Fri, 02 Jun 2017 01:43:18 GMT

Have you had some conclusions ? I'm having high step latency issue too.

Re: ISE authentication latency.

Joakim Backlund — Fri, 02 Jun 2017 07:56:43 GMT

Solution for my problem I had was to either downgrade to 2.1 patch 1 or upgrade to 2.2. TAC didn't have anything to come with except cosmetic bugs related to high latency. We could spot any real latencies related to AD authentication apart from the statistics in ISE but we had very high numbers on radius timeouts on the NADs.

We experienced many radius timeouts at 2.1 patch 2&3. I tested to revert back to 2.1 patch 1 and that solved it. If I remember correctly I upgraded directly to patch 3, not installing the explicit patch 2 and everything was working fine. As I had to go to 2.2 I also tested to upgrade a malfunctioning 2.1 patch 1,2&3 installation to 2.2 and that also worked out fine. Before I did the upgrade for the whole deployment I upgrade the test environment to 2.2 patch 1 and that worked fine as well with very few radius timeouts.

So, in short, the solution for me was to upgrade the whole deployment to 2.2 patch 1 as it resolved our problem.

Re: ISE authentication latency.

Ping Zhou — Fri, 02 Jun 2017 20:40:08 GMT

Thank you for sharing the info.

Re: ISE authentication latency.

Dustin Anderson — Fri, 02 Jun 2017 20:45:06 GMT

My issue is still pending, but Mobil Iron admitted to a bug in the 9.3 code. When ISE requests the status of a device, it sends it's whole database....every time...for every check. This is corrected in 9.4 and waiting on the admin to update it.

Re: ISE authentication latency.

Ping Zhou — Fri, 02 Jun 2017 20:49:19 GMT

Thank you. Good to hear that.