Switch sending Failure after Response Identity

Magroll · ‎02-04-2019

Hi all,

we have 3000 Clients, most of them working without any Problems.

But we have some strange issues with some MacBooks, sometimes.... :(

Most of the Time all is working fine, but sometimes we got some Problems, so the Client can't connect anymore. It happen 1-3 times per day with different clients. The only solution to fix the Problem, is to swap the Ethernet-Adapter of the Client with a other one, or to wait 2-3 days. After that the Client could connect again.

If the Problem occurs, Wireshark shows the following szenario:

The Switch is sending a "Request, Identity" and the Client (MacBook) is answering with "Response, Identity", after that the Switch response with "Failure".

Alle Packets looks fine. They look exactly as from working clients. The Radius live Log on the ISE 2.1.0.474 shows "Authentication failed" The Auth Method is shown as MAB!

How could this be, because the client is answering to the 802.1x Request with a 802.1x Response...

Currently I have no idea, where the Problem should be. ISE? Switch? ...?

Could this be a Bug on ISE?

Any Ideas to look further are welcome!

Damien Miller · ‎02-04-2019

When you switch adapters the mac address of the endpoint changes right? Do you have any rejected endpoint on ISE during this, the same client you are having issue with ideally.

After the eap radius response from the client, the switch should be forwarding an access request to ISE. Confirm this is being sent from the switch. ISE should be responding with a radius access challenge, it won't be visible if you are pcap'ing on the endpoint port as it's to and from the switch management radius source interface.

Magroll · ‎02-07-2019

@Damien Miller wrote:
When you switch adapters the mac address of the endpoint changes right?

Yes, that's the workaround.

Do you have any rejected endpoint on ISE during this, the same client you are having issue with ideally.

In the RADIUS Live log the Client shows Failure 22056 with the non working Adapter. Else the Policy shows that the client would fail with MAB. but the client is sending 802.1x Pakets, this is what I see via Wireshark.
Actually I've got a TCP Dump from ISE. There I could see a Access Request Radius Message. The Radius service-type is framed, and the NAS-Port Type is Ethernet. If this two criteria are met, the ISE Policy should do an 802.1x Authentication, but it doesn't. Direct after the Radius Access-Request from the Switch the ISE answers with Access-Reject. If I said before, normally all is working fine, (ISE sends an Access-Challenge) this problem happens only sometimes....

Till now, I didn't find any hint in a Dump, with a reason for this behavior... :(

Thanks & Regards,
Ramon

Magroll · ‎02-08-2019

Attached is a PDF Document, where I tried to put in all my info from my Dumps.

In the Table on the right side is the problemclient. On the left side a client without any problems.

Both are sending an Access-Request with 802.1x, but the one on the right side get an Failure.

Any Ideas are welcome...

Regards,

Ramon

hslai · ‎02-09-2019

If possible, try finding the first failure event of the endpoint failing the authentications and checking its auth detailed report.

Other than that, please open a case with Cisco TAC support to gather additional debug info.

Magroll · ‎02-12-2019

@hslai wrote:

If possible, try finding the first failure event of the endpoint failing the authentications and checking its auth detailed report.

Hi, I tried this but its very hard, because the Problem looks totally random to me. It's happen on random clients (only Macs) on random times. Actually I am planning to upgrade our Installation to SNS3515 Appliances and Version 2.4.

So i don't kow how much work I am using in the near Future to follow this problem. :o(

hslai · ‎02-15-2019

ISE keeps RADIUS authentication details reports for 7 days so you should be able to run the report and find the first failed incident. If not there, also check RADIUS Errors report, which kept for 5 days, under Operations > Reports > Reports > Diagnostics.

You may also consider configuring an external syslog target to collect the categories Failed Attempts and Passed Authentications.

paul · ‎02-10-2019

This response won't help you with the Mac issue, but your MAB policy authentication is setup incorrectly. In your PDF file I saw errors during MAB that said "subject not found in identity store". That shouldn't happen in the MAB policy set. In the authentication make sure you have the "User Not Found" error condition set to "Continue". Any time you use internal endpoints you should set that value to Continue, unless you are doing a rare completely whitelist setup.

Did you capture packets from the PSN side when this is happening? From the logs you sent only MAB is being seen on the ISE side for that MAC address.

Magroll · ‎02-12-2019

@paul wrote:

This response won't help you with the Mac issue, but your MAB policy authentication is setup incorrectly. In your PDF file I saw errors during MAB that said "subject not found in identity store". That shouldn't happen in the MAB policy set. In the authentication make sure you have the "User Not Found" error condition set to "Continue". Any time you use internal endpoints you should set that value to Continue, unless you are doing a rare completely whitelist setup.

Hi Paul, thank you for your Note, but we are doing a 802.1x certificate only authentication. Only the devices which are not capable of 802.1x with Certificates are doing MAB, so we create a Whitelist for that devices. I don't know why we should setup the Condition to "Continue"? I think this should be useful if we want to have a guest Account or something to unknown clients. Actually the guest access is running on extra equipment.

Or did I miss something else?

Regards,

Ramon