Solved: Re: ISE 802.1x auth error 5440 and 12935 - Page 2

TedB123 · ‎07-24-2023

hi

im trying to understand and figure out why our new ISE node is not authenticating devices.

the scenario is this...

we have an on Prem node which is v2.7, this node is to be decommissioned

deployed a new node in Azure which is v3.2

exported the config from v2.7 and imported into v3.2 - checking the config of the new node everything looks good, all certs are there, policy sets, profiles etc.. everything looks healthy.

my test lab is a meraki MX with an AP.

radius configuration points to the internal IP of the new node (meraki proxy not ticked)

firewall rules on the MX and our main firewall allow comms on port 1812,1813 - i can confirm im seeing traffic go thru successfully

when i look at the new nodes live logs i can see my laptops attempt to authenticate but fails with the following

12805	Extracted TLS ClientHello message
	12806	Prepared TLS ServerHello message
	12807	Prepared TLS Certificate message
	12808	Prepared TLS ServerKeyExchange message
	12809	Prepared TLS CertificateRequest message
	12810	Prepared TLS ServerDone message
	12505	Prepared EAP-Request with another EAP-TLS challenge
	11006	Returned RADIUS Access-Challenge
	11001	Received RADIUS Access-Request
	11018	RADIUS is re-using an existing session
	12504	Extracted EAP-Response containing EAP-TLS challenge-response
	12505	Prepared EAP-Request with another EAP-TLS challenge
	11006	Returned RADIUS Access-Challenge
	11001	Received RADIUS Access-Request
	11018	RADIUS is re-using an existing session
	12504	Extracted EAP-Response containing EAP-TLS challenge-response
	12505	Prepared EAP-Request with another EAP-TLS challenge
	11006	Returned RADIUS Access-Challenge
	11001	Received RADIUS Access-Request
	11018	RADIUS is re-using an existing session
	12504	Extracted EAP-Response containing EAP-TLS challenge-response
	12505	Prepared EAP-Request with another EAP-TLS challenge
	11006	Returned RADIUS Access-Challenge
	5440	Endpoint abandoned EAP session and started new ( Step latency=18680 ms)

Event	5440 Endpoint abandoned EAP session and started new
Failure Reason	5440 Endpoint abandoned EAP session and started new
Resolution	Verify known NAD or supplicant issues and published bugs. Verify NAD and supplicant configuration.
Root cause	Endpoint started new authentication while previous is still in progress. Most probable that supplicant on that endpoint stopped conducting the previous authentication and started the new one. Closing the previous authentication.

11001	Received RADIUS Access-Request
11017	RADIUS created a new session
11117	Generated a new session ID
15049	Evaluating Policy Group
15008	Evaluating Service Selection Policy
15048	Queried PIP - Normalised Radius.RadiusFlowType
15048	Queried PIP - Radius.Called-Station-ID
11507	Extracted EAP-Response/Identity
12500	Prepared EAP-Request proposing EAP-TLS with challenge
11006	Returned RADIUS Access-Challenge
11001	Received RADIUS Access-Request
11018	RADIUS is re-using an existing session
12502	Extracted EAP-Response containing EAP-TLS challenge-response and accepting EAP-TLS as negotiated
12800	Extracted first TLS record; TLS handshake started
12545	Client requested EAP-TLS session ticket
12542	The EAP-TLS session ticket received from supplicant while the stateless session resume is disabled. Performing full authentication

theres also mention of certificate issues...

12935 Supplicant stopped responding to ISE during EAP-TLS certificate exchange

Supplicant stopped responding to ISE during EAP-TLS certificate exchange

Verify that supplicant is configured properly to conduct a full EAP conversation with ISE. Verify that NAS is configured properly to transfer EAP messages to/from supplicant. Verify that supplicant or NAS does not have a short timeout for EAP conversation. Check the network that connects the Network Access Server to ISE. Verify that ISE local server certificate is trusted on supplicant. Verify that supplicant has a properly configured user/machine certificate.

when i change the radius settings back to the old node, authentication work.. so this tells me that the certs are all good.

so im a bit stuck as to what is missing here....

any suggestions on what else i can check?

i have raised a case with TAC and waiting for a reply.

cheers

Arne Bier · ‎07-28-2023

Well done mate. What a journey!! I guess this issue should have never occurred if PMTUD (path MTD discovery) was operating. Everything I have read says that all modern OS’s support it and it’s also done with UDP protocol. It relies on ICMP messages to inform the sender if its own MTU doesn’t fit along the way. So perhaps all along, the issue was that some device in the path was dropping these ICMP messages.
Either way, you sorted it. But if you happen to know that ICMP is deliberately being firewalled somewhere, then at least try to allow the ICMP messages relating to PMTUD (ICMP Type 3, Code 4).
Also, the common IOS command “no IP unreachables” also breaks PMTUD.

TedB123 · ‎07-31-2023

@Arne Bier wrote:
Well done mate. What a journey!! I guess this issue should have never occurred if PMTUD (path MTD discovery) was operating. Everything I have read says that all modern OS’s support it and it’s also done with UDP protocol. It relies on ICMP messages to inform the sender if its own MTU doesn’t fit along the way. So perhaps all along, the issue was that some device in the path was dropping these ICMP messages.
Either way, you sorted it. But if you happen to know that ICMP is deliberately being firewalled somewhere, then at least try to allow the ICMP messages relating to PMTUD (ICMP Type 3, Code 4).
Also, the common IOS command “no IP unreachables” also breaks PMTUD.

for sure something was dropping the packet .. in terms of our kit, theres only a couple of hops before it reaches the destination and we checked the FW and load balancers and there was nothing there that stood out.
thinking about the journey that the packet takes, based on the wireshark captures... i feel like the packet was arriving as you can see the access challenge and request in the capture, but it was the accept part that was being dropped somewhere ... most likely azure virtual network was causing the issue.

what bugs me about this is that theres no mention of this on the cisco ISE documentation.. surely they are aware of this, you'd expect some sort of note that changing the mtu like this might be needed.

anyways... its done for now... appreciate your help with this one

cheers