ISE Multiple Matching Accounts

[Marvin Rhoads]

I'm having an odd issue with one user getting multiple matching accounts on ISE. This started happening only after a recent upgrade from ISE 2.1 to 2.6 Patch 5. The 4-node ISE deployment is joined to a single AD forest. It's a single domain with no external trust relationships.

The username is unique in AD. Doing a test authentication with the Test User tool confirms it. Yet authentications happening as part of any of the defined policies (VPN, Wireless 802.1x or TACACS) all fail.

Here's an example below. As you can see, ISE initially says it finds a single matching account and then later in the process says there are multiple.

11001 	Received RADIUS Access-Request
  	11017 	RADIUS created a new session
  	15049 	Evaluating Policy Group
  	15008 	Evaluating Service Selection Policy
  	15048 	Queried PIP - Airespace.Airespace-Wlan-Id
  	15048 	Queried PIP - Radius.Called-Station-ID
  	15048 	Queried PIP - DEVICE.Device Type
  	11507 	Extracted EAP-Response/Identity
  	12500 	Prepared EAP-Request proposing EAP-TLS with challenge
  	11006 	Returned RADIUS Access-Challenge
  	11001 	Received RADIUS Access-Request ( [step latency=5039 ms] Step latency=5039 ms)
  	11018 	RADIUS is re-using an existing session
  	11042 	Received duplicate RADIUS request; retransmitting previous response
  	11001 	Received RADIUS Access-Request
  	11018 	RADIUS is re-using an existing session
  	12301 	Extracted EAP-Response/NAK requesting to use PEAP instead
  	12300 	Prepared EAP-Request proposing PEAP with challenge
  	11006 	Returned RADIUS Access-Challenge
  	11001 	Received RADIUS Access-Request
  	11018 	RADIUS is re-using an existing session
  	12302 	Extracted EAP-Response containing PEAP challenge-response and accepting PEAP as negotiated
  	12318 	Successfully negotiated PEAP version 0
  	12800 	Extracted first TLS record; TLS handshake started
  	12805 	Extracted TLS ClientHello message
  	12806 	Prepared TLS ServerHello message
  	12807 	Prepared TLS Certificate message
  	12808 	Prepared TLS ServerKeyExchange message
  	12810 	Prepared TLS ServerDone message
  	12811 	Extracted TLS Certificate message containing client certificate
  	12305 	Prepared EAP-Request with another PEAP challenge
  	11006 	Returned RADIUS Access-Challenge
  	11001 	Received RADIUS Access-Request
  	11018 	RADIUS is re-using an existing session
  	12304 	Extracted EAP-Response containing PEAP challenge-response
  	12305 	Prepared EAP-Request with another PEAP challenge
  	11006 	Returned RADIUS Access-Challenge
  	11001 	Received RADIUS Access-Request
  	11018 	RADIUS is re-using an existing session
  	12304 	Extracted EAP-Response containing PEAP challenge-response
  	12305 	Prepared EAP-Request with another PEAP challenge
  	11006 	Returned RADIUS Access-Challenge
  	11001 	Received RADIUS Access-Request
  	11018 	RADIUS is re-using an existing session
  	12304 	Extracted EAP-Response containing PEAP challenge-response
  	12305 	Prepared EAP-Request with another PEAP challenge
  	11006 	Returned RADIUS Access-Challenge
  	11001 	Received RADIUS Access-Request
  	11018 	RADIUS is re-using an existing session
  	12304 	Extracted EAP-Response containing PEAP challenge-response
  	12305 	Prepared EAP-Request with another PEAP challenge
  	11006 	Returned RADIUS Access-Challenge
  	11001 	Received RADIUS Access-Request
  	11018 	RADIUS is re-using an existing session
  	12304 	Extracted EAP-Response containing PEAP challenge-response
  	12305 	Prepared EAP-Request with another PEAP challenge
  	11006 	Returned RADIUS Access-Challenge
  	11001 	Received RADIUS Access-Request
  	11018 	RADIUS is re-using an existing session
  	12304 	Extracted EAP-Response containing PEAP challenge-response
  	12318 	Successfully negotiated PEAP version 0
  	12812 	Extracted TLS ClientKeyExchange message
  	12813 	Extracted TLS CertificateVerify message
  	12804 	Extracted TLS Finished message
  	12801 	Prepared TLS ChangeCipherSpec message
  	12802 	Prepared TLS Finished message
  	12816 	TLS handshake succeeded
  	12310 	PEAP full handshake finished successfully
  	12305 	Prepared EAP-Request with another PEAP challenge
  	11006 	Returned RADIUS Access-Challenge
  	11001 	Received RADIUS Access-Request
  	11018 	RADIUS is re-using an existing session
  	12304 	Extracted EAP-Response containing PEAP challenge-response
  	12313 	PEAP inner method started
  	11521 	Prepared EAP-Request/Identity for inner EAP method
  	12305 	Prepared EAP-Request with another PEAP challenge
  	11006 	Returned RADIUS Access-Challenge
  	11001 	Received RADIUS Access-Request
  	11018 	RADIUS is re-using an existing session
  	12304 	Extracted EAP-Response containing PEAP challenge-response
  	11522 	Extracted EAP-Response/Identity for inner EAP method
  	11806 	Prepared EAP-Request for inner method proposing EAP-MSCHAP with challenge
  	12305 	Prepared EAP-Request with another PEAP challenge
  	11006 	Returned RADIUS Access-Challenge
  	11001 	Received RADIUS Access-Request
  	11018 	RADIUS is re-using an existing session
  	12304 	Extracted EAP-Response containing PEAP challenge-response
  	11808 	Extracted EAP-Response containing EAP-MSCHAP challenge-response for inner method and accepting EAP-MSCHAP as negotiated
  	15041 	Evaluating Identity Policy
  	15048 	Queried PIP - Normalised Radius.RadiusFlowType
  	15048 	Queried PIP - Network Access.EapTunnel
  	15013 	Selected Identity Source - <redacted>
  	24430 	Authenticating user against Active Directory - <redacted>
  	24325 	Resolving identity - <redacted>
  	24313 	Search for matching accounts at join point - <redacted>
  	24315 	Single matching account found in domain - <redacted>
  	24323 	Identity resolution detected single matching account
  	24343 	RPC Logon request succeeded - <redacted>
  	24402 	User authentication against Active Directory succeeded - <redacted>
  	22037 	Authentication Passed
  	11824 	EAP-MSCHAP authentication attempt passed
  	12305 	Prepared EAP-Request with another PEAP challenge
  	11006 	Returned RADIUS Access-Challenge
  	11001 	Received RADIUS Access-Request
  	11018 	RADIUS is re-using an existing session
  	12304 	Extracted EAP-Response containing PEAP challenge-response
  	11810 	Extracted EAP-Response for inner method containing MSCHAP challenge-response
  	11814 	Inner EAP-MSCHAP authentication succeeded
  	11519 	Prepared EAP-Success for inner EAP method
  	12314 	PEAP inner method finished successfully
  	12305 	Prepared EAP-Request with another PEAP challenge
  	11006 	Returned RADIUS Access-Challenge
  	11001 	Received RADIUS Access-Request
  	11018 	RADIUS is re-using an existing session
  	12304 	Extracted EAP-Response containing PEAP challenge-response
  	24715 	ISE has not confirmed locally previous successful machine authentication for user in Active Directory
  	15036 	Evaluating Authorization Policy
  	24209 	Looking up Endpoint in Internal Endpoints IDStore - <redacted>
  	24211 	Found Endpoint in Internal Endpoints IDStore
  	24432 	Looking up user in Active Directory - <redacted>
  	24325 	Resolving identity - <redacted>
  	24313 	Search for matching accounts at join point - <redacted>
  	24320 	Multiple matching accounts in forest - <redacted>
  	24324 	Identity resolution detected multiple matching accounts
  	24417 	User's Groups retrieval from Active Directory failed - <redacted>
  	15048 	Queried PIP - <redacted>.ExternalGroups (4 times)
  	15016 	Selected Authorization Profile - DenyAccess
  	15039 	Rejected per authorization profile
  	12306 	PEAP authentication succeeded
  	11503 	Prepared EAP-Success
  	11003 	Returned RADIUS Access-Reject 

[Colby LeMaire]

That is a very odd issue.  What type of device is the user using?  Have you tried to have the user use another device?  It seems like there may be an issue with the supplicant.  ISE is receiving multiple requests from the supplicant and it took more than 5 seconds at one point for the supplicant to response (or ISE to see the response).  It's like the authentication is successful but then ISE sends another challenge for authentication.  Wonder if the supplicant itself is starting new sessions and ISE is merging the attempts together.  If this didn't start until the software was upgraded, then I would bet the issue is with ISE.  I know that versions 2.1 and before had a lot of database issues.  I would recommend a clean install and do not recover from backup.  Rebuild the configurations and policies from scratch.  If you don't want to do that, then I would recommend opening a TAC case.

[Marvin Rhoads]

It happens on multiple devices: managed and unmanaged computers, mobile phone - even NADs where the user is logging in with TACACS. Other users accessing from the same devices have no problem. So it's almost definitely not supplicant-related.

Rebuilding a production ISE deployment that supports several thousand users from scratch to (possibly) fix one user's login issues is not a viable option.

I will go via the TAC if no other options come up.

[Marvin Rhoads · Accepted solution]

After working with TAC  we ascertained that there was some aspect of the one user's two AD accounts (he has one for normal use and one for his admin role) that made them ambiguous in ISE's estimation. The AD trace file showed the two accounts being returned even though the CN and SAM and UPN were unique. Our resolution was to delete the second user account from AD.

I did learn that there are some ISE behavior changes and at least one BugID that can contribute to such a situation:

https://www.cisco.com/c/en/us/td/docs/security/ise/2-6/upgrade_guide/Upgrade_Journey/HTML/b_upgrade_post_upgrade_tasks_2_6.html#id_124359

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvf21978/?rfs=iqvred

The TAC engineer confirmed that they continue to encounter this bug even though the BugID indicates it is resolved since release 2.4. The suggested workarounds were of no avail to us.

Perhaps this will assist anybody else who encounters the same situation.

[hslai]

As discussed, the first round is for authentication and it uses the sAMAccount name for lookup. Then, the 2nd round is for authorization and it uses the UPN but assumes it could be UPN or email. The two accounts had the same email address so that appears the reason in getting 2 matches.

CSCvg56578 fix provides another registry to adjust whether to lookup UPN only, email only, or both.

[Marvin Rhoads]

Thanks Hsing-Tsu, that explanation makes sense.

I  find it kind of strange behavior that the same email address for two otherwise unique accounts would cause Authorization to fail as a default behavior. Perhaps the live log details could highlight that vs. having to dig into a very verbose trace file.

I will add that to my long list of "odd things about ISE that are Good To Know".

[andy!doesnt!like!uucp]

Hi Marvin

do u have "odd things about ISE that are Good To Know" shared somewhere in community? i'd highly appreciate it.

tnx in advance

[Marvin Rhoads]

@andy!doesnt!like!uucp that's more a figurative than an actual list.

[joeharb]

I am running into the same issue, I can't figure out why ISE see's the accounts as being the same, one is admin and the other is normal.  Removing either account is not really an option as we have several users that have the same setup.  Could you please explain how you did the AD trace, was that from ISE or AD itself?

We are using EAP-TLS for both machine and user authentication and testing migration to EAP-TEAP.

I don't mind to start a new thread on this if needed.

Thanks,

Joe

[Marcelo Morais]

Hi @joeharb 

 at Administration > System > Logging > Debug Log Configuration > select the Node ...  Log Level = Debug for:

Active Directory (get the info in the ad_agent.log)

identity-Store-AD (get the info in the ise-psc.log)

Note: to get the logs ... show logging application <log name>

Hope this helps !!!