PEAP failed SSL/TLS handshake because the client rejected the ISE local-certificate

[washingday]

I am troubleshooting a customer environment where a windows 7 client is refusing to authenticate with their ISE. There's another client where the certificate for the Root CA is functional and it authenticates successfully.

I haven't had the chance myself to learn much about ISE yet which is why I'm hoping for someone with more experience to steer me in the correct direction. Some debugging information collected from the switch authenticator:

Aug 2 11:50:05.013 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] Client MAC-address, Initialising Method dct state to 'Not run'
Aug 2 11:50:05.013 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] Adding method dct to runnable list for session 0xF80000AE
Aug 2 11:50:05.013 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] Client MAC-address, Initialising Method CTS Reauth Client state to 'Not run'
Aug 2 11:50:05.013 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] Adding method CTS Reauth Client to runnable list for session 0xF80000AE
Aug 2 11:50:05.013 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Processing SM CB request for 0xF80000AE: Event: New client notification (151)
Aug 2 11:50:05.013 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] Create attr list, session 0xF80000AE:
Aug 2 11:50:05.013 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] - adding MAC MAC-address
Aug 2 11:50:05.013 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] - adding Swidb 0x3860A3A4
Aug 2 11:50:05.013 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] - adding AAA_ID=D199
Aug 2 11:50:05.013 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] - adding Audit_sid=AC186AFB0000D1995C090312
Aug 2 11:50:05.013 UTC: AUTH-FEAT-IPDT-EVENT: [MAC-address, Gi1/0/7] NewClient: No entry for MAC-address. session 0xF80000AE
Aug 2 11:50:05.014 UTC: AUTH-FEAT-SISF-EVENT: [MAC-address, Gi1/0/7] No IPv6 binding found for MAC-address(0xF80000AE)
Aug 2 11:50:05.014 UTC: AUTH-FEAT-ACCT-EVENT: [MAC-address, Gi1/0/7] [Session 0xF80000AE] New client notification mac MAC-address
Aug 2 11:50:05.014 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] New client MAC-address - client handle 0x00000001 for SM Accounting Feature
Aug 2 11:50:05.014 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] New client MAC-address - client handle 0x00000001 for SVM
Aug 2 11:50:05.014 UTC: AUTH-FEAT-SWITCH-CORE-EVENT: [MAC-address, Gi1/0/7] new client domain 1, iif_id 0x0000000000000000 vlan 0
Aug 2 11:50:05.014 UTC: AUTH-FEAT-SWITCH-CORE-EVENT: [MAC-address, Gi1/0/7] no pre_auth_vlan provided by SM, usingvlan 52
Aug 2 11:50:05.014 UTC: AUTH-FEAT-SWITCH-CORE-EVENT: client struct with id = 0x9D0000AE created for MAC-address
Aug 2 11:50:05.014 UTC: AUTH-FEAT-SWITCH-CORE-EVENT: [Gi1/0/7] Find vlan on port: found vlan 52, user count 1 fwd count 0, client count 0, pending delete 0
Aug 2 11:50:05.014 UTC: AUTH-FEAT-SWITCH-CORE-EVENT: [Gi1/0/7] Find or alloc vlan: Updating vlan 52 New user count 2
Aug 2 11:50:05.014 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Single release event is posted from UNKNOWN
Aug 2 11:50:05.014 UTC: AUTH-FEAT-SWITCH-CORE-EVENT: [MAC-address, Gi1/0/7] Removed pre_auth_vlan 52 from SM
Aug 2 11:50:05.015 UTC: AUTH-EVENT: [Gi1/0/7] No MACs found
Aug 2 11:50:05.015 UTC: AUTH-EVENT: [Gi1/0/7] Client MAC count: 0, 0 not being deleted
Aug 2 11:50:05.015 UTC: AUTH-EVENT: [Gi1/0/7] No authorized client found in domain [DATA]
Aug 2 11:50:05.015 UTC: AUTH-EVENT: [Gi1/0/7] Domain authorized client count: 0
Aug 2 11:50:05.015 UTC: AUTH-EVENT: [Gi1/0/7] No authorized client found in domain [VOICE]
Aug 2 11:50:05.015 UTC: AUTH-EVENT: [Gi1/0/7] Domain authorized client count: 0
Aug 2 11:50:05.015 UTC: AUTH-EVENT: [Gi1/0/7] No authorized ctx found
Aug 2 11:50:05.015 UTC: AUTH-FEAT-SWITCH-CORE-EVENT: [Gi1/0/7] control enabled 1 cdp bypass enabled 1 static allow 1
Aug 2 11:50:05.015 UTC: AUTH-FEAT-SWITCH-CORE-EVENT: [Gi1/0/7] notify 1 oper allow 1 authorized 0 for DATA
Aug 2 11:50:05.015 UTC: AUTH-FEAT-SWITCH-CORE-EVENT: [Gi1/0/7] notify 0 oper allow 0 authorized 0 for VOICE
Aug 2 11:50:05.015 UTC: AUTH-FEAT-SWITCH-CORE-EVENT: [Gi1/0/7] Is domain valid: Voice vlan is invalid
Aug 2 11:50:05.015 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] New client MAC-address - client handle 0x9D0000AE for Switch PI
Aug 2 11:50:05.015 UTC: AUTH-FEAT-FFCP-EVENT: [MAC-address, Gi1/0/7] FFCP new client cb for MAC-address
Aug 2 11:50:05.016 UTC: AUTH-FEAT-FFCP-EVENT: [MAC-address, Gi1/0/7] IIF ID not set. Invoke FFCP add for MAC-address
Aug 2 11:50:05.016 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] IIF ID required for 0xF80000AE(MAC-address)
Aug 2 11:50:05.016 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] New client MAC-address - client handle 0x5E55FFC9 for Session Mgr FFCP Shim
Aug 2 11:50:05.016 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Added record to DB - hdl 0xF80000AE / MAC-address. 1 session(s) on IDB
Aug 2 11:50:05.016 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] Add record - adding MAC MAC-address
Aug 2 11:50:05.016 UTC: AUTH-FEAT-SWITCH-CORE-EVENT: [MAC-address, Gi1/0/7] attrib-change for id 0x9D0000AE
Aug 2 11:50:05.016 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] Add record - adding SWIDB GigabitEthernet1/0/7
Aug 2 11:50:05.016 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] Add record - adding AAA-ID D199
Aug 2 11:50:05.017 UTC: AUTH-FEAT-ACCT-EVENT: [MAC-address, Gi1/0/7] [Session 0xF80000AE] SM Notified attribute Add/Update aaa-unique-id 0000D199
Aug 2 11:50:05.017 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] Add record - adding AUDIT-ID AC186AFB0000D1995C090312
Aug 2 11:50:05.017 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] Add record - adding TARGET_SCOPE (Client)
Aug 2 11:50:05.017 UTC: AUTH-EVENT: Bound session (hdl 0xF80000AE) to policy (tgt 0x9B0100E8)
Aug 2 11:50:05.017 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] No attr list found
Aug 2 11:50:05.017 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] No identity attr list found
Aug 2 11:50:05.017 UTC: AUTH-EVENT: Handling client event RX_IDENTITY_UPDATE (19) for PRE, handle 0xF80000AE
Aug 2 11:50:05.017 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] IIF ID not set. Events for 0xF80000AE will be queued.
Aug 2 11:50:05.018 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Block events for MAC-address pending async IIF ID Set for unknown (0).
Aug 2 11:50:05.018 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Queueing event RX_IDENTITY_UPDATE(19) for 0xF80000AE - process in turn later
Aug 2 11:50:05.018 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Queued the event RX_IDENTITY_UPDATE for 0xF80000AE & pre crit status 0
Aug 2 11:50:05.018 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Queueing event SINGLE_ID_UPDATE(18) for 0xF80000AE - process in turn later
Aug 2 11:50:05.018 UTC: AUTH-EVENT: Handling client event SESSION_STARTED (8) for PRE, handle 0xF80000AE
Aug 2 11:50:05.018 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] IIF ID not set. Events for 0xF80000AE will be queued.
Aug 2 11:50:05.018 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Block events for MAC-address pending async IIF ID Set for unknown (0).
Aug 2 11:50:05.018 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Queueing event SESSION_STARTED(8) for 0xF80000AE - process in turn later
Aug 2 11:50:05.018 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Queued the event SESSION_STARTED for 0xF80000AE & pre crit status 0
Aug 2 11:50:05.023 UTC: AUTH-FEAT-FFCP-EVENT: [MAC-address, Gi1/0/7] auth_mgr FFCP callback called with oper 1 for 0xF80000AE
Aug 2 11:50:05.023 UTC: AUTH-FEAT-SWITCH-CORE-EVENT: [MAC-address, Gi1/0/7] attrib-change for id 0x9D0000AE
Aug 2 11:50:05.023 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Session method ctx missing for connection handle 21
Aug 2 11:50:05.024 UTC: AUTH-SYNC: [MAC-address, Gi1/0/7] Delay add/update sync of iif-id for MAC-address / 0xF80000AE
Aug 2 11:50:05.024 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Queued RESUME_PROCESSING event for 0xF80000AE(MAC-address) - IIF ID Set(0)
Aug 2 11:50:05.024 UTC: AUTH-FEAT-FFCP-EVENT: [MAC-address, Gi1/0/7] FFCP create callback for 0xF80000AE (iifid: 0x106978000000105 ), result 0x0
Aug 2 11:50:05.024 UTC: AUTH-EVENT: Handling ASYNC RESUME for handle 0xF80000AE
Aug 2 11:50:05.024 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Dequeueing event(s): block on IIF ID Set(0) / clear on IIF ID Set(0) for MAC-address
Aug 2 11:50:05.024 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Dequeueing message RX_IDENTITY_UPDATE
Aug 2 11:50:05.024 UTC: AUTH-EVENT: Handling client event RX_IDENTITY_UPDATE (19) for PRE, handle 0xF80000AE
Aug 2 11:50:05.025 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Policy event will be processed synchronously for 0xF80000AE
Aug 2 11:50:05.025 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Processing default action(s) for event RX_IDENTITY_UPDATE for session 0xF80000AE.
Aug 2 11:50:05.025 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Dequeueing message SINGLE_ID_UPDATE
Aug 2 11:50:05.025 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Received internal event SINGLE_ID_UPDATE (handle 0xF80000AE)
Aug 2 11:50:05.025 UTC: AUTH-SYNC: [MAC-address, Gi1/0/7] Delay remove sync of vlan-id for MAC-address / 0xF80000AE
Aug 2 11:50:05.025 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Dequeueing message SESSION_STARTED
Aug 2 11:50:05.025 UTC: AUTH-EVENT: Handling client event SESSION_STARTED (8) for PRE, handle 0xF80000AE
Aug 2 11:50:05.025 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] Create attr list, session 0xF80000AE:
Aug 2 11:50:05.025 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] - adding MAC MAC-address
Aug 2 11:50:05.025 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] - adding Swidb 0x3860A3A4
Aug 2 11:50:05.025 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] - adding AAA_ID=D199
Aug 2 11:50:05.026 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] - adding Audit_sid=AC186AFB0000D1995C090312
Aug 2 11:50:05.026 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] - adding IIF ID=0x106978000000105
Aug 2 11:50:05.026 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] Client MAC-address, Initialising Method dot1x state to 'Not run'
Aug 2 11:50:05.026 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] [UPDATE] Adding method dot1x to runnable list for session 0xF80000AE
Aug 2 11:50:05.026 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Processing SM CB request for 0xF80000AE: Event: Start a method (150)
Aug 2 11:50:05.026 UTC: dot1x_auth Gi1/0/7: initial state auth_initialize has enter
Aug 2 11:50:05.026 UTC: dot1x-sm:[MAC-address, Gi1/0/7] dot1x-clientID: initialising
Aug 2 11:50:05.026 UTC: dot1x_auth Gi1/0/7: during state auth_initialize, got event 0(cfg_auto)
Aug 2 11:50:05.026 UTC: @@@ dot1x_auth Gi1/0/7: auth_initialize -> auth_disconnected
Aug 2 11:50:05.026 UTC: dot1x-sm:[MAC-address, Gi1/0/7] dot1x-clientID: disconnected
Aug 2 11:50:05.026 UTC: dot1x_auth Gi1/0/7: idle during state auth_disconnected
Aug 2 11:50:05.026 UTC: @@@ dot1x_auth Gi1/0/7: auth_disconnected -> auth_restart
Aug 2 11:50:05.026 UTC: dot1x-sm:[MAC-address, Gi1/0/7] dot1x-clientID: entering restart
Aug 2 11:50:05.027 UTC: dot1x-ev:[MAC-address, Gi1/0/7] Sending create new context event to EAP for dot1x-clientID (MAC-address)
Aug 2 11:50:05.027 UTC: dot1x_auth_bend Gi1/0/7: initial state auth_bend_initialize has enter
Aug 2 11:50:05.027 UTC: dot1x-sm:[MAC-address, Gi1/0/7] dot1x-clientID: entering init state
Aug 2 11:50:05.027 UTC: dot1x_auth_bend Gi1/0/7: initial state auth_bend_initialize has idle
Aug 2 11:50:05.027 UTC: dot1x_auth_bend Gi1/0/7: during state auth_bend_initialize, got event 16383(idle)
Aug 2 11:50:05.027 UTC: @@@ dot1x_auth_bend Gi1/0/7: auth_bend_initialize -> auth_bend_idle
Aug 2 11:50:05.027 UTC: dot1x-sm:[MAC-address, Gi1/0/7] dot1x-clientID:entering idle state
Aug 2 11:50:05.027 UTC: dot1x-ev:[MAC-address, Gi1/0/7] Created a client entry (dot1x-clientID)
Aug 2 11:50:05.027 UTC: dot1x-ev:[MAC-address, Gi1/0/7] Dot1x authentication started for dot1x-clientID (MAC-address)
Aug 2 11:50:05.027 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Client MAC-address, Context changing state from 'Idle' to 'Running'
Aug 2 11:50:05.027 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Client MAC-address, Method dot1x changing state from 'Not run' to 'Running'
Aug 2 11:50:05.027 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Policy processing started for 0xF80000AE(MAC-address)
Aug 2 11:50:05.027 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Policy event will be processed synchronously for 0xF80000AE
Aug 2 11:50:05.028 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Authorization profile successfully applied for the event Identity Update
Aug 2 11:50:05.028 UTC: AUTH-EVENT: Raising ext evt AuthZ Success (21) on session 0xF80000AE, client (unknown) (0), hdl 0x00000000, attr_list 0x00000000
Aug 2 11:50:05.028 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Processing default action(s) for event SESSION_STARTED for session 0xF80000AE.
Aug 2 11:50:05.028 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Unblock events for MAC-address.
Aug 2 11:50:05.028 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Handling external PRE event AuthZ Success for context 0xF80000AE.
Aug 2 11:50:05.028 UTC: dot1x-sm:[MAC-address, Gi1/0/7] Posting !EAP_RESTART on Client dot1x-clientID
Aug 2 11:50:05.028 UTC: dot1x_auth Gi1/0/7: during state auth_restart, got event 6(no_eapRestart)
Aug 2 11:50:05.028 UTC: @@@ dot1x_auth Gi1/0/7: auth_restart -> auth_connecting
Aug 2 11:50:05.028 UTC: dot1x-sm:[MAC-address, Gi1/0/7] dot1x-clientID:enter connecting state
Aug 2 11:50:05.029 UTC: dot1x-sm:[MAC-address, Gi1/0/7] dot1x-clientID: restart connecting
Aug 2 11:50:05.029 UTC: dot1x-sm:[MAC-address, Gi1/0/7] Posting RX_REQ on Client dot1x-clientID
Aug 2 11:50:05.029 UTC: dot1x_auth Gi1/0/7: during state auth_connecting, got event 10(eapReq_no_reAuthMax)
Aug 2 11:50:05.029 UTC: @@@ dot1x_auth Gi1/0/7: auth_connecting -> auth_authenticating
Aug 2 11:50:05.029 UTC: dot1x-sm:[MAC-address, Gi1/0/7] dot1x-clientID: authenticating state entered
Aug 2 11:50:05.029 UTC: dot1x-sm:[MAC-address, Gi1/0/7] dot1x-clientID:connecting authenticating action
Aug 2 11:50:05.029 UTC: dot1x-sm:[MAC-address, Gi1/0/7] Posting AUTH_START for dot1x-clientID
Aug 2 11:50:05.029 UTC: dot1x_auth_bend Gi1/0/7: during state auth_bend_idle, got event 4(eapReq_authStart)
Aug 2 11:50:05.029 UTC: @@@ dot1x_auth_bend Gi1/0/7: auth_bend_idle -> auth_bend_request
Aug 2 11:50:05.029 UTC: dot1x-sm:[MAC-address, Gi1/0/7] dot1x-clientID:entering request state
Aug 2 11:50:05.029 UTC: AUTH-FEAT-MDA-EVENT: [Gi1/0/7] Is enabled: No
Aug 2 11:50:05.029 UTC: dot1x-ev:[Gi1/0/7] Sending EAPOL packet to group PAE address
Aug 2 11:50:05.029 UTC: dot1x-registry:registry:dot1x_ether_macaddr called
Aug 2 11:50:05.029 UTC: dot1x-ev:[Gi1/0/7] Sending out EAPOL packet
Aug 2 11:50:05.030 UTC: dot1x-packet:EAPOL pak Tx - Ver: 0x3 type: 0x0
Aug 2 11:50:05.030 UTC: dot1x-packet: length: 0x0005
Aug 2 11:50:05.030 UTC: dot1x-packet:EAP code: 0x1 id: 0x1 length: 0x0005
Aug 2 11:50:05.030 UTC: dot1x-packet: type: 0x1
Aug 2 11:50:05.030 UTC: dot1x-packet:[MAC-address, Gi1/0/7] EAPOL packet sent to client dot1x-clientID
Aug 2 11:50:05.030 UTC: dot1x-sm:[MAC-address, Gi1/0/7] dot1x-clientID:idle request action
Aug 2 11:50:05.034 UTC: dot1x-packet:[MAC-address, Gi1/0/7] Queuing an EAPOL pkt on Authenticator Q
Aug 2 11:50:05.034 UTC: dot1x-packet:EAPOL pak rx - Ver: 0x1 type: 0x0
Aug 2 11:50:05.034 UTC: dot1x-packet: length: 0x000F
Aug 2 11:50:05.035 UTC: dot1x-ev:[Gi1/0/7] Dequeued pkt: Int Gi1/0/7 CODE= 2,TYPE= 1,LEN= 15

Aug 2 11:50:05.035 UTC: dot1x-ev:[Gi1/0/7] Received pkt saddr =MAC-address , daddr = dest.mac, pae-ether-type = pae-ether
Aug 2 11:50:05.035 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Method dot1x is already in ctx method list.
Aug 2 11:50:05.035 UTC: dot1x-packet:EAPOL pak rx - Ver: 0x1 type: 0x0
Aug 2 11:50:05.035 UTC: dot1x-packet: length: 0x000F
Aug 2 11:50:05.035 UTC: AUTH-EVENT: Handling client event RX_METHOD_AGENT_FOUND (2) for PRE, handle 0xF80000AE
Aug 2 11:50:05.035 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] Create attr list, session 0xF80000AE:
Aug 2 11:50:05.035 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] - adding MAC MAC-address
Aug 2 11:50:05.035 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] - adding Swidb 0x3860A3A4
Aug 2 11:50:05.036 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] - adding AAA_ID=D199
Aug 2 11:50:05.036 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] - adding Audit_sid=AC186AFB0000D1995C090312
Aug 2 11:50:05.036 UTC: AUTH-DETAIL: [MAC-address, Gi1/0/7] - adding IIF ID=0x106978000000105
Aug 2 11:50:05.036 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Method dot1x is already in ctx method list.
Aug 2 11:50:05.036 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Ignoring start request for dot1x, as method is already running
Aug 2 11:50:05.036 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Policy event will be processed synchronously for 0xF80000AE
Aug 2 11:50:05.036 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Authorization profile successfully applied for the event Remote AuthC Failure
Aug 2 11:50:05.036 UTC: AUTH-EVENT: Raising ext evt AuthZ Success (21) on session 0xF80000AE, client (unknown) (0), hdl 0x00000000, attr_list 0x00000000
Aug 2 11:50:05.036 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Processing default action(s) for event RX_METHOD_AGENT_FOUND for session 0xF80000AE.
Aug 2 11:50:05.036 UTC: AUTH-DETAIL: No default action(s) for event RX_METHOD_AGENT_FOUND.
Aug 2 11:50:05.036 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Handling external PRE event AuthZ Success for context 0xF80000AE.
Aug 2 11:50:05.037 UTC: dot1x-sm:[MAC-address, Gi1/0/7] Posting EAPOL_EAP for dot1x-clientID
Aug 2 11:50:05.037 UTC: dot1x_auth_bend Gi1/0/7: during state auth_bend_request, got event 6(eapolEap)
Aug 2 11:50:05.037 UTC: @@@ dot1x_auth_bend Gi1/0/7: auth_bend_request -> auth_bend_response
Aug 2 11:50:05.037 UTC: dot1x-sm:[MAC-address, Gi1/0/7] dot1x-clientID:entering response state
Aug 2 11:50:05.037 UTC: dot1x-ev:[MAC-address, Gi1/0/7] Response sent to the server from dot1x-clientID
Aug 2 11:50:05.037 UTC: dot1x-sm:[MAC-address, Gi1/0/7] dot1x-clientID:request response action
Aug 2 11:50:05.038 UTC: AUTH-FEAT-SWITCH-PM-EVENT: [MAC-address, Gi1/0/7] mac addr process not notifying SM vlan 52
Aug 2 11:50:05.038 UTC: RADIUS/ENCODE(00000000):Orig. component type = Invalid
Aug 2 11:50:05.038 UTC: RADIUS(00000000): Config NAS IP: 0.0.0.0
Aug 2 11:50:05.038 UTC: RADIUS(00000000): Config NAS IPv6: ::
Aug 2 11:50:05.038 UTC: RADIUS(00000000): sending
Aug 2 11:50:05.038 UTC: AUTH-FEAT-SWITCH-PM-EVENT: [MAC-address, Gi1/0/7] Psecure datapkt: MacMAC-address:52 is not added by dot1x or is added as ask notifying dot1x
Aug 2 11:50:05.039 UTC: AUTH-FEAT-SWITCH-PM-EVENT: [MAC-address, Gi1/0/7] mac addr process not notifying SM vlan 52
Aug 2 11:50:05.039 UTC: AUTH-FEAT-WIRED-TUNNEL-ERROR: Client MAC-address doest not exist
Aug 2 11:50:05.039 UTC: RADIUS/ENCODE: Best Local IP-Address NASIP for Radius-Server IP-source
Aug 2 11:50:05.039 UTC: RADIUS(00000000): Send Access-Request to IP-source:1812 id 1645/123, len 255
Aug 2 11:50:05.039 UTC: RADIUS: authenticator AD HASH
Aug 2 11:50:05.039 UTC: RADIUS: User-Name [1] 12 "host/xxxx"
Aug 2 11:50:05.039 UTC: RADIUS: Service-Type [6] 6 Framed [2]
Aug 2 11:50:05.040 UTC: RADIUS: Vendor, Cisco [26] 27
Aug 2 11:50:05.040 UTC: RADIUS: Cisco AVpair [1] 21 "service-type=Framed"
Aug 2 11:50:05.040 UTC: RADIUS: Framed-MTU [12] 6 1500
Aug 2 11:50:05.040 UTC: RADIUS: Called-Station-Id [30] 19 "Authenticator MAC"
Aug 2 11:50:05.040 UTC: RADIUS: Calling-Station-Id [31] 19 "Client-MAC"
Aug 2 11:50:05.040 UTC: RADIUS: EAP-Message [79] 17
Aug 2 11:50:05.040 UTC: RADIUS: HASH [ host/OSN93]
Aug 2 11:50:05.040 UTC: RADIUS: Message-Authenticato[80] 18
Aug 2 11:50:05.040 UTC: RADIUS: HASH [ Ha'&@c]
Aug 2 11:50:05.040 UTC: RADIUS: EAP-Key-Name [102] 2 *
Aug 2 11:50:05.040 UTC: RADIUS: Vendor, Cisco [26] 49
Aug 2 11:50:05.040 UTC: RADIUS: Cisco AVpair [1] 43 "audit-session-id=AC186AFB0000D1995C090312"
Aug 2 11:50:05.040 UTC: RADIUS: Vendor, Cisco [26] 20
Aug 2 11:50:05.040 UTC: RADIUS: Cisco AVpair [1] 14 "method=dot1x"
Aug 2 11:50:05.040 UTC: RADIUS: NAS-IP-Address [4] 6 NASIP
Aug 2 11:50:05.041 UTC: RADIUS: NAS-Port-Id [87] 22 "GigabitEthernet1/0/7"
Aug 2 11:50:05.041 UTC: RADIUS: NAS-Port-Type [61] 6 Ethernet [15]
Aug 2 11:50:05.041 UTC: RADIUS: NAS-Port [5] 6 NAS-PORT
Aug 2 11:50:05.041 UTC: RADIUS(00000000): Sending a IPv4 Radius Packet
Aug 2 11:50:05.041 UTC: RADIUS(00000000): Started 5 sec timeout
Aug 2 11:50:05.044 UTC: RADIUS: Received from id 1645/123 IP-source:1812, Access-Reject, len 38
Aug 2 11:50:05.044 UTC: RADIUS: authenticator HASH
Aug 2 11:50:05.044 UTC: RADIUS: Message-Authenticato[80] 18
Aug 2 11:50:05.045 UTC: RADIUS: HASH [ +gL']
Aug 2 11:50:05.045 UTC: RADIUS(00000000): Received from id 1645/123
Aug 2 11:50:05.045 UTC: dot1x-ev:[MAC-address, Gi1/0/7] Received an EAP Fail
Aug 2 11:50:05.045 UTC: dot1x-sm:[MAC-address, Gi1/0/7] Posting EAP_FAIL for dot1x-clientID
Aug 2 11:50:05.045 UTC: dot1x_auth_bend Gi1/0/7: during state auth_bend_response, got event 10(eapFail)
Aug 2 11:50:05.045 UTC: @@@ dot1x_auth_bend Gi1/0/7: auth_bend_response -> auth_bend_fail
Aug 2 11:50:05.045 UTC: dot1x-sm:[MAC-address, Gi1/0/7] dot1x-clientID:exiting response state
Aug 2 11:50:05.046 UTC: dot1x-sm:[MAC-address, Gi1/0/7] dot1x-clientID:entering fail state
Aug 2 11:50:05.046 UTC: dot1x-sm:[MAC-address, Gi1/0/7] dot1x-clientID:response fail action
Aug 2 11:50:05.046 UTC: dot1x_auth_bend Gi1/0/7: idle during state auth_bend_fail
Aug 2 11:50:05.046 UTC: @@@ dot1x_auth_bend Gi1/0/7: auth_bend_fail -> auth_bend_idle
Aug 2 11:50:05.046 UTC: dot1x-sm:[MAC-address, Gi1/0/7] dot1x-clientID:entering idle state
Aug 2 11:50:05.046 UTC: dot1x-sm:[MAC-address, Gi1/0/7] Posting AUTH_FAIL on Client dot1x-clientID
Aug 2 11:50:05.046 UTC: dot1x_auth Gi1/0/7: during state auth_authenticating, got event 15(authFail)
Aug 2 11:50:05.046 UTC: @@@ dot1x_auth Gi1/0/7: auth_authenticating -> auth_authc_result
Aug 2 11:50:05.046 UTC: dot1x-sm:[MAC-address, Gi1/0/7] dot1x-clientID:exiting authenticating state
Aug 2 11:50:05.046 UTC: dot1x-sm:[MAC-address, Gi1/0/7] dot1x-clientID:entering authc result state
Aug 2 11:50:05.046 UTC: %DOT1X-5-FAIL: Authentication failed for client (MAC-address) on Interface Gi1/0/7 AuditSessionID AC186AFB0000D1995C090312
Aug 2 11:50:05.046 UTC: dot1x-packet:[MAC-address, Gi1/0/7] Added username in dot1x
Aug 2 11:50:05.046 UTC: dot1x-packet:[MAC-address, Gi1/0/7] Dot1x did not receive any key data
Aug 2 11:50:05.047 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Authc failure from Dot1X (1), status Cred Fail (1) / event fail (1)
Aug 2 11:50:05.047 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Highest prio method: INVALID, Authz method: INVALID, Conn hdl: dot1x
Aug 2 11:50:05.047 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Client MAC-address, Method dot1x changing state from 'Running' to 'Authc Failed'
Aug 2 11:50:05.047 UTC: AUTH-EVENT: Raised event RX_METHOD_AUTHC_FAIL (6) on handle 0xF80000AE
Aug 2 11:50:05.047 UTC: AUTH-EVENT: Raised event REMOVE_USER_PROFILE (16) on handle 0xF80000AE
Aug 2 11:50:05.047 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Queued AUTHC FAIL (Cred Fail) from Dot1X for session 0xF80000AE (MAC-address)
Aug 2 11:50:05.047 UTC: AUTH-EVENT: Handling client event RX_METHOD_AUTHC_FAIL (6) for PRE, handle 0xF80000AE
Aug 2 11:50:05.047 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Client MAC-address, Context changing state from 'Running' to 'Authc Failed'
SWITCH-name#
Aug 2 11:50:05.052 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Processing default action(s) for event RX_IDENTITY_UPDATE for session 0xF80000AE.
Aug 2 11:50:05.052 UTC: AUTH-EVENT: [MAC-address, Gi1/0/7] Handling external PRE event AuthZ Success for context 0xF80000AE.
Aug 2 11:50:05.052 UTC: dot1x-ev:[MAC-address, Gi1/0/7] Delete auth client (dot1x-clientID) message
Aug 2 11:50:05.052 UTC: dot1x-ev:Auth client ctx destroyed

- Any ideas of what the problem might be? 

[paul]

The problem is almost always on the client side. You need to look at the client authentication setup and validate how the server validation section is configured.  Is the server validation appears to be setup correct then ensure the root cert referenced in the server validation is the same as the root cert used to issue the ISE EAP Authentication cert.  It's all client side troubleshooting.

[washingday]

Yes the problem seems to be on the client side. However, comparing the settings in the certificate manager on an authenticated station, there's really no difference between them from what I can see. 

[paul · Accepted solution]

Don't compare the certificate manager. Look at the client's server validation screen in the 802.1x setup first. See if there is any difference there. Ideally this should all be GPO controlled so there shouldn't be a difference.