APs cannot join to WLC 2504 many logs %DTLS-3-HANDSHAKE_FAILURE:

[filiberto.aguirre]

Hi ,

We have a WLC 2504  runnning version 8.5.171.0 with 75 APs joined (with normal operation), several AP models are installed
AIR-AP3702I-UXK9, AIR-AP2702I-UXK9, AIR-CAP1702I-N-K9, AIR-CAP2702I-N-K9, AIR-AP2802I-N-K9,AIR-AP1572EAC-N-K9

yesterday suddenly 35 APs lost join to controller, when we review logs in WLC many logs regarding DTL handshake, in the APs there are also logs regarding DTLS erros

We have already followed the recomendations that was published on Field Notice: FN - 63942, we have applied the comands
ap cert-expiry-ignore mic enable
ap cert-expiry-ignore ssc enable

also we set set the WLC's clock back 1,2 years, but APs still can not join to WLC

any additional recommendation to clear this issue.

regards

WLC logs

*osapiBsnTimer: Dec 17 10:24:25.220: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:3231 Failed to complete DTLS handshake with peer 192.168.209.225
*osapiBsnTimer: Dec 17 10:24:24.212: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:3231 Failed to complete DTLS handshake with peer 192.168.209.212
*osapiBsnTimer: Dec 17 10:24:23.212: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:3231 Failed to complete DTLS handshake with peer 192.168.209.150
*osapiBsnTimer: Dec 17 10:23:46.408: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:3231 Failed to complete DTLS handshake with peer 192.168.209.216
*osapiBsnTimer: Dec 17 10:23:42.808: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:3231 Failed to complete DTLS handshake with peer 192.168.209.183
*osapiBsnTimer: Dec 17 10:23:33.796: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:3231 Failed to complete DTLS handshake with peer 192.168.209.230
*osapiBsnTimer: Dec 17 10:23:22.396: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:3231 Failed to complete DTLS handshake with peer 192.168.209.160
*osapiBsnTimer: Dec 17 10:23:20.596: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:3231 Failed to complete DTLS handshake with peer 192.168.209.242
*osapiBsnTimer: Dec 17 10:23:18.796: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:3231 Failed to complete DTLS handshake with peer 192.168.209.159
*osapiBsnTimer: Dec 17 10:23:04.188: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:3231 Failed to complete DTLS handshake with peer 192.168.209.161
*osapiBsnTimer: Dec 17 10:23:03.388: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:3231 Failed to complete DTLS handshake with peer 192.168.209.156
*spamApTask4: Dec 17 10:22:42.435: %CAPWAP-3-DTLS_CLOSED_ERR: capwap_ac_sm.c:7130 00:62:ec:35:0d:40: DTLS connection closed forAP 192:168:209:235 (54919), Controller: 192:168:209:253 (5246) Echo Timer Expiry
*osapiBsnTimer: Dec 17 10:22:22.764: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:3231 Failed to complete DTLS handshake with peer 192.168.209.217
*osapiBsnTimer: Dec 17 10:22:19.164: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:3231 Failed to complete DTLS handshake with peer 192.168.209.224
*osapiBsnTimer: Dec 17 10:22:16.764: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:3231 Failed to complete DTLS handshake with peer 192.168.209.234

AP logs

*Dec 17 07:25:57.987: DTLS_CLIENT_ERROR: ../capwap/base_capwap/dtls/base_capwap_dtls_connection_db.c:2214 Max retransmission count reached for Connection 0xDEE0D5C!
*Dec 17 07:26:19.351: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 192.168.209.253:5246
*Dec 17 07:26:19.455: AP has SHA2 MIC certificate - Using SHA2 MIC certificate for DTLS.
*Dec 17 07:26:19.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 192.168.209.253 peer_port: 5246
*Dec 17 07:26:49.115: DTLS_CLIENT_ERROR: ../capwap/base_capwap/dtls/base_capwap_dtls_connection_db.c:2214 Max retransmission count reached for Connection 0xED8B530!
*Dec 17 07:27:19.163: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 192.168.209.253:5246
*Dec 17 07:27:19.267: AP has SHA2 MIC certificate - Using SHA2 MIC certificate for DTLS.
*Dec 17 07:16:43.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 192.168.209.251 peer_port: 5246
*Dec 17 07:16:54.607: DTLS_CLIENT_ERROR: ../capwap/base_capwap/dtls/base_capwap_dtls_record.c:397 BD is not of DTLS Change Cipher Spec type
*Dec 17 07:16:54.607: %DTLS-5-SEND_ALERT: Send FATAL : Internal error Alert to 192.168.209.251:5246
*Dec 17 07:16:54.607: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 192.168.209.251:5246
*Dec 17 07:18:16.783: AP has SHA2 MIC certificate - Using SHA2 MIC certificate for DTLS.

[Scott Fella]

I don't see anything in regard to cert expiry in the log you have posted.  Anyways' you already entered the workaround commands, manually set the time and disabled ntp.  I would just look at one of the AP's and maybe factory reset it and take a look at the full output when you boot it up.  A simple test also would be to take one of the ap's  and connect it to the same vlan as the controller mangagement interface and see if the ap joins.

[filiberto.aguirre]

Hi Scott,

Thanks  a lot for your time and  recommendations.

I will factory reset one of the AP and check the output

regards

All APs and WLC are in the same network  192.168.209.X.

I will try 

[Scott Fella]

Be careful, because you listed some UX models which require provisioning with an app that is no longer supported.  The only way you can provision them is to make sure they are close to another UX access point so they can get their country code.

AIR-AP3702I-UXK9

AIR-AP2702I-UXK9

I would suggest you factory reset one of these:

AIR-CAP1702I-N-K9, AIR-CAP2702I-N-K9, AIR-AP2802I-N-K9

[Leo Laohoo]

---

@filiberto.aguirre wrote:

also we set set the WLC's clock back 1,2 years, but APs still can not join to WLC

---

Wait, what?  
Roll back the date to December 2, 2022 is enough.  1 to 2 years will cause the WLC to totally ignore the APs.

[marce1000]

 - Verify integrity of the intranet network , look at port counters for the access points and switch logs where they are connected to : in essence make sure the network does not have another problem which could lead to this , 

 M.

[filiberto.aguirre]

Hi marce1000,

After checking sw core ( 2 switches in stack  and stack power) found  that second switch had problems with its power supplies, switch was unstable and many flappling taking place on sw core but also on access switches.

The reload of the second switch clear the issue of the power supplies and also the problems with APs.

we hope second  switch keeps working stable from now on. 

regards