AP not joining Controller

[singh7881]

I have a Cisco 3700 Series AP and trying to connect it to a WLC 5508. I have getting a certificate validation failed error, however when I use the command "show crypto pki certificates" no such expired certificate shows up. Previously had an issue with certificate on this AP, and I made it so that the WLC ignores expiry of certificates through the CLI. 

*Mar 1 00:01:19.167: %CAPWAP-3-ERRORLOG: Go join a capwap controller
*May 10 14:52:36.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 172.27.10.5 peer_port: 5246
*May 10 14:52:38.235: %PKI-3-CERTIFICATE_INVALID_EXPIRED: Certificate chain validation has failed. The certificate (SN: 1468F48300000002CF39) has expired. Validity period ended on 00:51:11 UTC Apr 26 2024Peer certificate verification failed 001A

*May 10 14:52:38.235: %CAPWAP-3-ERRORLOG: Certificate verification failed!
*May 10 14:52:38.235: DTLS_CLIENT_ERROR: ../capwap/base_capwap/capwap/base_capwap_wtp_dtls.c:467 Certificate verified failed!
*May 10 14:52:38.235: %DTLS-5-SEND_ALERT: Send FATAL : Bad certificate Alert to 172.27.10.5:5246set_radio_pwr_mode: bad radio unit# 0
set_radio_pwr_mode: bad radio unit# 1

[Mark Elsen]

 - FYI :: https://www.cisco.com/c/en/us/support/docs/field-notices/639/fn63942.html
                  What software version is the 5508 running ?

[singh7881]

It is running version 8.0.140.0

[Rich R]

8.0.140.0 will not work reliably without constant workarounds being applied.  You obviously have not read the field notices below - please do so without further delay?

You need to update to 8.5.182.11 (link below) and apply the config for expired certs as per Field Notice: FN63942 APs and WLCs Fail to Create CAPWAP Connections Due to Certificate Expiration  You will need to set the WLC time back to allow the AP to join and download new software and config.  After that you can re-enable NTP for correct system time.

Before you update to 8.5.182.11 check the compatibility matrix (link below) to make sure all your AP models will still be supported on the new version.

If you don't update the software you'll just be wasting time trying to resolve these issues.

[singh7881]

Hi Rich, thanks for this solution. I will try to apply it as soon as possible. @jagan.chowdam asked me to find the Cisco SHA1 device cert certificate in the WLC and I can see that it expired on the date shown in the logs in my original post. One thing I am wondering though is why is that only 1 AP was affected? All our other APs are also Cisco 3700 Series. If a certificate in the WLC has expired shouldn't it affect all APs?

[Rich R]

Some of the x700 APs had SHA1 certs and some had SHA2 certs depending on when they were manufactured and that means they behave differently when handling certificates.

[jagan.chowdam]

Please refer the Field Notice: https://www.cisco.com/c/en/us/support/docs/field-notices/639/fn63942.html

And follow the Workaround/Solution mentioned.

You have AirOS Controller 5508. Use command and look for Cisco SHA1 device cert entry

show certificate all

Jagan Chowdam

/**Pls rate useful responses**/

[jeremiah.cox2]

I had success in temporarily rolling the system date back on the controller as the certificate on the AP was expired. Spent several hours on with TAC before we discovered this. NTP will adjust the date/time on next poll though so as soon as that AP loses connectivity it wont connect again. Ultimately we upgraded our APs and Controllers.

[Leo Laohoo]

---

@jeremiah.cox2 wrote:
Spent several hours on with TAC before we discovered this.

---

Da fuq?  Several hours???  

We are all volunteers in this forum.  None of us work for Cisco TAC but @Mark Elsen & @jagan.chowdam have provided the right answer/solution within 45 minutes after this thread went up. 

[Rich R]

The solution for anyone else reading this is to upgrade the 5508 to 8.5.182.11 (link below) and read through all the field notices below carefully to make sure you've applied all the configuration required to deal with expired certs.

This is a very well known problem so it should have taken TAC about 2 minutes to diagnose this!  The fact that it took hours reflects on the quality of many first line TAC staff these days.

Note that both WLC and AP certs can expire so even if AP cert has not expired, very old WLCs like 5508 may also have expired certs.  This is covered in the field notices.

[Mark Elsen]

 @Rich R >...The fact that it took hours reflects on the quality of many first line TAC staff these days.
                                    We are the best!

 M.