WLC 9800 DTLS session timeout for cisco 3700 access points

[hara12386]

Hello,

We are facing issues with cisco 3700 series access points that are not joining WLC 9800-CL and we have the following message:

*Jan 13 20:37:19.011: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 10.126.2.2:5246
*Jan 13 20:37:29.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 10.126.2.2 peer_port: 5246
*Jan 13 20:37:58.999: DTLS_CLIENT_ERROR: ../capwap/base_capwap/dtls/base_capwap_dtls_connection_db.c:2214 Max retransmission count reached for Connection 0xE14AF8C!

*Jan 13 20:38:29.011: Delete of backup image not donewith status 1
*Jan 13 20:38:29.011: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 10.126.2.2:5246
*Jan 13 20:38:39.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 10.126.2.2 peer_port: 5246
*Jan 13 20:39:08.999: DTLS_CLIENT_ERROR: ../capwap/base_capwap/dtls/base_capwap_dtls_connection_db.c:2214 Max retransmission count reached for Connection 0xDEFEDB4!
--More--  
*Jan 13 20:39:39.011: Delete of backup image not donewith status 1
*Jan 13 20:39:39.011: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 10.126.2.2:5246
*Jan 13 20:39:44.511: %CAPWAP-3-DHCP_RENEW: Could not discover WLC. Either IP address is not assigned or assigned IP is wrong. Renewing DHCP IP.
*Jan 13 20:39:47.587: %DHCP-6-ADDRESS_ASSIGN: Interface BVI1 assigned DHCP address 10.153.40.66, mask 255.255.255.192, hostname AP3c08.f66c.fa40

Cisco TAC not helping a lot as when we did the debugs they don't see the cause of DTLS session termination.

Many thanks for your help!

[MHM Cisco World]

Show wireless stats ap history 

Show wireless stats ap discovery 

Show wireless stats ap join summary 

Share output of above 

MHM

[marce1000]

  - Check 9800-CL software version , use advisory release (preferred) such as 17.9.4a
  - Have  a checkup of the controller configuration with the CLI command show tech wireless and feed the output into :
                                                                                                               Wireless Config Analyzer

  - Further debug AP (not) joining issues using : https://logadvisor.cisco.com/logadvisor/wireless/9800/9800APJoin
  - If you are on track review stats from :            https://www.cisco.com/c/en/us/support/docs/wireless/catalyst-9800-series-wireless-controllers/217738-monitor-catalyst-9800-kpis-key-performa.html#anc4

      More advanced dtls troubleshooting related commands :
show wireless stats ap join summary
show wireless dtls connections
show platform hardware chassis active qfp feature wireless capwap datapath statistics drop all
show platform hardware chassis active qfp feature wireless capwap datapath mac-address <APradio-mac> details
show platform hardware chassis active qfp feature wireless capwap datapath mac-address <APradio-mac> statistics
show platform hardware chassis active qfp feature wireless dtls datapath statistics all
show platform hardware chassis active qfp statistics drop all | inc Global | Wls 

 M.

[marce1000]

  - Added reply : https://community.cisco.com/t5/wireless/3702i-factory-reset/m-p/3196243#M208026

 M.

[hara12386]

Hello marce1000, thank you for your inputs, the reset factory has been performed without success.

Even the manual download of the IOS version has been performed on the AP but the DTLS session remains down.

I think that there is some kind of issue related to the certificate..

[marce1000]

- Review all other items from my initial reply too , especially the use of
WirelessAnalyzer.

M.

[marce1000]

  @hara12386      >...I think that there is some kind of issue related to the certificate..
         - Also note that for the cloud based 9800 controller you need : https://community.cisco.com/t5/wireless/unable-to-create-trustpoint-on-9800-cl/m-p/4098545#M198609

 M.

[Haydn Andrews]

https://www.cisco.com/c/en/us/support/docs/field-notices/639/fn63942.html 

[Rich R]

What version of software are you running on the WLC?
Hint: refer to TAC recommended link below and upgrade as per Marce's advice.

Read through all the Field Notices below if you think it's a certificate issue - there's more than one problem which could be impacting you. 

If it's FN63942 then you need to configure the WLC as per the field notice (C9800 Command to Accept Expired Certificates), disable NTP and change the WLC time back to before the cert expired, allow the AP to join and get the updated config and code download, then re-enable NTP.  But first - make sure WLC IOS-XE is up to date.

[hara12386]

Hello,

To comeback to this issue, below the workaround that I found:

- upgrade the access point to version 17.9.4a manually (using TFTP)

- Clear the configuration using : clear capwap ap all-config

- Reboot the access point 1 or 2 times

I am sure that it's a bug related to those access points but as they are not under support anymore so no other possible fixes.

Many thanks,

Hamid