topic Re: AP3802i wont join 5508 controller in Wireless

AP3802i wont join 5508 controller

Paul.Ostaszewski — Tue, 06 Jun 2023 01:19:00 GMT

I have a 5508 controller running 8.3.150 that I have four AIR-CAP3702I-A-K9 running on currently. I purchased four AIR-AP3802I-B-K9 to replace the four 3702's that I'm currently running.

My SHA1 cert is expired.

Certificate Name: Cisco SHA1 device cert

Subject Name :
C=US, ST=California, L=San Jose, O=Cisco Systems, CN=AIR-CT5508-K9-503de5aec9a0, emailAddress=support@cisco.com
Issuer Name :
O=Cisco Systems, CN=Cisco Manufacturing CA
Validity :
Start : Apr 8 13:39:08 2011 GMT
End : Apr 8 13:49:08 2021 GMT

I have disabled NTP and set the clock on the 5508 back to before the certificate expires.

I have also ran this command: config ap cert-expiry-ignore mic enable

I have a 25 license count for which 4 are being used by the 3702's.

Here is the debug from the controller:

*sshpmLscTask: Apr 08 04:03:02.699: sshpmLscTask: LSC Task received a message 4
*spamApTask5: Apr 08 04:03:34.746: 70:df:2f:05:0e:38 Failed to parse CAPWAP packet from 10.10.10.211:5248

*spamApTask5: Apr 08 04:03:44.230: sshpmGetCID: called to evaluate <cscoSha2IdCert>

*spamApTask5: Apr 08 04:03:44.230: sshpmGetCID: failed to find matching cert name cscoSha2IdCert

*spamApTask5: Apr 08 04:03:44.230: GetIDCert: Using SHA2 Id cert on WLC

*spamApTask5: Apr 08 04:03:44.230: sshpmGetCID: called to evaluate <cscoDefaultIdCert>

*spamApTask5: Apr 08 04:03:44.230: sshpmGetCID: Found matching ID cert cscoDefaultIdCert in row 2
*spamApTask5: Apr 08 04:03:44.230: Get Cert from CID: For CID 1db5a6a2 certType 1
*spamApTask5: Apr 08 04:03:44.230: Get Cert from CID: Found match of ID Cert in row 2
*spamApTask5: Apr 08 04:03:44.230: sshpmGetCID: called to evaluate <cscoSha2IdCert>

*spamApTask5: Apr 08 04:03:44.230: sshpmGetCID: failed to find matching cert name cscoSha2IdCert

*spamApTask5: Apr 08 04:03:44.231: GetDERIDKey: Using SHA2 Id cert Private Keys on WLC

*spamApTask5: Apr 08 04:03:44.231: sshpmGetCID: called to evaluate <cscoDefaultIdCert>

*spamApTask5: Apr 08 04:03:44.231: sshpmGetCID: Found matching ID cert cscoDefaultIdCert in row 2
*spamApTask5: Apr 08 04:03:44.231: GetPrivateKey: called to get key for CID 1db5a6a2

*spamApTask5: Apr 08 04:03:44.231: Private Key found row 2 KeyBufLen 2048 Keylen 1191 PrivateKeyPtr 0x2c4945b0

*spamApTask5: Apr 08 04:03:44.271: OpenSSL Get Issuer Handles: locking ca cert table

*spamApTask5: Apr 08 04:03:44.272: OpenSSL Get Issuer Handles: x509 subject_name /serialNumber=PID:AP3800 SN:FOC212448KU/O=Cisco/OU=ACT-2 Lite SUDI/CN=AP3800

*spamApTask5: Apr 08 04:03:44.272: OpenSSL Get Issuer Handles: issuer_name /O=Cisco/CN=ACT2 SUDI CA

*spamApTask5: Apr 08 04:03:44.272: OpenSSL Get Issuer Handles: CN AP3800

*spamApTask5: Apr 08 04:03:44.272: OpenSSL Get Issuer Handles: issuerCertCN ACT2 SUDI CA

*spamApTask5: Apr 08 04:03:44.272: OpenSSL Get Issuer Handles: Cert Name in subject is AP3800

*spamApTask5: Apr 08 04:03:44.272: OpenSSL Get Issuer Handles: Extracted cert issuer from subject name.

*spamApTask5: Apr 08 04:03:44.272: NMSP:: Algo name matched SHA256

*spamApTask5: Apr 08 04:03:44.272: ACT2 RSA SHA1 certificate

*spamApTask5: Apr 08 04:03:44.272: ACT2 dummy mac: MAC: 1122.3344.5566

*spamApTask5: Apr 08 04:03:44.272: OpenSSL Get Issuer Handles: Cert is issued by Cisco Systems.

*spamApTask5: Apr 08 04:03:44.272: Retrieving x509 cert for CertName cscoAct2RsaCaCert

*spamApTask5: Apr 08 04:03:44.272: sshpmGetCID: called to evaluate <cscoAct2RsaCaCert>

*spamApTask5: Apr 08 04:03:44.272: sshpmGetCID: Found matching CA cert cscoAct2RsaCaCert in row 8
*spamApTask5: Apr 08 04:03:44.272: Found CID 2b40476b for certname cscoAct2RsaCaCert

*spamApTask5: Apr 08 04:03:44.272: CACertTable: Found matching CID cscoAct2RsaCaCert in row 8 x509 0x2cc7be3c
*spamApTask5: Apr 08 04:03:44.273: Retrieving x509 cert for CertName cscoDefaultNewRootCaCert

*spamApTask5: Apr 08 04:03:44.273: sshpmGetCID: called to evaluate <cscoDefaultNewRootCaCert>

*spamApTask5: Apr 08 04:03:44.273: sshpmGetCID: Found matching CA cert cscoDefaultNewRootCaCert in row 4
*spamApTask5: Apr 08 04:03:44.273: Found CID 29307290 for certname cscoDefaultNewRootCaCert

*spamApTask5: Apr 08 04:03:44.273: CACertTable: Found matching CID cscoDefaultNewRootCaCert in row 4 x509 0x2cc7cd00
*spamApTask5: Apr 08 04:03:44.273: cscoAct2RsaCaCert: successfully added ACT2 RSA to store cert Verify User Certificate(?!)

*spamApTask5: Apr 08 04:03:44.279: Verify User Certificate: X509 Cert Verification return code: 1
*spamApTask5: Apr 08 04:03:44.279: Verify User Certificate: X509 Cert Verification result text: ok
*spamApTask5: Apr 08 04:03:44.279: sshpmGetCID: called to evaluate <cscoAct2RsaCaCert>

*spamApTask5: Apr 08 04:03:44.279: sshpmGetCID: Found matching CA cert cscoAct2RsaCaCert in row 8
*spamApTask5: Apr 08 04:03:44.279: Verify User Certificate: OPENSSL X509_Verify: AP Cert Verfied Using >cscoAct2RsaCaCert<

*spamApTask5: Apr 08 04:03:44.279: OpenSSL Get Issuer Handles: Check cert validity times (allow expired YES)
*spamApTask5: Apr 08 04:03:44.279: sshpmGetCID: called to evaluate <cscoDefaultIdCert>

*spamApTask5: Apr 08 04:03:44.279: sshpmGetCID: Found matching ID cert cscoDefaultIdCert in row 2
*spamApTask5: Apr 08 04:03:44.279: sshpmFreePublicKeyHandle: called with 0x1b0b5cc8

*spamApTask5: Apr 08 04:03:44.279: sshpmFreePublicKeyHandle: freeing public key

*spamApTask5: Apr 08 04:04:56.104: 70:df:2f:05:0e:38 Failed to parse CAPWAP packet from 10.10.10.211:5248

Here is the debug from the AP3802i:

[*04/08/2021 09:03:34.7848] CAPWAP State: Discovery
[*04/08/2021 09:03:34.7856] Got WLC address 192.168.1.10 from DHCP.
[*04/08/2021 09:03:34.7856] IP DNS query for CISCO-CAPWAP-CONTROLLER.zewsworld.com
[*04/08/2021 09:03:34.8559] Discovery Request sent to 192.168.1.10, discovery type DHCP(2)
[*04/08/2021 09:03:34.8600] Discovery Request sent to 255.255.255.255, discovery type UNKNOWN(0)
[*04/08/2021 09:03:34.8602] Discovery Response from 192.168.1.10
[*04/08/2021 09:03:44.0001] Started wait dtls timer (60 sec)
[*04/08/2021 09:03:44.0005]
[*04/08/2021 09:03:44.0005] CAPWAP State: DTLS Setup
[*04/08/2021 09:03:44.0409] dtls_verify_server_cert: Controller certificate verification successful
[*04/08/2021 09:03:44.7221]
[*04/08/2021 09:03:44.7221] CAPWAP State: Join
[*04/08/2021 09:03:44.7389] Sending Join request to 192.168.1.10 through port 5248
[*04/08/2021 09:04:41.0377]
[*04/08/2021 09:04:41.0377] CAPWAP State: DTLS Teardown
[*04/08/2021 09:04:41.1404] status 'upgrade.sh: Script called with args:[CANCEL]'
[*04/08/2021 09:04:41.1975] do CANCEL, part2 is active part
[*04/08/2021 09:04:41.2120] status 'upgrade.sh: Cleanup tmp files ...'
[*04/08/2021 09:04:41.2453] Dropping dtls packet since session is not established. Peer 192.168.1.10-5246, Local 10.10.10.211-5248, conn (nil)
[*04/08/2021 09:04:41.2454] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: DTLS Teardown(4).
[*04/08/2021 09:04:41.2455] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: DTLS Teardown(4).
[*04/08/2021 09:04:55.7907]
[*04/08/2021 09:04:55.7907] CAPWAP State: Discovery
[*04/08/2021 09:04:55.7915] Got WLC address 192.168.1.10 from DHCP.
[*04/08/2021 09:04:55.7915] IP DNS query for CISCO-CAPWAP-CONTROLLER.zewsworld.com
[*04/08/2021 09:04:55.8696] Discovery Request sent to 192.168.1.10, discovery type DHCP(2)
[*04/08/2021 09:04:55.8721] Discovery Request sent to 255.255.255.255, discovery type UNKNOWN(0)
[*04/08/2021 09:04:55.8743] Discovery Response from 192.168.1.10
[*04/08/2021 09:05:06.0001] Started wait dtls timer (60 sec)
[*04/08/2021 09:05:06.0005]
[*04/08/2021 09:05:06.0005] CAPWAP State: DTLS Setup
[*04/08/2021 09:05:06.0446] dtls_verify_server_cert: Controller certificate verification successful
[*04/08/2021 09:05:06.7043]
[*04/08/2021 09:05:06.7043] CAPWAP State: Join
[*04/08/2021 09:05:06.7116] Sending Join request to 192.168.1.10 through port 5248
[*04/08/2021 09:06:03.0353]
[*04/08/2021 09:06:03.0353] CAPWAP State: DTLS Teardown
[*04/08/2021 09:06:03.1216] status 'upgrade.sh: Script called with args:[CANCEL]'
[*04/08/2021 09:06:03.1782] do CANCEL, part2 is active part
[*04/08/2021 09:06:03.1927] status 'upgrade.sh: Cleanup tmp files ...'
[*04/08/2021 09:06:03.2260] Dropping dtls packet since session is not established. Peer 192.168.1.10-5246, Local 10.10.10.211-5248, conn (nil)
[*04/08/2021 09:06:03.2261] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: DTLS Teardown(4).
[*04/08/2021 09:06:03.2261] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: DTLS Teardown(4).

I think I've tried everything that's available to get these AP's to join. Anything I missed? I don't have support on this controller so upgrading to 8.5.182 or higher isn't really an option.

Here's more information:

(Cisco Controller) >show sysinfo

Manufacturer's Name.............................. Cisco Systems Inc.
Product Name..................................... Cisco Controller
Product Version.................................. 8.3.150.0
Bootloader Version............................... 1.0.20
Field Recovery Image Version..................... 7.6.101.1
Firmware Version................................. FPGA 1.7, Env 1.8, USB console 1.27
OUI File Update Time............................. Sun Sep 07 10:44:07 IST 2014

Build Type....................................... DATA + WPS

System Name......................................
System Location..................................
System Contact...................................
System ObjectID.................................. 1.3.6.1.4.1.9.1.1069
Redundancy Mode.................................. Disabled
IP Address....................................... 192.168.1.10
IPv6 Address..................................... ::
Last Reset....................................... Power on reset
System Up Time................................... 98 days 19 hrs 4 mins 29 secs
System Timezone Location......................... (GMT -6:00) Central Time (US and Canada)
System Stats Realtime Interval................... 5

--More-- or (q)uit
System Stats Normal Interval..................... 180

Configured Country............................... US - United States
Operating Environment............................ Commercial (0 to 40 C)
Internal Temp Alarm Limits....................... 0 to 65 C
Internal Temperature............................. +39 C
External Temperature............................. +21 C
Fan Status....................................... OK

State of 802.11b Network......................... Enabled
State of 802.11a Network......................... Enabled
Number of WLANs.................................. 2
Number of Active Clients......................... 35

OUI Classification Failure Count................. 8443

Burned-in MAC Address............................ 50:3D:E5:AE:C9:A0
Power Supply 1................................... Present, OK
Power Supply 2................................... Absent
Maximum number of APs supported.................. 500
System Nas-Id....................................
WLC MIC Certificate Types........................ SHA1

Any help would be greatly appreciated!

Thank you in advance

Re: AP3802i wont join 5508 controller

Leo Laohoo — Tue, 06 Jun 2023 01:53:13 GMT

Put NTP back on. The 3800 is not affected by the FNs.

Re: AP3802i wont join 5508 controller

Paul.Ostaszewski — Tue, 06 Jun 2023 14:35:33 GMT

With the time set correctly, the DTLS fails due to the certificate being
expired and the AP3802i will not join.

Re: AP3802i wont join 5508 controller

Leo Laohoo — Tue, 06 Jun 2023 23:46:36 GMT

Factory reset the AP using the command "capwap ap reset all".

Re: AP3802i wont join 5508 controller

Paul.Ostaszewski — Tue, 06 Jun 2023 23:56:33 GMT

What’s the username/password? I tried cisco/cisco and couldn’t get to the
command prompt.

Re: AP3802i wont join 5508 controller

Leo Laohoo — Wed, 07 Jun 2023 00:33:40 GMT

Power down the AP.
While still powered down, hold down the Mode button.
Keep holding the Mode button and apply power.
Keep the Mode button held down for 30 seconds and then release.

Re: AP3802i wont join 5508 controller

Paul.Ostaszewski — Wed, 07 Jun 2023 01:33:37 GMT

I've done that before, and I can't get to a command prompt without a username/password.

Re: AP3802i wont join 5508 controller

Paul.Ostaszewski — Fri, 09 Jun 2023 06:10:16 GMT

[*06/09/2023 06:08:13.7793] CAPWAP State: Discovery
[*06/09/2023 06:08:13.7801] Got WLC address 192.168.1.10 from DHCP.
[*06/09/2023 06:08:13.7801] IP DNS query for CISCO-CAPWAP-CONTROLLER.zewsworld.com
[*06/09/2023 06:08:13.8633] Discovery Request sent to 192.168.1.10, discovery type DHCP(2)
[*06/09/2023 06:08:13.8709] Discovery Request sent to 255.255.255.255, discovery type UNKNOWN(0)
[*06/09/2023 06:08:13.8710] Discovery Response from 192.168.1.10
[*06/09/2023 06:08:23.0000] Started wait dtls timer (60 sec)
[*06/09/2023 06:08:23.0004]
[*06/09/2023 06:08:23.0004] CAPWAP State: DTLS Setup
[*06/09/2023 06:08:23.0489] Certificate is expired
[*06/09/2023 06:08:23.0490] Certificate Start Date: Apr 8 13:39:08 2011 GMT
[*06/09/2023 06:08:23.0490] Certificate End Date: Apr 8 13:49:08 2021 GMT
[*06/09/2023 06:08:23.0490] display_verify_cert_status: Verify Cert: FAILED at 0 depth: certificate has expired
[*06/09/2023 06:08:23.0491] X509 OpenSSL Errors...
[*06/09/2023 06:08:23.0491]
[*06/09/2023 06:08:23.0492] 1956686832:error:0909006C:lib(9):func(144):reason(108):NA:0:Expecting: CERTIFICATE
[*06/09/2023 06:08:23.0492] 1956686832:error:0909006C:lib(9):func(144):reason(108):NA:0:Expecting: CERTIFICATE
[*06/09/2023 06:08:23.0492] 1956686832:error:0909006C:lib(9):func(144):reason(108):NA:0:Expecting: CERTIFICATE
[*06/09/2023 06:08:23.0492] 1956686832:error:0909006C:lib(9):func(144):reason(108):NA:0:Expecting: CERTIFICATE
[*06/09/2023 06:08:23.0492]
[*06/09/2023 06:08:23.0492]
[*06/09/2023 06:08:23.0492]
[*06/09/2023 06:08:23.0492] dtls_verify_server_cert: Controller certificate verification error
[*06/09/2023 06:08:23.0496] 1956686832:error:1416F086:lib(20):func(367):reason(134):NA:0:
[*06/09/2023 06:08:23.0496] dtls_process_packet: Error connecting TLS context ERR: 5
[*06/09/2023 06:08:23.0502] DTLS: Error while processing DTLS packet 0x55e65000.

Re: AP3802i wont join 5508 controller

Leo Laohoo — Fri, 09 Jun 2023 07:14:06 GMT

What is the serial number of this AP?

Re: AP3802i wont join 5508 controller

Paul.Ostaszewski — Fri, 09 Jun 2023 07:18:26 GMT

FCW2125JNSF

Re: AP3802i wont join 5508 controller

Leo Laohoo — Fri, 09 Jun 2023 08:14:39 GMT

Post the complete output to the command "sh version" taken from the AP.

Re: AP3802i wont join 5508 controller

Paul.Ostaszewski — Fri, 09 Jun 2023 16:49:09 GMT

AP70DF.2F05.0E38>sh version
Restricted Rights Legend

Use, duplication, or disclosure by the Government is subject to
restrictions as set forth in subparagraph (c) of the Commercial
Computer Software - Restricted Rights clause at FAR sec. 52.227-19 and
subparagraph (c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.

Cisco Systems, Inc.
170 West Tasman Drive
San Jose, California 95134-1706

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

This product contains some software licensed under the
"GNU General Public License, version 2" provided with
ABSOLUTELY NO WARRANTY under the terms of
"GNU General Public License, version 2", available here:
http://www.gnu.org/licenses/old-licenses/gpl-2.0.html

This product contains some software licensed under the
"GNU Library General Public License, version 2" provided
with ABSOLUTELY NO WARRANTY under the terms of "GNU Library
General Public License, version 2", available here:
http://www.gnu.org/licenses/old-licenses/lgpl-2.0.html

This product contains some software licensed under the
"GNU Lesser General Public License, version 2.1" provided
with ABSOLUTELY NO WARRANTY under the terms of "GNU Lesser
General Public License, version 2.1", available here:
http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html

This product contains some software licensed under the
"GNU General Public License, version 3" provided with
ABSOLUTELY NO WARRANTY under the terms of
"GNU General Public License, Version 3", available here:
http://www.gnu.org/licenses/gpl.html.

This product contains some software licensed under the
"GNU Affero General Public License, version 3" provided
with ABSOLUTELY NO WARRANTY under the terms of
"GNU Affero General Public License, version 3", available here:
http://www.gnu.org/licenses/agpl-3.0.html.

Cisco AP Software, (ap3g3), C3802, RELEASE SOFTWARE
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2023 by Cisco Systems, Inc.
Compiled Wed Aug 10 23:07:52 GMT 2022

ROM: Bootstrap program is U-Boot boot loader
BOOTLDR: U-Boot boot loader Version 2013.01-g2ee3a18aa (Jun 03 2022 - 11:27:17)

AP70DF.2F05.0E38 uptime is 0 days, 0 hours, 33 minutes
Last reload time : Sat Jun 24 23:59:59 UTC 2017
Last reload reason : unknown

cisco AIR-AP3802I-B-K9 ARMv7 Processor rev 1 (v7l) with 1028224/579380K bytes of memory.
Processor board ID FCW2125JNSF
AP Running Image : 17.6.4.56
Primary Boot Image : 17.6.4.56
Backup Boot Image : 0.0.0.0
Primary Boot Image Hash:
Backup Boot Image Hash:
1 Multigigabit Ethernet interfaces
1 Gigabit Ethernet interfaces
2 802.11 Radios
Radio Driver version : 9.0.5.5-W8964
Radio FW version : 9.1.8.1
NSS FW version : 2.4.28

Base ethernet MAC Address : 70:DF:2F:05:0E:38
Part Number : 73-017278-06
PCA Assembly Number : 000-00000-00
PCA Revision Number :
PCB Serial Number : FOC212448KU
Top Assembly Part Number : 068-100730-01
Top Assembly Serial Number : FCW2125JNSF
Top Revision Number : C0
Product/Model Number : AIR-AP3802I-B-K9

I'm also seeing a new error on the console of the AP:

[*06/09/2023 16:45:10.7731] No more AP manager addresses remain..
[*06/09/2023 16:45:10.7731] No valid AP manager found for controller 'ZEW-WLC-01' (ip: 192.168.1.10)

See the console output below:

[*06/09/2023 16:45:06.0238] CAPWAP State: DTLS Teardown
[*06/09/2023 16:45:06.1096] status 'upgrade.sh: Script called with args:[CANCEL]'
[*06/09/2023 16:45:06.1682] do CANCEL, part2 is active part
[*06/09/2023 16:45:06.1829] status 'upgrade.sh: Cleanup tmp files ...'
[*06/09/2023 16:45:06.2252] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: DTLS Teardown(4).
[*06/09/2023 16:45:06.2252] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: DTLS Teardown(4).
[*06/09/2023 16:45:10.7731] No more AP manager addresses remain..
[*06/09/2023 16:45:10.7731] No valid AP manager found for controller 'ZEW-WLC-01' (ip: 192.168.1.10)
[*06/09/2023 16:45:10.7731] Failed to join controller ZEW-WLC-01.
[*06/09/2023 16:45:10.7732] Failed to join controller.
[*06/09/2023 16:45:30.7776]
[*06/09/2023 16:45:30.7777] CAPWAP State: Discovery
[*06/09/2023 16:45:30.7784] Got WLC address 192.168.1.10 from DHCP.
[*06/09/2023 16:45:30.7784] IP DNS query for CISCO-CAPWAP-CONTROLLER.zewsworld.com
[*06/09/2023 16:45:30.8304] Discovery Request sent to 192.168.1.10, discovery type DHCP(2)
[*06/09/2023 16:45:30.8332] Discovery Request sent to 255.255.255.255, discovery type UNKNOWN(0)
[*06/09/2023 16:45:30.8333] Discovery Response from 192.168.1.10
[*06/09/2023 16:45:42.0000] Started wait dtls timer (60 sec)
[*06/09/2023 16:45:42.0013]
[*06/09/2023 16:45:42.0013] CAPWAP State: DTLS Setup
[*06/09/2023 16:45:42.0443] Certificate is expired
[*06/09/2023 16:45:42.0443] Certificate Start Date: Apr 8 13:39:08 2011 GMT
[*06/09/2023 16:45:42.0444] Certificate End Date: Apr 8 13:49:08 2021 GMT
[*06/09/2023 16:45:42.0444] display_verify_cert_status: Verify Cert: FAILED at 0 depth: certificate has expired
[*06/09/2023 16:45:42.0445] X509 OpenSSL Errors...
[*06/09/2023 16:45:42.0445]
[*06/09/2023 16:45:42.0445] 1956203504:error:0909006C:lib(9):func(144):reason(108):NA:0:Expecting: CERTIFICATE
[*06/09/2023 16:45:42.0445] 1956203504:error:0909006C:lib(9):func(144):reason(108):NA:0:Expecting: CERTIFICATE
[*06/09/2023 16:45:42.0445] 1956203504:error:0909006C:lib(9):func(144):reason(108):NA:0:Expecting: CERTIFICATE
[*06/09/2023 16:45:42.0445] 1956203504:error:0909006C:lib(9):func(144):reason(108):NA:0:Expecting: CERTIFICATE
[*06/09/2023 16:45:42.0445]
[*06/09/2023 16:45:42.0445]
[*06/09/2023 16:45:42.0445]
[*06/09/2023 16:45:42.0446] dtls_verify_server_cert: Controller certificate verification error
[*06/09/2023 16:45:42.0450] 1956203504:error:1416F086:lib(20):func(367):reason(134):NA:0:
[*06/09/2023 16:45:42.0450] dtls_process_packet: Error connecting TLS context ERR: 5
[*06/09/2023 16:45:42.0455] DTLS: Error while processing DTLS packet 0x54886000.

Re: AP3802i wont join 5508 controller

Paul.Ostaszewski — Fri, 09 Jun 2023 16:52:37 GMT

Leo,

I was also able to upgrade to 8.5.182.0 last night, and I still have the same issues.

Re: AP3802i wont join 5508 controller

karol.krzyzyk — Fri, 09 Jun 2023 21:09:19 GMT

Hello All

I have WLC 5508 with 8.5.182.0 software and start from last week I have crazy situation . In one location we have Primary WLC and in another Backup wlc. from last week half of our AP jump between backup and primary. this dive me crazy. .

on web log found

*spamApTask6: Jun 09 22:42:41.078: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 d0:c7:89:c6:c8:60: DTLS connection closed forAP 10:4:129:33 (50757), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask6: Jun 09 22:42:41.077: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: d0:c7:89:c6:c8:60
*spamApTask5: Jun 09 22:42:29.078: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 0c:68:03:38:78:30: DTLS connection closed forAP 10:4:129:34 (50675), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask5: Jun 09 22:42:29.077: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: 0c:68:03:38:78:30
*emWeb: Jun 09 22:41:32.388: %EMWEB-3-FORM_SUBMIT_CSRF_DETECTED: [PA]ews_form.c:1239 Form submit action failed. Cross Site Attack detected form_idx=256 url=/screens/banner.html formCsrfTbl[256]=1.
*spamApTask5: Jun 09 22:41:02.315: %LWAPP-3-VENDOR_PLD_VALIDATE_ERR: [PA]spam_lrad.c:12016 Validation of SPAM_VENDOR_SPECIFIC_PAYLOAD(185) with length=9 failed - AP d0:c7:89:c6:c8:60
*spamApTask3: Jun 09 22:40:50.980: %LWAPP-3-VENDOR_PLD_VALIDATE_ERR: [PA]spam_lrad.c:12016 Validation of SPAM_VENDOR_SPECIFIC_PAYLOAD(185) with length=9 failed - AP 0c:68:03:38:78:30
*spamApTask7: Jun 09 22:39:10.112: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 d0:c7:89:c6:c8:60: DTLS connection closed forAP 10:4:129:33 (50758), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask7: Jun 09 22:39:10.111: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: d0:c7:89:c6:c8:60
*spamApTask4: Jun 09 22:39:06.512: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 0c:68:03:af:9c:c0: DTLS connection closed forAP 10:4:17:22 (57758), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask4: Jun 09 22:39:06.511: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: 0c:68:03:af:9c:c0
*spamApTask4: Jun 09 22:39:00.112: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 0c:68:03:38:78:30: DTLS connection closed forAP 10:4:129:34 (50674), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask4: Jun 09 22:39:00.111: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: 0c:68:03:38:78:30
*spamApTask6: Jun 09 22:37:33.883: %LWAPP-3-VENDOR_PLD_VALIDATE_ERR: [PA]spam_lrad.c:12016 Validation of SPAM_VENDOR_SPECIFIC_PAYLOAD(185) with length=9 failed - AP 0c:68:03:af:9c:c0
*spamApTask6: Jun 09 22:37:32.788: %LWAPP-3-VENDOR_PLD_VALIDATE_ERR: [PA]spam_lrad.c:12016 Validation of SPAM_VENDOR_SPECIFIC_PAYLOAD(185) with length=9 failed - AP d0:c7:89:c6:c8:60
*spamApTask2: Jun 09 22:37:21.889: %LWAPP-3-VENDOR_PLD_VALIDATE_ERR: [PA]spam_lrad.c:12016 Validation of SPAM_VENDOR_SPECIFIC_PAYLOAD(185) with length=9 failed - AP 0c:68:03:38:78:30
*spamApTask6: Jun 09 22:35:27.912: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 d0:c7:89:c6:c8:60: DTLS connection closed forAP 10:4:129:33 (50757), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask6: Jun 09 22:35:27.911: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: d0:c7:89:c6:c8:60
*spamApTask5: Jun 09 22:35:16.912: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 0c:68:03:38:78:30: DTLS connection closed forAP 10:4:129:34 (50675), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask5: Jun 09 22:35:16.911: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: 0c:68:03:38:78:30
*spamApTask3: Jun 09 22:34:37.112: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 0c:68:03:af:9c:c0: DTLS connection closed forAP 10:4:17:22 (57757), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask3: Jun 09 22:34:37.111: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: 0c:68:03:af:9c:c0
*spamApTask5: Jun 09 22:33:50.769: %LWAPP-3-VENDOR_PLD_VALIDATE_ERR: [PA]spam_lrad.c:12016 Validation of SPAM_VENDOR_SPECIFIC_PAYLOAD(185) with length=9 failed - AP d0:c7:89:c6:c8:60
*spamApTask3: Jun 09 22:33:39.797: %LWAPP-3-VENDOR_PLD_VALIDATE_ERR: [PA]spam_lrad.c:12016 Validation of SPAM_VENDOR_SPECIFIC_PAYLOAD(185) with length=9 failed - AP 0c:68:03:38:78:30
*spamApTask5: Jun 09 22:33:05.824: %LWAPP-3-VENDOR_PLD_VALIDATE_ERR: [PA]spam_lrad.c:12016 Validation of SPAM_VENDOR_SPECIFIC_PAYLOAD(185) with length=9 failed - AP 0c:68:03:af:9c:c0
*spamApTask7: Jun 09 22:31:46.708: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 d0:c7:89:c6:c8:60: DTLS connection closed forAP 10:4:129:33 (50758), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask7: Jun 09 22:31:46.707: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: d0:c7:89:c6:c8:60
*spamApTask4: Jun 09 22:31:34.908: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 0c:68:03:38:78:30: DTLS connection closed forAP 10:4:129:34 (50674), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask4: Jun 09 22:31:34.907: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: 0c:68:03:38:78:30
*spamApTask4: Jun 09 22:30:13.508: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 0c:68:03:af:9c:c0: DTLS connection closed forAP 10:4:17:22 (57758), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask4: Jun 09 22:30:13.507: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: 0c:68:03:af:9c:c0
*spamApTask6: Jun 09 22:30:08.631: %LWAPP-3-VENDOR_PLD_VALIDATE_ERR: [PA]spam_lrad.c:12016 Validation of SPAM_VENDOR_SPECIFIC_PAYLOAD(185) with length=9 failed - AP d0:c7:89:c6:c8:60
*spamApTask2: Jun 09 22:29:57.574: %LWAPP-3-VENDOR_PLD_VALIDATE_ERR: [PA]spam_lrad.c:12016 Validation of SPAM_VENDOR_SPECIFIC_PAYLOAD(185) with length=9 failed - AP 0c:68:03:38:78:30
*spamApTask6: Jun 09 22:28:36.743: %LWAPP-3-VENDOR_PLD_VALIDATE_ERR: [PA]spam_lrad.c:12016 Validation of SPAM_VENDOR_SPECIFIC_PAYLOAD(185) with length=9 failed - AP 0c:68:03:af:9c:c0
*spamApTask6: Jun 09 22:28:04.708: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 d0:c7:89:c6:c8:60: DTLS connection closed forAP 10:4:129:33 (50757), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask6: Jun 09 22:28:04.707: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: d0:c7:89:c6:c8:60
*spamApTask5: Jun 09 22:27:53.708: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 0c:68:03:38:78:30: DTLS connection closed forAP 10:4:129:34 (50675), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask5: Jun 09 22:27:53.707: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: 0c:68:03:38:78:30
*spamApTask5: Jun 09 22:26:26.560: %LWAPP-3-VENDOR_PLD_VALIDATE_ERR: [PA]spam_lrad.c:12016 Validation of SPAM_VENDOR_SPECIFIC_PAYLOAD(185) with length=9 failed - AP d0:c7:89:c6:c8:60
*spamApTask3: Jun 09 22:26:15.548: %LWAPP-3-VENDOR_PLD_VALIDATE_ERR: [PA]spam_lrad.c:12016 Validation of SPAM_VENDOR_SPECIFIC_PAYLOAD(185) with length=9 failed - AP 0c:68:03:38:78:30
*spamApTask3: Jun 09 22:25:17.908: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 0c:68:03:af:9c:c0: DTLS connection closed forAP 10:4:17:22 (57757), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask3: Jun 09 22:25:17.907: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: 0c:68:03:af:9c:c0
*spamApTask7: Jun 09 22:24:22.708: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 d0:c7:89:c6:c8:60: DTLS connection closed forAP 10:4:129:33 (50758), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask7: Jun 09 22:24:22.707: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: d0:c7:89:c6:c8:60
*spamApTask5: Jun 09 22:24:08.162: %LWAPP-3-VENDOR_PLD_VALIDATE_ERR: [PA]spam_lrad.c:12016 Validation of SPAM_VENDOR_SPECIFIC_PAYLOAD(185) with length=9 failed - AP 0c:68:03:af:9c:c0
*spamApTask4: Jun 09 22:24:06.708: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 0c:68:03:38:78:30: DTLS connection closed forAP 10:4:129:34 (50674), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask4: Jun 09 22:24:06.707: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: 0c:68:03:38:78:30
*spamApTask6: Jun 09 22:22:44.459: %LWAPP-3-VENDOR_PLD_VALIDATE_ERR: [PA]spam_lrad.c:12016 Validation of SPAM_VENDOR_SPECIFIC_PAYLOAD(185) with length=9 failed - AP d0:c7:89:c6:c8:60
*spamApTask2: Jun 09 22:22:32.714: %LWAPP-3-VENDOR_PLD_VALIDATE_ERR: [PA]spam_lrad.c:12016 Validation of SPAM_VENDOR_SPECIFIC_PAYLOAD(185) with length=9 failed - AP 0c:68:03:38:78:30
*spamApTask4: Jun 09 22:21:15.308: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 0c:68:03:af:9c:c0: DTLS connection closed forAP 10:4:17:22 (57758), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask4: Jun 09 22:21:15.307: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: 0c:68:03:af:9c:c0
*spamApTask6: Jun 09 22:20:39.508: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 d0:c7:89:c6:c8:60: DTLS connection closed forAP 10:4:129:33 (50757), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask6: Jun 09 22:20:39.507: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: d0:c7:89:c6:c8:60
*spamApTask5: Jun 09 22:20:28.908: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 0c:68:03:38:78:30: DTLS connection closed forAP 10:4:129:34 (50675), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask5: Jun 09 22:20:28.907: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: 0c:68:03:38:78:30
*spamApTask6: Jun 09 22:19:40.102: %LWAPP-3-VENDOR_PLD_VALIDATE_ERR: [PA]spam_lrad.c:12016 Validation of SPAM_VENDOR_SPECIFIC_PAYLOAD(185) with length=9 failed - AP 0c:68:03:af:9c:c0
*spamApTask5: Jun 09 22:19:02.283: %LWAPP-3-VENDOR_PLD_VALIDATE_ERR: [PA]spam_lrad.c:12016 Validation of SPAM_VENDOR_SPECIFIC_PAYLOAD(185) with length=9 failed - AP d0:c7:89:c6:c8:60
*spamApTask3: Jun 09 22:18:50.592: %LWAPP-3-VENDOR_PLD_VALIDATE_ERR: [PA]spam_lrad.c:12016 Validation of SPAM_VENDOR_SPECIFIC_PAYLOAD(185) with length=9 failed - AP 0c:68:03:38:78:30
*spamApTask7: Jun 09 22:16:58.504: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 d0:c7:89:c6:c8:60: DTLS connection closed forAP 10:4:129:33 (50758), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask7: Jun 09 22:16:58.503: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: d0:c7:89:c6:c8:60
*spamApTask4: Jun 09 22:16:46.704: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 0c:68:03:38:78:30: DTLS connection closed forAP 10:4:129:34 (50674), Controller: 10:4:65:248 (5246) Echo Timer Expiry
*spamApTask4: Jun 09 22:16:46.703: %CAPWAP-3-ECHO_ERR: [PA]capwap_ac_sm.c:7875 Did not receive heartbeat reply; AP: 0c:68:03:38:78:30
*spamApTask3: Jun 09 22:16:46.304: %CAPWAP-3-DTLS_CLOSED_ERR: [PA]capwap_ac_sm.c:7130 0c:68:03:af:9c:c0: DTLS connection closed forAP 10:4:17:22 (57757), Controller: 10:4:65:248 (5246) Echo Timer Expiry

I don't know what is going on. tried reboot both WLCs , WAN line , FG, all, and not helped

Any sugestion ?

Re: AP3802i wont join 5508 controller

Leo Laohoo — Sat, 10 Jun 2023 02:28:36 GMT

@karol.krzyzyk,

This is a different situation. Please create a new thread so we do not get confused with troubleshooting.

Re: AP3802i wont join 5508 controller

Leo Laohoo — Sat, 10 Jun 2023 02:40:03 GMT

@Paul.Ostaszewski wrote:
I was also able to upgrade to 8.5.182.0 last night, and I still have the same issues.

How? I thought the AP was unable to join the controller?

Can we see the complete output to the WLC command of "show ap join stats detail 70DF.2F05.0E38"?

Re: AP3802i wont join 5508 controller

Paul.Ostaszewski — Sat, 10 Jun 2023 19:40:04 GMT

Leo,

Sorry, i should have been more clear. I updated the 5508 controller to 8.5.182.0, not the AP3802i.

here is the output of the command you requested:

(Cisco Controller) >show ap join stats detailed 70DF.2F05.0E38
No join information found for AP: 70:df:2f:05:0e:38

Re: AP3802i wont join 5508 controller

Leo Laohoo — Sun, 11 Jun 2023 02:17:08 GMT

@Paul.Ostaszewski wrote:
No join information found for AP: 70:df:2f:05:0e:38

This means the AP's join request has never reached the controller. Is there a FW somewhere? Can the AP ping the controller's IP address? Can the controller ping the AP's IP address?

Re: AP3802i wont join 5508 controller

Paul.Ostaszewski — Sun, 11 Jun 2023 03:35:26 GMT

No firewall. I even put the AP on the same vlan as the WLC management and
it won’t join.

The AP can ping the controller and the controller can ping the AP. They
definitely see each other.

Re: AP3802i wont join 5508 controller

Leo Laohoo — Sun, 11 Jun 2023 07:25:54 GMT

This is a 3800, right? Let's try something out:

Download the AP firmware for 17.11.1 and 17.10.1 and put both files into a TFTP server.

Next, console or remote into the AP and run the following commands:

archive download-sw /no-reboot tftp://<IP_ADDRESS>/ap3g3-k9w8-tar.153-3.JPO.tar

Wait for 2 minutes for the entire process to finish and continue with the next.

archive download-sw tftp://<IP_ADDRESS>/ap3g3-k9w8-tar.153-3.JPP.tar

After 2 minutes the AP will reboot automatically and then make attempts to join the controller. Make sure to plug a console cable in before it reboots.

Let's see what this will do.