topic Re: AIR-AP1562I-E-K9 not joining WLC 5508 in Wireless

AIR-AP1562I-E-K9 not joining WLC 5508

mhmservice — Mon, 05 Jul 2021 19:08:02 GMT

Hi all

I have 2x newly installed AIR-AP1562I-E-K9 that won't join a 5508 controller

I have added AP MACs to the "AP Policy" section of the web interface (never had to do that before for other APs)

"debug capwap errors enable" gives the following (not very helpful):

*spamApTask6: Jun 03 15:43:37.539: [SA] 4c:xx:xx:xx:xx:xx ApModel: AIR-AP1562I-E-K9

CDP on the switch with AP Connected shows as follows:

Device ID: AP4CE1.xxxx.xxxx
Entry address(es):
IP address: 10.x.x.x
Platform: cisco AIR-AP1562I-E-K9, Capabilities: Router Trans-Bridge
Interface: GigabitEthernet0/21, Port ID (outgoing port): GigabitEthernet0
Holdtime : 146 sec

Version :
Cisco AP Software, ap3g3-k9w8 Version: 8.3.143.0
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 2014-2015 by Cisco Systems, Inc.

advertisement version: 2
Power drawn: 29.900 Watts
Power request id: 27402, Power management id: 2
Power request levels are:29900 15400 0 0 0
Management address(es):
IP address: 10.x.x.x

Controller running the following software

(Cisco Controller) >show sysinfo

Manufacturer's Name.............................. Cisco Systems Inc.
Product Name..................................... Cisco Controller
Product Version.................................. 8.3.143.0
Bootloader Version............................... 1.0.20

Does anyone have any ideas to fix this? Other APs in the site work fine (mix of 1602 and 1702)

The AP is installed in an extremely inaccessible location in a remote branch so console connection isn't an easy option.. 😞

Re: AIR-AP1562I-E-K9 not joining WLC 5508

Sandeep Choudhary — Wed, 03 Jun 2020 15:20:39 GMT

paste the output of the command:

sh sysinfo from WLC

Check if time and date settings are corecct on WLC

Check if you add the correct AP mac address on wlc.

alos check the status (Monitoring>>Statistics>>AP Joint)

Regards

Dont forget to arte helpful posts

Re: AIR-AP1562I-E-K9 not joining WLC 5508

Leo Laohoo — Thu, 04 Jun 2020 01:15:36 GMT

Console into the AP and post the complete output to the command "sh capwap client rcb". The APs must've been purchased with MESH firmware.

Re: AIR-AP1562I-E-K9 not joining WLC 5508

mhmservice — Thu, 04 Jun 2020 07:59:16 GMT

Thanks for the reply,

Assuming that is the case what are the steps to fix it?

I will try to get someone on site to console to the AP but I can't ask them to do just that as I have to arrange a third party company to assist (site is in a remote country with no on site IT presence)

Re: AIR-AP1562I-E-K9 not joining WLC 5508

mhmservice — Thu, 04 Jun 2020 08:06:30 GMT

sh sysinfo:

(Cisco Controller) >show sysinfo

Manufacturer's Name.............................. Cisco Systems Inc.
Product Name..................................... Cisco Controller
Product Version.................................. 8.3.143.0
Bootloader Version............................... 1.0.20
Field Recovery Image Version..................... 7.6.101.1
Firmware Version................................. FPGA 1.7, Env 1.8, USB console 2.2
OUI File Update Time............................. Sun Sep 07 10:44:07 IST 2014

Build Type....................................... DATA + WPS

System Name...................................... WLC1
System Location.................................. 
System Contact................................... 
System ObjectID.................................. 1.3.6.1.4.1.9.1.1069
Redundancy Mode.................................. SSO
IP Address....................................... 10.4.1.8
IPv6 Address..................................... ::
Last Reset....................................... Software reset
System Up Time................................... 58 days 16 hrs 48 mins 16 secs
System Timezone Location......................... (GMT +1:00) Amsterdam, Berlin, Rome, Vienna
System Stats Realtime Interval................... 5

--More-- or (q)uit
System Stats Normal Interval..................... 180

Configured Country............................... Multiple Countries : CA,DE,RU,US,ZA
Operating Environment............................ Commercial (0 to 40 C)
Internal Temp Alarm Limits....................... 0 to 65 C
Internal Temperature............................. +38 C
External Temperature............................. +27 C
Fan Status....................................... OK

State of 802.11b Network......................... Enabled
State of 802.11a Network......................... Enabled
Number of WLANs.................................. 10
Number of Active Clients......................... 125

OUI Classification Failure Count................. 0

Burned-in MAC Address............................ xx:xx:xx:xx:xx:xx
Power Supply 1................................... Present, OK
Power Supply 2................................... Absent
Maximum number of APs supported.................. 500
System Nas-Id.................................... 
WLC MIC Certificate Types........................ SHA1

Screenshot of AP Join section:

Re: AIR-AP1562I-E-K9 not joining WLC 5508

Sandeep Choudhary — Thu, 04 Jun 2020 08:22:45 GMT

Are you sure you add the correct AP MAC address in WLC ?

I hope the AP1562 trying to join as MESH in WLC.

Mesh functionality for 1562 is not supported on 8.3. Mesh supported only from 8.4.

https://www.cisco.com/c/en/us/td/docs/wireless/controller/release/notes/crn83mr1.html#concept_A28403142D9D4CD2BE5C0B8F7A1434B9

Run the command on AP - CLI:

capwap ap mode local|flexconnect

change mode on AP as either local or flexconnect.

Regards

Dont forget to rate helpful posts

Re: AIR-AP1562I-E-K9 not joining WLC 5508

Rich R — Sun, 07 Jun 2020 17:02:42 GMT

Sandeep already provided the command to use above.
1562 usually defaults to mesh out of the box so if you didn't reconfigure them before installation then that is likely the problem like Leo and Sandeep said.

Re: AIR-AP1562I-E-K9 not joining WLC 5508

mhmservice — Thu, 11 Jun 2020 07:42:37 GMT

Hi,

That command doesn't exist on the AP

I managed to get on the console, im getting the following errors. It's set to static config as I tried to program in the WLC name just see if it helped,

[*06/10/2020 16:45:23.9415] CAPWAP State: Discovery
[*06/10/2020 16:45:23.9437] Discovery Request sent to 10.4.1.8, discovery type STATIC_CONFIG(1)
[*06/10/2020 16:45:24.0945] Discovery Request sent to 10.4.1.8, discovery type STATIC_CONFIG(1)
[*06/10/2020 16:45:24.0982] Discovery Request sent to 255.255.255.255, discovery type UNKNOWN(0)
[*06/10/2020 16:45:24.0983] Discovery Response from 10.4.1.8
[*06/10/2020 16:45:39.0069] Discovery Response from 10.4.1.8
[*06/10/2020 16:45:39.0000]
[*06/10/2020 16:45:39.0000] CAPWAP State: DTLS Setup
[*06/10/2020 16:45:39.0005] dtls_connectionDB_add_connection: Number of DTLS connections exceeded two
[*06/10/2020 16:45:39.2923] dtls_load_ca_certs: LSC Root Certificate not present
[*06/10/2020 16:45:39.2924]
[*06/10/2020 16:45:39.2951] dtls_verify_con_cert: Controller certificate verification error
[*06/10/2020 16:45:39.2951] dtls_process_packet: controller cert verification failed
[*06/10/2020 16:45:39.2955] DTLS: Received packet 0x26f1000 caused DTLS to close connection
[*06/10/2020 16:45:39.2955] sendPacketToDtls: DTLS: Closing connection 0x26c7a00.
[*06/10/2020 16:45:39.2955]
[*06/10/2020 16:45:39.2955] Lost connection to the controller, going to restart CAPWAP...
[*06/10/2020 16:45:39.2955]
[*06/10/2020 16:45:39.2956] Restarting CAPWAP State Machine.
[*06/10/2020 16:45:39.3002] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: DTLS Setup(3).
[*06/10/2020 16:45:39.3008] Failed to disconnect DTLS-CTRL session.
[*06/10/2020 16:45:39.3008]
[*06/10/2020 16:45:39.3008] CAPWAP State: DTLS Teardown
[*06/10/2020 16:45:39.3111] DTLS: Error while processing DTLS packet 0x26f3000.
[*06/10/2020 16:45:43.9413] No more AP manager addresses remain..
[*06/10/2020 16:45:43.9413] No valid AP manager found for controller 'WLC1' (ip: 10.4.1.8)
[*06/10/2020 16:45:43.9413] Failed to join controller WLC1.
[*06/10/2020 16:45:43.9413] Failed to join controller.
[*06/10/2020 16:45:39.0000]
[*06/10/2020 16:45:39.0000] CAPWAP State: DTLS Setup
[*06/10/2020 16:45:39.0002] dtls_new_connection: Connection 0x26c7a00 is already there for this server port 5246, Deleting it. Number of connections: 56
[*06/10/2020 16:45:39.0002]
[*06/10/2020 16:45:39.0004] dtls_connectionDB_add_connection: Number of DTLS connections exceeded two
[*06/10/2020 16:45:39.2955] dtls_load_ca_certs: LSC Root Certificate not present
[*06/10/2020 16:45:39.2955]
[*06/10/2020 16:45:39.2981] dtls_verify_con_cert: Controller certificate verification error
[*06/10/2020 16:45:39.2981] dtls_process_packet: controller cert verification failed
[*06/10/2020 16:45:39.2985] DTLS: Received packet 0x270a000 caused DTLS to close connection
[*06/10/2020 16:45:39.2985] sendPacketToDtls: DTLS: Closing connection 0x26c7a00.
[*06/10/2020 16:45:39.2985]
[*06/10/2020 16:45:39.2985] Lost connection to the controller, going to restart CAPWAP...
[*06/10/2020 16:45:39.2985]
[*06/10/2020 16:45:39.2986] Restarting CAPWAP State Machine.
[*06/10/2020 16:45:39.3033] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: DTLS Setup(3).
[*06/10/2020 16:45:39.3039] Failed to disconnect DTLS-CTRL session.
[*06/10/2020 16:45:39.3039]
[*06/10/2020 16:45:39.3039] CAPWAP State: DTLS Teardown
[*06/10/2020 16:45:39.3142] DTLS: Error while processing DTLS packet 0x26f5000.
[*06/10/2020 16:45:43.9415]
[*06/10/2020 16:45:43.9415] CAPWAP State: Discovery
[*06/10/2020 16:45:43.9427] Discovery Request sent to 10.4.1.8, discovery type STATIC_CONFIG(1)
[*06/10/2020 16:45:44.0941] Discovery Request sent to 10.4.1.8, discovery type STATIC_CONFIG(1)
[*06/10/2020 16:45:44.0950] Discovery Request sent to 255.255.255.255, discovery type UNKNOWN(0)
[*06/10/2020 16:45:44.0951] Discovery Response from 10.4.1.8
[*06/10/2020 16:45:58.0084] Discovery Response from 10.4.1.8
[*06/10/2020 16:45:58.0000]
[*06/10/2020 16:45:58.0000] CAPWAP State: DTLS Setup
[*06/10/2020 16:45:58.0032] dtls_connectionDB_add_connection: Number of DTLS connections exceeded two
[*06/10/2020 16:45:58.2967] dtls_load_ca_certs: LSC Root Certificate not present
[*06/10/2020 16:45:58.2967]
[*06/10/2020 16:45:58.2994] dtls_verify_con_cert: Controller certificate verification error
[*06/10/2020 16:45:58.2994] dtls_process_packet: controller cert verification failed
[*06/10/2020 16:45:58.2997] DTLS: Received packet 0x26f3000 caused DTLS to close connection
[*06/10/2020 16:45:58.2998] sendPacketToDtls: DTLS: Closing connection 0x26c7a00.
[*06/10/2020 16:45:58.2998]
[*06/10/2020 16:45:58.2998] Lost connection to the controller, going to restart CAPWAP...
[*06/10/2020 16:45:58.2998]
[*06/10/2020 16:45:58.2999] Restarting CAPWAP State Machine.
[*06/10/2020 16:45:58.3044] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: DTLS Setup(3).
[*06/10/2020 16:45:58.3050] Failed to disconnect DTLS-CTRL session.
[*06/10/2020 16:45:58.3051]
[*06/10/2020 16:45:58.3051] CAPWAP State: DTLS Teardown
[*06/10/2020 16:45:58.3158] DTLS: Error while processing DTLS packet 0x270a000.
[*06/10/2020 16:46:02.9413] No more AP manager addresses remain..
[*06/10/2020 16:46:02.9413] No valid AP manager found for controller 'WLC1' (ip: 10.4.1.8)
[*06/10/2020 16:46:02.9413] Failed to join controller WLC1.
[*06/10/2020 16:46:02.9413] Failed to join controller.
[*06/10/2020 16:45:58.0000]
[*06/10/2020 16:45:58.0000] CAPWAP State: DTLS Setup
[*06/10/2020 16:45:58.0002] dtls_new_connection: Connection 0x26c7a00 is already there for this server port 5246, Deleting it. Number of connections: 58
[*06/10/2020 16:45:58.0002]
[*06/10/2020 16:45:58.0004] dtls_connectionDB_add_connection: Number of DTLS connections exceeded two
[*06/10/2020 16:45:58.2907] dtls_load_ca_certs: LSC Root Certificate not present
[*06/10/2020 16:45:58.2907]
[*06/10/2020 16:45:58.2934] dtls_verify_con_cert: Controller certificate verification error
[*06/10/2020 16:45:58.2934] dtls_process_packet: controller cert verification failed
[*06/10/2020 16:45:58.2938] DTLS: Received packet 0x26f1000 caused DTLS to close connection
[*06/10/2020 16:45:58.2938] sendPacketToDtls: DTLS: Closing connection 0x26c7a00.
[*06/10/2020 16:45:58.2938]
[*06/10/2020 16:45:58.2938] Lost connection to the controller, going to restart CAPWAP...
[*06/10/2020 16:45:58.2938]
[*06/10/2020 16:45:58.2940] Restarting CAPWAP State Machine.
[*06/10/2020 16:45:58.2985] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: DTLS Setup(3).
[*06/10/2020 16:45:58.3001] Failed to disconnect DTLS-CTRL session.
[*06/10/2020 16:45:58.3001]
[*06/10/2020 16:45:58.3001] CAPWAP State: DTLS Teardown
[*06/10/2020 16:45:58.3093] DTLS: Error while processing DTLS packet 0x2708000.

Re: AIR-AP1562I-E-K9 not joining WLC 5508

Rich R — Thu, 11 Jun 2020 08:18:03 GMT

What command doesn't exist?
Did you set the AP to local mode?
I've checked my logs from doing the almost identical thing back in January and "capwap ap mode local" definitely solved the problem for me. After that the AP restarted, discovered and joined the WLC, downloaded the new software, and rebooted into normal operation.
The only difference is my AP started with 8.8.100.0 and was joining a WLC running 8.9.111.0 at the time.
8.3.143.0 is rather old so you could be hitting a bug that's been long since fixed in a later release.

Re: AIR-AP1562I-E-K9 not joining WLC 5508

Leo Laohoo — Wed, 10 Jun 2020 23:55:31 GMT

@mhmservice wrote:

IP Address....................................... 10.4.1.8

This is the Management IP address of the WLC.

@mhmservice wrote:

[*06/10/2020 16:45:43.9413] No valid AP manager found for controller 'WLC1' (ip: 192.168.10.10)

The AP is looking for a controller with the wrong Management IP address.

Is DHCP option 43 enabled?

Re: AIR-AP1562I-E-K9 not joining WLC 5508

mhmservice — Thu, 11 Jun 2020 07:42:59 GMT

Sorry I accidentally censored the IP

Where it says 192.168.10.10 it actually says 10.4.1.8, so its not connecting to the wrong controller, ive corrected the previous post now

With regards to the command "capwap ap mode local"... the AP doesn't support that 😞 I have a bunch of other commands like "capwap ap erase" and "capwap ap ip" but no "capwap ap mode"

Re: AIR-AP1562I-E-K9 not joining WLC 5508

Rich R — Thu, 11 Jun 2020 08:19:15 GMT

- With regards to the command "capwap ap mode local"... the AP doesn't support that 😞 I have a bunch of other commands like "capwap ap erase" and "capwap ap ip" but no "capwap ap mode"
Then I think you might need to consider upgrading to more recent code ...

Re: AIR-AP1562I-E-K9 not joining WLC 5508

Leo Laohoo — Thu, 11 Jun 2020 08:19:23 GMT

Post the complete output to the following command:

show capwap client rcb

Re: AIR-AP1562I-E-K9 not joining WLC 5508

mhmservice — Thu, 11 Jun 2020 10:54:16 GMT

I would if I could ... i have a lot of APs which are not supported past 8.3

Re: AIR-AP1562I-E-K9 not joining WLC 5508

mhmservice — Thu, 11 Jun 2020 10:54:55 GMT

I will get this and report back

Re: AIR-AP1562I-E-K9 not joining WLC 5508

Rich R — Thu, 11 Jun 2020 14:09:17 GMT

Then at least get onto the latest 8.3 release:
https://software.cisco.com/download/home/282600534/type/280926587/release/8.3.150.0

Re: AIR-AP1562I-E-K9 not joining WLC 5508

mhmservice — Fri, 12 Jun 2020 08:03:29 GMT

show capwap client rcb
AdminState : ADMIN_ENABLED
OperationState : DTLS SETUP
Name : AP4CE1.xxx.xxxx
SwVer : 8.3.143.0
HwVer : 1.0.0.0
MwarApMgrIp : 10.4.1.8
MwarName : WLC1
MwarHwVer : 0.0.0.0
Location : default location
ApMode : Local
ApSubMode : Not Configured
CAPWAP Path MTU : 576
CAPWAP UDP-Lite : Enabled
IP Prefer-mode : IPv4
AP Link DTLS Encryption : OFF
AP Tcp MSS Adjust : Disabled
LinkAuditing : disabled

Re: AIR-AP1562I-E-K9 not joining WLC 5508

mhmservice — Fri, 12 Jun 2020 08:40:51 GMT

I was affected by this bug

https://www.cisco.com/c/en/us/support/docs/field-notices/639/fn63942.html

Our SHA1 certificate expired in March 2020, I hadn't tried to connect any new APs since then

I worked around it by changing the time on the WLC back to 2019 and the APs joined instantly

I will try to get the fixed firmware mentioned installed

Re: AIR-AP1562I-E-K9 not joining WLC 5508

Rich R — Fri, 12 Jun 2020 08:54:45 GMT

Good that you finally solved it - had to point a few people towards that FN recently.
It didn't seem like the case for yours because you had said all other APs were working normally 🙂