Re: The APs are trying to connect

jvasilko · ‎03-23-2017

I am having an issue with getting some new 3702i APs to join a new 5520 WLC. I am trying to add about 180 Aps to this controller and none of them join. I see them trying to join in the AP Join section, but all are unsuccessful. I have another new 5520 at a different site that I had one AP try to connect to, and it successfully joined without issue.

Both controllers are running the same version of code (8.2.141) and have the same configuration aside from local IP addressing. The clock looks to be set to the proper time (using NTP) and the Aps can ping the WLC that will not allow them to join.

When I did a debug capwap event, I was immediately deluged with output from all of the Aps trying to join. One of the lines of output that was related to all was Disconnecting due to a wait join timer expiry event.

When I did a debug capwap error one of the lines associated to all of the APs was....unable to get AP bundled version, using controller version.

Any thoughts on what I am missing?

Leo Laohoo · ‎03-23-2017

Post the complete output to the following commands:

1. WLC: sh sysinfo;

2. WLC: sh time;

3. AP: sh version;

4. AP: sh ip interface brief; and

5. Reboot the AP. We want to see the entire boot-up process.

jvasilko · ‎03-24-2017

I have attached the sysinfo, time, and an AP boot sequence, did not have access to the AP (remotely located) to get the AP info you requested, hopefully the AP boot will tell you what you need to know.

Manufacturer's Name.............................. Cisco Systems Inc.
Product Name..................................... Cisco Controller
Product Version.................................. 8.2.141.0
RTOS Version..................................... 8.2.141.0
Bootloader Version............................... 8.1.102.0
Emergency Image Version.......................... 8.1.102.0

Build Type....................................... DATA + WPS

System Name...................................... US030-W-UA-B122-5520-01
System Location..................................
System Contact...................................
System ObjectID.................................. 1.3.6.1.4.1.9.1.2170
Redundancy Mode.................................. SSO
IP Address....................................... 10.31.39.5
IPv6 Address..................................... ::
System Up Time................................... 8 days 16 hrs 33 mins 1 secs
System Timezone Location......................... (GMT -6:00) Central Time (US and Canada)
System Stats Realtime Interval................... 5
System Stats Normal Interval..................... 180

--More-- or (q)uit
Configured Country............................... US - United States
Operating Environment............................ Commercial (10 to 35 C)
Internal Temp Alarm Limits....................... 10 to 38 C
Internal Temperature............................. +17 C
Fan Status....................................... OK

State of 802.11b Network......................... Enabled
State of 802.11a Network......................... Enabled
Number of WLANs.................................. 2
Number of Active Clients......................... 0

Burned-in MAC Address............................ 00:A2:EE:9D:24:C7
Power Supply 1................................... Present, OK
Power Supply 2................................... Present, Off
Maximum number of APs supported.................. 1500
System Nas-Id.................................... US030-W-UA-B122-5520-02
WLC MIC Certificate Types........................ SHA1/SHA2
Licensing Type................................... RTU

(Cisco Controller) >show time

Time............................................. Fri Mar 24 06:47:29 2017

Timezone delta................................... 0:0
Timezone location................................ (GMT -6:00) Central Time (US and Canada)

NTP Servers
    NTP Polling Interval.........................     3600

     Index     NTP Key Index                  NTP Server                Status          NTP Msg Auth Status
    ------- ----------------------------------------------------------------------------------------------
       1              0                               135.89.92.196     In Sync              AUTH DISABLED
       2              0                               135.89.92.132     Not Tried            AUTH DISABLED

AP BOOT:

IOS Bootloader - Starting system.
flash is writable
Tide XL MB - 40MB of flash
Xmodem file system is available.
flashfs[0]: 11 files, 2 directories
flashfs[0]: 0 orphaned files, 0 orphaned directories
flashfs[0]: Total bytes: 41158656
flashfs[0]: Bytes used: 7351296
flashfs[0]: Bytes available: 33807360
flashfs[0]: flashfs fsck took 10 seconds.
Base Ethernet MAC address: f4:0f:1b:04:23:30
Ethernet speed is 1000 Mb - FULL Duplex
Loading "flash:/ap3g2-rcvk9w8-mx/ap3g2-rcvk9w8-mx"...##########################

File "flash:/ap3g2-rcvk9w8-mx/ap3g2-rcvk9w8-mx" uncompressed and installed, entr
y point: 0x2003000
executing...

Secondary Bootloader - Starting system.
Tide XL MB - 40MB of flash
Xmodem file system is available.
flashfs[0]: 11 files, 2 directories
flashfs[0]: 0 orphaned files, 0 orphaned directories
flashfs[0]: Total bytes: 41158656
flashfs[0]: Bytes used: 7351296
flashfs[0]: Bytes available: 33807360
flashfs[0]: flashfs fsck took 10 seconds.
Base Ethernet MAC address: f4:0f:1b:04:23:30
Boot CMD: 'boot flash:/ap3g2-rcvk9w8-mx/ap3g2-rcvk9w8-xx;flash:/ap3g2-rcvk9w8-m
x/ap3g2-rcvk9w8-mx'
Loading "flash:/ap3g2-rcvk9w8-mx/ap3g2-rcvk9w8-xx"...######################
File "flash:/ap3g2-rcvk9w8-mx/ap3g2-rcvk9w8-xx" uncompressed and installed, entr
y point: 0x1003000
executing...

Restricted Rights Legend

Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.

cisco Systems, Inc.
170 West Tasman Drive
San Jose, California 95134-1706

Cisco IOS Software, C3700 Software (AP3G2-RCVK9W8-M), Version 15.2(4)JB1, RELEAS
E SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2013 by Cisco Systems, Inc.
Compiled Sat 16-Nov-13 10:46 by prod_rel_team

Tide XL MB - 40MB of flash
Initializing flashfs...

flashfs[2]: 11 files, 2 directories
flashfs[2]: 0 orphaned files, 0 orphaned directories
flashfs[2]: Total bytes: 40900608
flashfs[2]: Bytes used: 7351296
flashfs[2]: Bytes available: 33549312
flashfs[2]: flashfs fsck took 10 seconds.
flashfs[2]: Initialization complete.
flashfs[3]: 0 files, 1 directories
flashfs[3]: 0 orphaned files, 0 orphaned directories
flashfs[3]: Total bytes: 11999232
flashfs[3]: Bytes used: 1024
flashfs[3]: Bytes available: 11998208
flashfs[3]: flashfs fsck took 1 seconds.
flashfs[3]: Initialization complete.
Copying radio files from flash: to ram:
%Error opening flash:/ap3g2-rcvk9w8-mx/E2.bin (No such file or directory)
%Error opening flash:/ap3g2-rcvk9w8-mx/B2.bin (No such file or directory)
%Error opening flash:/ap3g2-rcvk9w8-mx/V2.bin (No such file or directory)
%Error opening flash:/ap3g2-rcvk9w8-mx/X2.bin (No such file or directory)
%Error opening flash:/ap3g2-rcvk9w8-mx/Y2.bin (No such file or directory)
%Error opening flash:/ap3g2-rcvk9w8-mx/R2.bin (No such file or directory)
%Error opening flash:/ap3g2-rcvk9w8-mx/Q2.bin (No such file or directory)
%Error opening flash:/ap3g2-rcvk9w8-mx/C2.bin (No such file or directory)
Uncompressing radio files...
File not found: flash:/ap3g2-rcvk9w8-mx/8004.img
File not found: flash:/ap3g2-rcvk9w8-mx/E5.bin
File not found: flash:/ap3g2-rcvk9w8-mx/B5.bin
File not found: flash:/ap3g2-rcvk9w8-mx/V5.bin
File not found: flash:/ap3g2-rcvk9w8-mx/X5.bin
File not found: flash:/ap3g2-rcvk9w8-mx/Y5.bin
File not found: flash:/ap3g2-rcvk9w8-mx/R5.bin
File not found: flash:/ap3g2-rcvk9w8-mx/Q5.bin
File not found: flash:/ap3g2-rcvk9w8-mx/C5.bin
File not found: flash:/ap3g2-rcvk9w8-mx/8006.img
...done Initializing flashfs.

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
no ip http server
^
% Invalid input detected at '^' marker.

use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco AIR-CAP3702I-A-K9 (PowerPC) processor (revision A0) with 204790K/57344K by
tes of memory.
Processor board ID FTX182079HE
PowerPC CPU at 800Mhz, revision number 0x2151
Last reset from power-on
LWAPP image version 7.6.1.118
1 Gigabit Ethernet interface

32K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address: F4:0F:1B:04:23:30
Part Number : 73-15243-01
PCA Assembly Number : 000-00000-00
PCA Revision Number :
PCB Serial Number : FOC18181ECF
Top Assembly Part Number : 068-05054-01
Top Assembly Serial Number : FTX182079HE
Top Revision Number : A0
Product/Model Number : AIR-CAP3702I-A-K9
% Please define a domain-name first.

Press RETURN to get started!

APAVC: Initial WLAN Buffers Given to System is 2500
APAVC: WlanPAKs 18174 RadioPaks 17566
set_radio_pwr_mode: bad radio unit# 0
set_radio_pwr_mode: bad radio unit# 1

*Mar 1 00:00:13.663: %LWAPP-3-CLIENTERRORLOG: Config load from flash failed. In
itialising Cfg

*Mar 1 00:00:15.707: %LINK-6-UPDOWN: Interface GigabitEthernet0, changed state
to up
*Mar 1 00:00:16.067: %SYS-5-RESTART: System restarted --
Cisco IOS Software, C3700 Software (AP3G2-RCVK9W8-M), Version 15.2(4)JB1, RELEAS
E SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2013 by Cisco Systems, Inc.
Compiled Sat 16-Nov-13 10:46 by prod_rel_team
*Mar 1 00:00:16.107: %LWAPP-3-CLIENTERRORLOG: Config load from flash failed. In
itialising Cfg

*Mar 1 00:00:16.107: %CAPWAP-3-ERRORLOG: Failed to load configuration from flas
h. Resetting to default configlwapp_crypto_init: MIC Present and Parsed Successf
ully

*Mar 1 00:00:16.227: %SSH-5-ENABLED: SSH 2.0 has been enabled
*Mar 1 00:00:17.095: %LINEPROTO-5-UPDOWN: Line protocol on Interface BVI1, chan
ged state to upERROR: Failed to configure ethernet promiscuous mode. Interface D
escriptor mismatch
*Mar 1 00:00:20.915: DPAA Initialization Complete
*Mar 1 00:00:20.915: %SYS-3-HARIKARI: Process DPAA INIT top-level routine exite
d
*Mar 1 00:00:21.915: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEth
ernet0, changed state to up
no bridge-group 1 source-learning
^
% Invalid input detected at '^' marker.
%Default route without gateway, if not a point-to-point interface, may impact pe
rformance
*Mar 1 00:00:34.859: %LWAPP-3-CLIENTERRORLOG: Config load from flash failed. In
itialising Cfg
set_radio_pwr_mode: bad radio unit# 0
set_radio_pwr_mode: bad radio unit# 1

*Mar 1 00:00:36.555: %CDP_PD-4-POWER_OK: 15.4 W power - NEGOTIATED inline power
sourceset_radio_pwr_mode: bad radio unit# 0
set_radio_pwr_mode: bad radio unit# 1

*Mar 1 00:00:45.727: %CDP_PD-4-POWER_OK: Full power - NEGOTIATED inline power s
ource
*Mar 1 00:00:58.483: %DHCP-6-ADDRESS_ASSIGN: Interface BVI1 assigned DHCP addre
ss 10.100.33.25, mask 255.255.254.0, hostname APf40f.1b04.2330

Translating "CISCO-CAPWAP-CONTROLLER.dow.com"...domain server (10.0.2.11)

*Mar 1 00:01:08.431: %CAPWAP-3-ERRORLOG: Did not get log server settings from D
HCP.
*Mar 1 00:01:08.443: %CAPWAP-3-ERRORLOG: Could Not resolve CISCO-CAPWAP-CONTROL
LER.dow.com
Not in Bound state.
*Mar 1 00:01:53.943: %CAPWAP-3-DHCP_RENEW: Could not discover WLC using DHCP IP
. Renewing DHCP IP.
*Mar 1 00:02:05.639: %CAPWAP-3-ERRORLOG: Invalid event 40 & state 2 combination
.
*Mar 1 00:02:05.719: %DHCP-6-ADDRESS_ASSIGN: Interface BVI1 assigned DHCP addre
ss 10.100.33.27, mask 255.255.254.0, hostname APf40f.1b04.2330

Translating "CISCO-CAPWAP-CONTROLLER.dow.com"...domain server (10.0.2.11)

*Mar 1 00:02:14.943: %CAPWAP-3-ERRORLOG: Did not get log server settings from D
HCP.
*Mar 1 00:02:14.955: %CAPWAP-3-ERRORLOG: Could Not resolve CISCO-CAPWAP-CONTROL
LER.dow.com
Not in Bound state.

User Access Verification

Username:
*Mar 1 00:02:50.455: %CAPWAP-3-DHCP_RENEW: Could not discover WLC using DHCP IP
. Renewing DHCP IP.
*Mar 1 00:03:01.875: %CAPWAP-3-ERRORLOG: Invalid event 40 & state 2 combination
.
*Mar 1 00:03:01.975: %DHCP-6-ADDRESS_ASSIGN: Interface BVI1 assigned DHCP addre
ss 10.100.33.242, mask 255.255.254.0, hostname APf40f.1b04.2330

Username: Cisco
Password:
Translating "CISCO-CAPWAP-CONTROLLER.dow.com"...domain server (10.0.2.11)

*Mar 1 00:03:11.455: %CAPWAP-3-ERRORLOG: Did not get log server settings from D
HCP.
*Mar 1 00:03:11.467: %CAPWAP-3-ERRORLOG: Could Not resolve CISCO-CAPWAP-CONTROL
LER.dow.com

APf40f.1b04.2330>en
Password:
APf40f.1b04.2330#
APf40f.1b04.2330#
APf40f.1b04.2330#
APf40f.1b04.2330#dir
Directory of flash:/

2 -rwx 337 Jan 1 1970 00:03:18 +00:00 info
3 -rwx 1048 Mar 1 1993 00:00:21 +00:00 private-multiple-fs
32 drwx 512 Mar 1 1993 00:03:29 +00:00 ap3g2-rcvk9w8-mx
4 -rwx 155 Jan 1 1970 00:00:45 +00:00 env_vars

40900608 bytes total (33549312 bytes free)
APf40f.1b04.2330#
Not in Bound state.
*Mar 1 00:03:46.967: %CAPWAP-3-DHCP_RENEW: Could not discover WLC using DHCP IP
. Renewing DHCP IP.
APf40f.1b04.2330#
*Mar 1 00:03:59.075: %CAPWAP-3-ERRORLOG: Invalid event 40 & state 2 combination
.
*Mar 1 00:03:59.127: %DHCP-6-ADDRESS_ASSIGN: Interface BVI1 assigned DHCP addre
ss 10.100.33.243, mask 255.255.254.0, hostname APf40f.1b04.2330

APf40f.1b04.2330#wr era
Translating "CISCO-CAPWAP-CONTROLLER.dow.com"...domain server (10.0.2.11)
ce
*Mar 1 00:04:07.967: %CAPWAP-3-ERRORLOG: Did not get log server settings from D
HCP.
*Mar 1 00:04:07.975: %CAPWAP-3-ERRORLOG: Could Not resolve CISCO-CAPWAP-CONTROL
LER.dow.com
^
% Invalid input detected at '^' marker.

APf40f.1b04.2330#dir
Directory of flash:/

2 -rwx 337 Jan 1 1970 00:03:18 +00:00 info
3 -rwx 1048 Mar 1 1993 00:00:21 +00:00 private-multiple-fs
32 drwx 512 Mar 1 1993 00:03:29 +00:00 ap3g2-rcvk9w8-mx
4 -rwx 155 Jan 1 1970 00:00:45 +00:00 env_vars

40900608 bytes total (33549312 bytes free)
APf40f.1b04.2330#

Leo Laohoo · ‎03-24-2017

*Mar 1 00:03:46.967: %CAPWAP-3-DHCP_RENEW: Could not discover WLC using DHCP IP
. Renewing DHCP IP.

Here's the reason: DHCP Option 43 is not configured.

jvasilko · ‎03-24-2017

I used the DNS entry cisco-capwap-controller.x.com as the alias for my controller address. I am seeing them attempt to join the controller, so they know how to get there.

Jerry Lemke · ‎03-24-2017

I didn't see from the APs perspective that it actually discovered the controller. From the controller what does is show under AP join statistics for these APs? Monitor>statistics> AP Join.

You could try to Prime the controller address into the AP using "capwap ap" commands from the AP CLI.

jvasilko · ‎03-24-2017

Each AP show up as 'not joined' and have a reason for unsuccessful attempt as failed to delete database entry.

I would normally prime these with the controller command, but that did not work either. They tried to join, but failed with the response above.

Leo Laohoo · ‎03-24-2017

Cisco IOS Software, C3700 Software (AP3G2-RCVK9W8-M)

Look at the IOS the AP is loading, it is the recovery (aka RCV) image. This means the AP has never seen a controller before.

If the AP has an IP address, enter this enable command: config ap primary- <WLC Name> <WLC Management IP address>

sandjose_cisco · ‎03-27-2017

were you able to get the debug capwap events from the WLC

check if DTLS is getting completed

show dtls connections

There are some cert issues with SHA2 , please have a check

http://www.cisco.com/c/en/us/support/docs/field-notices/639/fn63916.html

jvasilko · ‎03-27-2017

When I checked before, there was a successful DTLS connection being made for the APs. These APs are all brand new 3702i APs trying to connect to a brand new 5520 running 8.2.141.0 code, so I would hope that SHA2 certs are not the problem.

jvasilko · ‎03-27-2017

Found the issue this morning. I checked the licenses and they weren't activated. The GUI showed 1500 AP license, but they weren't accepted. Sort of a silly exercise to accept something that I obviously wanted, but its resolved now so I am good to .

jvasilko · ‎03-27-2017

The APs are trying to connect to the controller, they are getting denied for some reason, and that is the issue. I have entered the command directly on a test AP with the same results. SO the APs have seen the controller, they are just unable to join.

ibneali991 · ‎03-20-2024

Hello brother I am facing same issue. how you resolve this issue before kindly guide me thanks.

APs Not Joining a New 5520 Controller