Access Points No longer Join Controller

pinglis · ‎01-27-2017

Earlier in the week I upgrade our 5508 controller from 7.6.130.0 to 8.0.140.0.

Everything worked okay and all APs re-joined and have been working okay for a couple of days.

Today a number of access points disconnected and now will not reconnect. Then number is in increasing currently about 50 out of a total of 190 originally joined to the controller.

In the log I am seeing a stream of the following errors (actual mac and IP removed).

*spamApTask1: Jan 27 13:33:44.265: %CAPWAP-3-DTLS_DB_ERR: capwap_ac_sm.c:8724 mm:mm:mm:mm:mm:mm: Failed to create DTLS connection for AP x.x.x.x (25902).

The AP join status show the AP discovering the controller but never joining it.

Any ideas how to fix it and what may have caused it?

Sandeep Choudhary · ‎01-27-2017

paste the output of these commands:

sh sysinfo from WLC

sh version from AP.

Also paste the complete boot-up process from AP console.

Regards

Dont forget to rate helpful posts

pinglis · ‎01-27-2017

The issue is occurring on multiple access points of different models. The show version is from on example. The APs are in different countries to me and I have no access to the console

show sysinfo:

Manufacturer's Name.............................. Cisco Systems Inc.
Product Name..................................... Cisco Controller
Product Version.................................. 8.0.140.0
Bootloader Version............................... 1.0.1
Field Recovery Image Version..................... 6.0.182.0
Firmware Version................................. FPGA 1.3, Env 1.6, USB console 1.27
Build Type....................................... DATA + WPS
-----
System ObjectID.................................. 1.3.6.1.4.1.9.1.1069
Redundancy Mode.................................. Disabled
IP Address.......................................
IPv6 Address..................................... ::
Last Reset....................................... Software reset
System Up Time................................... 2 days 14 hrs 34 mins 29 secs
System Timezone Location......................... (GMT +1:00) Amsterdam, Berlin, Rome, Vienna
System Stats Realtime Interval................... 5
System Stats Normal Interval..................... 180

Configured Country............................... Multiple Countries:AT,BE,CH,DE,ES,FI,FR,GB,GR,IE,IN,IT,NL,NO,PK,PL,PT,RU,SE,ZA
Operating Environment............................ Commercial (0 to 40 C)
Internal Temp Alarm Limits....................... 0 to 65 C
Internal Temperature............................. +44 C
External Temperature............................. +23 C
Fan Status....................................... OK

State of 802.11b Network......................... Enabled
State of 802.11a Network......................... Enabled
Number of WLANs.................................. 6
Number of Active Clients......................... 266

Burned-in MAC Address............................
Power Supply 1................................... Present, OK
Power Supply 2................................... Present, OK
Maximum number of APs supported.................. 500
System Nas-Id....................................
WLC MIC Certificate Types........................ SHA1

Show version

Cisco IOS Software, C1240 Software (C1240-K9W8-M), Version 12.4(25e)JAP10, RELEA SE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2016 by Cisco Systems, Inc.
Compiled Mon 22-Aug-16 23:59 by prod_rel_team

ROM: Bootstrap program is C1240 boot loader
BOOTLDR: C1240 Boot Loader (C1240-BOOT-M) Version 12.4(13d)JA, RELEASE SOFTWARE (fc2)

INCU-AP01 uptime is 2 days, 14 hours, 28 minutes
System returned to ROM by power-on
System image file is "flash:/c1240-k9w8-mx.124-25e.JAP10/c1240-k9w8-mx.124-25e.J AP10"

-----

cisco AIR-LAP1242AG-A-K9 (PowerPCElvis) processor (revision A0) with 27638K/5120K bytes of memory.
Processor board ID FGL1448U0NA
PowerPCElvis CPU at 262Mhz, revision number 0x0950
Last reset from power-on
LWAPP image version 8.0.140.0
1 FastEthernet interface
2 802.11 Radio(s)

32K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address: C4:71:FE:EC:82:C8
Part Number                          : 73-9925-07
PCA Assembly Number                  : 800-26579-06
PCA Revision Number                  : A0
PCB Serial Number                    : FOC14464QR6
Top Assembly Part Number             : 800-29232-02
Top Assembly Serial Number           : FGL1448U0NA
Top Revision Number                  : A0
Product/Model Number                 : AIR-AP1242AG-A-K9

Configuration register is 0xF

Sandeep Choudhary · ‎01-27-2017

Hi,

WLC :

Your config:Countries:AT,BE,CH,DE,ES,FI,FR,GB,GR,IE,IN,IT,NL,NO,PK,PL,PT,RU,SE,ZA

AP:

Product/Model Number : AIR-AP1242AG-A-K9 (Its for American Regulatory domain)

You configured European(-E) country codes on WLC. Just add/configure "US" country code on your WLC to join this AP.

Regards

Dont forget to rate helpful posts

pinglis · ‎01-27-2017

I don't believe this is the cause of issue as we are seeing it across multiple access point with different regulatory domains include, e.g. -E-

Also in the Web GUI the controller is reporting the following supported Regulatory Domain:
802.11a/n/ac: (Indoor: -ACDENR, Outdoor: -DENR)
802.11b/g/n: (Indoor: -AER, Outdoor: -AENR)

I confirmed on a test controller as you select different countries additional domains are added to these list. For example India (where is access point is) adds the A for b/g/n

pinglis · ‎01-30-2017

Latest Status:

On Friday I rebooted the controller and all access points successfully re-joined.

On Saturday a couple of APs has lost connection

By Sunday all the access points (40+) that had a problem originally had lost connection and were unable to re-join

I have now downgraded the controller back to 7.6.130.0 and so far all APs have remained joined to the controller.

Sandeep Choudhary · ‎01-30-2017

It could be an issue of Compatibility.

May be you have old APs which dont support 8.0.140.0 version!!!

Here check the compatibility matrix of WLC and APs:

http://www.cisco.com/c/en/us/td/docs/wireless/compatibility/matrix/compatibility-matrix.html

Reargds

Dont forget to rate helpful posts