So i had to upgrade a Cisco WLC 4402 which is located in Sweden and has H-REAP AP's connected to it which are located in Norway.
Since i have done these kind of upgrade's a few times onsite and it went flawless i thought what would be the problem doing it remotely for once.
Uploaded the firmware to the WLC checked the configuration and AP's and rebooted the WLC.
After the reboot the 4 AP's from Norway join and download there new firmware, when the firmware download was finished they rebooted and i thought they joined successfully but they keep rebooting. Also when they are joined they are in local mode and i cant set them to H-REAP (its not selectable in the dropdown list only local,monitor,Rogue Detector and sniffer) and the AP's reboot after about 30 secs.
From what i can see in the debugging the AP joins and then suddenly the DTLS connection was closed.
And that process is in a continuos loop. The regulatory domains of the WLC are SE and NO so i think that should be OK and i would expect an regulatory domain or country setting error in the debugging if that was the issue.
There also is 1 other AP associated to the controller the difference with this one is that its is located on the site in Sweden and is in local mode and working fine.
The AP's and WLC are pingable 100% of the time
Below the debugging output:
*spamReceiveTask: Apr 23 11:59:00.133: ec:44:76:80:8c:c0 DTLS connection was closed
*spamReceiveTask: Apr 23 11:59:00.133: ec:44:76:80:8c:c0 DTLS connection closed event receivedserver (x:138:63:241/5246) client (x:138:32:104/42318)
*spamReceiveTask: Apr 23 11:59:00.133: ec:44:76:80:8c:c0 Entry exists for AP (x:138:32:104/42318)
*spamReceiveTask: Apr 23 11:59:00.133: ec:44:76:80:8c:c0 apfSpamProcessStateChangeInSpamContext: Deregister LWAPP event for AP ec:44:76:80:8c:c0 slot 0
*spamReceiveTask: Apr 23 11:59:00.133: ec:44:76:80:8c:c0 apfSpamProcessStateChangeInSpamContext: Deregister LWAPP event for AP ec:44:76:80:8c:c0 slot 1
*spamReceiveTask: Apr 23 11:59:00.134: ec:44:76:80:8c:c0 No AP entry exist in temporary database for x.138.32.104:42318
*apfReceiveTask: Apr 23 11:59:00.134: ec:44:76:80:8c:c0 Deregister LWAPP event for AP ec:44:76:80:8c:c0 slot 0
*apfReceiveTask: Apr 23 11:59:00.134: ec:44:76:80:8c:c0 Deregister LWAPP event for AP ec:44:76:80:8c:c0 slot 1
*spamReceiveTask: Apr 23 11:59:00.184: ec:44:76:80:8c:c0 Discovery Request from x.138.32.104:42319
*spamReceiveTask: Apr 23 11:59:00.184: ec:44:76:80:8c:c0 Join Priority Processing status = 1, Incoming Ap's Priority 1, MaxLrads = 25, joined Aps =0
*spamReceiveTask: Apr 23 11:59:00.184: ec:44:76:80:8c:c0 Discovery Response sent to x.138.32.104:42319
*spamReceiveTask: Apr 23 11:59:10.185: 00:15:62:17:db:41 DTLS connection not found, creating new connection for x:138:32:104 (42319) x:138:63:241 (5246)
*spamReceiveTask: Apr 23 11:59:11.106: 00:15:62:17:db:41 DTLS Session established server (x.138.63.241:5246), client (x.138.32.104:42319)
*spamReceiveTask: Apr 23 11:59:11.106: 00:15:62:17:db:41 Starting wait join timer for AP: x.138.32.104:42319
*spamReceiveTask: Apr 23 11:59:16.173: ec:44:76:80:8c:c0 Join Request from x.138.32.104:42319
*spamReceiveTask: Apr 23 11:59:16.174: 00:15:62:17:db:41 Deleting AP entry x.138.32.104:42319 from temporary database.
*spamReceiveTask: Apr 23 11:59:16.176: ec:44:76:80:8c:c0 Join Version: = 117504770
*spamReceiveTask: Apr 23 11:59:16.176: ec:44:76:80:8c:c0 Join resp: CAPWAP Maximum Msg element len = 101
*spamReceiveTask: Apr 23 11:59:16.177: ec:44:76:80:8c:c0 Join Response sent to x.138.32.104:42319
*spamReceiveTask: Apr 23 11:59:16.177: ec:44:76:80:8c:c0 CAPWAP State: Join
*spamReceiveTask: Apr 23 11:59:16.177: ec:44:76:80:8c:c0 capwap_ac_platform.c:1217 - Operation State 0 ===> 4
*apfReceiveTask: Apr 23 11:59:16.179: ec:44:76:80:8c:c0 Register LWAPP event for AP ec:44:76:80:8c:c0 slot 0
*apfReceiveTask: Apr 23 11:59:16.179: WARP IEs: (12)
*apfReceiveTask: Apr 23 11:59:16.179: [0000] dd 0a 00 c0 b9 01 00 00 00 08 01 01
*apfReceiveTask: Apr 23 11:59:16.180: ec:44:76:80:8c:c0 Register LWAPP event for AP ec:44:76:80:8c:c0 slot 1
*apfReceiveTask: Apr 23 11:59:16.180: WARP IEs: (12)
*apfReceiveTask: Apr 23 11:59:16.180: [0000] dd 0a 00 c0 b9 01 00 00 00 08 01 01
*spamReceiveTask: Apr 23 11:59:34.404: ec:44:76:80:8c:c0 DTLS connection was closed
*spamReceiveTask: Apr 23 11:59:34.404: ec:44:76:80:8c:c0 DTLS connection closed event receivedserver (x:138:63:241/5246) client (x:138:32:104/42319)
*spamReceiveTask: Apr 23 11:59:34.404: ec:44:76:80:8c:c0 Entry exists for AP (x:138:32:104/42319)
*spamReceiveTask: Apr 23 11:59:34.404: ec:44:76:80:8c:c0 apfSpamProcessStateChangeInSpamContext: Deregister LWAPP event for AP ec:44:76:80:8c:c0 slot 0
*spamReceiveTask: Apr 23 11:59:34.404: ec:44:76:80:8c:c0 apfSpamProcessStateChangeInSpamContext: Deregister LWAPP event for AP ec:44:76:80:8c:c0 slot 1
*spamReceiveTask: Apr 23 11:59:34.405: ec:44:76:80:8c:c0 No AP entry exist in temporary database for x.138.32.104:42319
*apfReceiveTask: Apr 23 11:59:34.406: ec:44:76:80:8c:c0 Deregister LWAPP event for AP ec:44:76:80:8c:c0 slot 0
*apfReceiveTask: Apr 23 11:59:34.406: ec:44:76:80:8c:c0 Deregister LWAPP event for AP ec:44:76:80:8c:c0 slot 1
*spamReceiveTask: Apr 23 11:59:34.455: ec:44:76:80:8c:c0 Discovery Request from x.138.32.104:42318
*spamReceiveTask: Apr 23 11:59:34.455: ec:44:76:80:8c:c0 Join Priority Processing status = 1, Incoming Ap's Priority 1, MaxLrads = 25, joined Aps =0
Similar topics i think:(date and time of the WLC is correct)
https://supportforums.cisco.com/discussion/11068001/cisco-access-point-unable-join-wlc-after-changing-local-monitor-mode
I also checked:
http://www.cisco.com/c/en/us/support/docs/field-notices/639/fn63942.html
But that doesnt seem to be the case.
