Problem adding Cisco Aironet 1815i to the controller

[bosman]

Hello everyone,

I have an issue adding Capwap ap's to the controller, It all started when I had to reset my ME ap since it was not transmitting WiFi signal, and after the WiFi issue is resolved and reconfigured the ME ap, I'm not able of adding the Capwap ap's to it,

Note that all ap's are on the same VLAN, and the Capwap ap's are discovered by the controller but not manageable,

Kindly find attached image, it's a screenshot of the ap's list,

Thank you,

Best Regards,

[marce1000]

                  - What is the controller model and software version ?

 M.

[bosman]

Model: AIR-AP1815I-E-K9

Software version: 8.5.105.0

[marce1000]

     -  Check controller logs when the AP's try to join and post the (capwap)-ap boot process.

 M.

 M.

[bosman]

Controller Logs

*httpImgDwnldTask1: Nov 24 11:12:55.060: %CAPWAP-3-PRED_ERR3: capwap_ac_img_dwnld.c:1697 LabAP: Curl returned error code= 28 , err_string: Timeout was reached
*httpImgDwnldTask0: Nov 24 11:12:50.020: %CAPWAP-3-PRED_ERR3: capwap_ac_img_dwnld.c:1697 HeadOffice: Curl returned error code= 28 , err_string: Timeout was reached
*CAPWAP DATA: Nov 24 11:12:29.898: %RRM-3-RRM_LOGMSG: rrmClient.c:1531 RRM LOG: iapp chd client stat , Unable to find AP b4:de:31:9b:7e:20
*Dot1x_NW_MsgTask_0: Nov 24 11:12:06.738: %DOT1X-3-INVALID_REPLAY_CTR: 1x_eapkey.c:452 Invalid replay counter from client 4c:dd:31:72:38:d9 - got 00 00 00 00 00 00 00 0c, expected 00 00 00 00 00 00 00 0d
*Dot1x_NW_MsgTask_0: Nov 24 11:12:06.423: %DOT1X-3-INVALID_REPLAY_CTR: 1x_eapkey.c:452 Invalid replay counter from client 66:29:ec:98:3a:85 - got 00 00 00 00 00 00 00 02, expected 00 00 00 00 00 00 00 03
*httpImgDwnldTask1: Nov 24 11:10:36.314: %CAPWAP-3-PRED_ERR3: capwap_ac_img_dwnld.c:1697 LabAP: Curl returned error code= 28 , err_string: Timeout was reached
*httpImgDwnldTask0: Nov 24 11:10:31.264: %CAPWAP-3-PRED_ERR3: capwap_ac_img_dwnld.c:1697 HeadOffice: Curl returned error code= 28 , err_string: Timeout was reached
*httpImgDwnldTask1: Nov 24 11:08:17.537: %CAPWAP-3-PRED_ERR3: capwap_ac_img_dwnld.c:1697 LabAP: Curl returned error code= 28 , err_string: Timeout was reached
*httpImgDwnldTask0: Nov 24 11:08:12.508: %CAPWAP-3-PRED_ERR3: capwap_ac_img_dwnld.c:1697 HeadOffice: Curl returned error code= 28 , err_string: Timeout was reached
*CAPWAP DATA: Nov 24 11:07:59.811: %RRM-3-RRM_LOGMSG: rrmClient.c:1531 RRM LOG: iapp chd client stat , Unable to find AP b4:de:31:9b:7e:20
*httpImgDwnldTask1: Nov 24 11:05:58.780: %CAPWAP-3-PRED_ERR3: capwap_ac_img_dwnld.c:1697 LabAP: Curl returned error code= 28 , err_string: Timeout was reached
*httpImgDwnldTask0: Nov 24 11:05:53.751: %CAPWAP-3-PRED_ERR3: capwap_ac_img_dwnld.c:1697 HeadOffice: Curl returned error code= 28 , err_string: Timeout was reached
*httpImgDwnldTask1: Nov 24 11:03:40.044: %CAPWAP-3-PRED_ERR3: capwap_ac_img_dwnld.c:1697 LabAP: Curl returned error code= 28 , err_string: Timeout was reached
*httpImgDwnldTask0: Nov 24 11:03:34.188: %CAPWAP-3-PRED_ERR3: capwap_ac_img_dwnld.c:1697 HeadOffice: Curl returned error code= 28 , err_string: Timeout was reached
*CAPWAP DATA: Nov 24 11:03:29.758: %RRM-3-RRM_LOGMSG: rrmClient.c:1531 RRM LOG: iapp chd client stat , Unable to find AP b4:de:31:9b:7e:20
*httpImgDwnldTask1: Nov 24 11:01:21.268: %CAPWAP-3-PRED_ERR3: capwap_ac_img_dwnld.c:1697 LabAP: Curl returned error code= 28 , err_string: Timeout was reached
*httpImgDwnldTask0: Nov 24 11:01:16.221: %CAPWAP-3-PRED_ERR3: capwap_ac_img_dwnld.c:1697 HeadOffice: Curl returned error code= 28 , err_string: Timeout was reached
*httpImgDwnldTask1: Nov 24 10:59:02.505: %CAPWAP-3-PRED_ERR3: capwap_ac_img_dwnld.c:1697 LabAP: Curl returned error code= 28 , err_string: Timeout was reached
*httpImgDwnldTask0: Nov 24 10:58:57.464: %CAPWAP-3-PRED_ERR3: capwap_ac_img_dwnld.c:1697 HeadOffice: Curl returned error code= 28 , err_string: Timeout was reached
*httpImgDwnldTask1: Nov 24 10:56:43.741: %CAPWAP-3-PRED_ERR3: capwap_ac_img_dwnld.c:1697 LabAP: Curl returned error code= 28 , err_string: Timeout was reached
*httpImgDwnldTask0: Nov 24 10:56:38.705: %CAPWAP-3-PRED_ERR3: capwap_ac_img_dwnld.c:1697 HeadOffice: Curl returned error code= 28 , err_string: Timeout was reached
*httpImgDwnldTask1: Nov 24 10:54:24.206: %CAPWAP-3-PRED_ERR3: capwap_ac_img_dwnld.c:1697 LabAP: Curl returned error code= 28 , err_string: Timeout was reached
*httpImgDwnldTask0: Nov 24 10:54:19.164: %CAPWAP-3-PRED_ERR3: capwap_ac_img_dwnld.c:1697 HeadOffice: Curl returned error code= 28 , err_string: Timeout was reached

_____________________________________________________

Capwap Boot Process

Kindly find attach Capwap.txt file with the boot process

_____________________________________________________

[Rich R]

Yes we need to see those console logs from the CAPWAP APs from power on.
Also the join stats from the ME WLC.
show ap join stats summary all
show ap join stats detailed <Cisco AP Mac>
and "show sysinfo" and "show ap image all" from the ME.

The APs are not showing IP addresses in the screenshot - just to confirm your DHCP is working and they're getting IP addresses?

8.5.105.0 is very old, lots of known bugs resolved since then - you should be considering an update to 8.5.182.0 or 8.10 latest (wait for 8.10.183.0 - due soon - if you want 8.10).

[bosman]

(Cisco Controller) >show ap join stats summary all

Number of APs.............................................. 3

Base Mac AP EthernetMac AP Name IP Address Status
a0:93:51:1e:29:e0 a0:93:51:1e:29:e0 HeadOffice 172.16.11.3 Joined
b4:de:31:27:46:80 00:5d:73:79:ee:48 MainAP▒ 172.16.11.2 Joined
b4:de:31:9b:7e:20 b4:de:31:9b:7e:20 LabAP 172.16.11.4 Joined

(Cisco Controller) >show ap join stats detailed a0:93:51:1e:29:e0

Sync phase statistics
- Time at sync request received............................ Not applicable
- Time at sync completed................................... Not applicable

Discovery phase statistics
- Discovery requests received.............................. 60
- Successful discovery responses sent...................... 60
- Unsuccessful discovery request processing................ 0
- Reason for last unsuccessful discovery attempt........... Not applicable
- Time at last successful discovery attempt................ Nov 24 11:42:54.656
- Time at last unsuccessful discovery attempt.............. Not applicable

Join phase statistics
- Join requests received................................... 19
- Successful join responses sent........................... 19
- Unsuccessful join request processing..................... 0
- Reason for last unsuccessful join attempt................ Not applicable
- Time at last successful join attempt..................... Nov 24 11:43:04.203
- Time at last unsuccessful join attempt................... Not applicable

Configuration phase statistics

--More-- or (q)uit
- Configuration requests received.......................... 0
- Successful configuration responses sent.................. 0
- Unsuccessful configuration request processing............ 0
- Reason for last unsuccessful configuration attempt....... Not applicable
- Time at last successful configuration attempt............ Not applicable
- Time at last unsuccessful configuration attempt.......... Not applicable

Last AP message decryption failure details
- Reason for last message decryption failure............... Not applicable

Last AP disconnect details
- Reason for last AP connection failure.................... Not applicable
- Last AP disconnect reason................................ Not applicable

Last join error summary
- Type of error that occurred last......................... None
- Reason for error that occurred last...................... Not applicable
- Time at which the last join error occurred............... Not applicable

AP disconnect details
- Reason for last AP connection failure.................... Not applicable
Ethernet Mac : a0:93:51:1e :29:e0 Ip Address : 172.16.11.3

--More-- or (q)uit

(Cisco Controller) >show ap join stats detailed a0:93:51:1e:29:e0

Sync phase statistics
- Time at sync request received............................ Not applicable
- Time at sync completed................................... Not applicable

Discovery phase statistics
- Discovery requests received.............................. 63
- Successful discovery responses sent...................... 63
- Unsuccessful discovery request processing................ 0
- Reason for last unsuccessful discovery attempt........... Not applicable
- Time at last successful discovery attempt................ Nov 24 11:45:13.427
- Time at last unsuccessful discovery attempt.............. Not applicable

Join phase statistics
- Join requests received................................... 19
- Successful join responses sent........................... 19
- Unsuccessful join request processing..................... 0
- Reason for last unsuccessful join attempt................ Not applicable
- Time at last successful join attempt..................... Nov 24 11:43:04.203
- Time at last unsuccessful join attempt................... Not applicable

Configuration phase statistics

--More-- or (q)uit
- Configuration requests received.......................... 0
- Successful configuration responses sent.................. 0
- Unsuccessful configuration request processing............ 0
- Reason for last unsuccessful configuration attempt....... Not applicable
- Time at last successful configuration attempt............ Not applicable
- Time at last unsuccessful configuration attempt.......... Not applicable

Last AP message decryption failure details
- Reason for last message decryption failure............... Not applicable

Last AP disconnect details
- Reason for last AP connection failure.................... Not applicable
- Last AP disconnect reason................................ Not applicable

Last join error summary
- Type of error that occurred last......................... None
- Reason for error that occurred last...................... Not applicable
- Time at which the last join error occurred............... Not applicable

AP disconnect details
- Reason for last AP connection failure.................... Not applicable
Ethernet Mac : a0:93:51:1e :29:e0 Ip Address : 172.16.11.3

--More-- or (q)uit
(Cisco Controller) >show ap join stats detailed b4:de:31:9b:7e:20

Sync phase statistics
- Time at sync request received............................ Not applicable
- Time at sync completed................................... Not applicable

Discovery phase statistics
- Discovery requests received.............................. 63
- Successful discovery responses sent...................... 63
- Unsuccessful discovery request processing................ 0
- Reason for last unsuccessful discovery attempt........... Not applicable
- Time at last successful discovery attempt................ Nov 24 11:44:57.928
- Time at last unsuccessful discovery attempt.............. Not applicable

Join phase statistics
- Join requests received................................... 21
- Successful join responses sent........................... 21
- Unsuccessful join request processing..................... 0
- Reason for last unsuccessful join attempt................ Not applicable
- Time at last successful join attempt..................... Nov 24 11:45:07.485
- Time at last unsuccessful join attempt................... Not applicable

Configuration phase statistics

--More-- or (q)uit
- Configuration requests received.......................... 0
- Successful configuration responses sent.................. 0
- Unsuccessful configuration request processing............ 0
- Reason for last unsuccessful configuration attempt....... Not applicable
- Time at last successful configuration attempt............ Not applicable
- Time at last unsuccessful configuration attempt.......... Not applicable

Last AP message decryption failure details
- Reason for last message decryption failure............... Not applicable

Last AP disconnect details
- Reason for last AP connection failure.................... Not applicable
- Last AP disconnect reason................................ Not applicable

Last join error summary
- Type of error that occurred last......................... None
- Reason for error that occurred last...................... Not applicable
- Time at which the last join error occurred............... Not applicable

AP disconnect details
- Reason for last AP connection failure.................... Not applicable
Ethernet Mac : b4:de:31:9b :7e:20 Ip Address : 172.16.11.4

--More-- or (q)uit
(Cisco Controller) >show ap join stats detailed b4:de:31:27:46:80

Sync phase statistics
- Time at sync request received............................ Not applicable
- Time at sync completed................................... Not applicable

Discovery phase statistics
- Discovery requests received.............................. 1
- Successful discovery responses sent...................... 1
- Unsuccessful discovery request processing................ 0
- Reason for last unsuccessful discovery attempt........... Not applicable
- Time at last successful discovery attempt................ Nov 24 01:10:50.097
- Time at last unsuccessful discovery attempt.............. Not applicable

Join phase statistics
- Join requests received................................... 1
- Successful join responses sent........................... 1
- Unsuccessful join request processing..................... 0
- Reason for last unsuccessful join attempt................ Not applicable
- Time at last successful join attempt..................... Nov 24 01:10:59.881
- Time at last unsuccessful join attempt................... Not applicable

Configuration phase statistics

--More-- or (q)uit
- Configuration requests received.......................... 5
- Successful configuration responses sent.................. 2
- Unsuccessful configuration request processing............ 0
- Reason for last unsuccessful configuration attempt....... Not applicable
- Time at last successful configuration attempt............ Nov 24 01:11:00.826
- Time at last unsuccessful configuration attempt.......... Not applicable

Last AP message decryption failure details
- Reason for last message decryption failure............... Not applicable

Last AP disconnect details
- Reason for last AP connection failure.................... Not applicable
- Last AP disconnect reason................................ Unknown failure reason

Last join error summary
- Type of error that occurred last......................... None
- Reason for error that occurred last...................... Not applicable
- Time at which the last join error occurred............... Not applicable

AP disconnect details
- Reason for last AP connection failure.................... Not applicable
Ethernet Mac : 00:5d:73:79 :ee:48 Ip Address : 172.16.11.2

--More-- or (q)uit
(Cisco Controller) >show sysinfo

Manufacturer's Name.............................. Cisco Systems Inc.
Product Name..................................... Cisco Controller
Product Version.................................. 8.5.105.0

System Name...................................... cisco1815i
System Location..................................
System Contact...................................
System ObjectID.................................. 1.3.6.1.4.1.9.1.2489
IP Address....................................... 172.16.11.5
Last Reset....................................... 0: unknown

System Up Time................................... 0 days 10 hrs 36 mins 30 secs
System Timezone Location......................... (GMT +2:00) Jerusalem
System Stats Realtime Interval................... 5
System Stats Normal Interval..................... 180

Configured Country............................... LB - Lebanon

State of 802.11b Network......................... Enabled
State of 802.11a Network......................... Enabled
Number of WLANs.................................. 4
Number of Active Clients......................... 8

--More-- or (q)uit

OUI Classification Failure Count................. 0

Burned-in MAC Address............................ 00:5D:73:79:EE:40
Maximum number of APs supported.................. 50
System Nas-Id....................................
WLC MIC Certificate Types........................ SHA1/SHA2

(Cisco Controller) >show ap image all

Total number of APs.............................. 3
Number of APs
Initiated....................................... 0
Downloading..................................... 0
Predownloading.................................. 0
Completed predownloading........................ 0
Not Supported................................... 0
Failed/Waiting to Predownload................... 0

Predownload Predownload Flexconnect Failure Download
AP Name Primary Image Backup Image Status Version Next Retry Time Retr y Count Predownload Reason Progress
------------------ -------------- -------------- --------------- -------------- ---------------- ---- -------- -------------- -------------------- ----------
MainAP 8.5.105.0 8.8.111.0 None None NA NA NA
LabAP 8.8.111.0 0.0.0.0 None None NA NA NA
HeadOffice 8.8.111.0 0.0.0.0 None None NA NA NA

________________________________________________________

Regards,

B Osman

[Rich R · Accepted solution]

So your MainAP is running 8.5.105.0 while the other 2 are running 8.8.111.0 which is why they aren't working and you obviously haven't set up a TFTP server for them to download the software from so they can never fully join.
But your quick fix here is that your MainAP has 8.8.111.0 in the backup partition so all you have to do is switch to the backup image then they'll all be on the same version and should work.

[bosman]

Thanks you,

On the controller I used the following command to restore the backup,

config ap image swap <ap name>

______________________________________

Regards,

B Osman

[Rich R]

So it's working now?

[bosman]

Yes it is working, thanks you!

I will be upgrading the firmware of the access points later