WLC cannot store all AP bundle images on its flash

business0001 · ‎08-02-2017

Hello,

at the moment I have 2x 1852i deployed which are booth equiped with the ME firmware. One of the APs is my ME controller. I have a SmartNet for those APs and running the 8.5 firmware, also a TFTP is running with the images.

Now I want to add an additional 2802i which I bought with CAP firmware. Problem is, the AP connects to the ME controller and tries to download the firmware. I monitored it also via CLI and it seems that the 2802i is stuck in a update loop or something like that.

Any ideas what I can do? Do you want a CLI log?

Leo Laohoo · ‎08-02-2017

Reboot the 1852 and try again.

business0001 · ‎08-03-2017

Did not change anything. AP still in reboot loop...

sremk · ‎08-03-2017

Hi,

Just check the software of 2802i and if it is different from 1852 software, then please follow the below procedure.

The MobExp "WLC" cannot store all AP bundle images on its flash. Therefore you have to host a TFTP server where you unzipped all the AP images (available on cisco.com in the 1850 section). Give this TFTP details in the MobExp WLC GUI. Newly joining APs will then download the code from the TFTP instead of from the MobExp WLC.

If the APs trying to join are already running capwap image on the same code, they can join despite lack of TFTP server as they don't need to download new code.

Thanks

Sreejith

patoberli · ‎08-04-2017

Hi Sreejith

Thanks for this post! As you work for Cisco, I have one suggestion/wish, please please please offer something different from the slow ass and not anymore developed and improved TFTP! SFTP maybe, or SCP, or even HTTPS, or is this also possible? (not having a MobExp here)

sremk · ‎08-04-2017

Hi Patoberli,

Understand your frustration.

I am afraid that we don't have that option at the moment and this should be considered under feature request.

Thanks

Sreejith

business0001 · ‎08-04-2017

That is the problem. Different firmware versions on both devices. 1850 controller is running 8.5.X and 2802 8.2.X.

BUT:

TFTP is configured and worked fine with an 1700i, which joined the controller after getting the firmware via TFTP..

So the question is, what is the problem with the 2802?

patoberli · ‎08-04-2017

I believe the 1700 series is already included in the on-board ME firmware, while the firmware for the 2800 series is a separate download.

Check this part here: http://www.cisco.com/c/en/us/td/docs/wireless/access_point/mob_exp/85/release_notes/b_ME_RN_85.html#sw-rls

So you need to download the file AIR-AP1850-K9-ME-8-5-103-0.zip and extract it to the TFTP server, probably into it's root directory.

business0001 · ‎08-04-2017

Thanks, but as mentioned before the ZIP file is extracted to my TFTP server and the controller provides this information. Thats how the 1700 got his update which I monitored via CLI.

As I can see via CLI at the 2802 it is downloading its firmware, but reboot and installation is faulty and ending in a reboot loop trying to update again.

patoberli · ‎08-04-2017

The thing is, the 1700 doesn't require that ZIP file, only the x8xx series and a few others do.

Anyway, could you share the console output from the 2802 while it's trying to update and reboot? Maybe it shows us some hints about what might be wrong.

business0001 · ‎08-05-2017

[*01/01/1970 00:00:07.1703] Made it into bootsh: Jul 28 2016 21:48:46
[*01/01/1970 00:00:07.1725] bootsh build T-449ffb2d842632edd9ea4c3f2e30425b28467356-g449ffb2d-kalairam
init started: BusyBox v1.20.2 (2016-07-28 21:20:49 PDT)

User Access Verification
Username: [*01/01/1970 00:00:12.9664]
[*01/01/1970 00:00:12.9664]
[*01/01/1970 00:00:12.9664]
[*01/01/1970 00:00:12.9664]
[*01/01/1970 00:00:12.9664] Authenticating ACT2 ID:AP2800
[*01/01/1970 00:00:12.9664] Authenticating SN:FDO20341KGX
[*01/01/1970 00:00:12.9668]
[*01/01/1970 00:00:12.9669] (ACT2Boot)[08/02/2017 22:50:11.3000] PCI: enabling device 0000:00:02.0 (0140 -> 0143)
Hardware is Cisco (c) authentic :-)
[*01/01/1970 00:00:12.9669]
[*08/22/2016 23:59:59.0055] Last reload time: Aug 2 22:50:11 2017
[*08/02/2017 22:50:11.0001] Setting system time Wed Aug 2 22:50:11 UTC 2017
[*08/02/2017 22:50:11.0111] /usr/bin/btldr_upgrade: Bootloader booted on master version 0x106 Bundle: none
[*08/02/2017 22:50:11.0111] Verifying shadow against master...
[*08/02/2017 22:50:11.0111] shadow version 106
[*08/02/2017 22:50:11.0111] No updates
[*08/02/2017 22:50:11.1592] Active version: 8.2.121.0
[*08/02/2017 22:50:11.1602] Backup version: 0.0.0.0
[*08/02/2017 22:50:12.5504] 0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;
[*08/02/2017 22:50:17.7138] Firmware download complete
[08/02/2017 22:50:17.8600] PCI: enabling device 0000:00:03.0 (0140 -> 0143)
[*08/02/2017 22:50:19.1086] 0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;
[*08/02/2017 22:50:23.5060] Firmware download complete
[*08/02/2017 22:50:23.9355] ifconfig: SIOCGIFFLAGS: No such device
[*08/02/2017 22:50:24.0505] Initializing NSS Firmware
[08/02/2017 22:50:27.1700] Disabling lock debugging due to kernel taint
[*08/02/2017 22:50:27.1840] stil[08/02/2017 22:50:27.2700] stile module dp init successfully
e_lm_ft_barbados[08/02/2017 22:50:27.3500] NBAR Label:BLD_MCP_DEV_LATEST_20150424_123021 : module licenseNBAR VERSION: 'Copyright (c) module build date: Jan 28 2016 at 10:43:16
2014-2015 by cisco Systems, Inc.' taints kernel.
[*08/02/2017 22:50:30.4485]
[*08/02/2017 22:50:30.4516] boot 13 build T-449ffb2d842632edd9ea4c3f2e30425b28467356-g449ffb2d-kalairam board barbados mac CC:16:7E:FA:ED:86
[*08/02/2017 22:50:30.4541] Module: act2_driver .text=0x7f000000        data=0x7f000770        bss=0x7f000a04
[*08/02/2017 22:50:30.4541] Module: aptrace .text=0x7f004000        data=0x7f008f64        bss=0x7f009198
[*08/02/2017 22:50:30.4541] Module: ap8x .text=0x7f00f000        data=0x7f0b6518        bss=0x7f0bbdc0
[*08/02/2017 22:50:30.4541] Module: mv_nss_api .text=0x7f287000        data=0x7f2d5c78        bss=0x7f2d7088
[*08/02/2017 22:50:30.4541] Module: proclikefs .text=0x7f2e6000        data=        bss=0x7f2e6b80
[*08/02/2017 22:50:30.4541] Module: stile_lm_ft .text=        data=        bss=
[*08/02/2017 22:50:30.4541] Module: stile_lm_dp .text=        data=        bss=
[*08/02/2017 22:50:30.4541] Module: mtdoops .text=0x7fb55000        data=0x7fb55dd8        bss=0x7fb55f50
[*08/02/2017 22:50:30.4541] Module: ansi_cprng .text=0x7fb5a000        data=0x7fb5a8bc        bss=0x7fb5aaf8
[*08/02/2017 22:50:30.4541] Module: crc_itu_t .text=0x7fb5e000        data=        bss=
[*08/02/2017 22:50:30.4541] Module[08/02/2017 22:50:31.7000] wired0 emac 2: link up
: rng_core .tex[08/02/2017 22:50:31.7700] wired0: link up
t=0x7fb62000        data=0x7fb62818        bss=0x7fb629d8
[*08/02/2017 22:50:30.4541] Module: udf .text=0x7fb66000        data=0x7fb75e7c        bss=0x7fb75ff8
[*08/02/2017 22:50:30.5625] Last reload reason : 0: unknown
[*08/02/2017 22:50:53.0540] Create symlink /dev/mic_cert result 0
[*08/02/2017 22:50:53.0540] Loading certificates from storage...
[*08/02/2017 22:50:53.0664] SKU to radio FW
[*08/02/2017 22:50:53.0984] Generating key, this may take a while...
[*08/02/2017 22:50:55.8763] Public key portion is:
[*08/02/2017 22:50:55.8763] ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdFiEoPCQSP0JWPuTQapyUNI7T99jwXwKeMgxFQCVry3IQEAaW49MvbMDgEFhcZ1GleHl3VajMjQf7hc2LyJv2lDf3STjWxaxZBG50D0zQld7bwDxG6RthbCjqj1kdZxpwsTDP3zCuFiEcclhWbt8Zq6DWzN825oOm3Hr5XswRXhva7nozjrNOpDqucX4JoQI8IQCJeUuqnnRfnA48aRIlOX7CKGbOGwXpeDdhxeupFrGEa5Wo/Z9V+2HGJICxJgcm0wdQZ6tB7JPVqbJl7PCX0D61Gvo+WmYOPZ2Jb+bT+iDPM8n0tNb+0QVuk8hwUCfBGJoi4IaA/zb/fvR2UnQt root@mCC167EFAED86
[*08/02/2017 22:50:55.8763] Fingerprint: md5 5f:d8:bd:65:91:27:94:f6:71:90:5a:1e:9f:8a:9a:d4
[*08/02/2017 22:50:55.9525] gzip -dc /etc/wcp.usermodule.template.gz | sed -e 's/__TX_POWER_XOR_5G_FILE__/\/radio_fw\/X5.csv/g' -e 's/__EVM_POWER_XOR_5G_FILE__/\/radio_fw\/EVM_X5.csv/g' -e 's/__TX_POWER_2G_FILE__/\/radio_fw\/R2.csv/g' -e 's/__TX_POWER_5G_FILE__/\/radio_fw\/R5.csv/g' -e 's/__EVM_POWER_2G_FILE__/\/radio_fw\/EVM_2.csv/g' -e 's/__EVM_POWER_5G_FILE__/\/radio_fw\/EVM_5.csv/g' -e 's/__DOMAIN_2G__/1/g' -e 's/__DOMAIN_5G__/12/g' -e 's/__PRODUCT_ID__/AIR-AP2802I-E-K9/g' -e 's/__AP_TYPE__/56/g' -e 's/__WIRELESS_MAC__/00:A2:EE:FB:47:C0/g' -e 's/__WIRED_MAC__/CC:16:7E:FA:ED:86/g' -e 's/__WIRED1_MAC__/CC:16:7E:FA:ED:86/g' | DMALLOC_OPTIONS=debug=0x4e48503,inter=1000,log=/tmp/wcp.dmalloc.log,limit=40000000 wcpd &
[*08/02/2017 22:50:57.2368] Stopped Radio 0
[*08/02/2017 22:50:57.8693] Stopped Radio 1
[*08/02/2017 22:50:59.9562] waiting for uplink IP address and reachable default gateway
[*08/02/2017 22:50:59.9562]
[*08/02/2017 22:51:01.8759] ethernet_port wired0, ip 10.30.0.181, netmask 255.255.254.0, gw 10.30.0.1, mtu 1500, bcast 10.30.1.255, dns1 10.30.0.10, dns2 10.20.10.10, domain XXXX, vid 0, static_ip_failover false, dhcp_vlan_failover false
[*08/02/2017 22:51:02.9925] /usr/bin/cisco_core.sh: Corefile /storage/cores//APCC16.7EFA.ED86__core-verify.8.2.121.0.2017-08-03-00-01-24.tgz deleted.
[*08/02/2017 22:51:03.0300] Password for 'Cisco' changed
[*08/02/2017 22:51:03.6894] DOT11_DRV[0]: set_channel Channel set to 1
[*08/02/2017 22:51:04.2849] DOT11_DRV[0]: set_channel Channel set to 1
[*08/02/2017 22:51:05.4654] DOT11_DRV[1]: set_channel Channel set to 36
[*08/02/2017 22:51:06.0738] DOT11_DRV[1]: set_channel Channel set to 36
[*08/02/2017 22:51:06.1841] Failed to load flex AP config from file. Default config will be used.
[*08/02/2017 22:51:06.1863] WTP IP Address = 10.30.0.181
[*08/02/2017 22:51:06.8538] DOT11_DRV[0]: set_channel Channel set to 1
[*08/02/2017 22:51:07.5078] DOT11_DRV[0]: set_channel Channel set to 1
[*08/02/2017 22:51:08.7023] DOT11_DRV[1]: set_channel Channel set to 36
[*08/02/2017 22:51:09.3827] DOT11_DRV[1]: set_channel Channel set to 36
[*08/02/2017 22:51:09.4931] Failed to load flex AP config from file. Default config will be used.
[*08/02/2017 22:51:09.4941] CAPWAP State: Discovery
[*08/02/2017 22:51:09.7251] Discovery Request sent to 10.20.1.22[08/05/2017 15:52:59.0000] device capwap0 entered promiscuous mode
2 with discovery type set to CAPWAP_DISCOVERY_TY[08/05/2017 15:52:59.1300] device capwap1 entered promiscuous mode
PE_STATIC_CONFIG(1)
[*08/02/2017 22:51:09.7267] Discovery Request sent to 255.255.255.255 with discovery type set to CAPWAP_DISCOVERY_TYPE_UNKNOWN(0)
[*08/02/2017 22:51:09.7270] CAPWAP State: Init
[*08/02/2017 22:51:09.7271] CAPWAP State: Discovery
[*08/02/2017 22:51:09.7739] Discovery Response from 10.20.1.222
[*08/05/2017 15:52:59.0000] CAPWAP State: DTLS Setup
[*08/05/2017 15:53:02.1416] CAPWAP State: Join
[*08/05/2017 15:53:02.1427] Sending Join request to 10.20.1.222 through port 5272
[*08/05/2017 15:53:06.8391] Join Response from 10.20.1.222
[*0[08/05/2017 15:53:06.9000] UBIFS: background thread "ubifs_bgt0_1" started, PID 8398
8/05/2017 15:53:06.8392] TLV-DEC-PROC: Info miss[08/05/2017 15:53:07.0300] UBIFS: mounted UBI device 0, volume 1, name "part2"
ing for 1363
[*[08/05/2017 15:53:07.1200] UBIFS: LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
08/05/2017 15:53[08/05/2017 15:53:07.2600] UBIFS: FS size: 65646592 bytes (62 MiB, 517 LEBs), journal size 8888320 bytes (8 MiB, 70 LEBs)
:06.8392] CAPWAP[08/05/2017 15:53:07.4100] UBIFS: reserved for root: 0 bytes (0 KiB)
State: Image Da[08/05/2017 15:53:07.5000] UBIFS: media format: w4/r0 (latest is w4/r0), UUID 7E6E3849-4DBC-41AE-B74B-D47197E82BA0, small LPT model
ta
[*08/05/2017 15:53:06.8694] do PRECHECK, part1 is active part
[08/05/2017 15:53:07.7300] UBIFS: un-mount UBI device 0, volume 1
[08/05/2017 15:53:07.8000] UBIFS: background thread "ubifs_bgt0_1" stops
[*08/05/2017 15:53:19.9801] .........Invalid event 50 & state 10 combination.
[*08/05/2017 15:55:00.8858] SM handler: Failed to process timer message. Event 50, state 10
[*08/05/2017 15:55:00.8858] Failed to handle timer message.
[*08/05/2017 15:55:00.8858] FSM failure while processing message with id: 1
[*08/05/2017 15:55:09.0672] ....................................................

patoberli · ‎08-07-2017

I've never seen this message and I also don't find any public bug in the toolkit.

I suggest to open a TAC.

You could try to manually upgrade the image (via console and a TFTP server), but it's probably better to have a TAC open.

business0001 · ‎08-07-2017

Yeah this is completely strange.

I have to buy a SmartNet for the 2802 first. An the moment I can not download any images to TFTP manuel or open a ticket.

Thank you for your help! :)

patoberli · ‎08-07-2017

Oh ok. Last chance would be this: https://supportforums.cisco.com/discussion/10941311/recovering-failed-ap

Not sure if this is still valid for the x8xx models, since they use a completely new software, but might be worth a try.

stoteda · ‎12-16-2017

Hello Sreejth,

It appears that I have the same issue as another user you helped out. I have the 1815i's in a network running the Mobility Express, and I attempted to have the 2802i join as an AP. It appears to connect to the controller but is not functional. It shows 0.0.0.0 as it's IP address, but I can see the MAC / IP binding set up properly in the DHCP tables of my DHCP server.

I suspect the AP is also attempting to download the image from the ME Controller, but cannot.

My question is whether I have to put the same revision of 2802i image as I am running on the 1815i, which would be 8.4.100. Or would you recommend putting upgrade image for both the 1815i and the 2802i.

Everything has been working fine, and I probably should have just added another 1815i. But I was attempting to add a higher performance AP for a busy area.

Thanks in advance!

Mobility Express Scenario - problems with 2802i