CSCvb25644 - AP3700 Randomly entering into continous boot loop due to Radio Fw image download error

[ABM Networking]

Does anyone know the work around for this? 

I can install any V7.x.x image and the AP works fine, install any 8.x.x image and the AP goes into the boot loop failing on the Radio FW Image download.

Tried updating the image via Rmon, is there a version I need to install to get this AP working with V8.2.151

[Jades Moreira]

I have the same problem with the AP 2702. Now I´ll try to recover the image via rommon. If I solve it I share what was done.
========================================================
*Mar 1 00:00:28.787: %DOT11-2-RESET_RADIO: Restarting Radio interface Dot11Radio1 due to FW: hostmem badmagic=00000000, vec=33 pc=D0109470, ev=1, @01B74AA9,s@01B72A16
*Mar 1 00:00:28.787: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio1 due to
*Mar 1 00:00:29.683: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio0 due to IOS reload
*Mar 1 00:00:30.483: %SYS-5-RELOAD: Reload requested by Init. Reload Reason:
Radio FW image download failed ....
IOS Bootloader - Starting system.
========================================================

[Leo Laohoo]

What controller firmware version are you using?  I am very familiar with this bug. 

There are two ways of fixing this: 

1.  Force the AP to boot the RCV image by deleting the CAPWAP image.  

2.  Force the AP to boot into ROMmon, format the flash, and upload the RCV image.  

In both methods, console access is the only way.  

These are the methods I've developed myself because I was encountering this bug when I would "loose" an AP every three days.  I've upgraded my controller to 8.5.103.0 and this issue has stopped.

Issue is present in 8.5.103.0/8.5.105.0.

[Jades Moreira]

Hi Leo Laohoo,

Thanks for help me.

See, the controller version is 8.3.122.0 but will be update to 8.3.130.0. Now I´ll try to recover AP via console/rommon with a new image.

[Leo Laohoo]

Cisco will be publishing new firmwares to fix KRACK WPA2 vulnerability.  I recommend you go with this fix.

Fix firmware are scheduled for release on 23 October 2017 (see image below). 

[Thomas Obbekaer Thomsen]

Im looking forward to hearing if the newest 8.3 solved the problem. :-)

We have multible 2700 that does this as well.

A hardreset of the AP (holding down the reset button) is the easy fix for us at this point.

[chris.brunt]

8.3.133.0 still getting a problem with 1x 3702 however I have fixed about 8x 1702 by formating flash and using recovery image.

Had about 40 APs fail in a boot loop with error like below

%CAPWAP-3-ERRORLOG: Validate Msg: msg type 4 does not supported payload 215
%LWAPP-3-CLIENTERRORLOG: Validate Msg: msg type 4 does not supported payload 215
%LWAPP-3-CLIENTERRORLOG: Validate Msg: error in unknown payload (received length = 9, payload type = 215)
%CAPWAP-3-ERRORLOG: Failed to validate vendor specific message element type 215 len 9.
%CAPWAP-3-ERRORLOG: Config status: Failed to validate vendor specific message element.
%CAPWAP-3-ERRORLOG: Failed to decode join response.

Able to fix them remotely by forcing them to an 8.2 version on another controller by using a vlan that has an ACL blocking the 8.3 controller IP. Then once they have downloaded 8.2, change vlan back to normal put them back to 8.3 and they work okay.

I have lost confidence with doing software updates as imagine having to console when you have 1000 APs in a hospital!

[Leo Laohoo]

Well, this issue is also present in 8.5.103.0/8.5.105.0. Will observe if this is fixed in 8.5.110.0.

[rudimocnik]

I am running 8.5.151.0 on IW3702 and one of the AP is experiencing this same issue. 

[Leo Laohoo]

---

@rudimocnik wrote:

*Mar  1 00:00:23.951: %SYS-5-RELOAD: Reload requested by Init. Reload Reason: 
Radio FW image download failed ....

---

 No, this issue is totally different. 

Break into ROMMON and post the complete output to the command "dir flash:".

[rudimocnik]

ap: dir flash:
Directory of flash:/

2    -rwx  279       <date>               info
3    -rwx  64        <date>               sensord_CSPRNG0
4    -rwx  146       <date>               capwap-saved-config
5    -rwx  0         <date>               config.txt
30   drwx  2496      <date>               ap3g2-k9w8-mx.153-3.JD16
7    -rwx  280       <date>               lwapp_officeextend.cfg
6    drwx  640       <date>               configs
8    -rwx  146       <date>               capwap-saved-config-bak
9    -rwx  64        <date>               sensord_CSPRNG1
10   -rwx  64159     <date>               event.log
11   -rwx  61044     <date>               lwapp_non_apspecific_reap.cfg
18   -rwx  73831     <date>               event.capwap
13   -rwx  106       <date>               pnp-saved-config
26   -rwx  0         <date>               ce
23   -rwx  141597    <date>               event.r1
28   -rwx  95008     <date>               lwapp_reap.cfg.bak
25   -rwx  142272    <date>               event.r0
42   -rwx  95008     <date>               lwapp_reap.cfg
29   -rwx  12312     <date>               private-multiple-fs
43   -rwx  230       <date>               env_vars

25785344 bytes available (15373312 bytes used)

[Leo Laohoo]

In ROMMON, issue this command: boot flash:/ap3g2-rcvk9w8-mx/ap3g2-rcvk9w8-mx

[rudimocnik]

I received the same error by trying to boot that.

I fixed the issue by formating flash: and following this guide.

https://community.cisco.com/t5/wireless-mobility-documents/access-point-rommon-recovery-ap-prompt-recovery-example/ta-p/3119495

[Thomas Obbekaer Thomsen]

Did anyone notice that this bug us now status : terminated, and there is no fix.
So this is a hardware problem ?

[Leo Laohoo]

The bug, and other iteration, is present in 8.0 to 8.5.
The biggest question is how to find the missing APs and fix them.