1530AP boot loop - recovery?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-28-2022 11:30 AM - edited 05-28-2022 11:50 AM
1530AP is stuck in a boot loop after changing high availability settings to point to a new controller running 8.3.150.0
The old controller is still online (version 8.0.152.0). Did the same thing with APs including 1530s and they are all registered, upgraded and working.
The 1530AP shows up on the new controller with ip address 0.0.0.0 even though it has an ip address.
It won't upgrade and won't go back to the old controller. On the WLC I get AP Disassociated. Base Radio MAC x:x:x:x ApName - APblahblah
Packet captures show it doing discovery, and it seems to negotiate capwap/dtls with the new controller.
The AP syslogs show:
%CAPWAP-5-DTLSREQSUCC: DTLS connection created sucessfully peer_ip: x.x.x peer_port: 5246
%CAPWAP-5-SENDJOIN: sending Join Request to x.x.x.x
%SYS-5-RELOAD: Reload requested by CAPWAP CLIENT. Reload Reason: Reload.
Edit: fixed it by putting on the same vlan as the old controller and adding an ACL on the AP switch port blocking access to the new controller. The AP then registered back on the old controller.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-28-2022 12:08 PM
Hi
1530 is outdoor AP. They need to be permited on the WLC. SECURITY > AP Policy
Add the AP to the authorization list.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-28-2022 03:03 PM
The AP MIC is already set in AP policy on the new controller.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-29-2022 06:40 AM
Could be a corrupt flash problem preventing it from downloading the new IOS from the new WLC.
https://www.cisco.com/c/en/us/support/docs/field-notices/703/fn70330.html
You can try upgrading the IOS yourself manually after checking the flash.
https://software.cisco.com/download/home/284620606/type/280775090/release/15.3.3-JD17
Please click Helpful if this post helped you and Select as Solution (drop down menu at top right of this reply) if this answered your query.
------------------------------
TAC recommended codes for AireOS WLC's and TAC recommended codes for 9800 WLC's
Best Practices for AireOS WLC's, Best Practices for 9800 WLC's and Cisco Wireless compatibility matrix
Check your 9800 WLC config with Wireless Config Analyzer using "show tech wireless" output or "config paging disable" then "show run-config" output on AireOS and use Wireless Debug Analyzer to analyze your WLC client debugs
Field Notice: FN63942 APs and WLCs Fail to Create CAPWAP Connections Due to Certificate Expiration
Field Notice: FN72424 Later Versions of WiFi 6 APs Fail to Join WLC - Software Upgrade Required
Field Notice: FN72524 IOS APs stuck in downloading state after 4 Dec 2022 due to Certificate Expired
- Fixed in 8.10.196.0, latest 9800 releases, 8.5.182.12 (8.5.182.13 for 3504) and 8.5.182.109 (IRCM, 8.5.182.111 for 3504)
Field Notice: FN70479 AP Fails to Join or Joins with 1 Radio due to Country Mismatch, RMA needed
How to avoid boot loop due to corrupted image on Wave 2 and Catalyst 11ax Access Points (CSCvx32806)
Field Notice: FN74035 - Wave2 APs DFS May Not Detect Radar After Channel Availability Check Time
Leo's list of bugs affecting 2800/3800/4800/1560 APs
Default AP console baud rate from 17.12.x is 115200 - introduced by CSCwe88390
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-29-2022 12:55 PM
Looks like the new AP code is indeed corrupt.
1530 is registered to old WLC and I enabled SSH
From ap:
#archive upload-sw /version ap1g3-k9w8-mx.153-3.JD17 ftp://x.x.x.x/test.tar
With bash cli:
tar xf test.tar cd ap1g3-k9w8-mx.153-3.JD17 $ sha512sum -c < file_hashes CA2.bin: OK CA5.bin: OK CO2.bin: OK CO5.bin: OK RO2.bin: OK RO5.bin: OK ap1g3-k9w8-mx.153-3.JD17: FAILED img_sign_rel.cert: OK img_sign_rel_sha2.cert: OK info: OK sha512sum: WARNING: 1 computed checksum did NOT match
To recap, recovery required:
1. create an ACL on switch port so the AP could not talk to the new WLC. This broke the boot loop and let it revert to old WLC
2. When the AP finally reverted to the old WLC, I enabled SSH
3. log in to AP, rename corrupt file to ap1g3-k9w8-mx.153-3.JD17.bad to make sure re-downloading didn't hit the same bad spot in flash
4. copy good file, delete bad file
5. reconfigure for new WLC
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-29-2022 02:02 PM
Did you fsck the flash? (read the links I shared)
If not it will just keep causing more problems for that AP until you do!
You're lucky you only had 1 AP with the problem. When upgrading from an affected version of code you should be checking every AP as per the field notice - simply prevents problems like this.
Please click Helpful if this post helped you and Select as Solution (drop down menu at top right of this reply) if this answered your query.
------------------------------
TAC recommended codes for AireOS WLC's and TAC recommended codes for 9800 WLC's
Best Practices for AireOS WLC's, Best Practices for 9800 WLC's and Cisco Wireless compatibility matrix
Check your 9800 WLC config with Wireless Config Analyzer using "show tech wireless" output or "config paging disable" then "show run-config" output on AireOS and use Wireless Debug Analyzer to analyze your WLC client debugs
Field Notice: FN63942 APs and WLCs Fail to Create CAPWAP Connections Due to Certificate Expiration
Field Notice: FN72424 Later Versions of WiFi 6 APs Fail to Join WLC - Software Upgrade Required
Field Notice: FN72524 IOS APs stuck in downloading state after 4 Dec 2022 due to Certificate Expired
- Fixed in 8.10.196.0, latest 9800 releases, 8.5.182.12 (8.5.182.13 for 3504) and 8.5.182.109 (IRCM, 8.5.182.111 for 3504)
Field Notice: FN70479 AP Fails to Join or Joins with 1 Radio due to Country Mismatch, RMA needed
How to avoid boot loop due to corrupted image on Wave 2 and Catalyst 11ax Access Points (CSCvx32806)
Field Notice: FN74035 - Wave2 APs DFS May Not Detect Radar After Channel Availability Check Time
Leo's list of bugs affecting 2800/3800/4800/1560 APs
Default AP console baud rate from 17.12.x is 115200 - introduced by CSCwe88390
