Cisco AIR3602 - continual reboots

[SimonWoodcock8849]

Hi,

I had setup in a test lab a autonomous AIR3602i, running the code version "ap3-g2-k9w7-tar.153-3.J14,tar" - Been fine for about 3 months. Today, one started rebooting continuously - okay, so I had another, powered it up, had a base configuration on, but okay all good. So I configured it as per the first. Just finished, rebooted to test, and that has now started rebooting continuously. Now I'm confused, because I have a third which runs a 'production' network, and that has a different configuration, but works fine! 

Anyone seen this behavior before? The two that reboot I can't escape character in to do any diagnostics.continuous reboot, Aironet Access Points

[Scott Fella]

Post the output from the console when the ap is powered on until the ap reboots.  The output will typically have the answer to the issue.

[Leo Laohoo]

Post the complete output to the following commands: 

1. sh version
2. dir

[SimonWoodcock8849]

Hi both, thanks for prompt response. I have tried booting these up and basically what I have found is:

1. 3602 boots normally not attached to original PoE port, on a standard Cisco Mains supply (for this device) (captured in 3602_normal_boot.txt)

2. 3602 is stable when not connected to any clients, and the config (with no IP/passwords) is in config.txt

3. When I then connect a client to the 3602, then the unit crashes. Both do this (I have two) this is captured in 3602_crash.txt

So for background:

1. I use a 192.168.210.x/24 VLAN, it's an internal test VLAN

2. I have tested this connected to the original LAN port with PoE, LAN port with standard Cisco mains PSU, check with another port on switch (SG series switch), not connected to switch and this is the same behavior on all scenarios

3. This error crops up when a client is connected only, otherwise the 3602 is stable

4. The firmware level is up to date

5. This has been running for about 2 months okay, happened after a power failure yesterday, but second 3602 was configured from the first

Regards,

[Scott Fella]

You mentioned this previously, "Now I'm confused, because I have a third which runs a 'production' network, and that has a different configuration, but works fine!"  What is the difference between this working ap and the other?  

[SimonWoodcock8849]

Hi,

So I have three of these devices:

1. Main WLAN network, setup with specific  SSID, on own VLAN on Cisco SG500 switch, WPA etc for day to day usage, Access port on switch, single VLAN, just autonomous WAP for 1 VLAN access

2. Test network - for me to work with, but become part of a separate WLAN because with lockdown we needed more devices connected as a wider family and I needed to expand my home network out - same logic in setup, i.e. on own VLAN on SG500, this one is a Trunk not Access port, but one VLAN, but logic is pretty much the same as 1.

3. A spare device, ready in case one of the other two failed

So number 2 failed. So I took the config off of it, put it on number 3, and tried again. This one fails as well, same symptoms, continuous reboots.

But, the WAP in '1.' is still fine!!

I'm guessing something in the config of 2 and 3 has hit an undocumented bug (or is a bit of my config I've got wrong and somehow looping reboots on client connection).

I've attached the remaining controllers config.

They all run the same code level, and are set to autonomous. 

Regards,

[Rich R]

- You've misquoted your software version - it's ap3g2-k9w7-mx.153-3.JF14 (not J14) which is very recent.  However there is a JF14i out now which you could try.

- Presume you're aware these APs are almost end of support and went end of software maintenance 29 Dec 2017 End-of-Sale and End-of-Life Announcement for the Cisco Aironet 3600 Series - Cisco so Cisco stopped testing the software on these APs 3.5 years ago so use at own risk.

- Yes it's very likely a bug, something on your device triggering it.  There are plenty of bugs for similar crashes - mostly terminated (Cisco won't fix them) some which they have tried to fix (seems there are multiple causes).  Most have no workarounds but 2 of them suggest disabling PMF will reduce the frequency of crashes.  Some are also related to the AC module so if you have that installed try removing it. https://www.cisco.com/c/en/us/td/docs/wireless/access_point/15-3-3/configuration/guide/cg15-3-3/cg15-3-3-chap12-wdsroam-rm.html

- Otherwise try disabling features or even try downgrading to an older release but this is the risk with using end of life hardware with modern devices.

[SimonWoodcock8849]

Hi sorry been away for a few days. I'll have a go at this. Yes It does look like I have hit a bug in the image software. I'll check, I didn't think there was an updated version for these units. If there is I'll update. 

I am planning to upgrade these, but was hoping to later this year. I have put two other devices in, and with a slightly different configuration these are both fine. 

I will make the configuration change and see if this works as well. Thanks for this. Will update.

Regards,

Simon

[Leo Laohoo]

Post the complete output to the following commands: 

1. sh version
2. dir

[SimonWoodcock8849]

Please see below:

AIR1#sh ver
Cisco IOS Software, C3600 Software (AP3G2-K9W7-M), Version 15.3(3)JF14, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2021 by Cisco Systems, Inc.
Compiled Tue 09-Feb-21 00:51 by prod_rel_team

ROM: Bootstrap program is C3600 boot loader
BOOTLDR: C3600 Boot Loader (AP3G2-BOOT-M) LoaderVersion 12.4(23)JY, RELEASE SOFT WARE (fc1)

AIR1 uptime is 4 hours, 3 minutes
System returned to ROM by power-on
System image file is "flash:/ap3g2-k9w7-mx.153-3.JF14/ap3g2-k9w7-xx.153-3.JF14"
Last reload reason:

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco AIR-SAP3602I-E-K9 (PowerPC) processor (revision A0) with 188398K/60928K by tes of memory.
Processor board ID FCZ1632J1E9
PowerPC CPU at 800Mhz, revision number 0x2151
Last reset from power-on
1 Gigabit Ethernet interface
2 802.11 Radios

32K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address: D4:8C:B5:E4:79:5E
Part Number : 73-14521-02
PCB Serial Number : FOC16303BSB
Top Assembly Part Number : 800-35852-02
Top Assembly Serial Number : FCZ1632J1E9
Top Revision Number : C0
Product/Model Number : AIR-CAP3602I-E-K9

Configuration register is 0xF

AIR1#dir
Directory of flash:/

2 -rwx 285 Jan 1 1970 04:13:11 +00:00 info
3 drwx 2496 Jan 1 1970 04:23:24 +00:00 ap3g2-k9w7-mx.153-3.JF14
234 -rwx 11584 Jul 31 2012 14:20:38 +01:00 event.log
243 -rwx 2608 Aug 27 2012 10:37:51 +01:00 config.txt
244 -rwx 2072 Aug 27 2012 10:37:51 +01:00 private-multiple-fs
245 -rwx 450 Mar 1 1993 00:00:31 +00:00 env_vars
246 -rwx 2608 Aug 27 2012 10:37:51 +01:00 config.txt.bak
247 -rwx 1915 Aug 27 2012 10:37:51 +01:00 private-config
248 -rwx 4 Oct 3 2012 16:18:49 +01:00 FOC16310PF0

31739904 bytes total (17964544 bytes free)

[Scott Fella]

Have you tried to take the working ap config, edit it and use that on the other two that are having issues?  If you look at the console output when the ap reboots, does that tell you why it rebooted?  Have you tried to clear the nvram?

Hold reset for over 30 seconds to get to ROMMON

ap:delete flash:private-config

ap:delete flash:private-multiple-fs

ap:boot

Your AP will be factory default

[SimonWoodcock8849]

Hi, yes, the second unit exhibited the same symptoms when config was loaded.

[Scott Fella]

You used the config from the ap that is still running fine?  Did you factory reset with the instructions that I provided?  If you still have an issue, well... time to look for a replacement for those old ap's.  If budget is an issue, you can look for a used 2504 controller on ebay and centralize the management.  I'm not a fan of autonomous ap's anymore and the support for them is no longer being worked on.  

[SimonWoodcock8849]

Hi, thanks Scott. Would a AIR-CT5508-K9 do the job okay? I think I may have one I was given around somewhere....

[SimonWoodcock8849]

Sorry is there a reference for access points vs WLC compatibility please? I can't seem to find one, sure one did exist somewhere. Thanks