AP sudden restarting

[Moudar]

Hi,

I got 2 APs 2700 joining WLC 5520 which runs 8.5.181. The APs sometimes unjoin and rejoin the WLC.

x.x.x.x represents the WLC's IP address.

Logs:

*Nov 24 13:58:46.759: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 10.0.0.147:5246
*Nov 24 13:58:46.763: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio0 due to the reason code 27
*Nov 24 13:58:46.763: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio1 due to the reason code 27
*Nov 24 13:58:46.763: %WIDS-6-DISABLED: IDS Signature is removed and disabled.
*Nov 24 13:58:46.891: %CAPWAP-5-AP_EASYADMIN_INFO: AP Easy Admin information - EASY_ADMIN is not set, turn off easy admin service!

*Nov 24 13:58:46.891: %CAPWAP-5-AP_EASYADMIN_INFO: AP Easy Admin information - Easy Admin is not enabled, turn it off!

*Nov 24 13:58:46.915: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio0 due to the reason code 39
*Nov 24 13:58:46.915: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio1 due to the reason code 39
*Nov 24 13:58:46.927: %CLEANAIR-6-STATE: Slot 0 down
*Nov 24 13:58:46.931: %CLEANAIR-6-STATE: Slot 1 down
*Nov 24 13:58:46.931: %LINK-5-CHANGED: Interface Dot11Radio0, changed state to administratively down
*Nov 24 13:58:46.931: %LINK-5-CHANGED: Interface Dot11Radio1, changed state to administratively down
*Nov 24 13:58:46.931: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio0 due to the reason code 10
*Nov 24 13:58:46.935: %LINK-6-UPDOWN: Interface Dot11Radio0, changed state to up
*Nov 24 13:58:47.539: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio1 due to the reason code 10
*Nov 24 13:58:47.567: %LINK-6-UPDOWN: Interface Dot11Radio1, changed state to up
*Nov 24 13:58:47.931: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to down
*Nov 24 13:58:47.959: %LINK-6-UPDOWN: Interface Dot11Radio1, changed state to down
*Nov 24 13:58:47.967: %LINK-5-CHANGED: Interface Dot11Radio1, changed state to reset
*Nov 24 13:58:48.951: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to up
*Nov 24 13:58:48.959: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio1, changed state to down
*Nov 24 13:58:48.995: %LINK-6-UPDOWN: Interface Dot11Radio1, changed state to up
*Nov 24 13:58:49.007: %LINK-6-UPDOWN: Interface Dot11Radio0, changed state to down
*Nov 24 13:58:49.015: %LINK-5-CHANGED: Interface Dot11Radio0, changed state to reset
*Nov 24 13:58:49.995: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio1, changed state to up
*Nov 24 13:58:50.007: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to down
*Nov 24 13:58:50.035: %LINK-6-UPDOWN: Interface Dot11Radio0, changed state to up
*Nov 24 13:58:51.035: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to up
*Nov 24 13:59:17.567: AP has SHA2 MIC certificate - Using SHA2 MIC certificate for DTLS.

*Nov 24 13:59:18.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 10.0.0.147 peer_port: 5246
*Nov 24 13:59:18.247: %CAPWAP-5-DTLSREQSUCC: DTLS connection created sucessfully peer_ip: X.X.X.X peer_port: 5246
*Nov 24 13:59:18.247: %CAPWAP-5-SENDJOIN: sending Join Request to X.X.X.X
*Nov 24 13:59:23.247: %CAPWAP-5-SENDJOIN: sending Join Request to X.X.X.X
*Nov 24 13:59:23.415: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio0 due to the reason code 56
*Nov 24 13:59:23.419: %LINK-6-UPDOWN: Interface Dot11Radio0, changed state to down
*Nov 24 13:59:23.427: %LINK-5-CHANGED: Interface Dot11Radio0, changed state to reset
*Nov 24 13:59:24.047: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio1 due to the reason code 56
*Nov 24 13:59:24.051: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio0 due to the reason code 10
*Nov 24 13:59:24.055: %CAPWAP-5-JOINEDCONTROLLER: AP has joined controller 5520-1
*Nov 24 13:59:24.123: %LINK-6-UPDOWN: Interface Dot11Radio0, changed state to up
*Nov 24 13:59:24.527: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to down
*Nov 24 13:59:24.783: %LINK-6-UPDOWN: Interface Dot11Radio1, changed state to down
*Nov 24 13:59:24.791: %LINK-5-CHANGED: Interface Dot11Radio1, changed state to reset
*Nov 24 13:59:24.863: %WIDS-6-ENABLED: IDS Signature is loaded and enabled
*Nov 24 13:59:25.123: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio1, changed state to down
*Nov 24 13:59:25.775: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to up
*Nov 24 13:59:25.823: %DOT11-6-DFS_SCAN_START: DFS: Scanning frequency 5300 MHz for 60 seconds.
*Nov 24 13:59:25.827: %LINK-6-UPDOWN: Interface Dot11Radio1, changed state to up
*Nov 24 13:59:25.835: %LINK-6-UPDOWN: Interface Dot11Radio0, changed state to down
*Nov 24 13:59:25.843: %LINK-5-CHANGED: Interface Dot11Radio0, changed state to reset
*Nov 24 13:59:26.827: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio1, changed state to up
*Nov 24 13:59:26.835: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to down
*Nov 24 13:59:26.867: %LINK-6-UPDOWN: Interface Dot11Radio0, changed state to up
*Nov 24 13:59:27.867: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to up
*Nov 24 13:59:46.747: %CLEANAIR-6-STATE: Slot 0 enabled
*Nov 24 13:59:48.799: %CLEANAIR-6-STATE: Slot 1 enabled
*Nov 24 14:00:26.875: %DOT11-6-DFS_SCAN_COMPLETE: DFS scan complete on frequency 5300 MHz
*Nov 24 14:27:47.087: %DOT11-4-FLUSH_DEAUTH: Consecutive tx fail 500+: deauth ac67.5de4.3851
*Nov 28 10:49:30.819: %DOT11-4-FLUSH_DEAUTH: Consecutive tx fail 500+: deauth 847b.57c8.c8a0

ON the logs i could see something "administratively down" does this mean same as in switches (someone shutted down that !) if so, where are the logs?

What does the last 2 logs mean?

What does "%CAPWAP-5-AP_EASYADMIN_INFO: AP Easy Admin information - EASY_ADMIN is not set, turn off easy admin service!"

Mean?

[Scott Fella]

Does you ap join and is joined for a while and then drops off or does the ap never really join successfully.

ON the logs i could see something "administratively down" does this mean same as in switches (someone shutted down that !) if so, where are the logs?

No

What does the last 2 logs mean?

What does "%CAPWAP-5-AP_EASYADMIN_INFO: AP Easy Admin information - EASY_ADMIN is not set, turn off easy admin service!"

Here are a list of reset codes.  It can be due to DFS radar.

AP Radio Reset Codes - Cisco

Easy Admin info can be found here:

FlexConnect AP Easy Admin (cisco.com)

[Moudar]

The AP joins successfully for a while like 3 days and then suddenly do this again and again!

DFS channels are not used, I use only 36, 40, 44, 48, 52, 56, 60, 64!  Extended UNII-2 channels are disabled!

The AP is in local mode why should it gets Flexconnect logs, as EASY_ADMIN?

[Leo Laohoo]

On the AP, post the complete output to the command "dir".

[Moudar]

Directory of flash:/

    2  -rwx         282   Jan 1 1970 00:10:21 +00:00  info
    3  -rwx         368  Nov 24 2022 13:59:25 +00:00  capwap-saved-config
    4  -rwx       57967   Jul 3 2022 15:07:43 +00:00  event.log
    5  -rwx      128409  May 10 2017 09:59:23 +00:00  event.r1
   38  drwx         576   Jan 1 1970 00:10:20 +00:00  ap3g2-rcvk9w8-mx
    6  -rwx          64   Jul 3 2022 15:07:36 +00:00  sensord_CSPRNG0
    7  -rwx      128560   Jan 4 2018 23:00:35 +00:00  event.r0
   77  drwx           0   Mar 1 1993 00:01:07 +00:00  configs
   78  -rwx           0  Dec 11 2017 17:12:17 +00:00  config.txt
   79  -rwx          64   Jul 3 2022 15:07:36 +00:00  sensord_CSPRNG1
    8  -rwx         381  Nov 29 2022 05:53:44 +00:00  env_vars
    9  -rwx       12312  Nov 28 2022 16:49:13 +00:00  private-multiple-fs
  145  -rwx         368  Nov 28 2022 16:49:13 +00:00  capwap-saved-config-bak
   82  drwx        2368   Jul 3 2022 15:07:15 +00:00  ap3g2-k9w8-mx.153-3.JF15

40900608 bytes total (18640384 bytes free)

[Moudar]

I got another AP where users are complaining, the AP is 2800 and got about 800 logs this week!!!

Nov 24 07:39:19 mrvlfwd: Bad conn type 0 in msg proc
Nov 24 07:39:34 mrvlfwd: Bad conn type 0 in msg proc
Nov 24 07:39:39 mrvlfwd: Bad conn type 0 in msg proc
Nov 24 07:39:44 mrvlfwd: Bad conn type 0 in msg proc
Nov 24 07:39:54 mrvlfwd: Bad conn type 0 in msg proc
Nov 24 07:39:59 mrvlfwd: Bad conn type 0 in msg proc
Nov 24 07:40:09 mrvlfwd: Bad conn type 0 in msg proc
Nov 24 07:40:19 mrvlfwd: Bad conn type 0 in msg proc
Nov 24 07:40:34 mrvlfwd: Bad conn type 0 in msg proc
Nov 24 07:43:44 mrvlfwd: Bad conn type 0 in msg proc
Nov 24 07:44:04 mrvlfwd: Bad conn type 0 in msg proc
Nov 24 07:44:14 mrvlfwd: Bad conn type 0 in msg proc
Nov 24 07:49:49 mrvlfwd: Bad conn type 0 in msg proc
Nov 24 09:25:29 mrvlfwd: Bad conn type 0 in msg proc
Nov 24 10:09:29 mrvlfwd: Bad conn type 0 in msg proc
Nov 24 13:54:10 mrvlfwd: Bad conn type 0 in msg proc
Nov 24 19:29:41 mrvlfwd: Bad conn type 0 in msg proc
Nov 24 19:31:27 mrvlfwd: Bad conn type 0 in msg proc
Nov 24 23:55:23 mrvlfwd: Bad conn type 0 in msg proc
Nov 25 08:20:58 mrvlfwd: Bad conn type 0 in msg proc
Nov 25 08:50:59 mrvlfwd: Bad conn type 0 in msg proc
Nov 25 10:06:14 mrvlfwd: Bad conn type 0 in msg proc

Users are complaining like "we loose the connection" on IPADS. IPADS are running 16.1 and 16.1.. I don't know if the problem is in the AP or the wifi problem with Ipads? 

[Leo Laohoo]

---

@Moudar wrote:

    4  -rwx       57967   Jul 3 2022 15:07:43 +00:00  event.log
    5  -rwx      128409  May 10 2017 09:59:23 +00:00  event.r1

---

Can you attach these two files?

[Moudar]

What portion of these files do you need to look at?

There are many IPs and MACs that i don't want to disclose in open yard 🙂

[Leo Laohoo]

---

@Moudar wrote:

What portion of these files do you need to look at?

---

I want to see everything.  

---

@Moudar wrote:

There are many IPs and MACs that i don't want to disclose in open yard 🙂

---

Raise a TAC Case.

[Moudar]

July 3 was a WLC upgrading from 8.3 to 8.5

[Rich R]

So probably nothing signifcant in those files then.  The radio crash was 2017 and the reload was your software upgrade.

From the info you've provided on the 2700 (which I think is 8.5.182.0 not 8.5.181.0) I'd say your main problem is the CAPWAP session to the controller dropping - check your connectivity between the AP and WLC for packet drops.

The ipad problems on the 2800 are likely to be completely separate - problem could be AP or ipad or which options you have enabled on the WLAN.  Test with disabling features and get over the air captures for problem clients.  You might need to get Cisco TAC to help with that.  But even if it is a WLC/AP bug you won't get a fix in 8.5.182.0.  You would need to move to 8.10 (which might even fix it) but that's rapidly approaching the end of software maintenance milestone (31-01-2023) https://www.cisco.com/c/en/us/products/collateral/wireless/5520-wireless-controller/eos-eol-notice-c51-744430.html so the soon-to-be-released 8.10.18x.0 will possibly be the last release so if it's not fixed already then unlikely to be.  The only reason for more releases will be major security vulnerabilities until 30-01-2025.