topic Re: WLC Memory Leak Reboot in Wireless

WLC Memory Leak Reboot

colossus1611 — Mon, 05 Jul 2021 19:35:03 GMT

Hi All,

I have a slow memory leak noticable on WLC, and have scheduled a WLC reboot in 8 hours time to fix it temporarily. The OS is N-1 and there are no bug fixes released by Cisco to address this memory leak as yet for this OS - 8.10.112.0

Question is - if I reboot WLC, will my APs reboot too? Because I would not want them to reboot.

I got told rebooting from GUI would not reboot APs, but rebooting from CLI would reboot APs as well, but I couldn't find any Cisco documentation to verify this.

I would appreciate if someone can please confirm based on experience/know how of this.

Thanks.

Re: WLC Memory Leak Reboot

Mark Elsen — Wed, 30 Sep 2020 06:55:45 GMT

- Yes , AP's will reboot in all circumstances regardless of methodology being used.

Re: WLC Memory Leak Reboot

colossus1611 — Wed, 30 Sep 2020 06:59:36 GMT

Oh that's no good then. I thought I was playing safe by rebooting off GUI, but maybe not.

Re: WLC Memory Leak Reboot

Leo Laohoo — Wed, 30 Sep 2020 07:01:44 GMT

What do you mean by "The OS is N-1".

If the WLC is not upgrade nor downgrading then the APs will NOT reboot with the controller reloads.

Re: WLC Memory Leak Reboot

Mark Elsen — Wed, 30 Sep 2020 07:03:15 GMT

- If the memory leak is very weak (...) and controller doesn't start complaining with error messages, then consider keeping production state on at all times.

Re: WLC Memory Leak Reboot

colossus1611 — Wed, 30 Sep 2020 07:08:30 GMT

Hi Leo,

Sorry with N-1 , I just meant we are on OS one below the latest release by Cisco - 10.10.112.0.

Okay , that's music to my ears then if it doesn't reboot APs except for upgrading or downgrading WLC.

So basically irrespective of whether I do it from CLI or GUI, it shouldn't matter in this case and APs should stay UP all the time.

We have Flexconnect in the environment and some sites being 24x7, I don't want to cause an outage to them, with APs being able to migrate to secondary controller while Primary reboots.

Re: WLC Memory Leak Reboot

colossus1611 — Wed, 30 Sep 2020 07:10:30 GMT

Hi Marce,

The memory leak seems to be taking away 2% every day for now it seems and we are left with 5% memory as I write this.

Not sure what you mean by keeping production on at all times - we have redundant WLC, so all APs are expected to migrate across; I just don't want them to reboot.

PID: AIR-CT5520-K9
Total System Memory.............................. (32380916 KB) 31621 MB
Total System Free Memory......................... (1639548 KB) 1601 MB (5 %)

Re: WLC Memory Leak Reboot

Mark Elsen — Wed, 30 Sep 2020 07:16:37 GMT

- Keeping production on = not rebooting. However the availability of a HA/SSO setup was not denoted in your initial post. Rebooting becomes more flexible then, access points will then not reboot either.

Re: WLC Memory Leak Reboot

colossus1611 — Wed, 30 Sep 2020 07:26:08 GMT

Thanks Marce1000.

Yes I forgot to mention about the HA setup. However, as per Leo's post, it seems it would not reboot APs irrespective of whether WLCs are in HA setup or standalone?

Though in my case with HA setup, it doesn't matter now that both of you have confirmed that APs won't reboot, I just want it clarified so we can mark the right post as a solution for future reference for any one reading through this.

Would it not reboot APs only if WLCs are in HA, or would it not reboot APs even on a standalone WLC if we aren't upgrading or downgrading the WLC?

Re: WLC Memory Leak Reboot

Scott Fella — Wed, 30 Sep 2020 09:02:26 GMT

If you are running N+1, you should write a script to move the aps to the secondary and then wait until the aps all move and reboot the primary. Once the primary comes back up, then move the aps back to the primary. Then you know for a fact that your environment is stable during your reboot.

Re: WLC Memory Leak Reboot

colossus1611 — Wed, 30 Sep 2020 11:30:07 GMT

Thanks Scott. I might look into script for next time around, a bit late for the change in 3 hours time I suppose.

I also noticed (something that I don't remember seeing with previous versions of OS on WLC) the reboot and restart options under Commands on GUI.

I presume the reboot is what the deem as the Fast Restart and other is the hard reboot, is it? What's the difference technically still, I am not sure.

Re: WLC Memory Leak Reboot

colossus1611 — Wed, 30 Sep 2020 14:30:12 GMT

Hi All,

For reference, I rebooted the primary WLC via GUI using 'reboot' (as opposed to restart) and the APs moved swiftly across to secondary WLC and had no reboot of their own. The setup this customer has for redundancy is using Mobility Group.

When primary WLC came back up, all APs moved back to it due to 'AP Fallback' feature enabled, which is enabled by default.

Hope that helps.

Re: WLC Memory Leak Reboot

Scott Fella — Wed, 30 Sep 2020 15:10:26 GMT

You can review this post for the answer:

https://community.cisco.com/t5/wireless-security-and-network/reboot-vs-restart-on-5520/td-p/3737426

Re: WLC Memory Leak Reboot

Rich R — Wed, 30 Sep 2020 17:43:33 GMT

Just for the record 10.10.112.0 does not exist - I guess you meant 8.10.112.0 which is deferred (Cisco no longer support it due to serious known bugs).

And if that is the case that is NOT N-1 as you call it.

There's been 3 releases since that and the current one is 8.10.130.0.

Do you actually have a bug ID for the memory leak or are you just hoping it will magically get fixed?

Normally a memory leak like that would be fairly high priority and get fixed in the next maintenance release so if there is a bug for it I'd expect it to be fixed in 8.10.130.0.

If you haven't raised a TAC case and got a bug ID for the memory leak then unlikely it will ever get fixed unless somebody else finds it and reports it.

Re: WLC Memory Leak Reboot

colossus1611 — Fri, 02 Oct 2020 04:58:06 GMT

Hi rrudling,

Thanks for the input, yes I indeed got that code version wrong as well as the fact that it is not the latest stable version. I meant 8.10.112.0 but overlooked if this was latest or not (checked in July and it was) due to the urgency with reboot due to memory leak.

The memory leak issue did get fixed with the WLC restart for now.

But your input did alert me to the fact that we need code upgrade, so I am now going to schedule that now on these WLCs. Thanks again. 🙂

Re: WLC Memory Leak Reboot

patoberli — Tue, 06 Oct 2020 06:53:43 GMT

Just for clarification, there are two possible redundancy modes, one is SSO (it's the better generally) and the other is two standalone WLC in a mobility group and with specific configuration on the APs (they need to have both WLC configured in their high availability tab).

For SSO, just make a redundancy failover, all APs will move to the second WLC without any traffic interruption (normally, or an extremely short one). In this case the previous WLC will reboot and then be the redundant secondary. Downside of SSO, if you do a software upgrade, both reboot at the same time, including all APs.

For two WLC (this is N+1), if you reboot the "primary" WLC, the APs will move to the second after a short time, this will cause a short traffic interruption. Advantage here is, you can upgrade the "secondary" to a new software release and then join some APs to the secondary to test the stability. If APs of the same building are split between the two WLC, then this can cause roaming issues/delays.

[edit]

I forgot one big downside of the N+1 variant. With N+1 both WLC require enough licenses to handle all APs if the other WLC is rebooting/broken. With SSO, only the primary needs AP licenses, the secondary will inherit them.

Re: WLC Memory Leak Reboot

colossus1611 — Sat, 03 Oct 2020 10:03:29 GMT

Thanks patoberli.

We have a redundant setup with Mobility Group setup - good to know that this means I can test secondary WLC first with the upgrade.