APs randomly unregister from WLC 5520

[adrianv3r@hotmail]

Hi all,

Last year I implemented two WLC 5520 with HA SSO (Site A Primary and Site B Secondary) . The wireless network has been working for about a year without a problem.

There are 497 APs registered to the WLC (AIR-CAP2702I-A-K9, AIR-CAP2602I-A-K9, AIR-CAP3702I-A-K9) that have been switchover from the Primary to Secondary unit and viceversa without any issue. Several switchover have been done manually without any problem.

Since AireOS 8.2.166.0 was deprecated by Cisco, we decided to upgrade the WLC to version 8.3.143.0. The WLC was working fine since one day the Secondary Unit became Active (Site B). This was no planned.

The real problem ocurred when we decided to switchover again to the Primary Unit (Site A) . The switchover was done normally and the 496 APs registered normally to the Primary WLC. (This is where things went strange). After 25-30mins several APs started to unregister from the Primary WLC at the point that only 370 from 496 APs kept registered to the WLC.

We forced switchover again to the Secondary WLC and then the 496 APs registered again. To keep it short, now if the Primary unit becomes the active unit, only 370 APs kept registered to the WLC for about 25-30mins (always the same APs, Several models 2702, 2602,3702). The APs that are unregistered are pingable and we have AP management via SSH. The unregistered APs are unable to ping the WLC mangement interface. If a site has 4 APs for example, 2 APs unregister from the WLC and the other two keep registered.

We decided to upgrade the WLC to versión 8.5.135. Same behavior. So, at this moment all the 496 APs are registered to the Secondary unit (Active unit for the moment).

I want to emphasize that switchover has been made several times in the past with no issue. No changes have been made to the network. The wireless network has been working fine for a year.

Thanks.

[Haydn Andrews]

Have you done any debugs on the WLC to look at why the APs are not joining?

Just confirming when on secondary WLC that all is fine, this is only on the primary WLC.

Can the WLC ping the APs?

Good Guide to investigate AP join issues:

https://www.cisco.com/c/en/us/support/docs/wireless/5500-series-wireless-controllers/119286-lap-notjoin-wlc-tshoot.html

From the sounds of it, it sounds like a bug, have you raised a TAC case?

[adrianv3r@hotmail]

Hi Hayden,

When APs are joined to the Primary WLC and they unregister, no ping between APs and WLC. Pings are fine when the Secondary WLC becomes active.

We are planning to upgrade to release 8.5.140 and if the behavior remains the same, we are going to downgrade to release 8.2.170.0 since the WLC was working fine with release 8.2.166.

If no success, the next step is to open a TAC case.

[Leo Laohoo]

I think this is due to Field Notice: FN - 70330 - Cisco IOS Access Point Stranded Due to Flash Corruption Issue - Workaround Provided.

[adrianv3r@hotmail]

Thanks Leo.

As I mentioned, the 496 APs are operational and registered as long as the Secondary WLC remains active. The output from the command SHOW AP IMAGE ALL shows the APs also were updated to 8.5.135.0 just fine.

Also, when the Primary WLC is forced switchover to become active, only 370 APs remain registered to the WLC. The same APs and quantity unregistered everytime. When the Primary WLC is active, the 496 APs remain registered to the WLC for around 25-30min. After that period of time, APs start to unregister and only 370 APs remain registered to the Primary WLC.

BTW, all the APs are in FlexConnect mode. And the WLC are configured in HA SSO.

The Primary WLC worked fine with release 8.3.143.0 for some time until this happened. We upgraded the WLC to release 8.5.135 but no success, same behavior. Next step is to upgrade to release .8.5.140.

[adrianv3r@hotmail]

We upgraded the Aire OS version to 8.5.140, same behavior. Also we swap the WLCs, so the Primary WLC is installed at the site B. No issues at Site B no matter if the WLC is Primary o Secondary. It seems that issue is only happening when the production is at Site A.

[patoberli]

Maybe there is a latency or packet loss issue between the APs and site A.
Can you console into one AP while it's unsuccessfully trying to associate to the WLC site A? Maybe you see something there.