11-01-2023 07:59 AM
Hi folks,
Running a Catalyst 9800-40 WLC. 1463 WAP's connected, predominantly C9136I WAP's. 8,817 active clients with 4,796 clients on our SSID using PEAP authentication via ISE on the back end.
We are observing clients will frequently disconnect from the network despite having strong RSSI and SNR levels. The behavior manifests as 'Incorrect Wi-Fi Password', even though the user has been previously connected with no issues and their wireless profile is saved on their phone. I have seen this exclusively on iPhone devices at this time. The DNAC log timestamps seem to lineup with the log 'Client has requested it be deleted'
To remediate, users can simply hit cancel and wait, and they will reconnect to the network. I am working a TAC case parallel to this community post. Just wanted to throw this out there in case other people were seeing this in their environment.
09-26-2024 08:43 AM
Hi Rich,
I have upgraded to version 17.12.4 because I don't have anything to rely on as Cisco TAC has abandoned the case asking me to raise on Apple. I am not sure whether those bugs are the ones related to the issue, but let's keep up checking users after upgrade.
09-26-2024 09:20 AM
I actually haven't realized until i saw your update this morning but...after updating to 17.12.4 I don't think I've seen the issue on my campus.
How about you?
09-26-2024 11:10 AM
Good to hear that. How long have you updated your devices?
09-27-2024 02:57 AM - edited 10-10-2024 08:00 AM
We upgraded to 17.12.4 + APSP1 (has to be requested from TAC) on Tuesday. We saw some AP crashes within minutes of upgrade due to "Beacon Stuck Reset Radio" even though it was middle of the night with no users and a few more since then so that seems to be a new regression - looks like CSCwm58430 - will open a TAC case for them! Apart from that no user problems reported so far and seems to be pretty stable.
ps: just had a closer look - also a few AP crashes due to "kernel panic" which are obviously not resolved by APSP1. Looks like they're using a lot of Meraki code now so I suspect they've imported bugs from the Meraki code which were not there before (note purely my opinion based on how much Meraki gets mentioned in the stack trace of the crash files)
FYI: 17.12.4 APSP1 fixes:
CSCwj72985 multiple wcpd crash during longevity test with ap in flex-LA/LS mode
CSCwj77042 Kernel Panic at "pc : splitmac_api_add_client+0x68/0x498[umac]"SF#07186679
Note that 17.12.4 APSP2 has been published now. It mentions an extra fix in APSP1 (not mentioned in APSP1 release note) which is:
CSCwj39057 9130: Traffic loss and delays due to perceived channel utilization and interference
APSP2 fixes:
CSCwj66264 Half Duplex Mismatch messages seen on mGig port of 9300, 9400 switches
FYI: 17.12.4 resolves all the PSIRT advisories which affect 9800 which were announced the other day.
Update 10-10-2024: TAC believe the "Beacon Stuck Reset Radio" crashes are caused by CSCwm58430 - devs are still scratching their heads. The suggested workaround is to disable individual channels on specific APs which I have no intention of doing at this point without further explanation from DE but apparently that stabilised the APs for another customer! So they're saying on 17.12.4 certain APs won't work reliably on certain channels!
10-01-2024 06:53 AM
@Rich R- Have you had a chance to install APSP2 yet? How's that going? Any updates on the AP crashes? I'm gearing up to upgrade to 17.12.4/APSP2 and am wondering if there's anything I should watch out for. (I mean, there will inevitably be something to watch out for; the question is what.)
In the lab, I was unable to join a PSK test SSID on the 5 GHz radio of an 1815W on 17.12.4 APSP2. That was yesterday, and the WLC and APs were up for exactly 1 week following the upgrade/APSP at that point. Rebooting the AP resolved that. I'm going to do daily tests of that radio to make sure it keeps working; could have been a fluke or hardware issue.
10-01-2024 10:09 AM
Hi @eglinsky2012 - we're not using any mgig on the 17.12.4 WLCs at the moment so no need for the SP2 fix. So we'll stick with APSP1 for now.
10-10-2024 09:02 AM
@Rich R wrote:Hi @eglinsky2012 - we're not using any mgig on the 17.12.4 WLCs at the moment so no need for the SP2 fix. So we'll stick with APSP1 for now.
APSP3 is out for the following bugs:
CSCwk33521 Multiple 913x/916x AP Kernel Crashes (SF 07238396) CSCwk58876 Multiple 9166 AP Kernel Crashes (SF 07238396) CSCwm13005 Router Advertisement packets from clients result in ipv6 gateway change on the Access Point
Since we upgraded to 17.12.4/APSP2 on Monday, we have had no crash logs generated to the WLC crash for any 9100 series APs. Only a handful of older APs. And we don't use IPv6. So, for now I'll skip APSP3. Just thought I'd mention it for others' benefit.
10-10-2024 10:23 AM
Thanks for the update @eglinsky2012
Naturally they didn't update the release notes! Feedback submitted.
10-10-2024 11:37 AM
@Rich R wrote:Thanks for the update @eglinsky2012
Naturally they didn't update the release notes! Feedback submitted.
Yeah, it's not in the "Release Notes for 17.12.4" link, but it is in the "README" page if you hover over the file name on the downloads site.
10-18-2024 06:15 AM
After I have updated WLC to 17.12.4 it seems that the issue of Apple devices giving the message "Incorrect password" and asking for authentication all the time has stopped. Now we have been facing an issue where WLC suddenlly crashes and switchover to standby, the standby crashes in the same way switchover to the previous active and so on, keep flapping and meanwhile APs won't JOIN the WLC an Wifi environment remains out.
Cisco TAC has advised me update to 17.14.1. Has anyone updated for that version?
10-18-2024 08:26 AM
- @listcsbgnetsecurity >... Now we have been facing an issue where WLC suddenly crashes
When that happens issue the command : show version | inc reload
M.
10-18-2024 12:48 PM
In order to recover the environment, I have manually rebooted. When occur again I will take this command, and I will post here.
10-18-2024 02:03 PM
I'm not aware of anybody using 17.14.1 and I would not generally recommend using a limited support release.
Did TAC give a specific reason for recommending 17.14.1 - for example did it include a specific bug fix which they think is relevant to the problem?
If not then I would be very suspicious that they are just hoping it will help and it won't make any difference.
If they do point to a specific bug fix then ask them to provide that fix as a SMU for 17.12.4 rather than forcing you to upgrade to 17.14.1.
10-21-2024 08:41 AM
TAC has advised this version to fix CSCwk14917 bug. WLC is suddenly crashing.
10-21-2024 09:07 AM
CSCwk14917 is tagged as Duplicate meaning it was not actually fixed but the fix is supposed to be in another bug.
As seems to be common these days dev/TAC have not linked it to the bug which is supposed to have the fix so there is no way to verify what version (if any) might have the fix in it.
Please ask your TAC engineer to confirm which bug is supposed to fix this issue (because it is not CSCwk14917)?
And then ask them to make sure that CSCwk14917 is linked to the other bug as it should be?
Let us know once you have the correct bug ID which has the actual fix?
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide