Corp users are getting stuck in 8021X_REQD

[Noovi]

Hello Team,

we have Cisco 5520 WLC and we have upgraded WLC to 8.10.183.0 image version.

After this upgrade, few region started complaining issue in connecting and as per invistigation we found that user are stuck in 8021X_REQD state.

When we checked logs in ISE, we found error like 'suplicant stopped responding to ISE'

We already checked with CISCO TAC from wireless and ISE end but no any findings from them.

Anyone has similar issues at your end?

[Noovi]

Hello Leo,

thanks.

Any links/advisoiries for this communication?

[Leo Laohoo]

None so far.

[JPavonM]

@Leo Laohoo  do you have additional updates from this issue?

Which conditions do the issue needs to appear? Which adapters are impacted?

Regards.

[Leo Laohoo]

No, Google would not provide us the Intel case number. 

However, I found an issue with Realtek RTL88xx wireless NIC:  CSCwf03870 &/or Realtek Release Notes

The Realtek bug is an issue with Realtek (regardless of brand & model).

[Noovi]

Meanwhile, Cisco identify bug in 8.10.183.0 image for 8021X_REQD issue and fix for this issue is image 8.10.185.0

But funny thing is 8.10.185.0 is having another issue for AP reboot due to Kernel Panic and issue is still not fixed.

[Leo Laohoo]

APs suffering from Kernel Panic or crashes due to FIQ/NMI is nothing to be of any concern because after the APs crash they return to a normal state (until the next crash).

[Rich R]

Like Leo says there are dozens of AP crash bugs.  Most of those are not actually new - they just keep opening new bugs for them as we (customers) keep reporting them.  Over time the numbers will come down hopefully.  They're unlikely to get fixed in AireOS now.  You're still better off upgrading to 8.10.185.0 to get the other fixes - those open caveats will also be affecting 8.10.183.0 as well in most cases so you're not making them any worse with the upgrade.

[LenarFA]

Could you, please, provide bug ID?

[Noovi]

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCwe91301

[Noovi]

Reply from TAC

Here is quick summary of our call till now:

• APs are crashing due to Kernel panic error message:

<4>[04/18/2023 04:28:33.3290] CPU: 1 PID: 3486 Comm: wl1-kthrd Tainted: P           O    4.1.52 #5

<4>[04/18/2023 04:28:33.4310] Hardware name: Broadcom-v8A (DT)

<4>[04/18/2023 04:28:33.4940] task: ffffffc003bac680 ti: ffffffc06c61c000 task.ti: ffffffc06c61c000

<4>[04/18/2023 04:28:33.5960] PC is at __udelay+0x38/0x48

<4>[04/18/2023 04:28:33.6550] LR is at osl_delay+0x2c/0x48 [hnd]

***

<0>[04/18/2023 04:28:35.0460] Kernel panic - not syncing: softlockup: hung tasks

<4>[04/18/2023 04:28:35.1290] CPU: 1 PID: 3486 Comm: wl1-kthrd Tainted: P           O L  4.1.52 #5

<4>[04/18/2023 04:28:35.2300] Hardware name: Broadcom-v8A (DT)

We already have opened a defect for this issue 2 weeks ago:

AP crash with PC & LR is at __udelay+0x30/0x48 CS00012289938

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCwe91301

Following defect affects AP models 9105, 9115 and 9120 and it is found on versions 8.10.183 and 17.3.6 code. 18xx/28xx/38xx and 48xx APs should not be affected.

Possible workaround:

1. Move APs from AireOS to 9800 WLC with code 17.6.4 or 17.9.3
2. Change one of he 9120 to EWC mode with 17.9.3 code and move all other APs to that AP.

[LenarFA]

Thanks for the info, but I meant this - "Meanwhile, Cisco identify bug in 8.10.183.0 image for 8021X_REQD issue and fix for this issue is image 8.10.185.0". There is an official bug ID for "user are stuck in 8021X_REQD state" in 8.10.183.0? I have the same issue with wlc55200 (AireOS 8.10.183.0 ) and 9115ax.

[Noovi]

Ohh Sorry..i shared bug for AP reboot due to Kernel panic issue in 8.10.185.

For dot1xrequired issues, please find below. and actually i think TAC shared more than 1 bug

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCwd59921

Thank you for your time. As discussed on call, we have identified this is to be a known issue and is fixed in 8.10.185.0 version. I have shared the release notes and links to the bugs below.

8.10.185.0 Release Notes:

https://www.cisco.com/c/en/us/td/docs/wireless/controller/release/notes/crn810mr9.html

Please check under “Resolved Caveats” for ID: CSCwb82694. Details for the bug shared below.

https://bst.cisco.com/bugsearch/bug/CSCwb82694

The above bug is internally related to bug ID: CSCwd40914. Details for the bug shared below.

https://bst.cisco.com/bugsearch/bug/CSCwd40914

Please upgrade the WLC to 8.10.185.0 to fix this issue.