Specific SSID Wireless Clients Suddenly Lose Connectivity

[fdharmawan]

Hi Guys,

Today I encountered an issue that is very strange. So on each AP I broadcasted 3 SSIDs. Today, an issue happened only to clients that is connected specifically to one of the SSIDs, let's say SSID A. The clients were still connected to the SSID but could not connect anywhere. But clients that are connected to SSID B and C were fine.

Upon investigating further, I found the following logs on the AP.
Oct 31 07:08:33 kernel: [*10/31/2023 06:53:40.6210] DOT11_DRV[1]: Stopped Radio 1
Oct 31 07:08:33 kernel: [*10/31/2023 07:08:33.4022] DOT11_DRV[0]: Stopped Radio 0
Oct 31 07:08:35 kernel: [*10/31/2023 07:08:35.2170] DOT11_CFG[1]: Starting radio 1
Oct 31 07:08:36 kernel: [*10/31/2023 07:08:36.3949] DOT11_DRV[1]: Started Radio 1
Oct 31 07:08:36 kernel: [*10/31/2023 07:08:36.6747] DOT11_CFG[0]: Starting radio 0
Oct 31 07:08:37 kernel: [*10/31/2023 07:08:37.5321] DOT11_DRV[0]: Started Radio 0

The AP itself is 3802 running on 8.10.162.0. I collected logs on PI, I could not find any anomalies as well. Have someone experienced the same?

[marce1000]

 - Is the AP attached to a controller ? If so what is the controller model and software version running on it ?

 M.

[fdharmawan]

Hi Marce,

Yes, it is attached to a controller. The controller itself is 5520 running on 8.10.162.0.

[marce1000]

  - Upgrade the controller to https://software.cisco.com/download/home/286284738/type/280926587/release/8.10.190.0  
     and try again , 

 M.

[fdharmawan]

Hi Marce,

Thank you. Let me arrange the upgrade.

[Rich R]

Like Marce says - step no 1 is to upgrade to the latest software release (see TAC recommended link below).
Also refer to Leo's link for known bugs on those APs.

Stopped radios would affect all SSIDs so that might or might not be relevant to your issue and there are many different causes for that including radio firmware hang or crash.
As the client is associated to the SSID the radio must be up.
What's the difference between the working and not working WLANs?

[fdharmawan]

Hi Rich,

The affected SSID is using dot1x auth. The other two SSIDs are using dot1x auth and web auth.

[Rich R]

So affected SSID is using dot1x auth and working SSID is using dot1x auth - what's the difference between them?

[fdharmawan]

The affected one is for laptops, while the unaffected one is for mobile devices.

[Rich R]

That tells me nothing about the difference in config between the 2 WLANs!
So again I ask: what's the difference between the config of the 2 WLANs?

[fdharmawan]

Hi Rich,

Only a bit difference.
1. SSID that was having an issue only serves laptops. The SSID A only serves mobile devices.
2. Even though both SSIDs uses dot1x, the certificates are different.
3. The dot1x authentications are handled by ISE, though both SSID has different set of policies on ISE.
4. Both SSIDs are using different VLANs

I could not think of any differences left, since both are pretty much the same. It's just the segregation.

[Rich R]

- Upgrade to 8.10.190.0

> the certificates are different.
Make sure the clients and ISE both trust the certificates so they are recognised as valid.

> both SSID has different set of policies on ISE
Check the logs on ISE?

> The affected one is for laptops
Have you made sure that the laptop OS and driver versions are 100% up to date?

Get client debugs from the WLC during failure and use the Debug Analyzer.

[fdharmawan]

Hi Rich,

I am sure 100%. Because only clients that are connected to that specific AP was being dropped. Thousands other of laptops are fine connected to hundreds different APs. I was just not sure why it was only affecting 1 SSID on 1 particular AP. Never seen like this one before.

[ammahend]

Restart the AP, i have seen issues where the AP does not bridge radio traffic to APs ethernet interface leading to traffic drop, disconnections etc, this was sporadic and does not effect every AP, your issue sounds very similar, please confirm if issue continues after reload. As suggested move to recommended code and upgrade laptop firmware also. 

[Rich R]

Ah yes that's new info - you hadn't mentioned that it was only affecting a single.  Agree with @ammahend - restarting the AP is likely to resolve the issue, almost definitely caused by one of the many bugs and if you're lucky it may be fixed in 8.10.190.0.