iPhones unstable on wifi

[Rene Mueller]

Hi Folks,

I am running EWC on 9115AXI Access Points. I randomly see that Apple iPhones are losing their Internet connection while they are sill on the wifi network. As a workaround I have to turn the wifi on the iPhone off an on again. This happens also when iPhones stayed really next to an AP. I don't see this kind of issue with Android devices.

I had this experience already with 1832i Mobility Express and also with 2802i Mobility Express.

In every setup (1830, 2802, 9115) I setup best practices on the ME or EWC controller with no success.

Anyone an idea what might be the issue?

[Rich R]

Definitely try 17.12.6 because it also includes the fix for CSCwp07242.  Note that although the bug refers to 9105 it actually potentially affects 9105, 9115 & 9120 because they all use similar Broadcom chipset and drivers (also shown under affected Products).  It's also fixed in 17.15.4.

[MicheleDS]

Hi,

installed 17.12.6 during the weekend. will let you know in the next couple of days if it fixes the issue with iOS devices.

Not sure that the CSCwp07242 apply to our environment since we have issues only with iOS devices, Android and Windows system works without issues. (anyway, any issue fixed is good )

thanks,

Michele

[Rich R]

@MicheleDS you should be aware that Cisco have removed (deferred)17.12.6 and 17.15.4 in the last few days due to a critical defect (I believe it's with mdns) so we can expect to see them replaced with 17.12.6a and 17.15.4a in the next few weeks so you should be prepared to upgrade from 17.12.6 to the replacement version once it is released.

[MicheleDS]

that's why I usually wait a couple of weeks before installing a freshly released version

anyway, APs seem to be working fine at the moment (just a few working hours at the moment) so I'll keep this version until the fixed one is released.

thanks for the information,

bye

Michele

[MicheleDS]

Today after 5 days of AP working as expected the issue with iOS devices and 5ghz radio started to happen again
as usual turning off 5ghz radio and then back on fixed the issue.....until it happens again

Any hints on what can we look at? 

here the log of one client when the issue appear:

2025/09/11 13:00:59.469951316 {wncd_x_R0-0}{1}: [apmgr-bssid] [9200]: (ERR): Failed to Get bssid hdl. Apmgr opctxt bssid handle is NULL.xxxxxxx40
2025/09/11 13:00:59.469977616 {wncd_x_R0-0}{1}: [sanet-shim-translate] [9200]: (ERR): xxxxxxxea6 :Could not get bssid handle
2025/09/11 13:00:59.688691665 {wncd_x_R0-0}{1}: [client-orch-sm] [9200]: (note): MAC: xxxxxxxea6 Client delete initiated. Reason: CO_CLIENT_DELETE_REASON_CLIENT_USER_TRIGGERED_DISASSOC, details: , fsm-state transition 88|8b|93|15|1a|1b|2d|38|47|49|4b|4d|56|65|67|88|8b|93|15|1a|1b|2d|38|47|49|4b|4d|56|65|67|88|b0|
2025/09/11 13:00:59.688928207 {wncd_x_R0-0}{1}: [client-orch-sm] [9200]: (note): MAC: xxxxxxxea6 Delete mobile payload sent for BSSID: xxxxxxx4f WTP mac: xxxxxxx40 slot id: 1
2025/09/11 13:00:59.688964267 {wncd_x_R0-0}{1}: [client-orch-state] [9200]: (note): MAC: xxxxxxxea6 Client state transition: S_CO_RUN -> S_CO_DELETE_IN_PROGRESS
2025/09/11 13:00:59.690230700 {wncd_x_R0-0}{1}: [dpath_svc] [9200]: (note): MAC: xxxxxxxea6 Client datapath entry deleted for ifid 0x9000002b
2025/09/11 13:00:59.690644105 {wncd_x_R0-0}{1}: [sanet-shim-translate] [9200]: (note): MAC: xxxxxxxea6 Session manager disconnect event called, session label: 0x60000579
2025/09/11 13:00:59.691447413 {wncd_x_R0-0}{1}: [sisf-tdl] [9200]: (ERR): DAD Pending Rec Del Failed fe80::10de:7336:8bc4:7006:-2147483626
2025/09/11 13:00:59.693759277 {wncd_x_R0-0}{1}: [client-auth] [9200]: (ERR): MAC: xxxxxxxea6 Failed to build flex client cache payload for FT-PSK. Couldn't get client AKM.
2025/09/11 13:00:59.696338003 {wncd_x_R0-0}{1}: [client-orch-state] [9200]: (note): MAC: xxxxxxxea6 Client state transition: S_CO_DELETE_IN_PROGRESS -> S_CO_DELETED
2025/09/11 13:01:45.677593827 {wncd_x_R0-0}{1}: [client-orch-sm] [9200]: (note): MAC: xxxxxxxea6 Association received. BSSID xxxxxxx4f, WLAN dsrad, Slot 1 AP xxxxxxx40, CiscoAP1, Site tag default-site-tag, Policy tag DS Policy, Policy profile dsrad, Switching Local, Socket delay 0ms
2025/09/11 13:01:45.677986611 {wncd_x_R0-0}{1}: [client-orch-state] [9200]: (note): MAC: xxxxxxxea6 Client state transition: S_CO_INIT -> S_CO_ASSOCIATING
2025/09/11 13:01:45.678906880 {wncd_x_R0-0}{1}: [dot11] [9200]: (note): MAC: xxxxxxxea6 Association success. AID 12, Roaming = False, WGB = False, 11r = True, 11w = True Fast roam = False
2025/09/11 13:01:45.679125622 {wncd_x_R0-0}{1}: [client-orch-state] [9200]: (note): MAC: xxxxxxxea6 Client state transition: S_CO_ASSOCIATING -> S_CO_L2_AUTH_IN_PROGRESS
2025/09/11 13:01:45.679632348 {wncd_x_R0-0}{1}: [client-auth] [9200]: (note): MAC: xxxxxxxea6 ADD MOBILE sent. Client state flags: 0x1 BSSID: MAC: xxxxxxx4f capwap IFID: 0x9000000d, Add mobiles sent: 1
2025/09/11 13:01:45.685212005 {wncd_x_R0-0}{1}: [client-auth] [9200]: (note): MAC: xxxxxxxea6 L2 Authentication initiated. method DOT1X, Policy VLAN 22, AAA override = 0 , NAC = 0
2025/09/11 13:01:45.688211316 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [9200]: (note): Authentication Success. Resolved Policy bitmap:11 for client xxxxxxxea6
2025/09/11 13:01:45.844477663 {wncd_x_R0-0}{1}: [sanet-shim-miscellaneous] [9200]: (ERR): authc policy update from SANet vlan 22
2025/09/11 13:01:45.844840686 {wncd_x_R0-0}{1}: [client-auth] [9200]: (note): MAC: xxxxxxxea6 L2 Authentication Key Exchange Start. Resolved VLAN: 22, Audit Session id: F000A8C0000005EC38DCA8D6
2025/09/11 13:01:45.855606017 {wncd_x_R0-0}{1}: [client-keymgmt] [9200]: (note): MAC: xxxxxxxea6 EAP Key management successful. AKM:FT-DOT1X Cipher:CCMP WPA Version: WPA3
2025/09/11 13:01:45.855991161 {wncd_x_R0-0}{1}: [client-orch-sm] [9200]: (note): MAC: xxxxxxxea6 Mobility discovery triggered. Client mode: Flex - Local Switching
2025/09/11 13:01:45.855999761 {wncd_x_R0-0}{1}: [client-orch-state] [9200]: (note): MAC: xxxxxxxea6 Client state transition: S_CO_L2_AUTH_IN_PROGRESS -> S_CO_MOBILITY_DISCOVERY_IN_PROGRESS
2025/09/11 13:01:45.857008792 {wncd_x_R0-0}{1}: [mm-client] [9200]: (note): MAC: xxxxxxxea6 Mobility Successful. Roam Type None, Sub Roam Type MM_SUB_ROAM_TYPE_NONE, Client IFID: 0x9000002b, Client Role: Local PoA: 0x9000000d PoP: 0x0
2025/09/11 13:01:45.857508217 {wncd_x_R0-0}{1}: [client-auth] [9200]: (note): MAC: xxxxxxxea6 ADD MOBILE sent. Client state flags: 0x2 BSSID: MAC: xxxxxxx4f capwap IFID: 0x9000000d, Add mobiles sent: 1
2025/09/11 13:01:45.857575817 {wncd_x_R0-0}{1}: [client-orch-state] [9200]: (note): MAC: xxxxxxxea6 Client state transition: S_CO_MOBILITY_DISCOVERY_IN_PROGRESS -> S_CO_DPATH_PLUMB_IN_PROGRESS
2025/09/11 13:01:45.857982962 {wncd_x_R0-0}{1}: [dot11] [9200]: (note): MAC: xxxxxxxea6 Client datapath entry params - ssid:dsrad,slot_id:1 bssid ifid: 0x0, radio_ifid: 0x90000008, wlan_ifid: 0xf0400001
2025/09/11 13:01:45.858156923 {wncd_x_R0-0}{1}: [dpath_svc] [9200]: (note): MAC: xxxxxxxea6 Client datapath entry created for ifid 0x9000002b
2025/09/11 13:01:45.858236864 {wncd_x_R0-0}{1}: [client-orch-state] [9200]: (note): MAC: xxxxxxxea6 Client state transition: S_CO_DPATH_PLUMB_IN_PROGRESS -> S_CO_IP_LEARN_IN_PROGRESS

and just wait forever in this state (IP LEARN IN PROGRESS) until the client timeout the IP request.

thanks,

Michele

[Mark Elsen]

  - @MicheleDS     Check the output from the CLI command :
                                        WLC 9800# show wireless stats trace-on-failure | ex :_0

  M.

[MicheleDS]

Hi,

since yesterday evening:

CiscoWLC1#show wireless stats trace-on-failure | ex :_0
----------------------------------------------------------
Wireless Trace On Failure Statistics
----------------------------------------------------------
021. Client MAB authentication failure...................: 5
023. Client stage timeout................................: 3
034. SANET AUTHC failure.................................: 3

Thanks,

Michele

[Mark Elsen]

  - @Rene Mueller   The output is too sparse to draw any conclusions ;what software version are you using on the
                               EWC  controller ?
                               Do not use session timeout value of 0 If maximum session timeout is desired, use 86400

 M.

[Rene Mueller]

Running version is 17.12.5 on EWC. 

How can I find the session timeout setting?

[Mark Elsen]

 - Go to the WLAN profile for the particular wlan. Then Advanced tab where you can find the session timeout setting ,

 M.

[Rene Mueller]

wlan session timeout was set to 28800, I changed this now to 86400.

[Leo Laohoo]

---

@MicheleDS wrote:

Today after 5 days of AP working as expected the issue with iOS devices and 5ghz radio started to happen again
as usual turning off 5ghz radio and then back on fixed the issue.....until it happens again

---

Confirms my recommendation of a daily reboot of the APs.  

Long-story-short:  There have been several bugs where the AP radio would, routinely, drop traffic.  These bugs, which I have started calling them as "features", have been in existence since 8.10.X.X days and have since migrated to IOS-XE.  Worse hit are Cheetah OS APs.  I do not see such behaviour with APs on classic IOS.  

Rebooting the controller or upgrading the controller firmware does not make any difference:

1.  It does not fix the issue (regardless what TAC or the developers say/promise); 

2.  Instead of getting acquainted with one bug, after upgrading there will be more bugs to get "familiar" with.  

[MicheleDS]

Even if it's not a "solution" I think i might start to apply it, it's been month now and we need to fix it somehow.

I'll try to script an AP reboot every early morning and see if it "fix" the issue....at least until a really fixed fw comes out!

thanks

Michele

[Leo Laohoo]

---

@MicheleDS wrote:
I'll try to script an AP reboot every early morning and see if it "fix" the issue

---

I've got one even better:  What is/are the model of the switch in the fleet? 

IF the switches are Catalyst 3750X, 3650/3850 or Catalyst 9k there is a feature called EnergyWise.

energywise domain [BLAH] security shared-secret [PASSWORD] energywise importance 60
!  Interface
interface range 
 energywise level 10 recurrence importance 70 time-range POWER_ON
 energywise level  0 recurrence importance 70 time-range POWER_OFF
 energywise importance 60

!  Time Range Power ON
time-range POWER_ON
 absolute start 00:00 12 September 2025
   periodic weekdays 8:00 to 21:00

!  Time Range Power OFF
time-range POWER_OFF
 absolute start 00:00 12 September 2025
   periodic weekdays 21:00 to 23:59
   periodic weekdays  0:00 to  8:00

[Rich R]

You could also use EEM applets (available on all platforms) with the "event timer cron" option.

event manager applet reset-ap authorization bypass
description shut then no shut interfaces to power cycle APs
event timer cron name int-admin-down-up cron-entry "00 02 * * *"
action 1.0 cli command "enable"
action 2.0 cli command "conf t"
action 3.0 cli command "int range Gig x/y - z"
action 4.0 cli command "shutdown"
action 5.0 wait 10
action 6.0 cli command "no shutdown"
action 7.0 cli command "end"