topic Re: Catalyst 9800-CL client losing connectivity in Wireless

Catalyst 9800-CL client losing connectivity

Jeff A. — Mon, 05 Jul 2021 19:58:54 GMT

I've got a strange issue which I'm not quite able to find the root cause for.

There is one particular client type on my network (iPad) which will, after some time, loose network connectivity as well as its IPv6 address. The affected client usually loses connectivity after being on the network for a few days. Once connectivity is lost, nothing on the network can be pinged, either on IPv4 or IPv6. I don't know if the IPv6 address is just the first one to fall or not as the IPv4 address is sticky. Performing a DHCP lease renew does not bring the client back online, instead the client must be disconnected from the wireless network and re-connected before restoring connectivity.

I have a dual-stack LAN where the IPv6 addresses are provided to clients using an external router via RA. AP's are in FlexConnect mode with local switching but central authentication. The issue appears to re-occur every 3-5 days of the device being connected to the network. Since the device does not leave the premises, it will eventually fail. Either other devices do not appear to be affected (Macs and iPhones) or there is some timeout that is occurring at the time when the affected clients are in use. Next time it re-occurs I will attempt to find out how long, if at all, it would take for the client to have its connectivity restored.

I was initially experiencing this as a more widespread issue across iPhone devices where even the IPv4 address was lost, and per the advice of another discussion thread on this forum I modified the Session Timeout on the WLAN to 0 seconds, which appears to have resolved that. Now it just appears to affect iPad devices, but I'm not sure why they would be losing connectivity and IPv6 addressing but not IPv4 addressing.

Any insight or suggestions would be appreciated.

Re: Catalyst 9800-CL client losing connectivity

Mark Elsen — Sat, 09 Jan 2021 17:08:30 GMT

- Make sure the 9800-cl uses a recent and or advisory software release. Same for the OS on the ipad(s). For the rest you will need client debugging if the problem persists.

Re: Catalyst 9800-CL client losing connectivity

Jeff A. — Sun, 10 Jan 2021 00:31:32 GMT

Thanks for the advice. Installed version is 17.3.2a. I can't go newer as there is 3702i hardware which are not supported in 17.4+. iPads are running the latest software from Apple.

I've got the following logs out of the controller for one of the affected clients:

2021/01/09 13:17:25.084596 {wncd_x_R0-0}{1}: [client-keymgmt] [22681]: (ERR): MAC: 7a46.532f.1d50  Keymgmt: Failed to eapol key m5 retrasmit failure. Max retries for M5 over
2021/01/09 13:17:25.084754 {wncd_x_R0-0}{1}: [client-orch-sm] [22681]: (note): MAC: 7a46.532f.1d50  Client move to idle state, delete reason: CO_CLIENT_DELETE_REASON_GROUP_KEY_UPDATE_TIMEOUT, BSSID MAC: 005d.73fc.316f, WTP MAC: 005d.73fc.3160
2021/01/09 13:17:25.084894 {wncd_x_R0-0}{1}: [client-orch-state] [22681]: (note): MAC: 7a46.532f.1d50  Client state transition: S_CO_RUN -> S_CO_IDLE
2021/01/09 13:17:30.619127 {wncd_x_R0-0}{1}: [dot11] [22681]: (ERR): MAC: 7a46.532f.1d50  Failed to parse disassoc/deauth payload, continuing
2021/01/09 13:17:30.619310 {wncd_x_R0-0}{1}: [client-orch-sm] [22681]: (note): MAC: 7a46.532f.1d50  Client delete initiated. Reason: CO_CLIENT_DELETE_REASON_DEAUTH_OR_DISASSOC_REQ, fsm-state transition 82|8a|13|17|18|28|33|42|44|46|48|4d|5c|5e|7f|82|8a|13|17|18|28|33|42|44|46|48|4d|5c|5e|7f|26|a5|
2021/01/09 13:17:30.619501 {wncd_x_R0-0}{1}: [client-orch-sm] [22681]: (note): MAC: 7a46.532f.1d50  Delete mobile payload sent forbssid: 005d.73fc.316f WTP mac: 005d.73fc.3160 slot id: 1 
2021/01/09 13:17:30.619515 {wncd_x_R0-0}{1}: [client-orch-state] [22681]: (note): MAC: 7a46.532f.1d50  Client state transition: S_CO_IDLE -> S_CO_DELETE_IN_PROGRESS
2021/01/09 13:17:30.620202 {wncd_x_R0-0}{1}: [sisf-gleaner] [22681]: (note): Wireless unassociation received from CO for mac: 7a46.532f.1d50, Vlan: 1, Zone-id: 0x00000000
2021/01/09 13:17:30.620442 {wncd_x_R0-0}{1}: [dpath_svc] [22681]: (note): MAC: 7a46.532f.1d50  Client datapath entry deleted for ifid 0xa0000003
2021/01/09 13:17:30.624985 {wncd_x_R0-0}{1}: [epm-misc] [22681]: (ERR): [0000.0000.0000:unknown] auth mgr get vn called
2021/01/09 13:17:30.624990 {wncd_x_R0-0}{1}: [epm-misc] [22681]: (ERR): [0000.0000.0000:unknown] misc_plugin_get_vn: session_hdl invalid
2021/01/09 13:17:30.625209 {wncd_x_R0-0}{1}: [svm] [22681]: (ERR): SVM-ERR: SVM wlan apply cb: session ctx missing
2021/01/09 13:17:30.625574 {wncd_x_R0-0}{1}: [auth-mgr] [22681]: (ERR): [7a46.532f.1d50:capwap_9000000b] Failed to search/create timer main rec while timer stop
2021/01/09 13:17:30.625941 {wncd_x_R0-0}{1}: [client-auth] [22681]: (ERR): MAC: 7a46.532f.1d50  Failed to build flex client cache payload for FT-PSK.  Couldn't get client AKM.
2021/01/09 13:17:30.626255 {mobilityd_R0-0}{1}: [pmkcache] [23940]: (ERR): MAC: 7a46.532f.1d50  Pmkcache delete failed, failed to delete record
2021/01/09 13:17:30.626279 {mobilityd_R0-0}{1}: [mm-client] [23940]: (ERR): MAC: 7a46.532f.1d50  Unable to delete the pmk cache entry
2021/01/09 13:17:30.626458 {wncd_x_R0-0}{1}: [client-orch-state] [22681]: (note): MAC: 7a46.532f.1d50  Client state transition: S_CO_DELETE_IN_PROGRESS -> S_CO_DELETED
2021/01/09 13:17:32.037675 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:17:44.099515 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:17:50.954518 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:17:57.432450 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:18:51.477664 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:18:57.908915 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:19:04.083663 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:19:10.398110 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:19:16.496865 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:19:22.911627 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:19:29.046010 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:19:35.434377 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:19:41.941880 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:19:48.409182 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:20:02.865193 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:20:09.056933 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:20:21.800390 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:20:27.959038 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:20:34.397126 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:20:40.510762 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:20:46.907268 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:20:53.009574 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:20:59.413283 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:21:13.010136 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:21:19.405501 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:21:25.550859 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:21:31.911832 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:21:38.408697 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:21:44.486199 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:21:50.951909 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:21:57.413649 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:22:03.905254 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:22:10.040770 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:22:16.433622 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:22:22.832226 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:22:28.933392 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:22:35.432812 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:22:41.913943 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:22:48.078957 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:22:54.415487 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:23:09.000130 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:23:15.409524 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:23:22.324293 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:23:28.496041 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:23:34.946522 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:23:41.409425 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:23:47.517111 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:23:53.915339 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:24:00.404798 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:24:06.574749 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:24:21.169851 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:24:27.433042 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:24:33.906697 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:24:40.081154 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:24:46.429554 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:24:52.918278 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:24:59.100370 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:25:05.423422 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:25:11.896816 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:25:18.090198 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:25:24.400662 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:25:30.511117 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:25:36.908249 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:25:43.008178 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:25:49.402634 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:25:55.576610 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:26:01.901824 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:26:08.014189 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:26:14.415074 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:28:41.789770 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:28:47.953127 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:28:58.056534 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:30:02.745527 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:30:08.955036 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:30:15.440062 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:30:21.919163 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:30:30.712725 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:30:36.943405 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:30:43.122591 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null 
2021/01/09 13:30:49.437699 {wncd_x_R0-0}{1}: [dot11k] [22681]: (ERR): MAC: 7a46.532f.1d50  IEEE80211v: client record null

So it looks like there is an issue with key management, but I was hoping that moving to no session timeout would have resolved that for the clients. Here is my WLAN config:

WLAN Profile Name     : Network1
================================================
Identifier                                     : 1
Description                                    : 
Network Name (SSID)                            : TheNetworkName
Status                                         : Enabled
Broadcast SSID                                 : Enabled
Advertise-Apname                               : Disabled
Universal AP Admin                             : Disabled
Max Associated Clients per WLAN                : 0
Max Associated Clients per AP per WLAN         : 0
Max Associated Clients per AP Radio per WLAN   : 200
OKC                                            : Enabled
Number of Active Clients                       : 18
CHD per WLAN                                   : Enabled
WMM                                            : Allowed
WiFi Direct Policy                             : Disabled
Channel Scan Defer Priority:
  Priority (default)                           : 5
  Priority (default)                           : 6
Scan Defer Time (msecs)                        : 100
Media Stream Multicast-direct                  : Disabled
CCX - AironetIe Support                        : Disabled
Peer-to-Peer Blocking Action                   : Disabled
Radio Policy                                   : All
DTIM period for 802.11a radio                  : 
DTIM period for 802.11b radio                  : 
Local EAP Authentication                       : Disabled
Mac Filter Authorization list name             : Disabled
Mac Filter Override Authorization list name    : Disabled
Accounting list name                           : 
802.1x authentication list name                : Disabled
802.1x authorization list name                 : Disabled
Security
    802.11 Authentication                      : Open System
    Static WEP Keys                            : Disabled
    Wi-Fi Protected Access (WPA/WPA2/WPA3)     : Enabled
        WPA (SSN IE)                           : Disabled
        WPA2 (RSN IE)                          : Enabled
            MPSK                               : Disabled
            AES Cipher                         : Enabled
            CCMP256 Cipher                     : Disabled
            GCMP128 Cipher                     : Disabled
            GCMP256 Cipher                     : Disabled
            Randomized GTK                     : Disabled
        WPA3 (WPA3 IE)                         : Disabled
        Auth Key Management
            802.1x                             : Disabled
            PSK                                : Enabled
            CCKM                               : Disabled
            FT dot1x                           : Disabled
            FT PSK                             : Enabled
            Dot1x-SHA256                       : Disabled
            PSK-SHA256                         : Disabled
            SAE                                : Disabled
            OWE                                : Disabled
            SUITEB-1X                          : Disabled
            SUITEB192-1X                       : Disabled
    CCKM TSF Tolerance (msecs)                 : 1000
    OWE Transition Mode                        : Disabled
    OSEN                                       : Disabled
    FT Support                                 : Enabled
        FT Reassociation Timeout (secs)        : 20
        FT Over-The-DS mode                    : Disabled
    PMF Support                                : Optional
        PMF Association Comeback Timeout (secs): 1
        PMF SA Query Time (msecs)              : 200
    Web Based Authentication                   : Disabled
    Conditional Web Redirect                   : Disabled
    Splash-Page Web Redirect                   : Disabled
    Webauth On-mac-filter Failure              : Disabled
    Webauth Authentication List Name           : Disabled
    Webauth Authorization List Name            : Disabled
    Webauth Parameter Map                      : Disabled
Band Select                                    : Enabled
Load Balancing                                 : Disabled
Multicast Buffer                               : Disabled
Multicast Buffers (frames)                     : 0
IP Source Guard                                : Disabled
Assisted-Roaming
    Neighbor List                              : Enabled
    Prediction List                            : Disabled
    Dual Band Support                          : Disabled
IEEE 802.11v parameters
    Directed Multicast Service                 : Enabled
    BSS Max Idle                               : Enabled
        Protected Mode                         : Disabled
    Traffic Filtering Service                  : Disabled
    BSS Transition                             : Enabled
        Disassociation Imminent                : Disabled
            Optimised Roaming Timer (TBTTS)    : 40
            Timer (TBTTS)                      : 200
        Dual Neighbor List                     : Disabled
    WNM Sleep Mode                             : Disabled
802.11ac MU-MIMO                               : Enabled
802.11ax parameters
    OFDMA Downlink                             : Enabled
    OFDMA Uplink                               : Enabled
    MU-MIMO Downlink                           : Enabled
    MU-MIMO Uplink                             : Enabled
    BSS Target Wake Up Time                    : Enabled
    BSS Target Wake Up Time Broadcast Support  : Enabled
mDNS Gateway Status                            : Bridge
WIFI Alliance Agile Multiband                  : Disabled
Device Analytics
    Advertise Support                          : Enabled
    Share Data with Client                     : Disabled
Client Scan Report (11k Beacon Radio Measurement)
    Request on Association                     : Disabled
    Request on Roam                            : Disabled
WiFi to Cellular Steering                      : Disabled

Is there anything that sticks out as being misconfigured that could be causing the issue/errors?

Re: Catalyst 9800-CL client losing connectivity

Grendizer — Sun, 10 Jan 2021 03:39:16 GMT

You need to do two things:

1) Disable FT completely from that WLAN, FT will not save that much of time if the WLAN is PSK.

2) Do not use session timeout value of 0 , If maximum session timeout is desired, use 86400 instead of 0.

Re: Catalyst 9800-CL client losing connectivity

Jeff A. — Sun, 10 Jan 2021 04:30:36 GMT

I’ll give that a go and report back. Since it takes some days before the issue re-appears I may not be able to confirm for a little while.

Re: FT - I’m assuming that turning off FT but leaving on adaptive should be fine? I specifically wanted to deploy Cisco wireless for the Apple device compatibility benefits.

Re: timeout - Could you explain why the session timeout should not be 0? I read that it’s fine to be 0 for PSK but still learning the more advanced aspects of wireless.

Re: Catalyst 9800-CL client losing connectivity

Scott Fella — Sun, 10 Jan 2021 07:54:01 GMT

Don’t always drink the cool aid. The more you work on wireless the more you will see things break or not work because of features. I run the 9800-CL and also have session timeout set to zero but I don’t have 802.1kvr enabled. Now I don’t have issue with iPads at all, but iPhones seem to prompt for password a few times a day when using psk. Take a look at the Apple forums or just search and you will find threads regarding that. Now your issue where the iPad just stops working after days... I don’t think it’s a configuration issue but maybe more of a device issue. Do these iPads have a certain config or iOS that they are running or are these personal iPads and all different models? Maybe take one and factory reset it and only add the wireless profile and see if it stays connected for over a week.

Re: Catalyst 9800-CL client losing connectivity

Jeff A. — Sun, 10 Jan 2021 11:14:53 GMT

Thanks Scott. There's nothing special about the iPad configs. Most of them are in use by individuals, but the test device I have is an iPad Pro, factory restored, no iCloud data, latest OS. Exhibits the same behaviour as older and newer iPad devices, as explained above.

I've turned off FT for now but have left adaptive enabled. Will see if that makes a difference. If it does not, then I'll modify the session timeout as above and re-test.

My first priority is network stability, second is the ability to roam across the network with minimal packet loss (none would be ideal, but one can dream), and finally data speeds need to be over 100mbps on every compatible device, ideally 300mbps.

Are there any other pointers you might have based on the WLAN config that I have posted, Scott?

Re: Catalyst 9800-CL client losing connectivity

Scott Fella — Sun, 10 Jan 2021 11:53:01 GMT

Well when you say stability, it doesn’t mean that you still want features that may break something. I have a new gen iPad Pro with no issues on mine running various code from 16.x all the way up to 17.4. So what I would say is, disable FT, and try. Your wish of no pings during a roam is indeed a wish and thinking that FT will help, is also a wish. I say that because, people will complain more if their device doesn’t connect, or drops and they have to manually connect again, is slow, or the bars are low. Users don’t complain when roaming, only network folks:)
I don’t see how this would be an issue with your config since it takes 5+ days until the device hangs and this doesn’t happen to any other devices. Are these iPads personal and do they go home with the user or do they stay in the building?

Re: Catalyst 9800-CL client losing connectivity

Jeff A. — Sun, 10 Jan 2021 11:55:50 GMT

The troublesome devices stay in the building. The ones that go home and come back have no issues connecting, presumably because they are not in the building for longer than 12 hours at a time.

Re: Catalyst 9800-CL client losing connectivity

Scott Fella — Sun, 10 Jan 2021 12:24:01 GMT

That is what I figured. Maybe look at your dhcp and make sure that the lease are still valid. Just seems like it’s not a wireless issue to me.

Re: Catalyst 9800-CL client losing connectivity

Jeff A. — Sun, 10 Jan 2021 12:49:58 GMT

I've checked my dhcpd logs but there is nothing in there which references the client apart from successful DHCP renewals.

I could always try to plug an iPad in by USB-C Ethernet to the network for a week and see if the behaviour occurs there too.

If not wireless, could it be something on the network switch even? If so, any pointers to what I could check there?

Re: Catalyst 9800-CL client losing connectivity

Grendizer — Sun, 10 Jan 2021 17:51:08 GMT

For Zero session-timeout check CSCvs73917 https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvs73917

and for the FT, as i said before, you will not get that much of enabling it with PSK, so adaptive or full 11r will not save the client that much of time when roaming, yes 11r is very helpful when doing .1x but not with PSK (saving hundred milliseconds with .1x versus a few milliseconds with PSK). The rule of thumb is to disable any feature you don't use it or don't need it to limit the exposure to bugs. hope that helpful...

Re: Catalyst 9800-CL client losing connectivity

Scott Fella — Sun, 10 Jan 2021 18:44:01 GMT

Just try to eliminate variables. Maybe it’s ipv6, does this happen on Ethernet, does this happen if you bring the iPad home or switch to a different network from time to time. I have an iPad Pro I use and never had any issue with it. I run a setup at home with 6 access point in FlexConnect. The only issue I have is with iOS 14, the devices prompt to enter the password a few time a day. So try to isolate the issue by seeing what works and doesn’t work with these iPads you have. Maybe it’s best to have them restart every night. Have you posted in the Apple community forums?

Re: Catalyst 9800-CL client losing connectivity

Scott Fella — Sun, 10 Jan 2021 18:51:01 GMT

Well hopefully they fix that in the next version. Setting that to zero was like best practice in AireOS and folks moving to the 9800 would keep that setting.

Re: Catalyst 9800-CL client losing connectivity

Jeff A. — Sun, 10 Jan 2021 21:57:58 GMT

Thanks for the bug details - I'll keep an eye out for updates that fix this in 17.3. I can't move to 17.4 as 3702i AP's are no longer supported there.

In the meantime, I've set the timeout to 86400 and will see how it performs.

Re: Catalyst 9800-CL client losing connectivity

Jeff A. — Sun, 10 Jan 2021 21:59:56 GMT

It's funny you say that - I had issues with IPv6 RA from my ISP which is what I thought was causing these issues. Since working with the ISP and fixing that mess up in my router, I'm only having this last issue with the iPads, otherwise everything is smooth sailing. (except for one Sonos speaker having issues with AirPlay, but Sonos are renowned for their instability on enterprise networks so I'm not bothering to troubleshoot that further)

Re: Catalyst 9800-CL client losing connectivity

Scott Fella — Mon, 11 Jan 2021 03:28:50 GMT

Set session timer to something other than 0 like grendizer mentioned. This fixed my issue but that is different from what you are seeing but will probably prevent other issues.

Re: Catalyst 9800-CL client losing connectivity

Jeff A. — Sat, 16 Jan 2021 12:54:44 GMT

Since disabling FT completely, including adaptive mode, and ensuring the session timeout was not 0 but instead set to 86400, the issues have not re-occurred. Thank you to all who helped and provided advice.

I am curious to understand more as to why these two items caused the issues experienced. Does anyone have any resources to help me dig deeper into those aspects?

Re: Catalyst 9800-CL client losing connectivity

Scott Fella — Sat, 16 Jan 2021 17:27:11 GMT

Well the best thing is to try disabling one at a time. The session timer is a bug which was mentioned in the thread. So maybe if you want FT, enable that feature and see. Bugs will be issues with the wireless, but you have to also understand that device manufacturers might implement other features differently and possibly not work or work well.
What I have noticed since I set session timer to 86400 is that I’m no longer seeing my Apple devices asking for my password. However, my iPhone at times seem to not have network or is slow and I still have to toggle the wireless off and on.

Re: Catalyst 9800-CL client losing connectivity

Jeff A. — Sat, 16 Jan 2021 23:28:35 GMT

Thanks Scott. Funny I just noticed that a HomePod dropped offline in that manner. The end user devices have been solid so that's the priority, but I'll probably downgrade the controller version, maybe even go back to the starred version, to see if that helps with that overall stability.