Hello Community.
Users are suddenly disconnected from wireless. They stop reaching network services such as Internet and other network services. Windows wireless notification shows laptop is still connected to the SSID but "No Internet, secured". At the same time from ipconfig the Wi-Fi Adapter received and auto-assigend IP 169.254.X.X and a wrong default gateway (which actually exists but on a different WLAN).
From the radioactive traces I found these logs
2022/05/23 15:42:40.116 client-orch-sm Client made a new Association to an AP/BSSID: BSSID 2cf8.2cf8.2cf8, old BSSID 0000.0000.0000, WLAN smtcwireless, Slot 1 AP 2cf8.2cf8.2cf0, AP001
Association received. BSSID 2cf8.2cf8.2cf8, old BSSID 0000.0000.0000, WLAN smtcwireless, Slot 1 AP 2cf8.2cf8.2cf0, AP001
2022/05/23 15:42:40.116 client-orch-state __unknown__ Client state transition: S_CO_INIT -> S_CO_ASSOCIATING
2022/05/23 15:42:40.116 dot11 Association success for client, assigned AID is: 4 Association success. AID 4, Roaming = False, WGB = False, 11r = False, 11w = True
2022/05/23 15:42:40.117 client-orch-state __unknown__ Client state transition: S_CO_ASSOCIATING -> S_CO_L2_AUTH_IN_PROGRESS
2022/05/23 15:42:40.117 sanet-shim-translate __unknown__ 3868.5555.4444 :Requested session policy with MAC not present with EPM
2022/05/23 15:42:40.117 client-auth __unknown__ ADD MOBILE sent. Client state flags: 0x71 BSSID: MAC: 2cf8.2cf8.2cf8 capwap IFID: 0x9000016b
2022/05/23 15:42:40.121 client-auth __unknown__ L2 Authentication initiated. method DOT1X, Policy VLAN 0,AAA override = 0 , NAC = 0
2022/05/23 15:42:40.122 ewlc-infra-evq __unknown__ Authentication Success. Resolved Policy bitmap:11 for client 3868.5555.4444
2022/05/23 15:42:40.348 client-auth Starting EAPOL 4-Way Handshake L2 Authentication Key Exchange Start. Resolved VLAN: 80, Audit Session id: 79040C0A00000AE7F18C14D0
2022/05/23 15:42:40.416 client-keymgmt Negotiated the following encryption mechanism: AKM:DOT1X Cipher:CCMP WPA2
EAP Key management successful. AKM:DOT1X Cipher:CCMP WPA2
2022/05/23 15:42:40.416 client-orch-sm __unknown__ Mobility discovery triggered. Client mode: Local
2022/05/23 15:42:40.416 client-orch-state Starting Mobility Anchor discovery for client Client state transition: S_CO_L2_AUTH_IN_PROGRESS -> S_CO_MOBILITY_DISCOVERY_IN_PROGRESS
2022/05/23 15:42:40.417 mm-client __unknown__ Mobility Successful. Roam Type None, Sub Roam Type MM_SUB_ROAM_TYPE_NONE, Previous BSSID MAC: 0000.0000.0000 Client IFID: 0xa000001e, Client Role: Local PoA: 0x9000016b PoP: 0x0
2022/05/23 15:42:40.417 client-auth __unknown__ ADD MOBILE sent. Client state flags: 0x72 BSSID: MAC: 2cf8.2cf8.2cf8 capwap IFID: 0x9000016b
2022/05/23 15:42:40.417 client-orch-state __unknown__ Client state transition: S_CO_MOBILITY_DISCOVERY_IN_PROGRESS -> S_CO_DPATH_PLUMB_IN_PROGRESS
2022/05/23 15:42:40.417 dot11 __unknown__ Client datapath entry params - ssid:smtcwireless,slot_id:1 bssid ifid: 0x90000171, radio_ifid: 0x90000169, wlan_ifid: 0xf0400050
2022/05/23 15:42:40.418 dpath_svc __unknown__ Client datapath entry created for ifid 0xa000001e
2022/05/23 15:42:40.418 client-orch-state Entering IP learn state Client state transition: S_CO_DPATH_PLUMB_IN_PROGRESS -> S_CO_IP_LEARN_IN_PROGRESS
2022/05/23 15:42:43.315 client-iplearn Client got IP: 10.10.80.9, discovered through: DHCP Client IP learn successful. Method: DHCP IP: 10.10.80.9
2022/05/23 15:42:43.316 client-orch-state Client reached RUN state, connection completed. Client state transition: S_CO_IP_LEARN_IN_PROGRESS -> S_CO_RUN
2022/05/23 16:12:43.588 client-keymgmt Negotiated the following encryption mechanism: AKM:DOT1X Cipher:CCMP WPA2
EAP Key management successful. AKM:DOT1X Cipher:CCMP WPA2
2022/05/23 16:12:43.588 client-auth __unknown__ ADD MOBILE sent. Client state flags: 0x72 BSSID: MAC: 2cf8.2cf8.2cf8 capwap IFID: 0x9000016b
2022/05/23 16:12:43.588 client-orch-state __unknown__ Client state transition: S_CO_RUN -> S_CO_RUN
2022/05/23 16:42:44.679 client-keymgmt Negotiated the following encryption mechanism: AKM:DOT1X Cipher:CCMP WPA2
EAP Key management successful. AKM:DOT1X Cipher:CCMP WPA2
2022/05/23 16:42:44.679 client-auth __unknown__ ADD MOBILE sent. Client state flags: 0x72 BSSID: MAC: 2cf8.2cf8.2cf8 capwap IFID: 0x9000016b
2022/05/23 16:42:44.679 client-orch-state __unknown__ Client state transition: S_CO_RUN -> S_CO_RUN
2022/05/23 17:12:45.799 client-keymgmt Negotiated the following encryption mechanism: AKM:DOT1X Cipher:CCMP WPA2 EAP Key management successful. AKM:DOT1X Cipher:CCMP WPA2
2022/05/23 17:12:45.799 client-auth __unknown__ ADD MOBILE sent. Client state flags: 0x72 BSSID: MAC: 2cf8.2cf8.2cf8 capwap IFID: 0x9000016b
2022/05/23 17:12:45.799 client-orch-state __unknown__ Client state transition: S_CO_RUN -> S_CO_RUN
2022/05/23 17:35:05.870 avc-stats __unknown__ Received stats record for app 'myapp'(app-id: 0xd0004bb), client MAC: 3868.5555.4444 , SSID 'corpwireless', direction egress (1), WLAN ID <not provided>, #bytes 109, #packets 1
2022/05/23 17:35:06.135 auth-mgr __unknown__ [0000.0000.0000:unknown] Session info 0x55fffca49ac8 hdl 0x49000b18 client hdl 0 cur hdl 0xd7000b18 withclient name BM
2022/05/23 17:35:06.135 sisf-packet __unknown__ RX: ARP from interface capwap_9000016b on vlan 80 Source MAC: 3868.5555.4444 Dest MAC: ffff.ffff.ffff ARP REQUEST, ARP sender MAC: 3868.5555.4444 ARP target MAC: 0000.0000.0000 ARP sender IP: 10.10.80.9, ARP target IP: 192.168.1.1,
2022/05/23 17:35:06.136 sisf-packet __unknown__ TX: ARP from interface capwap_9000016b on vlan 80 Source MAC: 3868.5555.4444 Dest MAC: ffff.ffff.ffff ARP REQUEST, ARP sender MAC: 3868.5555.4444 ARP target MAC: 0000.0000.0000 ARP sender IP: 10.10.80.9, ARP target IP: 192.168.1.1,
2022/05/23 17:35:06.953 auth-mgr __unknown__ [0000.0000.0000:unknown] Session info 0x55fffca49ac8 hdl 0x49000b18 client hdl 0 cur hdl 0xd7000b18 withclient name BM
2022/05/23 17:35:06.953 sisf-packet __unknown__ RX: ARP from interface capwap_9000016b on vlan 80 Source MAC: 3868.5555.4444 Dest MAC: ffff.ffff.ffff ARP REQUEST, ARP sender MAC: 3868.5555.4444 ARP target MAC: 0000.0000.0000 ARP sender IP: 10.10.80.9, ARP target IP: 192.168.1.1,
2022/05/23 17:35:06.953 sisf-packet __unknown__ TX: ARP from interface capwap_9000016b on vlan 80 Source MAC: 3868.5555.4444 Dest MAC: ffff.ffff.ffff ARP REQUEST, ARP sender MAC: 3868.5555.4444 ARP target MAC: 0000.0000.0000 ARP sender IP: 10.10.80.9, ARP target IP: 192.168.1.1,
2022/05/23 17:35:07.871 avc-stats __unknown__ Received stats record for app 'app2'(app-id: 0xd0004b8), client MAC: 3868.5555.4444 , SSID 'corpwireless', direction egress (1), WLAN ID <not provided>, #bytes 87, #packets 1
2022/05/23 17:35:07.959 auth-mgr __unknown__ [0000.0000.0000:unknown] Session info 0x55fffca49ac8 hdl 0x49000b18 client hdl 0 cur hdl 0xd7000b18 withclient name BM
2022/05/23 17:35:07.959 sisf-packet __unknown__ RX: ARP from interface capwap_9000016b on vlan 80 Source MAC: 3868.5555.4444 Dest MAC: ffff.ffff.ffff ARP REQUEST, ARP sender MAC: 3868.5555.4444 ARP target MAC: 0000.0000.0000 ARP sender IP: 10.10.80.9, ARP target IP: 192.168.1.1,
2022/05/23 17:35:07.960 sisf-packet __unknown__ TX: ARP from interface capwap_9000016b on vlan 80 Source MAC: 3868.5555.4444 Dest MAC: ffff.ffff.ffff ARP REQUEST, ARP sender MAC: 3868.5555.4444 ARP target MAC: 0000.0000.0000 ARP sender IP: 10.10.80.9, ARP target IP: 192.168.1.1,
*****From here tons of avc-status logs and nothing else
And I haven't seen this before until today there are a lot, lot of avc-status messages: for example: Received stats record for app 'myapp'(app-id: 0xd0004bb), client MAC: 3868.5555.4444 , SSID 'corpwireless', direction egress (1), WLAN ID <not provided>, #bytes 109, #packets 1
Not sure if this is informative only or if had to do.
I restarted the appliance and the APs last Friday night, but still seeing the problem.
From what I undertsand is tht the state S_CO_RUN means that the clent is fully connected, then some minutes later the ARP logs came in.
