9800-CL, delay beetwen RUN state and passing traffic

[LenarFA]

Hi,

looking info for issue. Running wlc 9800-Cl, software release 17.9.5, flexconnect deployment. Sometimes client going to RUN state, but traffic is not passing. After 3-4 min. client reassociate and traffic is passing.
Via log could see that:

1) client going to RUN state at 18:15:51.500029705 (client recive ip from dhcp, traffic not passing)

2) after that at 18:18:51.397990309 wlc delete client (Reason: CO_CLIENT_DELETE_REASON_MN_DHCP_TIMEOUT)

3) client reassociate and traffic is passing

[LenarFA]

wlc log attached

[marce1000]

  - Use commands from https://www.cisco.com/c/en/us/support/docs/wireless/catalyst-9800-series-wireless-controllers/217738-monitor-catalyst-9800-kpis-key-performa.html#anc5
    to get detailed insights

    - Have a checkup of the 9800-CL configuration with the CLI command show tech wireless and feed the output from that into
      Wireless Config Analyzer  use the full command above it does now work with a simple show tech

    M.

[marce1000]

   - Added (2) : I have let your wic-log.txt parsed by Wireless Debug Analyzer
    (check all flag was checked)
    (check below) - keep using Debug Analyzer when configuration changes are made and or check for improvement

    + Consider 17.12.3  because of being the latest advisory release      

Time	Task	Translated
Connection attempt #1
Connection attempt #2
2024/08/05 18:15:51.169	client-orch-sm	Client made a new Association to an AP/BSSID: BSSID ec01.d526.790c, WLAN PIK.Tesla, Slot 1 AP ec01.d526.7900, APA00F.3718.4F40, Site tag HeadOffice-SiteTag, Policy tag PIK.Parking-PolicyTag, Policy profile FlexConnect-2102-PolicyProfile, Switching Local, Socket delay 0ms
2024/08/05 18:15:51.170	dot11	Association success for client, assigned AID is: 1
2024/08/05 18:15:51.401	client-keymgmt	Negotiated the following encryption mechanism: AKM:PSK Cipher:CCMP WPA Version: WPA2
2024/08/05 18:15:51.401	client-auth	Client successfully completed Pre-shared Key authentication. Assigned VLAN: 2102
2024/08/05 18:15:51.401	client-orch-sm	Policy profile is configured for local switching
2024/08/05 18:15:51.401	client-orch-state	Starting Mobility Anchor discovery for client
2024/08/05 18:15:51.403	client-orch-state	Entering IP learn state
2024/08/05 18:15:51.499	client-iplearn	Client got IP: 10.99.67.178, discovered through: DHCP
2024/08/05 18:15:51.500	client-orch-state	Client reached RUN state, connection completed.
2024/08/05 18:18:51.397	client-orch-sm	Controller initiated client deletion with code: CO_CLIENT_DELETE_REASON_MN_DHCP_TIMEOUT. Explanation: DHCP required is enabled, and client never completed DHCP negotiation. Actions: May happen during normal scenarios, if client roams out of coverage during onboarding, or goes to sleep or is turned off, during onboarding. If seen on large counts per VLAN, do client debugging and check DHCP Server pool and status
Connection attempt #3
2024/08/05 18:19:03.098	client-orch-sm	Client roamed to a new AP/BSSID: BSSID ec01.d526.790c, WLAN PIK.Tesla, Slot 1 AP ec01.d526.7900, APA00F.3718.4F40, Site tag HeadOffice-SiteTag, Policy tag PIK.Parking-PolicyTag, Policy profile FlexConnect-2102-PolicyProfile, Switching Local, Socket delay 0ms
2024/08/05 18:19:03.098	dot11	Association success for client, assigned AID is: 2
2024/08/05 18:19:03.329	client-keymgmt	Negotiated the following encryption mechanism: AKM:PSK Cipher:CCMP WPA Version: WPA2
2024/08/05 18:19:03.329	client-auth	Client successfully completed Pre-shared Key authentication. Assigned VLAN: 2102
2024/08/05 18:19:03.329	client-orch-sm	Policy profile is configured for local switching
2024/08/05 18:19:03.329	client-orch-state	Starting Mobility Anchor discovery for client
2024/08/05 18:19:03.332	client-orch-state	Entering IP learn state
2024/08/05 18:19:03.757	client-iplearn	Client got IP: 10.99.67.178, discovered through: IP Snooping
2024/08/05 18:19:03.758	client-orch-state	Client reached RUN state, connection completed.

[LenarFA]

Hi, thanks for reply.

"Controller initiated client deletion with code: CO_CLIENT_DELETE_REASON_MN_DHCP_TIMEOUT. Explanation: DHCP required is enabled, and client never completed DHCP negotiation"

At first time client got IP from DHCP, so that is, the problem of "DHCP negotiation" is the point of view from WLC?

"+ Consider 17.12.3  because of being the latest advisory release"

Considering that a new release 17.12.4 has already been released and published list of resolved bugs in 17.12.4 isn't it better to upgrade directly on it?

[marce1000]

   >....At first time client got IP from DHCP, so that is, the problem of "DHCP negotiation" is the point of view from WLC?
   + Take a go with the mentioned procedure concerning Wireless Config Analyzer  first and check if anything comes up
      related to that.

   + 17.12.4 is as good as the 'next advisory' (indeed)

   M.

[marce1000]

   - Added (3) : make sure that the DHCP server is reachable on the intended WLAN/VLAN pair either for local (flexconnect)
     switching or central switching (CAPWAP through the controller)

  M.

[MHM Cisco World]

What is l3 auth you use?

I think there is two status' one before CoA and other after CoA

MHM

[LenarFA]

Hi,

Four SSID, all FlexConnect with unique VLAN termination. All - 802.11r enabled. 2 of them - 802.1X & FT+802.1X, 2 of them PSK & FT+PSK. It is tested deployment before migrating to prod from AireOS. So i quickly periodically connect to all four SSID and test connection stability. Sometimes client associate to SSID, got IP, going to RUN state, but traffic not passed. After 3 min. client re-associate due WLC deletion and traffic traffic starts to be transmitted. Just updated to 17.12.4 same symptoms.

For PSK SSID WLC delete client after 3 min. from client going to RUN state with this error -  CO_CLIENT_DELETE_REASON_MN_DHCP_TIMEOUT

For 802.1X SSID WLC delete client after 3 min. from client going to RUN state with this error - CO_CLIENT_DELETE_REASON_CLIENT_EAP_TIMEOUT_FAILURE

[marce1000]

@LenarFA >............................CO_CLIENT_DELETE_REASON_CLIENT_EAP_TIMEOUT_FAILURE
                  For this one I found https://bst.cloudapps.cisco.com/bugsearch/bug/CSCwh36960
                  check if applicable to your environment , the bug report has no Known Fixed Releases
                  mentioned , you way want to report to TAC , (if desired)

 M.

[MHM Cisco World]

Hi 

Thetr are three options in policy profile

1- central switching 

2- central authentication 

3- central dhcp 

Can you conform the settings for each SSID

MHM

[LenarFA]

All SSID are FlexConnect with unique VLAN termination, so in wireless policy profile - no central switching, no central dhcp.

For example:

for 802.1X wlan:
wireless profile policy PolicyProfile-1
aaa-override
accounting-list ISE-1
no central dhcp
no central switching
dhcp-tlv-caching
http-tlv-caching
ipv4 arp-proxy
ipv4 dhcp required
radius-profiling
session-timeout 86400
vlan Null
no shutdown

for PSK:
wireless profile policy PolicyProfile-2
no central dhcp
no central switching
dhcp-tlv-caching
http-tlv-caching
ipv4 arp-proxy
ipv4 dhcp required
session-timeout 86400
vlan XXX
no shutdown

[Rich R]

Also note https://bst.cloudapps.cisco.com/bugsearch/bug/CSCwj45141 which is a regression in 17.9.4 APSP8 and 17.9.5 as per TAC recommended doc (link below) which you should always consult.

Again the recommendation - upgrade to 17.12.3 or 17.12.4. 
17.12.4 will only be promoted to recommended version after TAC have observed it being deployed and running without any major problems for at least 4 weeks but nothing to stop you trying it if you have tested it.  I'm considering 17.12.4 myself (already using in lab) due to a number of bugs which still affect 17.12.3 which are a concern.

[LenarFA]

Hello,

just to update the topic. Removing the "DHCP Required" setting from all policy profiles fixed the issue. Everything connects and reconnects stably.

[MHM Cisco World]

Please last Q' are you use central authc for these SSID?

MHM