CAPWAP tunnel interface flapping

[adamcisco]

Hi there,

We have a single AP out of about two dozen that is often disjoining from our WLC. We have noticed that it disjoins every 15 minutes or so and then re-joins again.

We have tested both patch leads on either side and of course the whole line. Physically the connection is fine as the WLC states that the uptime is over 100 days but the join date & time changes as one would expect due to the flapping.

The port is configured the same as the other ports on the same switch connecting to APs of the same model. APs are C9120AXE-Z on ver 17.3.1.9 and the WLC is a 9800 on ver 17.3.1.

The below are logs sent to our Syslog server:

2021-11-11 16:24:56 Local7.Debug *WLC Name* 102952: Nov 11 16:24:56: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] is cts enabled:FALSE
2021-11-11 16:24:56 Local7.Notice *WLC Name* 102953: 8w5d: %LINEPROTO-5-UPDOWN: Line protocol on Interface Capwap22, changed state to down
2021-11-11 16:24:57 Local7.Debug *WLC Name* 102954: Nov 11 16:24:56: AUTH-FEAT-SWITCH-PM-EVENT: [Ca22] Link down
2021-11-11 16:24:57 Local7.Debug *WLC Name* 102955: Nov 11 16:24:56: AUTH-FEAT-SWITCH-PM-EVENT: [Ca22] Port state change
2021-11-11 16:24:57 Local7.Debug *WLC Name* 102956: Nov 11 16:24:56: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] is cts enabled:FALSE
2021-11-11 16:24:57 Local7.Debug *WLC Name* 102957: Nov 11 16:24:56: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] is mka uplink port:FALSE
2021-11-11 16:24:57 Local7.Debug *WLC Name* 102958: Nov 11 16:24:56: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] is cts enabled:FALSE
2021-11-11 16:24:57 Local7.Debug *WLC Name* 102959: Nov 11 16:24:56: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] is mka uplink port:FALSE
2021-11-11 16:24:57 Local7.Debug *WLC Name* 102960: Nov 11 16:24:56: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] Disabling dot1x in switch shim
2021-11-11 16:24:57 Local7.Debug *WLC Name* 102961: Nov 11 16:24:56: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] Disable on port: sb get failed
2021-11-11 16:24:58 Local7.Notice *WLC Name* 102962: 8w5d: %CAPWAPAC_SMGR_TRACE_MESSAGE-5-AP_JOIN_DISJOIN: Chassis 1 R0/0: wncd: AP Event: Session-IP: 192.168.0.171[5256] Mac: *MAC ADDR* CAPWAP DTLS session closed for AP, cause: DTLS server session shutdown
2021-11-11 16:24:58 Local7.Notice *WLC Name* 102963: 8w5d: %CAPWAPAC_SMGR_TRACE_MESSAGE-5-AP_JOIN_DISJOIN: Chassis 2 R0/0: wncd: AP Event: AP Name: *AP NAME*, MAC: *MAC ADDR* Disjoined
2021-11-11 16:25:22 Local7.Debug *WLC Name* 102964: Nov 11 16:25:22: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] is cts enabled:FALSE
2021-11-11 16:25:22 Local7.Debug *WLC Name* 102965: Nov 11 16:25:22: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] is mka uplink port:FALSE
2021-11-11 16:25:22 Local7.Notice *WLC Name* 102966: 8w5d: %LINEPROTO-5-UPDOWN: Line protocol on Interface Capwap22, changed state to up
2021-11-11 16:25:22 Local7.Debug *WLC Name* 102967: Nov 11 16:25:22: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] is cts enabled:FALSE
2021-11-11 16:25:22 Local7.Debug *WLC Name* 102968: Nov 11 16:25:22: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] is mka uplink port:FALSE
2021-11-11 16:25:22 Local7.Debug *WLC Name* 102969: Nov 11 16:25:22: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] is cts enabled:FALSE
2021-11-11 16:25:22 Local7.Debug *WLC Name* 102970: Nov 11 16:25:22: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] is mka uplink port:FALSE
2021-11-11 16:25:22 Local7.Debug *WLC Name* 102971: Nov 11 16:25:22: AUTH-FEAT-SWITCH-PM-EVENT: [Ca22] Link coming up
2021-11-11 16:25:22 Local7.Debug *WLC Name* 102972: Nov 11 16:25:22: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] is cts enabled:FALSE
2021-11-11 16:25:22 Local7.Debug *WLC Name* 102973: Nov 11 16:25:22: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] is mka uplink port:FALSE
2021-11-11 16:25:22 Local7.Debug *WLC Name* 102974: Nov 11 16:25:22: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] is cts enabled:FALSE
2021-11-11 16:25:22 Local7.Debug *WLC Name* 102975: Nov 11 16:25:22: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] is mka uplink port:FALSE
2021-11-11 16:25:22 Local7.Debug *WLC Name* 102976: Nov 11 16:25:22: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] is cts enabled:FALSE
2021-11-11 16:25:22 Local7.Debug *WLC Name* 102977: Nov 11 16:25:22: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] is cts enabled:FALSE
2021-11-11 16:25:22 Local7.Debug *WLC Name* 102978: Nov 11 16:25:22: AUTH-FEAT-SWITCH-PM-EVENT: [Ca22] link up deferred handler
2021-11-11 16:25:22 Local7.Debug *WLC Name* 102979: Nov 11 16:25:22: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] is cts enabled:FALSE
2021-11-11 16:25:22 Local7.Debug *WLC Name* 102980: Nov 11 16:25:22: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] is mka uplink port:FALSE
2021-11-11 16:25:22 Local7.Debug *WLC Name* 102981: Nov 11 16:25:22: AUTH-FEAT-SWITCH-CORE-EVENT: [Ca22] Null sb; port-control-auto not enabled!
2021-11-11 16:25:23 Local7.Notice *WLC Name* 102982: 8w5d: %CAPWAPAC_SMGR_TRACE_MESSAGE-5-AP_JOIN_DISJOIN: Chassis 1 R0/0: wncd: AP Event: AP Name: *AP NAME*, MAC: *MAC ADDR* Joined

Any idea at what might be causing this?

Thanks,

Adam

[Leo Laohoo]

Reboot the AP. 

If this issue continues, upgrade the firmware of the controller to something more recent, like 17.3.3 or 17.3.4

[adamcisco]

Rebooted / Factory Reset the AP and re-joined to the WLC but still continues to disassociate.

We'll try upgrading the firmware of the controller.

Thanks for your response.

[marce1000]

 - Have a look at  the interface counters , on the interface the particular ap is connected to. And or configure 'logging event link-status' , then check the switch logs too when this happens.

 M.

[adamcisco]

Interface counters look similar to other physical interfaces with no failures or errors. 

I had set the logging event link-status on the physical interface but hadn't returned anything further of use. Also tried a different port on the same 9300 switch but issue persists.

Thanks for your response.

[MHM Cisco World]

WLC heartbeat timeout
but why you mention that there is flapping.

[adamcisco]

Heartbeat Timeout is set to 30 secs (which is set in the AP Join Profile, therefore it is the same across all joined APs).

Is flapping not the correct terminology for an interface that is continuously going up and down?

Thanks for your response.