Cisco access points C9115AXI are getting disconnected ramdomly

[gustavo.garcia@iosoffices.com]

Hello Community, I do have a Cisco C9115AXI-A wireless network, one of them is getting the controller functionality and another 6 APs are providing devices connectivity service. all of them are getting disconnected randomly.

I'm using software release IOS XE Software, Version 17.03.04c

Does someone have the same issue?, does someone have a solution for this?

Thanks in advance.

[Leo Laohoo]

---

gustavo.garcia@iosoffices.com wrote:

all of them are getting disconnected randomly.

---

Start with the basics:  Examine the uptime of each APs and determine if the APs crashed or rebooted.  

[gustavo.garcia@iosoffices.com]

Thank you Leo, actually, a continous ping never stops, it is only the contact with controller supervision which falls:

%CAPWAPAC_SMGR_TRACE_MESSAGE-3-WLC_GEN_ERR: Chassis 2 R0/0: wncd: Error in Session-IP: 10.137.8.19[5259] Mac: e44e.2d40.c5c0 Heartbeat timer expiry for AP. Close CAPWAP DTLS session

%CAPWAPAC_SMGR_TRACE_MESSAGE-5-AP_JOIN_DISJOIN: Chassis 2 R0/0: wncd: AP Event: AP Name: AP3-S125, MAC: a49b.cd29.d8b8 Disjoined

%PEER_SELECTION-5-EWC_PEER_SELECTION_EVENT: Chassis 2 R0/0: wncd: HA Event: Non-Candidate AP 'AP3-S125' is no longer a peer

%CAPWAPAC_SMGR_TRACE_MESSAGE-5-AP_JOIN_DISJOIN: Chassis 2 R0/0: wncd: AP Event: Session-IP:10.137.8.19[5259] CAPWAP DTLS session closed for AP, cause: DTLS handshake error

%CAPWAP_IMGDWNLD_TRACE_MESSAGE-5-CAPWAPIMGDWNLD_EWC_AP_AP_LIST_EVENTS: Chassis 2 R0/0: wncd: List Event: AP with wtp_mac e44e.2d40.c5c0 for Image Type ap1g7 is added to Master AP list

[Leo Laohoo]

Is the 9800 in an HA?

[ammahend]

This version has a known issue with 9115, there is a good chance you might be hitting this bug.

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvy51818

[gustavo.garcia@iosoffices.com]

Thank you @ammahend, I'll try updating software

[Rich R]

@ammahend CSCvy51818 is terminated and doesn't even mention IOS-XE.  Maybe you're thinking of some other bug?

Like @Leo Laohoo says you need to go back to basics and see whether the APs are crashing or just losing connectivity then work from there.

Crashes = software bug -> update IOS-XE.

Connectivity problems -> troubleshoot your network connectivity.

[gustavo.garcia@iosoffices.com]

Thank you @Rich R it is not a connectivity issue, because a continous ping never falls, the AP is just loosing communication to cntroller and disconnects all the devices connected at that point to it.

[patoberli]

I also suggest upgrading (please note, you also need to install an SMU for 17.3.5 on the controller).

Regarding the error message "Heartbeat timer expiry for AP." This typically means that the AP lost the connection to the controller. This could happen if the latency is to high (more than 300 ms) for example. This can be temporary caused if there is an overloaded link between the AP and the controller.

[ammahend]

look at the open caveat in IOS XE Amsterdam 17.3.4c

sorry should have pasted this link before, I am not recommending by any means to jump to conclusion and we should follow procedure, but based on code he is running there is a good change he might be hitting the bug.

https://www.cisco.com/c/en/us/td/docs/wireless/controller/ewc/17-3/rel-notes/ewc-rn-17-3-x.html#Cisco_Reference.dita_dc77575e-1ef4-4687-a033-3c380dd7b8d3

[Rich R]

So you have to understand how to read those @ammahend - those were open (still under investigation) at the time that version was released and like that one can be closed subsequently.

More useful to look at the resolved caveats and open caveats for the latest release - 17.3.5a in this case.

For example https://bst.cloudapps.cisco.com/bugsearch/bug/CSCwa12278 fixed.

Looking in bug tool there are actually numerous AP fixes (also relevant to 9115) in 17.3.5 so I would certainly consider using that anyway.

[gustavo.garcia@iosoffices.com]

Thank you very much @patoberli, @ammahend and @Rich R. I'll try again to upgrade to 17.3.5, I tried but still back to 03.04c, don't know why, I didn't find the reasson at logs.

Let you know.

Cheers!

[stuart.pannell]

Hi Gustavo, did you get a fix for this issue? was it AP/Controller related? I am having the same issues at a larger scale and have been banging my head trying to fix it. We have swapped out cabling, switches, AP's no fails on ping tests but random drops during the day. I think its down to congestion on either the 9200L switches or the AP's themselves. 

[Rich R · Accepted solution]

@stuart.pannell 17.3 code is almost end of life now and things have moved on a lot since then.

Current recommended code (see TAC recommended link below) is 17.9.4 + SMUs + APSPs.
Also check your config with the config analyzer - link below.
If you still see the issue then open a TAC case so they can capture all the relevant data and escalate with dev team.

[stuart.pannell]

I have checked the config via the analyser and also we are running code 17.9.3 with the latest APSP for that version. I have a taC case open so hopefully will get to the bottom of it soon.