AP Reboots - Max Retransmission to AP

[CiscoAnonymous123]

Hi, I have an ongoing TAC on this, but no luck so far. 

Any tips on what to do?

Technical information

• WLC 9800 version 17.9.3
• 4 problematic AP's out of 57 deployed on the WLC
  • Around 10 AP's on this current site, the other 6 is working fine.
• 2 is 1852 and 2 is 9115
• The problem is at the same site, but on two different Aruba Switches
• Network is Gigabit and on same L2/Vlan, HPE Aruba 2530 switches

Description of the problem

I've got a call from the school experiencing random wi-fi problems throughout the day. I then checked the WLC and could see that multiple AP's at the school only had uptime of 5 minutes.   I then checked the switches and so on, and everything seems to be working just fine.  The following days I monitored the WLC more closely and can see the same 4 AP's are rebooting 4-6 times through the day. Now I contacted Cisco. 

Cisco's changes

They changed the Retransmit timers, but still same problem.

Cisco's conclusion

The conclusion (so far) from Cisco is that this is a network issue and we need to check switches, ports and cabling. Okay, we will do that, but do that really make sense? We just migrated from the old WLC 2504 to this 9800 and the 1852 AP's was working fine on the old WLC, nothing else has changed. (same AP's, cables, switches and so on)

My tests, findings and thougts (so far)

• As far as I know, this problem started after deployment of the new WLC
• Other AP's on the same location (and same switches) is working fine
• I checked our monitoring and bandwidth usage on AP ports and Uplink ports is low
• I have a continuous ping running to the WLC, switches and AP, but the only ping-drop is when the AP is not Joined. 
• The strange part - the reboot of the AP's happened on the excactly the same time on different days. 
  
  • See a picture from the switches when the AP's is going off-line and On-line
  • No other ports on the switches is going off-line and on-line, only the problematic AP'

[Leo Laohoo]

---

@CiscoAnonymous123 wrote:
The conclusion (so far) from Cisco is that this is a network issue and we need to check switches, ports and cabling.

---

What region is the TAC Case assigned to, is it in TAC "Asia"?

Check the bootflash of the 9800 if there are any crash reports.  The filenames should end with ".crash".

[CiscoAnonymous123]

Hi, I cannot see the location of the TAC, but it's located in Global CX Center/Netherlands as far as I can see. 

I've checked the /bootflash, but no crashlogs. 

What is by best logging options to gather most information? Either via WLC or the AP to see what is happening around the time of Max Retransmissions.   I'm NOT a expert on Cisco products, so maybe stupid questions.  It's so many options for everthing, so easy to get lost. 

[CiscoAnonymous123]

A little visualization of what just happend.   The same AP's just rebooted. No trouble on the network at the same momemt. 

[CiscoAnonymous123]

Uplink between WLC og AP problem-site location

[Leo Laohoo]

---

@CiscoAnonymous123 wrote:
Hi, I cannot see the location of the TAC, but it's located in Global CX Center/Netherlands as far as I can see. 

---

Look at the signature block of the TAC engineer.  If the TAC Case is assigned to TAC "Asia", get it reassigned to TAC NAM/LATAM.  The men and women at TAC NAM/LATAM are better because TAC "Asia" is overwhelmed.  

[marce1000]

 - You can follow-up on overall AP health using commands from : https://www.cisco.com/c/en/us/support/docs/wireless/catalyst-9800-series-wireless-controllers/217738-monitor-catalyst-9800-kpis-key-performa.html#anc4

   To investigate (potential) AP  joining issues use : https://logadvisor.cisco.com/logadvisor/wireless/9800/9800APJoin

  (Always advised) : Have a checkup of the WLC 9800 configuration with the CLI command show tech wireless ; 
                                                                                                          feed the output into Wireless Config Analyzer

   Consider upgrading to latest advisory release : 17.9.4a

  M.

[eglinsky2012]

To get the syslog archive from the AP, you can copy the syslog directory from the AP to a TFTP or SCP server:

ap#copy syslogs ?
WORD Enter a File name
scp: Use SCP protocol
tftp: Use TFTP protocol

ap#copy syslogs tftp:?
WORD URL in format A.B.C.D[/dir][/filename]

The result will be a .tgz file containing a series of files containing various logs. There are some without a file extension, just open those with Notepad (if on a PC); those are the ones that contain the type of logs you'd see with "show log".

Thanks to Rich R who mentioned this in another thread recently.

[CiscoAnonymous123]

A short follow up from me. 

I was at the location this Friday and removed a POE-Injektor. AP is now directly connected to the POE+ Switch. So far none of the AP's have rebooted. For now I don't now if the problem is fixed, but isn't very strange if 1 problematic was impacting multiple AP's on two different switches?