Re: Max retransmissions & interference threshold

Jim Mueller · ‎05-01-2013

We're getting complaints about a specific 1131AG in the field only being intermittently accessible. WCS shows both the a and b/g interfaces randomly bouncing, sometimes the reason for the bounce is unknown other times it says it's because it lost connection to the controller. I can ssh into the WCS but the engineer who setup the AP's doesn't seem to have enabled remote access via telnet or ssh.

We are also seeing these events:

802.11a interface of AP * is down: Controller 172.17.0.10 Reason: Max retransmissions for the AP have reached.

Interference threshold violation reported by '802.11b/g' interface of AP *, connected to Controller '172.17.0.10'.

I know I can adjust the threshold percentage, but that would only seem to mask the issue.

WCS gives this version info on the AP:

Versions

Software Version	6.0.202.0
Boot Version	12.3.8.0

Inventory Information

AP Type	CAPWAP
AP Model	AIR-LAP1131AG-A-K9
IOS Version	12.4(21a)JHC
AP Certificate Type	Manufacture Installed
AP Serial Number	*

Unique Device Identifier (UDI)

Name	Cisco AP
Description	Cisco Wireless Access Point
Product Id	AIR-LAP1131AG-A-K9
Version Id	V01
Serial Number	*

<BR type="_moz" mozdirty>WCS CLI shows this version info:

(Cisco Controller) show>sysinfo

Manufacturer's Name.............................. Cisco Systems Inc.
Product Name..................................... Cisco Controller
Product Version.................................. 6.0.202.0
Bootloader Version............................... 1.0.1
Field Recovery Image Version..................... 6.0.182.0
Firmware Version................................. FPGA 1.3, Env 1.6, USB console 1.27
Build Type....................................... DATA + WPS

System Name...................................... FLDS-5508-01
System Location..................................
System Contact...................................
System ObjectID.................................. 1.3.6.1.4.1.9.1.1069
IP Address....................................... 172.17.0.10
Last Reset....................................... Software reset
System Up Time................................... 247 days 5 hrs 0 mins 52 secs
System Timezone Location......................... (GMT -5:00) Eastern Time (US and Canada)
Current Boot License Level....................... base
Current Boot License Type........................ Permanent
Next Boot License Level.......................... base
Next Boot License Type........................... Permanent

Configured Country............................... US - United States

~~More~~ or (q)uit
Operating Environment............................ Commercial (0 to 40 C)
Internal Temp Alarm Limits....................... 0 to 65 C
Internal Temperature............................. +35 C
External Temperature............................. +20 C
Fan Status....................................... OK

State of 802.11b Network......................... Enabled
State of 802.11a Network......................... Enabled
Number of WLANs.................................. 3
3rd Party Access Point Support................... Disabled
Number of Active Clients......................... 100

Burned-in MAC Address............................ 2C:54:2D:72:74:40
Power Supply 1................................... Present, OK
Power Supply 2................................... Present, OK
Maximum number of APs supported.................. 75

Thanks!

Jim

Jim Mueller · ‎05-01-2013

It doesn't appear I can edit the original post, but I need to clarify a couple of things. I can access the WCS web gui, access the WLC via SSH, but no known method of accessing the remote AP without traveling to the site (3 hrs one way) and using the console cable.

David Watkins · ‎05-03-2013

Hi Jim,

It looks like the AP is simply loosing connectivity with the WLC. The max retransmission is most likely the keep-alives once the DTLS connection is established with the WLC. Once these heartbeats are not talking two-way, the AP/WLC will both complain that the AP has reached max transmissions. The WLC will remove the AP entry and the AP will re-start the discovery/join process.

In a scenario like this you will most likely need to configure a capture at the AP and WLC mgmt interface to verify if the keep-alive messages are being sent/received properly.

Since you stated this is about 3 hours away, what is the latency you experience between the AP and WLC when it is "up". Is it possible that the AP cannot reach the WLC in a timely manner? You could enable link latency once the AP joins and monitor the values you are seeing.

If the AP is operating in local mode, it would be ideal to maintain no more than a 100 ms round-trip time. If this cannot be achieved, you may consider configuring the AP for FlexConnect mode if the infrastructure design can allow this.

Jim Mueller · ‎05-09-2013

Thanks for the reply, David.

I do not know how to configure a capture at the AP & WLC to verify the keep-alive messages. I enabled link latency on the AP and for the brief amount of time it's been enabled the range is 17-19ms. The round-trip ping latency averages 22ms between the core switch the WLC is connected to and the remote AP. The AP is operating in local mode.

Jim

Scott Fella · ‎05-09-2013

You should be fine with that RRT. On the WLC GUI, go to Wireless and click on the AP and enable ssh or telnet.

Thanks,

Scott

Help out other by using the rating system and marking answered questions as "Answered"

-Scott
*** Please rate helpful posts ***

Jim Mueller · ‎05-09-2013

I enabled telnet & ssh on the AP. Are there other diagnostic steps now that it's enabled?

Scott Fella · ‎05-09-2013

Well you really should run debugs from the WLC. You have some show commands available and you should be able to do a show dir and see if you have any crash logs.

Thanks,

Scott

Help out other by using the rating system and marking answered questions as "Answered"

-Scott
*** Please rate helpful posts ***

Jim Mueller · ‎05-30-2013

Show dir is not a supported command. Browsing through the Show help, I notice eventlog & msglog commands but nothing which indicated to me it referenced a local filesystem.

Saravanan Lakshmanan · ‎05-18-2013

Get show log, more event.log from AP for analaysis.

Are just AP disconnecting from WLC?

//Enable debug capwap on AP and WLc for that AP.

//Check AP join status on WLC and AP.

What's the AP's up and associated time?

Did the AP crash?

//check AP crash log from WLC

Are only Radio resetting?

//AP log will show this.

AP code reference:-

https://supportforums.cisco.com/docs/DOC-33185

Are any rogue feature like rldp, auto/manual containment enabled?

//check WLC config.

Jim Mueller · ‎05-30-2013

The AP has two crashlogs in flash:, but they are from 9/24/12 and 12/18/12.

Edit: Added attachments in later post and cleaned up this post.

Insufficient info to enable debug capwap on AP:

---

2067-LakewoodShores#debug capwap
% Incomplete command.

2067-LakewoodShores#debug capwap ?
am      KAM-AP debug
client CAPWAP Client Debug Options
ids     WIDS debug
mcast   CAPWAP MCAST Debug Options
mfp     CAPWAP MFP Debug Options
reap    CAPWAP REAP Debug Options
rm      CAPWAP RM Debug Options

2067-LakewoodShores#debug capwap

---

Can you be as specific as possible on the commands to generate the information you're requesting please?

Scott Fella · ‎05-30-2013

Jim,

Can you attach the show run-config... make sure you attach it and not paste it in the thread as the las post makes it difficult to view it from an iphone.

On the AP, enter the following:

debug capwap console CLI

dir

and post the dir

I would delete the crash files also then issue

undebug all

Thanks,

Scott

Help out other by using the rating system and marking answered questions as "Answered"

-Scott
*** Please rate helpful posts ***

Jim Mueller · ‎05-30-2013

Hi Scott,

I don't see a method on this forum to directly attach a document to a post other than embedding an image or a video...? In the interim, the remainder of your request is below.

2067-LakewoodShores#debug capwap console CLI
Debug already enabled

2067-LakewoodShores#dir
Directory of flash:/

    2 -rwx      295424 Dec 18 2012 12:04:05 +00:00 crashinfo_20121218-120402
    3 -rwx      298851 Sep 24 2012 16:28:51 +00:00 crashinfo_20120924-162848
    4 drwx         128   Jan 1 1970 00:02:37 +00:00 c1130-rcvk9w8-mx
   36 drwx         320 Aug 27 2012 15:55:58 +00:00 c1130-k9w8-mx.124-21a.JHC
   34 -rwx       97500 May 14 2013 14:05:46 +00:00 event.log
    8 -rwx        6168 May 29 2013 23:03:53 +00:00 private-multiple-fs
   10 -rwx         189 May 29 2013 23:03:52 +00:00 env_vars

15482880 bytes total (7936512 bytes free)
2067-LakewoodShores#del flash:crash*
Delete filename [crash*]?
Delete flash:crashinfo_20121218-120402? [confirm]
Delete flash:crashinfo_20120924-162848? [confirm]
2067-LakewoodShores#dir
Directory of flash:/

    4 drwx         128   Jan 1 1970 00:02:37 +00:00 c1130-rcvk9w8-mx
   36 drwx         320 Aug 27 2012 15:55:58 +00:00 c1130-k9w8-mx.124-21a.JHC
   34 -rwx       97500 May 14 2013 14:05:46 +00:00 event.log
    8 -rwx        6168 May 29 2013 23:03:53 +00:00 private-multiple-fs
   10 -rwx         189 May 29 2013 23:03:52 +00:00 env_vars

15482880 bytes total (8530944 bytes free)
2067-LakewoodShores#undebug all
All possible debugging has been turned off
2067-LakewoodShores#

Scott Fella · ‎05-30-2013

On the top right when you repluy, use the advanced editor... this will have am Attach Files.

Thanks,

Scott

Help out other by using the rating system and marking answered questions as "Answered"

-Scott
*** Please rate helpful posts ***

Jim Mueller · ‎05-30-2013

Files attached.

Scott Fella · ‎05-30-2013

So what exactly is happening? Only this ap is showing that it goes down/up? How often is this happening? What do you see in the WLC log and what do you see on the WLC > Monitor > AP Join stats for that AP.

Thanks,

Scott

Help out other by using the rating system and marking answered questions as "Answered"

-Scott
*** Please rate helpful posts ***