If a wireless client is having problems roaming, it may be necessary to perform a "deep dive" data capture. This will involve:
- capturing information regarding the environment
- reproducing the problem, while collecting:
- packet capture from the client under test
- wireless sniff from channels in use near the client
- debugs from the APs involved
- debugging from the wireless LAN controller(s) to which the AP(s) are joined
- possibly also wired capture from the APs' and/or WLC's switchports
In more detail:
- capturing information regarding the environment
- collect information on the client under test:
- manufacturer, model
- operating system
- wireless adapter: manufacturer, model, firmware version
- supplicant, supplicant version
- if the client is running Windows: get the output of "netsh wlan show all"
- collect information on the wireless infrastructure:
- WLC model, software version
- complete "show run-config" output from the WLC (to be used with WLC Config Analyzer)
- a floormap or similar showing the location where the roaming test is performed, with AP names labeled (e.g. a floor map screenshot from WCS/NCS/PI)
- site survey data from the location under test can be helpful
- reproducing the problem, while collecting:
- packet capture from the client under test
- be sure that the client is NTP synced to true time
- nonpromiscuous packet capture from the client's wireless adapter (e.g. Wireshark, tshark, tcpdump)
- wireless sniff from channels in use near the client
- if there is possibility of significant non-802.11 interference in the environment (more likely in 2.4GHz than in 5GHz), then get a a spectrum capture near the location of concern during the the time of concern. This could be done with Spectrum Expert with CleanAir APs (or with a Cognio Cardbus card), or with Chanalyzer from Metageek.
- debugs from the APs involved
- ideally, you will set up a syslog server that all APs can talk to, and configure the APs to send their debug output to that server. This will enable all AP debugs to be nicely sequenced.
- alternatively, you can run terminal sessions into all of the APs:
- telnet or ssh into all APs in the roaming path
- log each telnet/ssh session
- set "terminal monitor" in each session
- enter the following commands into each AP:
- For IOS APs:
debug capwap console cli ! (hidden)
config t
no logging console
logging buffered 1000000 debugging
logging w.x.y.z !(syslog server)
logging trap debug
line vty 0 4
exec-timeout 240
end
debug dot11 dot11radio0 monitor address hhhh.hhhh.hhhh
debug dot11 dot11radio0 monitor probe
debug dot11 dot11radio0 trace print mgmt client
(if using 5GHz, use "dot11radio1" in addition to / instead of "dot11radio0")
debug capwap client mgmt
- For 2800/3800 series APs:
exec-timeout 720
config ap client-trace address add <client_MAC-address>
config ap client-trace filter all enable
config ap client-trace output console-log enable
config ap client-trace start
debug client <client_MAC-address>
terminal monitor
- debugging from the wireless LAN controller(s) to which the AP(s) are joined
- telnet/ssh into the WLC(s)
- log each telnet/ssh session
- make sure that your WLC is NTP synced, with the minimum 3600 second interval
- enter the following commands:
config session timeout 160
debug client hh:hh:hh:hh:hh:hh
debug mobility handoff enable (only if multiple WLCs are in use)
- possibly also wired capture from the APs' and/or WLC's switchports
- start a continuous ping (e.g. a Windows ping -t) to the client under test, from a wired station in or near the wireless client's subnet
- now reproduce the problem.
- make a note of what time the problem happened, where the client under test was when the problem occured, what symptoms the client experienced (ping latency, ping drops), etc.
- stop the packet capture on the client under test, the wireless sniffer(s), the debugs on the APs, the debugging on the WLCs, the wired sniffers (if in use)
- from each AP, collect: "show controller dot11radio0" (or dot11radio1)
- from the WLC, collect: "show msglog"
- collect all of:
- the note describing the event
- the writeup describing the client device
- the show run-config output from the WLC(s)
- the map showing the area of interest
- the nonpromiscuous capture from the client adapter
- any other info from the client
- the wireless sniffer captures (unfiltered, but do not include gigantic files that do not correspond to the time of interest)
- the spectrum capture, if any
- the AP log files
- the WLC debugs/log files
- the wired sniffer capture(s) if in use
- zip all of these up together
- Upload this data to your TAC case
