Introduction
This document helps in gathering logs and troubleshooting of issues related to WAAS. These logs and basic troubleshooting is crucial before contacting to any escalated support.
What Information is Required
- A detailed problem description
- All the information / results from your troubleshooting efforts
- Versions of the involved device(s) – both from site having an issue at edge or core and corresponding core and edge, including WAE(s), router(s), switch(es), server(s), client(s), application(s), etc. Also include any ip addresses of affected devices (client machines having issues & servers accessing)
- The sysreport(s) from the WAE(s) involved
- The show tech-support from the router(s) / switch(es) doing redirection and to which WAEs they are connected
- Related WCCP CLIs and information
- Duplication steps
- Configuration details (including screen captures, database backups, etc.)
- Diagrams (logical network diagram and wiring diagram)
- Evidence of the problem, such as crash files, packet capture, screen shots, error messages, DRE_EVENT_TRACES, etc.
Retrieve Required Information
On each WAE, capture the following commands:
term len 0
show clock detail
show tech-support
show wccp gre
show wccp router
show wccp wide-area-engine
show wccp flows tcp-promiscuous detail
NOTE: Run the above commands 3-4 times in succession to see if counters are incrementing.
If problem can be narrowed down to a specific client ip, please run the following commands also:
show tfo connection client-ip <client_ip_having_issue> detail
show statistics dre connection client-ip <client_ip_having_issue> active detail
show statistics dre detail
NOTE: Run the above commands 3-4 times in succession to see if counters are incrementing. Also, the above ‘detail’ option on the tfo & dre commands is hidden & cannot be tabbed out.
On the device that the WAE is directly connected to (core switch/wan-core/wan router), capture the following commands:
term len 0
show clock detail
show tech-support
show ip wccp
show ip wccp 61 service
show ip wccp 62 service
show ip wccp 61 detail
show ip wccp 62 detail
show ip wccp 61 internal
show ip wccp 62 internal
show ip wccp interface detail
show tcam counts
show mls stat
show mls netflow table detail
show mls netflow ip count
show mls netflow ip sw-installed count
show mls netflow ip sw-installed detail
show fm interface <interface connecting to WAE>
show ip wccp 61 mask
show ip wccp 62 mask
show ip wccp 61 merge
show ip wccp 62 merge
show tcam interface <WAE vlan – usually 5> acl in ip
show tcam interface <WAE vlan – usually 5> acl out ip
show tcam interface <WAE vlan – usually 5> acl in ip detail
show tcam interface <WAE vlan – usually 5> acl out ip detail
For possible IOS issues please capture debugs for wccp on the device that the WAE is connected to (switch / router):
debug ip wccp events
debug ip wccp packets
You can now submit the output from these commands to the escalation engineer.
Gather the Sysreport and any Crashdumps from the WAE
copy sysreport disk ______.tar.gz
To enable ftp on the WAE run the command:
inetd enable ftp
FTP from the command line on the NMS to the WAE address and use the following commands to get the file on your NMS account directory:
hash
bin
get <filename>
bye
Verify WAAS Image
Use following command:
wae# show version
Enable WAAS Logging
Use following command:
wae(config)# logging console enable
wae# show logging
Gather Packet Captures
It is recommended to capture 1-2 mins if the packet capture is not filtered; and 5 mins if it is filtered by IP addresses.
Read filter for specific hosts (looks inside GRE)
tethereal –I eth0 -s 1600 -w _____.cap –R "ip.addr eq x.x.x.x and ip.addr eq x.x.x.x"
Create rolling captures {files files with 1MB file size}
tethereal –I eth0 -s 1600 –b filesize:1024 –b files:5 -w _____.cap
Basic Troubleshooting
Issue command "show ip wccp" and check for the following.
On platforms that use software-based redirection, verify that the Total Packets s/w Redirected counters are incrementing in the above command output. On platforms that use hardware-based redirection, these counters should not be incrementing much. If you are seeing these counters increment significantly on hardware-based platforms, WCCP could be misconfigured on the router (WCCP GRE is processed in software by default), or the router could be falling back to software redirection due to hardware resources issues such as running out of TCAM resources. More investigation is required if you see these counters incrementing on a hardware-based platform, which could lead to high CPU usage.
The Total Packets Denied Redirect counter increments for packets that match the service group but do not match the redirect list.
The Total Authentication failures counter increments for packets that are received with the incorrect service group password.
Related Information
Cisco WAAS Troubleshooting Guide for Release 4.1.3 and Late
