Introduction
Ethanalyzer is a Cisco NX-OS protocol analyzer tool based on the Wireshark (formerly Ethereal) open source code. Ethanalyzer is a command-line version of Wireshark that captures and decodes packets. You can use Ethanalyzer to troubleshoot your network and analyze the control-plane traffic. This document covers configuration of Ethanalyzer, examples of its implementation and Ethanalyzer usage together with ACLs "log" option to sniff data plane traffic.
Configuration
To configure Ethanalyzer, use the following commands:
Command | Purpose |
ethanalyzer local interface | Captures packets sent or received by the supervisor and provides detailed protocol information. |
ethanalyzer local interface inband | Captures packets sent or received by the supervisor and provides detailed protocol information in the inband and outband interfaces. |
ethanalyzer local interface mgmt | Captures packets sent or received by the supervisor and provides detailed protocol information in the management interfaces. |
ethanalyzer local interface {inband | mgmt} brief | Captures packets sent or received by the supervisor and provides a summary of protocol information. |
ethanalyzer local interface {inband | mgmt} limit-captured-frames | Limits the number of frames to capture. |
ethanalyzer local interface {inband | mgmt} limit-frame-size | Limits the length of the frame to capture. |
ethanalyzer local interface {inband | mgmt} capture-filter | Filters the types of packets to capture. |
ethanalyzer local interface {inband | mgmt} display-filter | Filters the types of captured packets to display. |
ethanalyzer local interface {inband | mgmt} decode-internal | Decodes the internal frame header for Cisco NX-OS. Note Do not use this option if you plan to analyze the data using Wireshark instead of Ethanalyzer. |
ethanalyzer local interface {inband | mgmt} write | Saves the captured data to a file. |
ethanalyzer local read | Opens the captured data file and analyzes it. |
Ethanalyzer does not capture data traffic that Cisco NX-OS forwards in the hardware but you can use ACLs with log option as a workaround (see the corresponding paragraph below).
Ethanalyzer uses the same capture filter syntax as tcpdump and uses the Wireshark display filter syntax.
See the Wireshark weekly tips for helpful hints on using the tool.
Filtering
Two types of filters are supported:
1. capture-filter: standard tcmdump capture filter syntax
tcp/udp port <>, src/dst portrange <>, src/dst port <>, src/dst host <>, src/dst net <>, less/greater <>, vlan etc …
2. display-filter: standard wireshark display filter syntax
ip.src==x.x.x.x, frame.len > x, http.request.uri == http://www.wireshark.org/
Capture output
Captured packet data can be written to different locations for easy debugging:
Nexus7000# ethanalyzer local read ?
bootflash:, logflash:, slot0:, usb1:, usb2:, volatile:
1. Write the ethanalyzer ouput to a file on bootflash:
Nexus7000# ethanalyzer local interface inband write bootflash:sniff_capture
2. Read the ethanalyzer ouput from a file on bootflash: (You can also add a “pipe” to the end):
Nexus7000# ethanalyzer local read bootflash:sniff_capture
File logging limitations and enhancements
The file size is set to 10485. As a result even if you have enough space on flash, you will drop packets if you exceed this size.
Switch# ethanalyzer local interface inband capture-filter " ip proto 89" limit-captured-frames 0 write logflash:ospf.pcap
Capturing on inband
81342
2560 packets dropped
ACLs and Ethanalyzer for Data Plane Sampling
The Ethanalyzer captures only traffic on CPU, so seems as unsuitable solution for the data plane traffic analysis. However, this limitation can be avoided with a use of ACL logging to sample specific packets from data plane.
Nexus 7000 ACLs features:
- Nexus 7000 ACLs enforced by hardware forwarding engine at wire rate;
- Sizeable pool of hardware resources available for classification -64K CL TCAM entries;
- ACLs can match based on granular IP- or MAC-based parameters,including L4 protocol and port numbers, packet length, etc.
When we use ACLs and the “log” keyword, access control entries (ACEs) with log keyword cause system to punt a copy of matching packets to supervisor CPU. Key point is that original traffic forwarded or dropped in hardware with no performance penalty. Note that punted copies subjected to hardware rate limiter, forwarding engine hardware enforces rate to avoid saturating inband interface/CPU.
hardware rate-limit access-list-log command adjusts rate (100 pps by default).
"log" keyword can be used both with permit or deny ACEs and system syslogs punted packets:
n7010# 2008 Sep 15 19:21:00 n7010 %$ VDC-1 %$ %ACLLOG-6-ACLLOG_FLOW_INTERVAL: Source IP: 10.1.1.2, Destination IP: 10.1.2.2, Source Port: 1024, Destination Port: 40000, Source Interface: Ethernet1/1, protocol: "TCP"(6), Hit-count = 6224
Full Packet Analysis
1. Define ACL entry with logging to match traffic of interest
ip access-list acl-cap
permit tcp 10.1.1.3/32 10.1.2.2/32 eq 5000 log
permit ip any any
2. Attach ACL to interface
interface e1/1
ip access-group acl-cap in
3. Define ethanalyzer capture and/or display filter to capture just the subject traffic
ethanalyzer local interface inband capture-filter “tcp port 5000”
All packets matching log ACEs are punted (with rate limiting) – use capture/display filters to see only a subset of traffic matching log ACEs.
4. View captured traffic on-switch, or copy to PC/workstation for GUI analysis
Example – Brief Decode On-Switch
n7010# ethanalyzer local interface inband brief capture-filter "tcp port 5000" limit-cap 3
Capturing on eth0
2008-09-16 18:36:08.315736 10.1.1.3 -> 10.1.2.2 TCP 5000 > 5000 [ACK] Seq=0 Ack=0 Win=4096 Len=454
2008-09-16 18:36:08.988747 10.1.1.3 -> 10.1.2.2 TCP [TCP Retransmission] 5000 > 5000 [ACK] Seq=0 Ack=0 Win=4096 Len=454
2008-09-16 18:36:09.873360 10.1.1.3 -> 10.1.2.2 TCP [TCP Retransmission] 5000 > 5000 [ACK] Seq=0 Ack=0 Win=4096 Len=454
3 packets captured
n7010#
Example – Full Decode On-Switch
n7010# ethanalyzer local interface inband capture-filter "tcp port 5000" limit-captured-frames 1 | no-more
Capturing on eth0
Frame 1 (508 bytes on wire, 508 bytes captured)
Arrival Time: Sep 16, 2008 18:33:12.138957000
[Time delta from previous captured frame: 1221615192.138957000 seconds]
[Time delta from previous displayed frame: 1221615192.138957000 seconds]
[Time since reference or first frame: 1221615192.138957000 seconds]
Frame Number: 1
Frame Length: 508 bytes
Capture Length: 508 bytes
[Protocols in frame: eth:ip:tcp:data]
Ethernet II, Src: 00:1b:54:c2:40:41 (00:1b:54:c2:40:41), Dst: 00:10:94:00:00:02 (00:10:94:00:00:02)
<…>
Internet Protocol, Src: 10.1.1.3 (10.1.1.3), Dst: 10.1.2.2 (10.1.2.2)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
<…>
Source: 10.1.1.3 (10.1.1.3)
Destination: 10.1.2.2 (10.1.2.2)
Transmission Control Protocol, Src Port: 5000 (5000), Dst Port: 5000 (5000), Seq: 0, Ack: 0, Len: 454
Source port: 5000 (5000)
Destination port: 5000 (5000)
<…>
Example – Write Data to File
n7010# ethanalyzer local interface inband capture-filter "tcp port 5000" limit-captured-frames 50 write bootflash:test.cap //Use “write” option to send captured frames to flash file instead of print onscreen.
Capturing on eth0
50
n7010# dir test.cap
26224 Sep 16 18:40:08 2008 test.cap
Usage for bootflash://sup-local
1138716672 bytes used
671182848 bytes free
1809899520 bytes total
n7010# copy bootflash: scp: //Copy cap file off-box to a PC/workstation to view the file contents in GUI version of wireshark.
Enter source filename: test.cap
Enter vrf (If no input, current vrf 'default' is considered): management
Enter hostname for the scp server: dove
Enter username: tstevens
Password: ***
test.cap 100% 26KB 25.6KB/s 00:00
n7010#
Filter Examples
1. This example shows captured data (limited to four packets) on the management interface:
switch(config)# ethanalyzer local interface mgmt brief limit-captured-frames 4
Capturing on eth1
2008-02-18 13:21:21.841182 172.28.230.2 -> 224.0.0.2 HSRP Hello (state Standy)
2008-02-18 13:21:21.842190 10.86.249.17 -> 172.28.231.193 TCP 4261 > telnet [AC] Seq=0 Ack=0 Win=64475 Len=0
2008-02-18 13:21:21.843039 172.28.231.193 -> 10.86.249.17 TELNET Telnet Data ..
2008-02-18 13:21:21.850463 00:13:5f:1c:ee:80 -> ab:00:00:02:00:00 0x6002 DEC DN
Remote Console
4 packets captured
2. This example shows detailed captured data for one HSRP packet:
switch(config)# ethanalyzer local interface mgmt capture-filter "udp port 1985"
limit-captured-frames 1
Capturing on eth1
Frame 1 (62 bytes on wire, 62 bytes captured)
Arrival Time: Feb 18, 2008 13:29:19.961280000
[Time delta from previous captured frame: 1203341359.961280000 seconds]
[Time delta from previous displayed frame: 1203341359.961280000 seconds]
[Time since reference or first frame: 1203341359.961280000 seconds]
Frame Number: 1
Frame Length: 62 bytes
Capture Length: 62 bytes
[Frame is marked: False]
[Protocols in frame: eth:ip:udp:hsrp]
Ethernet II, Src: 00:00:0c:07:ac:01 (00:00:0c:07:ac:01), Dst: 01:00:5e:00:00:02
(01:00:5e:00:00:02)
Destination: 01:00:5e:00:00:02 (01:00:5e:00:00:02)
Address: 01:00:5e:00:00:02 (01:00:5e:00:00:02)
.... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
Source: 00:00:0c:07:ac:01 (00:00:0c:07:ac:01)
Address: 00:00:0c:07:ac:01 (00:00:0c:07:ac:01)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
Type: IP (0x0800)
Internet Protocol, Src: 172.28.230.3 (172.28.230.3), Dst: 224.0.0.2 (224.0.0.2)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0xc0 (DSCP 0x30: Class Selector 6; ECN: 0x00)
1100 00.. = Differentiated Services Codepoint: Class Selector 6 (0x30)
.... ..0. = ECN-Capable Transport (ECT): 0
.... ...0 = ECN-CE: 0
Total Length: 48
Identification: 0x0000 (0)
Flags: 0x00
0... = Reserved bit: Not set
.0.. = Don't fragment: Not set
..0. = More fragments: Not set
Fragment offset: 0
Time to live: 1
Protocol: UDP (0x11)
Header checksum: 0x46db [correct]
[Good: True]
[Bad : False]
Source: 172.28.230.3 (172.28.230.3)
Destination: 224.0.0.2 (224.0.0.2)
User Datagram Protocol, Src Port: 1985 (1985), Dst Port: 1985 (1985)
Source port: 1985 (1985)
Destination port: 1985 (1985)
Length: 28
Checksum: 0x8ab9 [correct]
[Good Checksum: True]
[Bad Checksum: False]
Cisco Hot Standby Router Protocol
Version: 0
Op Code: Hello (0)
State: Active (16)
Hellotime: Default (3)
Holdtime: Default (10)
Priority: 105
Group: 1
Reserved: 0Authentication Data: Default (cisco)
Virtual IP Address: 172.28.230.1 (172.28.230.1)
1 packets captured
3. This example uses a display filter to show only those HSRP packets that have an active HSRP state:
switch(config)# ethanalyzer local interface mgmt brief display-filter "hsrp.state==Active" limit-captured-frames 2
Capturing on eth1
2008-02-18 14:35:41.443118 172.28.230.3 -> 224.0.0.2 HSRP Hello (state Active)
2008-02-18 14:35:44.326892 172.28.230.3 -> 224.0.0.2 HSRP Hello (state Active)
2 packets captured
4. Other filter examples:
ethanalyzer local interface mgmt capture-filter “dst host 172.16.185.1”
ethanalyzer local interface inband capture-filter “stp”
ethanalyzer local interface inband decode-internal capture-filter “stp”
ethanalyzer local interface inband capture-filter “stp” limit-frame-size 64
ethanalyzer local interface inband capture-filter “icmp and host 10.10.10.1” limit-captured-frames 1000 write bootflash:icmp
