Packet Capture on VMware ESXi using the pktcap-uw tool

[lesquive]

Important notes:

• Available for ESXi 5.5 and later.

• This feature only captures packets in one direction and defaults to inbound.

• You can run a capture on the physical network card, vmkernel, or the virtual switchport of a particular virtual machine.

• By default the captured packets are displayed on the terminal. It’s easier if you save the capture and transfer it via SFTP to view via wireshark https://www.wireshark.org/

Enabling SSH on ESXi host

1. You need to enable SSH on the ESXi host that is managing your VMs. This is done by logging in directly to the ESXi via CIMC or KVM switch:

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2004746

1. Access your ESXi host via SSH.

Capturing packets on physical interface, VMkernel, and VM switchport:

1. Confirm the virtual port number & client name of the device you wish to run the capture on:

Command: net-stats –l

1. If you wish to run a packet capture on vmk0 and export the output to a file named CAPTURE.pcap in the /tmp/ directory of your host:

Command: pktcap-uw --vmk vmk0 -o /tmp/CAPTURE.pcap

*Notice how as I press enter the number of packets increases from 35 >> 41 >> 48 >> and finally 55.

To cancel the capture I press: Ctrl + c

1. If you wish to run a packet capture on vmnic3 and export the output to a file named Capture.pcap in the /tmp/ directory of your host:

Command: pktcap-uw --uplink vmnic3 -o /tmp/CAPTURE.pcap

1. If you wish to run a packet capture on one of your virtual machines and export the output to a file named Capture.pcap in the /tmp/ directory of your host:

Command: pktcap-uw --switchport <switchportnumber> -o /tmp/CAPTURE.pcap

Exporting captures to an SFTP server:

Now we need to export CAPTURE.pcap to an external SFTP server for easy viewing via WIRESHARK. I’m using an SFTP on Ubuntu, but you can use any SFTP you would like.

• Log into the SFTP server.

• Run the command: sftp <esxi-user-ID>@<ESXi-IP-Address>. For example in my case it will be: sftp root@192.168.213.236

• Run the command: get /tmp/CAPTURE.pcap

At this point you should have been able to download the capture to your SFTP:

Changing from the default inbound to outbound direction:

It’s very important to remember that this feature only captures traffic one way. Up until this point we have been capturing traffic in the default inbound direction. To Capture packets on the outbound direction:

Command: --dir 1

pktcap-uw --vmk vmk0 --dir 1 -o /tmp/CAPTURE.pcap

pktcap-uw --uplink vmnic3 --dir 1 -o /tmp/CAPTURE.pcap

pktcap-uw --switchport 50331657 --dir 1 -o /tmp/CAPTURE.pcap

References:

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2051814

http://www.virten.net/2015/10/esxi-network-troubleshooting-with-tcpdump-uw-and-pktcap-uw/

[beandrew]

If you want to do bi-directional captures using pktcap-uw then you can do the following command:

pktcap-uw --switchport 33554495 --dir 0 -o /tmp/33554495_in.pcap & \
pktcap-uw --switchport 33554495 --dir 1 -o /tmp/33554495_out.pcap &

You can also do this in one line if you remove the "\" and return:

pktcap-uw --vmk vmk0 --dir 0 -o /tmp/vmk0_in.pcap & pktcap-uw --vmk vmk0 --dir 1 -o /tmp/vmk0_out.pcap &

Important note: Because "&" cause the process to be run in the background you'll need to

manually kill the process after starting it. The easiest way to do this is to kill all instances of pktcap-uw:

kill $(lsof |grep pktcap-uw |awk '{print $1}'| sort -u)    

You can verify all pcaps are ended by running the following command: 

lsof |grep pktcap-uw |awk '{print $1}'| sort -u

References:
http://networkgeekstuff.com/networking/minipost-capturing-bidirectional-traffic-of-virtual-machine-vms-on-vmware-esx-6-x-host/
https://kb.vmware.com/s/article/2051814

[orlandoguy82]

Great article. +5 

This helped me isolate a SIP issues coming from a CUCM VM. I will add SSH can be enabled from Vsphere under the Configuration -> Software pane -> Security Profile -> Properties -SSH -> start SSH

[beandrew]

These captures just got so much easier as of ESXi 6.7 because they finally added --dir 2 which finally allows for both inbound and outbound captures within the same file!

Using the example above for ESXi 6.5 and earlier we can see this is now reduced to the following command in ESXi 6.7+
pktcap-uw --switchport 33554495 --dir 2 -o /tmp/33554495_InandOut.pcap

https://kb.vmware.com/s/article/2051814