Nexus 1000V erspan not working after upgrade to 5.2(1)SV3(3.1)

[riziv]

Hello all,

We recently upgraded our Nexus1000V ( for VMware) from v4.2.1.SV2.2.2 to version 5.2(1)SV3(3.1) and since then, when executing a remote span to my machine, the traffic never arrives at destination anymore.

The configuration looks to be correct according to the conf guide of the new version :

monitor session 3 type erspan-source
  erspan-id 999
  destination ip 10.21.51.1
  ip ttl 64
  source interface Vethernet30 both
  source interface Vethernet121 both
  source interface Vethernet650 both
  no shut

N1K# sh monitor
Session  State        Reason                  Description
-------  -----------  ----------------------  --------------------------------
1        down         Session admin shut
2        down         Session admin shut
3        up           The session is up
4        down         Session admin shut

Do you have any idea why it does not work anymore ?

Regards

Frédéric

[Kirk J]

Greetings.

Please run the following:

• show monitor internal errors
• show monitor internal event-history msgs
• show monitor internal info global-info
• show monitor internal mem-stats

Ping the ERSPAN IP destination from the host VMKernel NIC.

vmkping dest-id

Use the vempkt command to capture packets on the VMKernel NIC LTL and ensure ERSPAN packets are being sent. Use the vemlog debug sfspan d command so that the ERSPAN packets appear in the vempkt capture log

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus1000/sw/5_2_1_s_v_3_1_1/TroubleshootGuide/n1000v_trouble/n1000v_TS_17span.html

Let see what some of the output shows.

Thanks,

Kirk...

[riziv]

Hello Kirk,

Please find attached the result of all the show command. (see debugLogs.txt)
If you already see errors from the provided info, please let me know.

vmkping to the destination ip is well successful :

 # vmkping 10.21.51.1
PING 10.21.51.1 (10.21.51.1): 56 data bytes
64 bytes from 10.21.51.1: icmp_seq=0 ttl=124 time=1.152 ms
64 bytes from 10.21.51.1: icmp_seq=1 ttl=124 time=0.803 ms
64 bytes from 10.21.51.1: icmp_seq=2 ttl=124 time=0.860 ms

 The vem well show the span configured  :

VEM SOURCE IP: 10.14.10.47   --> This is my VMKernel interface with L3control capability allowing erspan

HW SSN ID   ERSPAN ID   HDR VER   DST LTL/IP
        3         999         2   10.21.51.1
RX Ltl Sources :65,
TX Ltl Sources :65,
RX Vlan Sources :
TX Vlan Sources :
Source Filter :

When using vempkt capture , I cannot get output packet matching the filter I configured :

 vempkt capture all-stages vlan 507 ltl 65 --> The LTL of the source interface is 65 on vlan 507

vempkt start

But the vempkt show info seems not to see increasing packet in the "Available Packet Entries" counters and no packet is captured :

vempkt show info
                 Enabled  : Yes
    Total Packet Entries  : 48327
  Wrapped Packet Entries  : 48327
     Lost Packet Entries  : 48329
  Skipped Packet Entries  : 9945
Available Packet Entries  : 0
     Packet Capture Size  : 10000000
     Packet Capture Mode  : Un Reliable
 Stop After Packet Entry  : Not Specified

Thanks in advance

Frédéric