Nexus - command output interpretation - show system internal snmp event-history pktdump

[axa-wongjeff]

Performed a 'show tech-support snmp' on a Nexus switch to investigate an issue with SNMP.

One of the commands listed in the tech support output lists what could be packets causing the issue.  There is an event that looks like a remote IP address of the following: 

16.100.113.4

16.100.205.204

16.113.249.188

I did a packet capture on the traffic going in and out of the switch interfaces and did not find any SNMP packets with those outside addresses. Looking for assistance in interpreting this output to confirm if these are truly outside IP addresses trying to do an SNMP function on the switch. At this point I feel these are neither ingress or egress packets. It's just something internal to the Nexus.

`show system internal snmp event-history pktdump`

1) Event:E_DEBUG, length:224, at 846968 usecs after Fri Mar 30 16:04:46 2012

    [107]

SNMPPKTSTRT: 3.000000 162 1157652703.000000 1157652703.000000 0.000000 0.000000 0 4 3 2 0 172.20.35.87  0 0 0 0.000000 0.000000 16.100.113.4 remote ip,v4: snmp_4401_172.20.35.87  11  0  11 admin 5 0 0 0x107ae674 86

2) Event:E_DEBUG, length:346, at 846900 usecs after Fri Mar 30 16:04:46 2012

    [107] 1157652703.000000:iso.3.6.1.2.1.31.1.1.1.6.167772161 = Counter64: 10061161472 iso.3.6.1.2.1.31.1.1.1.10.167772161 = Counter64: 889856360 iso.3.6.1.2.1.2.2.1.13.167772161 = Counter32: 0 iso.3.6.1.2.1.2.2.1.19.167772161 = Counter32: 0 iso.3.6.1.2.1.2.2.1.14.167772161 = Counter32: 0 iso.3.6.1.2.1.2.2.1.20.167772161 = Counter32: 0  SNMPPKTEND

3) Event:E_DEBUG, length:46, at 846635 usecs after Fri Mar 30 16:04:46 2012

    [107] SNMP PDU[ReqId:1157652703.000000 cmd:162]

4) Event:E_DEBUG, length:226, at 842551 usecs after Fri Mar 30 16:04:46 2012

    [107]

SNMPPKTSTRT: 3.000000 160 1157652703.000000 1157652703.000000 0.000000 0.000000 0 4 3 2 0 172.20.35.87  0 0 0 0.000000 0.000000 16.100.205.204 remote ip,v4: snmp_4401_172.20.35.87  11  0  11 admin 5 0 0 0x107ae674 86

5) Event:E_DEBUG, length:280, at 842482 usecs after Fri Mar 30 16:04:46 2012

    [107] 1157652703.000000:iso.3.6.1.2.1.31.1.1.1.6.167772161 = NULL iso.3.6.1.2.1.31.1.1.1.10.167772161 = NULL iso.3.6.1.2.1.2.2.1.13.167772161 = NULL iso.3.6.1.2.1.2.2.1.19.167772161 = NULL iso.3.6.1.2.1.2.2.1.14.167772161 = NULL iso.3.6.1.2.1.2.2.1.20.167772161 = NULL  SNMPPKTEND

6) Event:E_DEBUG, length:46, at 842245 usecs after Fri Mar 30 16:04:46 2012

    [107] SNMP PDU[ReqId:1157652703.000000 cmd:160]

7) Event:E_DEBUG, length:226, at 532789 usecs after Fri Mar 30 16:04:44 2012

    [107]

SNMPPKTSTRT: 3.000000 162 1157652626.000000 1157652626.000000 0.000000 0.000000 0 4 3 2 0 172.20.35.87  0 0 0 0.000000 0.000000 16.113.249.188 remote ip,v4: snmp_4401_172.20.35.87  11  0  11 admin 5 0 0 0x1078f644 86

8) Event:E_DEBUG, length:501, at 532722 usecs after Fri Mar 30 16:04:44 2012

    [107] 1157652626.000000:iso.3.6.1.2.1.31.1.1.1.6.17874944 = Counter64: 42837088 iso.3.6.1.2.1.31.1.1.1.10.17874944 = Counter64: 42837204 iso.3.6.1.2.1.2.2.1.13.17874944 = Counter32: 0 iso.3.6.1.2.1.2.2.1.19.17874944 = Counter32: 0 iso.3.6.1.2.1.2.2.1.14.17874944 = Counter32: 0 iso.3.6.1.2.1.2.2.1.20.17874944 = Counter32: 0 iso.3.6.1.2.1.31.1.1.1.6.34078720 = Counter64: 11767736126 iso.3.6.1.2.1.31.1.1.1.10.34078720 = Counter64: 2465339074 iso.3.6.1.2.1.2.2.1.13.34078720 = Counter32: 0  SNMPPKTEND

9) Event:E_DEBUG, length:173, at 532711 usecs after Fri Mar 30 16:04:44 2012

    [107] 1157652626.000000:iso.3.6.1.2.1.2.2.1.19.34078720 = Counter32: 0 iso.3.6.1.2.1.2.2.1.14.34078720 = Counter32: 0 iso.3.6.1.2.1.2.2.1.20.34078720 = Counter32: 0  VARBIND

[phiharri · Accepted solution]

Greetings,

The output in 'show system internal snmp event-history pktdump' is a result of external SNMP traffic and not internal to the system. The output format may differ a bit between NX-OS versions. I would look at the values after "remote ip" as the source of these packets, eg. this packet comes from 172.20.35.87:

SNMPPKTSTRT: 3.000000 162 1157652703.000000 1157652703.000000 0.000000  0.000000 0 4 3 2 0 172.20.35.87  0 0 0 0.000000 0.000000 16.100.113.4  remote ip,v4: snmp_4401_172.20.35.87  11  0  11 admin 5 0 0 0x107ae674  86 

If these events are happening frequently, I'd suggest running Ethanalyser on both inband and mgmt ports simultaneously in two terminals, if you're not certain whether this traffic arrives via inband or out-of-band interfaces:

ethanalyzer local in mgmt capture-fi 'port 161' limit-cap 0
ethanalyzer local in inband capture-fi 'port 161' limit-cap 0

Hope this helps,

/Phil