Hello everyone,
here is a brief description of a problem:
On a Cisco 7600 with SUP720 BXL sudden increase of CPU utilization (from 35% to 98%) appeared.
This utilisation happend and still happens on the SP CPU (not RP) and it is interrupt based.
router-sp#sh proc cpu
CPU utilization for five seconds: 99%/82%; one minute: 98%; five minutes: 98%
I also got output from ibc
router-sp#show ibc
Interface information:
Interface IBC0/0(idb 0x44E47588)
Hardware is Mistral IBC (revision 5)
5 minute rx rate 24000 bits/sec, 44 packets/sec
5 minute tx rate 68000 bits/sec, 122 packets/sec
1186057298 packets input, 80461312123 bytes
1179948920 broadcasts received
3254930405 packets output, 224120381448 bytes
85350832 broadcasts sent
0 Inband input packet drops
0 Bridge Packet loopback drops
0 Packets CEF Switched, 0 Packets Fast Switched
0 Packets SLB Switched, 0 Packets CWAN Switched
IBC resets = 2; last at 00:28:58.792 CET Wed Nov 30 2011
Using SPAN I collected packets that were punted to SP CPU.
Statistics showed that 96,82% of all pakets were STP BPDUs and second next were CDP packets with 0,43% share.
It can be seen from the collected packets that 70% off all STP BPDUs are from 2 source MAC addresses.
Since this layer 2 network consists of few hundreds Cisco switches (series 7600, 6500, 2960, 3560,3750, and Linksys) it is very hard to trace this 2 MAC addreses.
questions:
-----------------
1) is there a way to somehow trace these two MAC addreses/switches besides loggin into every switch in the network?
Problem is that these 2 MAC addresses don't appear in CAM or ARP tables on network, And also, as far as I know, huge problem is that 2960,3560 series for BID (bridge identifier) use special MAC address which can be seen only with "show version" (and this MAC is also source MAC address for BPDU)
2) regarding the output of "show ibc", the traffic comming and going to/from SP CPU shouldn't be significant for this CPU utilisation of 98%.
5 minute rx rate 24000 bits/sec, 44 packets/sec
5 minute tx rate 68000 bits/sec, 122 packets/sec
That leads me to conclusion that this could be some kind of loop in processing on CPU and reload could help. Have anyone seen maybe something like this? Am I misled or 122 BPDUs per second can realy overwhelm CPU?
3) Generaly have anyone experienced problem with high SP CPU utilisation on SUP720 And what was the cause usualy?
(and if someone can recommend what else should I look for)
Thanks in advance,
A.
