4500 High Cpu / K2CpuMan Review / Host Learning

[_networker_]

Hello,

there`s a 4500 Switch with High Cpu Load:

#show proc cpu sorted | ex 0.0
CPU utilization for five seconds: 88%/3%; one minute: 92%; five minutes: 93%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
  41  1147238344  45841172      25026 72.39% 78.83% 79.02%   0 Cat4k Mgmt LoPri
  40  4102120428 671891694       6105  6.47%  6.26%  6.26%   0 Cat4k Mgmt HiPri
  80  14929834522273994121          0  3.19%  3.09%  3.08%   0 Spanning Tree

The following command indicates that the CPU is getting many packets to process

#show platform health | ex " 0.0"
                     %CPU   %CPU    RunTimeMax   Priority  Average %CPU  Total
                     Target Actual Target Actual   Fg   Bg 5Sec Min Hour  CPU
K2CpuMan Review       30.00  78.76     30     91  100  500  115 108   81  86617:42

#show platform cpu packet statistics

[...]

Packets Received by Packet Queue

Queue                  Total           5 sec avg 1 min avg 5 min avg 1 hour avg
---------------------- --------------- --------- --------- --------- ----------
Esmp                       11440169709       121       133       107         98
L2/L3Control                 991904978         8         6         7          0
Host Learning               3226484847      5077      5981      4745       4596

[...]

There`s a very high number of new learned MAC adresses. I`ve checked the MAC Table:

#show mac address-table count
MAC Entries for all vlans:
Dynamic Unicast Address Count:                  97
Static Unicast Address (User-defined) Count:    0
Static Unicast Address (System-defined) Count:  12
Total Unicast MAC Addresses In Use:             109
Total Unicast MAC Addresses Available:          32768
Multicast MAC Address Count:                    68
Total Multicast MAC Addresses Available:        16384

Furthermore i`ve checked if the network is unstable with many Topology Changes Notification. But this is not the case.

I used show spanning-tree detail | inc ieee|occurr|from|is exec and the latest TCN is from yesterday. There was indeed a Change, so this is normal behaviour.

What can i do next?

What could be the the reason for "Host Learning" ? According to Management the CPU suddenly increased from 20% to 93%!

[InayathUlla Sharieff]

https://supportforums.cisco.com/document/12298401/troubleshooting-high-cpu-3750

Try debuging and enable mac move notificiation and check for the result.

Also if you seeing TCN findout from were exactly its happening.

[_networker_]

Hm, the document you posted seems to be for an 3750 Switch, but this is a 4500!?

Regarding "mac move notification": Is this a CPU intensive Feature, or are there any other risk by enabling this feature?

[Rajeev Sharma]

Hey,

The document will work with 4500 as well. Regarding mac-move you may turn it on, there are no risks enabling in this feature. You may also collect the SPAN of Host Learning queue:

http://www.cisco.com/en/US/products/hw/switches/ps663/products_tech_note09186a00804cef15.shtml#tools

HTH.

Regards,

RS.

[_networker_]

The mac-move feature doesn't shows any entries. I expected to see many entries, because the Host Learning queue still shows an 1 hour average of 3819!

Regarding SPAN i don't have a SPAN Destination port at the moment.

Another question: There are two Core-Switches and some distribution switches are connected to each of the Core-Switch. On the trunks between the core-Switches and Distribution-Switches there are only some vlan allowed via "trunk allowed vlan".

Furthermore i can see that there are different spanning-tree root bridges for the same VLAN, e.g. if a VLAN is not allowed to a Distribution Switch, this Switch will be the root instead of the desired first Core-Switch. Is this a normal behaviour?

Must the trunk allowed configuration match on the link between core- and Distribution Switch?

[Rajeev Sharma]

Hey,

If we dont have SPAN destination port you may use the built in sniffer capture to check the packet hitting the CPU; last option in that link:http://www.cisco.com/en/US/products/hw/switches/ps663/products_tech_note09186a00804cef15.shtml#tool2

Regarding your query on spanning tree root bridges - There should be only one root bridge per vlan.

And allowed vlan on trunks - It should be same across the core and distribution links, check the logical topology for one or two vlans for better understanding.

HTH.

Regards,

RS.

[_networker_]

Hello,

i`ve done "debug platform packet all receive buffer" but the output is unexpected, because i expected to see the Event "Host Learning"!?

There`re many Packets displayed from the same Interface/RxVlan and Destination IP, but the Event is "SA Miss"! The Source-IP is always different out of our network. SrcMac is from the SVI this switch uses in that VLAN, dst Mac and DST-IP is unknown to me (It's not used)

What does that mean?

Is that normal behaviour that the Device is arping the dst-IP int this vlan, but nobody answers?

How can i interpret the data from the debug any further?