High CPU Utilization on Cisco IOS for Catalyst 4500

[Claudio Cubillos]

Hi all,

We have a Switch Catalyst 4500, model WS-C4510R+E, with 2 Supervisors, models V-10GE, which is showing high CPU values all times.

When this occures, the switch shows 95-100% for total cpu for a few hours at a time.

show proccesses cpu shows process Cat4k Mgmt LoPri with an unusual cpu value of 56%

The show platform heath command gave me a high reading for K2L2 Address Table R which is curently 9,45% but the last hour is around a 20% average CPU

I've read the document http://www.cisco.com/en/US/products/hw/switches/ps663/products_tech_note09186a00804cef15.shtml

But none of the mentioned common reasons seems to be the cause.

Does anyone know what the process K2L2 Address Table R means and what could be the cause of this peak?

Thanks in advance.

[acampbell]

Claudio,

Your issue is most likely to be being caused by
Host MAC Address flapping.

To help you locate the issue enable mac address-table notification change.

This will let you see if MAC flap alarms are being logged.

conf t
!
mac address-table notification change mac-move
!
end

show log

look for alarms like:-

"%C4K_EBM-4-HOSTFLAPPING".......

This will alert to mac addresses that are appearing on
more than 1 interface.

The usual causes are wrongly configured etherchannels
but not always at the switc side so you may need to talk with
your server team if you locate the issue to server nics etc.

see this post
https://supportforums.cisco.com/docs/DOC-2116

Hope this helps

Regards,
Alex.
Please rate useful posts.

[Kyujin Choi]

 Thanks for this information. I just made one etherchannel, I don't see any problem. but I found iosd after I made an etherchannel from 4506. So I am curious why you mentioned etherchannel problem. 

 Currently two interfaces are joined as one etherchannel. 

 Update-------------

 I disabled new etherchannel, but I could see still iosd. So it was not related to etherchannel in my case. 

[ixholla69]

I have 2 4500E's one with a Sup6E and another with two Sup7E Engines.

I monitor their CPU Utilization and noticed that I see a strange 1 hour interval every 6 hours or so that increases my 2nd 4500 with the Sup7E's to almost 90%!!! it then drops down to normal. It does this like I say, every 6 hours.

My 4500 with the Sup6E does not do this.

[InayathUlla Sharieff]

Hi Claudio,

We need to look whats happening on the switch during the peak?

Is there any chance for you to log a CIsco TAC case so that we login to your device and check what causing the CPU to go high?

Else here is TS method :

====================

1- Check the CPU utilization (Show proc cpu sorted | ex 0.00) >>Check which process is utilizing more CPU usage.

2- Check STP to see if there are any loops (show spann det | in ieee|from|occur|is exec) >> Check for any STP changes for VLAN , If yes then track that vlan and see which inteface is sending and recieving high packets or any drops on that particular interface.

3-Check the CPU packets Stats,

4-Use the following inbuilt non impacting debug to find out the packets going to the CPU(menioned in the same high CPU troubleshooting document)

Switch#debug platform packet all receive buffer

platform packet debugging is on

Switch#show platform cpu packet buffered

Focus on finding any common traits in the packets like srcmac etc.Once we know the bulk of the traffic goingto the CPU the source can be shut off

to bring the CPU down.

Hope that helps,

REgards

Inayath

*Plz rate all usefull posts.

[avvotyakov]

hi all, i have a same problem, but maybe i now the reson this problem. i have 6 4500(4-4510R+E sup7-e(dual sup), 2-4507R+E sup7-le(single sup)). managment interfaces sup(Out-of-Band interfaces) i connect to same switch in vlan X, and some time i see strange situation, if managment interface receiev pacet whith dst ip addres in same network that managment interfaces (but dst mac addres not addres managment interfaces) and have arp dst ip, managment interfaces route this pacet with decrease ttl and change src mac adres to own, and we have l3 loop!!  

[subhojithalder198]

Hi,

Pls check the TCAM Utilization on the switch

[jockew]

A customer of mine has two 4507R+E with 1 Sup7-E in them.

We had an issue last year with high CPU on both of them. We found that via sh proc cpu there was an error in the calculation of th total CPU.

There is a proicess called iosd, this process starts a lot of forked processes. When we did a sh proc cpu on the switches the iosd process had alreade added all the child/forked processes but the sh proc cpu command added all the child/firked processes once more to the calculation of the total CPU.

Regards.

[gerrit vink]

Jockew,

We have the same issue with two 4500 switches , around 55 % cpu time only for the IOSD proces.

Can you tell us how you fixt this issue ?

Catalyst 4500 L3 Switch Software (cat4500e-UNIVERSALK9-M), Version 03.04.01.SG RELEASE SOFTWARE (fc3)

[jockew]

There was no fix needed, because this is an cosmetic bug.

The sh proc cpu command adds the cpu usage of all of the child processes of iosd, to the total.

iosd is already an calculated value of all the child processes cpu usage, so the child processes cpu usage gets counted twice.

Regards,

Jocke W

[gerrit vink]

Hi Jocke,

Thanks for your response,

Regards,

Gerrit

[ibrahimbadr4669]

Hi Claudio,

what's the root cause of the high utilization