cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
8620
Views
0
Helpful
4
Replies

High CPU utilization on a stack of 3750E

f_flagothier
Level 1
Level 1

Hello everyone,

I have a very strange problem on a stack of 8 Catalyst 3750E-48.

The configuration is however pretty simple.

Two VLANS (20 and 24) with inter-vlan routing (no ACL, QoS, whatsoever).

Machines in the the Vlan 20 can see each others with no problem. Same in Vlan 24.

However machines from Vlan 20 can see the ones from Vlan 24 only intermittently!

There is nothing in the log (no interface going up or down that could explain this)

I had a look at the CPU and it seems kinda high although I'm not sure this is the cause of my problems...

CPU utilization for five seconds: 45%/26%; one minute: 40%; five minutes: 45%

PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

  91    17770597  17477172       1016  7.18%  5.83%  5.85%   0 HLFM address lea

111    13703094   1203510      11385  2.07%  1.75%  1.73%   0 hpm counter proc

151     5449427  18566548        293  0.95%  0.77%  0.99%   0 Hulc LED Process

162     6082598   2006746       3031  0.63%  0.76%  0.75%   0 HRPC qos request

327     3248183   1627290       1996  0.63%  0.50%  0.51%   0 SNMP ENGINE

161     3099356    126352      24529  0.47%  0.38%  0.37%   0 HQM Stack Proces

197     2148272   4136461        519  0.47%  0.55%  0.54%   0 ADJ resolve proc

337       23342      4248       5494  0.31%  0.04%  0.53%   1 Virtual Exec

325     1299928   3259491        398  0.31%  0.23%  0.18%   0 IP SNMP

  46     1283570    145905       8797  0.31%  0.16%  0.16%   0 Compute load avg

235      564101    713775        790  0.15%  0.14%  0.14%   0 IPC LC Message H

193      776933   2715979        286  0.15%  0.14%  0.20%   0 IP Input

  39     1095189    657808       1664  0.15%  0.12%  0.13%   0 Per-Second Jobs

112     2258147   7973142        283  0.15%  0.19%  0.24%   0 HRPC pm-counters

  75      445786    178510       2497  0.15%  0.15%  0.12%   0 Adjust Regions

152      413958    466565        887  0.15%  0.09%  0.09%   0 HL3U bkgrd proce

  76     3404112    630469       5399  0.15%  0.04%  0.04%   0 hrpc -> response

  17           0         1          0  0.00%  0.00%  0.00%   0 IFS Agent Manage

  16      888674  24596449         36  0.00%  0.00%  0.00%   0 EEM ED Syslog

  15         200        17      11764  0.00%  0.00%  0.00%   0 Entity MIB API

  18         303     10565         28  0.00%  0.00%  0.00%   0 IPC Dynamic Cach

  14           0         1          0  0.00%  0.00%  0.00%   0 Policy Manager

It seems that there are lot of interrupts and I cannot see why

On this page, Cisco says that it might be caused by "punted packets"

http://cisco.biz/en/US/docs/switches/lan/catalyst3750/software/troubleshooting/cpu_util.html#wp1028395

The sdm is set to "desktop routing" and there is no TCAM exhaustion :

MyEVS#sh platform tcam utilization

CAM Utilization for ASIC# 0                      Max            Used

                                             Masks/Values    Masks/values

Unicast mac addresses:                       3292/3292        447/447

IPv4 IGMP groups + multicast routes:         1120/1120          1/1

IPv4 unicast directly-connected routes:      3072/3072        150/150

IPv4 unicast indirectly-connected routes:    8144/8144        102/102

IPv4 policy based routing aces:               498/498          13/13

IPv4 qos aces:                                474/474          21/21

IPv4 security aces:                           972/972          32/32

The switch is on its own and as I said the L3 config is very simple (all network directly connected)

MyEVS#sh ip route

     172.16.0.0/24 is subnetted, 3 subnets

C       172.16.202.0 is directly connected, FastEthernet0

C       172.16.24.0 is directly connected, Vlan24

C       172.16.20.0 is directly connected, Vlan20

MyEVS#

Maybe is it caused by a lot of multicast traffic ? (I know some application intensively used on this network is based on multicast)

But is my CPU hogged and could drop traffic between Vlan because of that ?

If not.. have you any idea where it could come from ?

Thank you very much for any help

1 Accepted Solution

Accepted Solutions

podhillo
Level 1
Level 1

These can be the probable causes of High CPU utilization.

3750 High CPU in HRPC qos request

IOS 12.2(44)SE4/SE5 - AAA memory allocation failure, high CPU or crash.

IP address assignment to a shutdown port causes high CPU

High cpu in HLFM address lea with MAC address learning disabled

High CPU indicated by 'Collection process' on Cat3750 after enable IPv6

High CPU usage due to renewal of DHCP Snooping database

These all are IOS bugs.

View solution in original post

4 Replies 4

Ganesh Hariharan
VIP Alumni
VIP Alumni

Hello everyone,

I have a very strange problem on a stack of 8 Catalyst 3750E-48.

The configuration is however pretty simple.

Two VLANS (20 and 24) with inter-vlan routing (no ACL, QoS, whatsoever).

Machines in the the Vlan 20 can see each others with no problem. Same in Vlan 24.

However machines from Vlan 20 can see the ones from Vlan 24 only intermittently!

There is nothing in the log (no interface going up or down that could explain this)

I had a look at the CPU and it seems kinda high although I'm not sure this is the cause of my problems...

CPU utilization for five seconds: 45%/26%; one minute: 40%; five minutes: 45%

PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

  91    17770597  17477172       1016  7.18%  5.83%  5.85%   0 HLFM address lea

111    13703094   1203510      11385  2.07%  1.75%  1.73%   0 hpm counter proc

151     5449427  18566548        293  0.95%  0.77%  0.99%   0 Hulc LED Process

162     6082598   2006746       3031  0.63%  0.76%  0.75%   0 HRPC qos request

327     3248183   1627290       1996  0.63%  0.50%  0.51%   0 SNMP ENGINE

161     3099356    126352      24529  0.47%  0.38%  0.37%   0 HQM Stack Proces

197     2148272   4136461        519  0.47%  0.55%  0.54%   0 ADJ resolve proc

337       23342      4248       5494  0.31%  0.04%  0.53%   1 Virtual Exec

325     1299928   3259491        398  0.31%  0.23%  0.18%   0 IP SNMP

  46     1283570    145905       8797  0.31%  0.16%  0.16%   0 Compute load avg

235      564101    713775        790  0.15%  0.14%  0.14%   0 IPC LC Message H

193      776933   2715979        286  0.15%  0.14%  0.20%   0 IP Input

  39     1095189    657808       1664  0.15%  0.12%  0.13%   0 Per-Second Jobs

112     2258147   7973142        283  0.15%  0.19%  0.24%   0 HRPC pm-counters

  75      445786    178510       2497  0.15%  0.15%  0.12%   0 Adjust Regions

152      413958    466565        887  0.15%  0.09%  0.09%   0 HL3U bkgrd proce

  76     3404112    630469       5399  0.15%  0.04%  0.04%   0 hrpc -> response

  17           0         1          0  0.00%  0.00%  0.00%   0 IFS Agent Manage

  16      888674  24596449         36  0.00%  0.00%  0.00%   0 EEM ED Syslog

Hi,

As per the out put the CPU utilization is around 45 % avg any how check out the below link for possible cause  for high cpu in stack based switches 3750.

http://www.cisco.com/en/US/products/hw/switches/ps5023/products_tech_note09186a00807213f5.shtml

Hope to Help !!

Ganesh.H

podhillo
Level 1
Level 1

Please update the IOS version currently runing on the stack..

podhillo
Level 1
Level 1

These can be the probable causes of High CPU utilization.

3750 High CPU in HRPC qos request

IOS 12.2(44)SE4/SE5 - AAA memory allocation failure, high CPU or crash.

IP address assignment to a shutdown port causes high CPU

High cpu in HLFM address lea with MAC address learning disabled

High CPU indicated by 'Collection process' on Cat3750 after enable IPv6

High CPU usage due to renewal of DHCP Snooping database

These all are IOS bugs.

what are these bug-id?

Review Cisco Networking for a $25 gift card