Hello,
I have a Cisco 3850 stack with 2 switches, 3.7.0E IOS, and they are experiencing high CPU peak.
Peaks are cyclic and take place one a minute during 15 seconds more or less, I have checked processes:
111111111111111
6666600000000000000066666
211111333336666600000000000000077777222222222233333111112222
100 ***************
90 ***************
80 ***************
70 *************************
60 *************************
50 *************************
40 *************************
30 *************************
20 *************************
10 *************************
0....5....1....1....2....2....3....3....4....4....5....5....
0 5 0 5 0 5 0 5 0 5
CPU% per second (last 60 seconds)
PCC-COM-SWT#show processes cpu sorted | exclude 0.00
Core 0: CPU utilization for five seconds: 6%; one minute: 8%; five minutes: 21%
Core 1: CPU utilization for five seconds: 68%; one minute: 20%; five minutes: 15%
Core 2: CPU utilization for five seconds: 1%; one minute: 20%; five minutes: 10%
Core 3: CPU utilization for five seconds: 1%; one minute: 6%; five minutes: 5%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
10639 1905347 13642207 1147 17.09 8.93 10.23 0 iosd
5943 813160 35212827 51 1.89 1.47 1.53 1088 fed
5945 1943420 41293167 47 0.19 0.07 0.08 0 stack-mgr
6498 1863200 12307674 151 0.10 0.25 0.26 0 ffm
5944 1032440 58072118 17 0.05 0.07 0.06 0 platform_mgr
6501 1006550 1134373 887 0.05 0.05 0.05 0 cpumemd
10634 2831480 23240118 12 0.05 0.12 0.13 0 wcm
PCC-COM-SWT#show processes cpu detailed process iosd | exclude 0.00
Core 0: CPU utilization for five seconds: 3%; one minute: 8%; five minutes: 20%
Core 1: CPU utilization for five seconds: 99%; one minute: 32%; five minutes: 18%
Core 2: CPU utilization for five seconds: 3%; one minute: 17%; five minutes: 10%
Core 3: CPU utilization for five seconds: 3%; one minute: 7%; five minutes: 6%
PID T C TID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
(%) (%) (%)
10639 L 1910407 136422258 1147 24.8 10.2 10.4 0 iosd
10639 L 1 10639 2044974 121471752 0 24.5 9.89 10.2 0 iosd
10639 L 0 11578 3849380 10222258 0 0.34 0.27 0.23 0 iosd.fastpath
7 I 1915240 150788 0 0.66 0.33 0.33 0 Check heaps
22 I 1223640 2932338 0 0.77 0.33 0.22 0 CMI IOSd task
289 I 1334423 4678902 0 96.7 37.3 37.7 0 IGMP Input
358 I 2153310 56211 0 0.77 0.44 0.44 0 QoS stats process
I checked IGMP traffic and even though there isn´t so much IGMP traffic received, between 40 and 50 groups, each time he sends a queries the CPU goes up until all reports are processed. I have already checked but I found no issues/bugs related
Any ideas on what may be going on?
Thanks!
