We have a stack of 3850 Switches which are suffering from high proccessor which causes netwrok outages until a reload.
Seems to be caused when a site vunerability scan is run every week - which is always followed by the processor increasing until getting to 100% eventually and a reboot resolves.
The highest process is around 4%
IOS has been upgraded which slowed the issue down but didnt resolve it.
show tech done prior to reload and some info below:
Any help would be appreciated - TAC also investigating but not very forthcoming
SWITCH#sho processes cpu sorted | exclude 0.0
CPU utilization for five seconds: 63%/53%; one minute: 65%; five minutes: 66%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
227 19189758 23145028 829 5.12% 4.65% 4.61% 0 Spanning Tree
530 6980444 100314727 69 1.12% 1.18% 1.12% 0 IP SLAs XOS Even
73 2565105 6431825 398 0.79% 0.61% 0.69% 0 IOSD ipc task
203 780213 2256556 345 0.31% 0.37% 0.37% 0 IP Input
191 1189410 862820 1378 0.31% 0.30% 0.29% 0 CDP Protocol
117 1599486 6826746 234 0.31% 0.45% 0.47% 0 IOSXE-RP Punt Se
35 2464635 3855755 639 0.31% 0.39% 0.39% 0 ARP Input
228 1247369 4356306 286 0.23% 0.28% 0.27% 0 UDLD
423 20051 6366 3149 0.15% 1.25% 2.73% 2 SSH Process
199 594753 13072577 45 0.15% 0.11% 0.10% 0 IPAM Manager
SWITCH1#sho proc cpu hist
666666666666665555566666666666666666666666666666666666666666
111122222222229999911111111113333300000222222222222222000005
100
90
80
70
60 **********************************************************
50 **********************************************************
40 **********************************************************
30 **********************************************************
20 **********************************************************
10 **********************************************************
0....5....1....1....2....2....3....3....4....4....5....5....6
0 5 0 5 0 5 0 5 0 5 0
CPU% per second (last 60 seconds)
666666666666666666666666666666666666666666666666666666666666
657666777595765776677765445546684565767566679646766638564455
100
90
80
70 ************************ ** *** ************* ***** ***
60 ##########################################################
50 ##########################################################
40 ##########################################################
30 ##########################################################
20 ##########################################################
10 ##########################################################
0....5....1....1....2....2....3....3....4....4....5....5....6
0 5 0 5 0 5 0 5 0 5 0
CPU% per minute (last 60 minutes)
* = maximum CPU% # = average CPU%
666667776691121299999822221211121111111121312222211111121111112121222272
998970007928909556447207519096819998879979096741089988827899881929007551
100 ** *
90 * *****
80 * ******
70 *********** ******
60 ########### ******
50 ########### ******
40 ########### ******
30 ########### *****#* ** * * ** **
20 ###########*****######************************************************
10 ######################################################################
0....5....1....1....2....2....3....3....4....4....5....5....6....6....7..
0 5 0 5 0 5 0 5 0 5 0 5 0
CPU% per hour (last 72 hours)
* = maximum CPU% # = average CPU%
Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
1 34 WS-C3850-24XS 16.3.3 CAT3K_CAA-UNIVERSALK9 INSTALL
* 2 56 WS-C3850-48T 16.3.3 CAT3K_CAA-UNIVERSALK9 INSTALL
3 56 WS-C3850-48T 16.3.3 CAT3K_CAA-UNIVERSALK9 INSTALL
4 56 WS-C3850-48T 16.3.3 CAT3K_CAA-UNIVERSALK9 INSTALL
