08-30-2024 06:50 AM
Hi folks,
3 servers are connected to Nexus 5548 (acting as a L2 TOR switch), some are complaining about traffic slowness, so checked cpu utilization was not bad
You can see for 5 min it is 16%, that is good in case of nexus
TOR01# show processes cpu
CPU utilization for five seconds: 11%/0%; one minute: 15%; five minutes: 16%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
----- ----------- -------- ----- ------ ------ ------ --- -----------
1 7170 32735 0 0.00% 0.00% 0.00% - init
2563 10 30 0 0.00% 0.00% 0.00% - portmap
2572 10 6 1 0.00% 0.00% 0.00% - rpc.statd
2600 0 1 0 0.00% 0.00% 0.00% - rpc.mountd
2623 12670 350417 0 0.00% 0.00% 0.00% - sysmgr
3548 80 16116 0 0.00% 0.00% 0.00% - insmod
4108 10 3 3 0.00% 0.00% 0.00% - xinetd
4109 10 4 2 0.00% 0.00% 0.00% - tftpd
4110 4950 171320 0 0.00% 0.00% 0.00% - syslogd
4111 30 25 1 0.00% 0.00% 0.00% - sdwrapd
4112 3604950 881654 4 1.06% 1.14% 1.12% - pfma
4136 70 1581 0 0.00% 0.00% 0.00% - klogd
4182 930 41882 0 0.00% 0.00% 0.00% - vshd
4183 130 6835 0 0.00% 0.00% 0.00% - smm
4184 1630 161625 0 0.00% 0.00% 0.00% - psshelper
4185 230 4213 0 0.00% 0.00% 0.00% - mmode
4186 150 2758 0 0.00% 0.00% 0.00% - lmgrd
4187 1200 36891 0 0.00% 0.00% 0.00% - licmgr
4188 76330 23373 3 0.00% 0.21% 0.15% - fs-daemon
4189 16580 287001 0 0.00% 0.00% 0.00% - feature-mgr
4190 530 32391 0 0.00% 0.00% 0.00% - fcfwd
4191 870 5534 0 0.00% 0.00% 0.00% - confcheck
4192 570 24083 0 0.00% 0.00% 0.00% - capability
4193 470 59659 0 0.00% 0.00% 0.00% - bloggerd
4194 1600 161639 0 0.00% 0.00% 0.00% - psshelper_gsvc
4204 1040 5607 0 0.00% 0.00% 0.00% - cisco
4206 16460 1504672 0 0.00% 0.00% 0.00% - clis
4235 2420 37738 0 0.00% 0.00% 0.00% - port-profile
4236 600 25215 0 0.00% 0.00% 0.00% - provision
4244 1000 49502 0 0.00% 0.00% 0.00% - zschk
4245 980 46194 0 0.00% 0.00% 0.00% - xmpp
4246 1040 55799 0 0.00% 0.00% 0.00% - xmlma
4247 940 48338 0 0.00% 0.00% 0.00% - vmm
4248 940 20618 0 0.00% 0.00% 0.00% - vman
4249 1430 58226 0 0.00% 0.00% 0.00% - vdc_mgr
4250 1080 82620 0 0.00% 0.00% 0.00% - ttyd
4251 580 459156 0 0.48% 0.47% 0.47% - sysinfo
4252 7580 55509 0 0.00% 0.00% 0.00% - sunnyvale
4253 10510 96428 0 0.00% 0.00% 0.00% - statsprofiler
4254 50 109 0 0.00% 0.00% 0.00% - sksd
4255 82540 21354 3 0.00% 0.02% 0.02% - sensor
4257 370 5842 0 0.00% 0.00% 0.00% - res_mgr
4258 1710 164036 0 0.00% 0.00% 0.00% - plugin
4259 2100 58236 0 0.00% 0.00% 0.00% - pltfm_config
4260 3300 32610 0 0.00% 0.00% 0.00% - plog_sup
4261 3890 74231 0 0.00% 0.00% 0.00% - pfstat
4262 240 5704 0 0.00% 0.00% 0.00% - patch-installer
4263 690 22159 0 0.00% 0.00% 0.00% - obfl
4264 16320 191411 0 0.00% 0.00% 0.00% - nohms
4265 29910 662958 0 0.00% 0.00% 0.00% - nfp
4266 480 17761 0 0.00% 0.00% 0.00% - mvsh
4267 960 46942 0 0.00% 0.00% 0.00% - mcm
4268 1160 48404 0 0.00% 0.00% 0.00% - lim
4269 2190 163243 0 0.00% 0.00% 0.00% - ledmgr
4270 6290 180342 0 0.09% 0.00% 0.00% - l2rib
4271 6070 71515 0 0.00% 0.00% 0.00% - ifmgr
4272 3310 33456 0 0.00% 0.00% 0.00% - idehsd
4273 490 17345 0 0.00% 0.00% 0.00% - evms
4275 430 13253 0 0.00% 0.00% 0.00% - evmc
4276 7820 59013 0 0.00% 0.01% 0.01% - ethpc
4277 490 11739 0 0.00% 0.00% 0.00% - eth_dstats
4278 200 726 0 0.00% 0.00% 0.00% - core-dmon
4280 3879030 11969037 0 0.38% 1.08% 1.17% - carmelusd
4281 1700 89096 0 0.00% 0.00% 0.00% - bootvar
4282 1730 33659 0 0.00% 0.00% 0.00% - bios_daemon
4283 2060 127933 0 0.00% 0.00% 0.00% - bfdc
4284 36090 2597758 0 8.20% 8.88% 8.88% - bcm_usd
4285 490 16323 0 0.00% 0.00% 0.00% - athertonusd
4286 500 20504 0 0.00% 0.00% 0.00% - ascii-cfg
4287 350 15408 0 0.00% 0.00% 0.00% - adbm
4289 2160 161816 0 0.00% 0.00% 0.00% - securityd
4290 1390 17215 0 0.00% 0.00% 0.00% - PMon
4291 6740 194360 0 0.00% 0.00% 0.00% - cert_enroll
4292 1900 162306 0 0.00% 0.00% 0.00% - aaa
4302 220 26640 0 0.00% 0.00% 0.00% - l3vm
4303 170 174681 0 0.00% 0.00% 0.00% - urib
4319 4680 6184 0 0.00% 0.00% 0.00% - aclmgr
4320 970 49671 0 0.00% 0.00% 0.00% - eem_policy_dir
4321 20 20 1 0.00% 0.00% 0.00% - core-client
4344 257060 868182 0 0.00% 0.07% 0.08% - fwm
4347 8570 155954 0 0.00% 0.00% 0.00% - qd
4348 4248860 1969935 2 0.09% 1.42% 1.47% - statsclient
4351 1787850 6115676 0 0.09% 0.29% 0.62% - snmpd
4364 180 12768758 0 0.00% 0.02% 0.02% - adjmgr
4366 190 52451 0 0.00% 0.00% 0.00% - u6rib
4369 10320 18574 0 0.00% 0.00% 0.00% - copp
4373 200 9342 0 0.00% 0.00% 0.00% - npacl
4415 950 48133 0 0.00% 0.00% 0.00% - sal
4419 300 2938175 0 0.09% 0.17% 0.17% - arp
4421 160 90167 0 0.00% 0.01% 0.01% - icmpv6
4422 370 9953749 0 0.19% 0.18% 0.20% - pktmgr
4437 2290 172817 0 0.00% 0.00% 0.00% - callhome
4450 18020 125021 0 0.00% 0.00% 0.00% - lldp
4459 430 7470785 0 0.09% 0.11% 0.17% - netstack
4514 4110 164405 0 0.00% 0.00% 0.00% - radius
4517 470 8962 0 0.00% 0.00% 0.00% - acllog
4518 10180 89256 0 0.00% 0.00% 0.00% - cdp
4519 91300 1680029 0 0.00% 0.02% 0.02% - cfs
4520 160 16100696 0 0.09% 0.02% 0.02% - ecp
4521 290 209197 0 0.00% 0.00% 0.00% - igmp
4522 20 15 1 0.00% 0.00% 0.00% - ip_dummy
4523 20 9 2 0.00% 0.00% 0.00% - ipv6_dummy
4524 250 426081 0 0.00% 0.00% 0.00% - mrib
4526 960 48650 0 0.00% 0.00% 0.00% - otm
4527 180 68862 0 0.00% 0.00% 0.00% - rpm
4528 20 165 0 0.00% 0.00% 0.00% - tcpudp_dummy
4563 3300 165344 0 0.00% 0.00% 0.00% - tacacs
4569 170 1434 0 0.00% 0.00% 0.00% - dcos-xinetd
4595 1462070 223311 6 0.00% 0.34% 0.43% - afm
4597 520 14112 0 0.00% 0.00% 0.00% - m2rib
4598 73100 1131542 0 0.00% 0.04% 0.03% - stp
4609 1120 48903 0 0.00% 0.00% 0.00% - iscm
4610 240 7446 0 0.00% 0.00% 0.00% - mcastfwd
4619 350 1901 0 0.00% 0.00% 0.00% - ptplc
4622 1290 47992 0 0.00% 0.00% 0.00% - monitor
4623 2170 11626 0 0.00% 0.00% 0.00% - ntp
4625 5660 315749 0 0.00% 0.00% 0.00% - ntpd
4626 1470 35581 0 0.00% 0.00% 0.00% - eth_port_channel
4627 154060 271318 0 0.00% 0.01% 0.04% - vlan_mgr
4628 6620 76411 0 0.00% 0.00% 0.00% - ipqosmgr
4629 640 25152 0 0.00% 0.00% 0.00% - lacp
4630 693940 1574287 0 0.00% 0.08% 0.22% - ethpm
4632 6330 86888 0 0.00% 0.00% 0.00% - interface-vlan
4633 550 13084 0 0.00% 0.00% 0.00% - vpc
4634 520 8637 0 0.00% 0.00% 0.00% - vtp
4636 36480 65929 0 0.00% 0.00% 0.00% - btcm
4637 964540 397653 2 0.38% 0.30% 0.29% - udld
4642 9420 75446 0 0.00% 0.00% 0.00% - ipfib
4643 6010 30183 0 0.00% 0.00% 0.00% - mfdm
4644 2360 53766 0 0.00% 0.00% 0.00% - ufdm
4908 10 32 0 0.00% 0.00% 0.00% S0 getty
4910 10 5 2 0.00% 0.00% 0.00% - sh
4915 40 46 0 0.00% 0.00% 0.00% - libvirtd
26592 550 6574 0 0.00% 0.01% 0.01% - dcos_sshd
26596 1070 3318 0 0.00% 0.04% 0.03% 0 vsh
26956 660 8067 0 0.00% 0.00% 0.00% - dcos_sshd
26968 1110 3032 0 0.00% 0.00% 0.00% 1 vsh
BUT same time when run show processes cpu history this command i find cpu utilization was upto 90%
Am i reading it correct or wrong or how to read these graphs
TOR01# show processes cpu history
3 1 11 1111 111111 21146211111111 1 11211 111 1 1 113111
768818609838447025020946180758213310977417517556917949112064
100
90
80
70
60 #
50 ##
40 # ##
30 # ### # #
20 # # # # ## ##### # ## ### # #
10 ############################################################
0....5....1....1....2....2....3....3....4....4....5....5....
0 5 0 5 0 5 0 5 0 5
CPU% per second (last 60 seconds)
# = average CPU%
533335434484534534356422354353544256355355524534444552454323
477980223343035385533869304010886848244570122415858045877391
100
90
80 *
70 * *
60 * * * * * *
50 * * * * ** * *** * * *** ** ** *** * ****** ***
40 ******* ***** ******** ** * *** ** ****** ** ****** ***
30 ******************************************* ****************
20 #****#****#****#****#****#****#****#****#****#****#****#****
10 ############################################################
0....5....1....1....2....2....3....3....4....4....5....5....
0 5 0 5 0 5 0 5 0 5
CPU% per minute (last 60 minutes)
* = maximum CPU% # = average CPU%
1
76668676687678686768666768676676678767798770
22753498507671929735678767627262160467840840
100 *
90 * * * *
80 * * ** ** * * * * * * ** ***** *
70 * *** ************ ********** * ***********
60 ********************************************
50 ********************************************
40 ********************************************
30 ********************************************
20 *******************************************#
10 ############################################
0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.
0 5 0 5 0 5 0 5 0 5 0 5 0
CPU% per hour (last 72 hours)
* = maximum CPU% # = average CPU%
Please tell me how to read these graphs
Thanks!!!
08-30-2024 06:54 AM
Indeed cpu is high utilize up to 90%
This can slow the traffic to us not traffic pass use tcam
So why traffic is pass to cpu?
One of case is mtu mismatch, check mtu along the path
MHM
08-30-2024 05:03 PM
"BUT same time when run show processes cpu history this command i find cpu utilization was upto 90%
Am i reading it correct or wrong or how to read these graphs"
Actually, it appears in the last 72 hours your CPU peaked at 100%, but doing what, we don't know.
Those graphs show both peak (instantaneous) CPU utilization and average CPU utilization.
As to "slow" traffic, on a switch, CPU utilization doesn't necessarily correlate with data throughput, whether high or low.
For CPU, besides how "busy" it is, we want to know how busy it was doing what (like information from your show processes CPU).
BTW, when using the show processes CPU, often it's useful to use the "sorted" option, if provided, and/or exclude lines with "0.00% 0.00% 0.00%".
For your stats, that would be something like:
4284 36090 2597758 0 8.20% 8.88% 8.88% - bcm_usd
4348 4248860 1969935 2 0.09% 1.42% 1.47% - statsclient
4280 3879030 11969037 0 0.38% 1.08% 1.17% - carmelusd
4112 3604950 881654 4 1.06% 1.14% 1.12% - pfma
4351 1787850 6115676 0 0.09% 0.29% 0.62% - snmpd
4251 580 459156 0 0.48% 0.47% 0.47% - sysinfo
4595 1462070 223311 6 0.00% 0.34% 0.43% - afm
4637 964540 397653 2 0.38% 0.30% 0.29% - udld
4630 693940 1574287 0 0.00% 0.08% 0.22% - ethpm
4422 370 9953749 0 0.19% 0.18% 0.20% - pktmgr
4419 300 2938175 0 0.09% 0.17% 0.17% - arp
4459 430 7470785 0 0.09% 0.11% 0.17% - netstack
4188 76330 23373 3 0.00% 0.21% 0.15% - fs-daemon
4344 257060 868182 0 0.00% 0.07% 0.08% - fwm
4627 154060 271318 0 0.00% 0.01% 0.04% - vlan_mgr
4598 73100 1131542 0 0.00% 0.04% 0.03% - stp
26596 1070 3318 0 0.00% 0.04% 0.03% 0 vsh
4255 82540 21354 3 0.00% 0.02% 0.02% - sensor
4364 180 12768758 0 0.00% 0.02% 0.02% - adjmgr
4519 91300 1680029 0 0.00% 0.02% 0.02% - cfs
4520 160 16100696 0 0.09% 0.02% 0.02% - ecp
4276 7820 59013 0 0.00% 0.01% 0.01% - ethpc
4421 160 90167 0 0.00% 0.01% 0.01% - icmpv6
26592 550 6574 0 0.00% 0.01% 0.01% - dcos_sshd
BTW, totaling those columns roughly matches the summary totals. Also, for those processes, nothing obvious, I believe, that would dramatically slow throughput.
Many things can impact data transfer rates beyond switch CPU utilization. Insufficient information to identify cause of server user performance complaints, or to even verify those complaints are valid.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide