03-15-2011 09:09 AM
Hello
We are seeing 100% CPU via GUI and "show system resource"
Software Version: 5.0(4)
SE-SANSW002# show system resources
Load average: 1 minute: 1.23 5 minutes: 1.28 15 minutes: 1.27
Processes : 174 total, 2 running
CPU states : 62.4% user, 37.6% kernel, 0.0% idle
Memory usage: 1033004K total, 518296K used, 514708K free
SE-SANSW002# show processes cpu sort
PID Runtime(ms) Invoked uSecs 1Sec Process
----- ----------- -------- ----- ------ -----------
3023 102917 0 0 71.5% ivr
2798 59896 0 0 15.5% fcns
2780 17441 0 0 5.8% acl
2835 10238 0 0 3.9% arbiter
1 42108 0 0 0.0% init
2 15561 0 0 0.0% ksoftirqd/0
3 2082 0 0 0.0% desched/0
4 915 0 0 0.0% events/0
5 27 0 0 0.0% khelper
10 0 0 0 0.0% kthread
25 816 0 0 0.0% kblockd/0
43 0 0 0 0.0% khubd
86 4692 0 0 0.0% pdflush
87 6138 0 0 0.0% pdflush
88 0 0 0 0.0% kswapd0
89 0 0 0 0.0% aio/0
661 0 0 0 0.0% exec-osm/0
1038 224 0 0 0.0% kjournald
1043 105 0 0 0.0% kjournald
1585 2294 0 0 0.0% kjournald
1592 2 0 0 0.0% kjournald
1606 4270 0 0 0.0% kjournald
1802 3 0 0 0.0% portmap
1808 13 0 0 0.0% rpc.statd
1823 75 0 0 0.0% nfsd
1824 78 0 0 0.0% nfsd
1825 135 0 0 0.0% nfsd
1826 133 0 0 0.0% nfsd
1827 117 0 0 0.0% nfsd
1828 113 0 0 0.0% nfsd
1829 162 0 0 0.0% nfsd
1830 162 0 0 0.0% nfsd
1831 0 0 0 0.0% lockd
1832 0 0 0 0.0% rpciod
1837 10 0 0 0.0% rpc.mountd
1841 43149 0 0 0.0% sysmgr
2098 0 0 0 0.0% mping-thread
2150 3358 0 0 0.0% redun_kthread
2176 0 0 0 0.0% usd_mts_kthread
2205 11896 0 0 0.0% ls-notify-mts-t
2300 2572 0 0 0.0% sdip-mts-thread
2390 44075 0 0 0.0% kfu_fsm-app-137
2391 0 0 0 0.0% kfu_mts-app-137
2430 49 0 0 0.0% tftpd
2431 33964 0 0 0.0% syslogd
2432 243 0 0 0.0% sdwrapd
2433 48073 0 0 0.0% platform
2490 15912 0 0 0.0% xbar_driver_usd
2491 75275 0 0 0.0% babycaesar
2492 35 0 0 0.0% pfm_dummy
2683 8 0 0 0.0% klogd
2701 11955 0 0 0.0% syslogd
2703 2455 0 0 0.0% syslogd
2708 10237 0 0 0.0% vshd
2709 103 0 0 0.0% smm
2710 4042 0 0 0.0% lmgrd
2711 98201 0 0 0.0% licmgr
2712 29965 0 0 0.0% fs-daemon
2713 23845 0 0 0.0% feature-mgr
2714 11214 0 0 0.0% fcfwd
2715 91 0 0 0.0% confcheck
2716 18959 0 0 0.0% capability
2717 40185 0 0 0.0% aam
2720 12817 0 0 0.0% smm
2721 8686 0 0 0.0% smm:active-time
2728 25101 0 0 0.0% cisco
2734 73 0 0 0.0% smm:smm-main-th
2735 46 0 0 0.0% smm:smm-sysmgr-
2736 19242 0 0 0.0% clis
2742 62639 0 0 0.0% wwn
2743 54837 0 0 0.0% vsan
2744 23496 0 0 0.0% ttyd
2745 5968 0 0 0.0% sysinfo
2746 20575 0 0 0.0% span
2747 32 0 0 0.0% sksd
2748 92395 0 0 0.0% sensor_usd
2749 42319 0 0 0.0% scheduler
2751 4258 0 0 0.0% plugin
2752 14769 0 0 0.0% plog_sup
2753 13 0 0 0.0% pktmgr_dummy
2754 91049 0 0 0.0% ntp
2755 76896 0 0 0.0% smm:smm-cli-thr
2758 13 0 0 0.0% netstack_dummy
2759 13802 0 0 0.0% mvsh
2760 19 0 0 0.0% mping_server
2761 78187 0 0 0.0% memmon
2765 14 0 0 0.0% ip_dummy
2768 92884 0 0 0.0% idehsd
2770 41266 0 0 0.0% fc2d
2771 13261 0 0 0.0% evms
2772 10331 0 0 0.0% evmc
2773 60916 0 0 0.0% dev_log_sup
2774 1098 0 0 0.0% dcos-xinetd
2775 5121 0 0 0.0% crdcfg_server
2776 163 0 0 0.0% core-dmon
2777 17357 0 0 0.0% cimserver
2778 8681 0 0 0.0% bios_daemon
2779 8064 0 0 0.0% ascii-cfg
2782 54660 0 0 0.0% SystemHealth
2783 64 0 0 0.0% securityd
2784 44904 0 0 0.0% cert_enroll
2785 44900 0 0 0.0% aaa
2786 10034 0 0 0.0% obfl
2787 32835 0 0 0.0% cdp
2788 77542 0 0 0.0% rib
2790 65075 0 0 0.0% radius
2791 13 0 0 0.0% ipv6_dummy
2795 12175 0 0 0.0% device-alias
2796 45058 0 0 0.0% epp
2797 35262 0 0 0.0% fcdomain
2799 3608 0 0 0.0% rdl
2800 14464 0 0 0.0% rlir
2801 18355 0 0 0.0% rscn
2805 12 0 0 0.0% getty
2808 19 0 0 0.0% gettylogin1
2815 39957 0 0 0.0% fspf
2816 13036 0 0 0.0% fspf
2817 4 0 0 0.0% fspf
2818 127 0 0 0.0% fspf
2819 28538 0 0 0.0% flogi
2820 3359 0 0 0.0% scsi-target
2824 71101 0 0 0.0% fcs
2825 27577 0 0 0.0% ipfc
2841 12089 0 0 0.0% fcc
2849 28628 0 0 0.0% port-channel
2851 98447 0 0 0.0% ipconf
2852 21243 0 0 0.0% dstats
2853 53357 0 0 0.0% cfs
2855 67682 0 0 0.0% vrrp-eng
2858 32488 0 0 0.0% vrrp-cfg
2859 31002 0 0 0.0% zone
2861 12161 0 0 0.0% ohms_sup
2862 15689 0 0 0.0% ohms_sup
2949 64709 0 0 0.0% snmpd
2957 54708 0 0 0.0% callhome
2995 20223 0 0 0.0% xbar_client
3001 20592 0 0 0.0% xbar
3003 22866 0 0 0.0% ethport
3004 18023 0 0 0.0% port-resources
3008 968 0 0 0.0% port
3012 53993 0 0 0.0% fdmi
3014 65602 0 0 0.0% module
3015 13759 0 0 0.0% qos
3022 4435 0 0 0.0% fc-redirect
3029 85 0 0 0.0% ExceptionLog
3030 23956 0 0 0.0% bootvar
3032 17382 0 0 0.0% tcap
3033 21107 0 0 0.0% vni
3034 12404 0 0 0.0% expl
3035 0 0 0 0.0% expl
3036 64035 0 0 0.0% fc-tunnel
3037 39252 0 0 0.0% ipacl
3107 9 0 0 0.0% getty
3265 13419 0 0 0.0% clis
3266 28 0 0 0.0% clis:clis-cli-t
3267 47 0 0 0.0% clis:clis-nvdb-
3268 5 0 0 0.0% clis:clis-seria
3359 34154 0 0 0.0% wdpunch_thread
13802 657 0 0 0.0% dcos_sshd
13805 192 0 0 0.0% vsh
13864 7 0 0 0.0% more
13865 3 0 0 0.0% vsh
13866 43 0 0 0.0% ps
30844 14 0 0 0.0% tcpudp_dummy
30845 96445 0 0 0.0% mcast
30846 11474 0 0 0.0% fscm
30847 115 0 0 0.0% fcanalyzer
30889 222 0 0 0.0% dhcpd
30894 0 0 0 0.0% rpcapd
30898 28 0 0 0.0% lmreread
30906 58527 0 0 0.0% ntpd
30919 65496 0 0 0.0% thttpd
30920 10970 0 0 0.0% snmpd
30921 25615 0 0 0.0% snmpd
31038 32573 0 0 0.0% mts-sync-thr
CPU util : 65.3% user, 34.7% kernel, 0.0% idle
Please note that CPU util is per switch, whereas process cpu util is per cpu
E-SANSW002# show processes memory
PID MemAlloc MemLimit MemUsed StackBase/Ptr Process
----- -------- ---------- ---------- ----------------- ----------------
1 147456 0 1667072 7ffffe50/7ffff950 init
2 0 0 0 0/0 ksoftirqd/0
3 0 0 0 0/0 desched/0
4 0 0 0 0/0 events/0
5 0 0 0 0/0 khelper
10 0 0 0 0/0 kthread
25 0 0 0 0/0 kblockd/0
43 0 0 0 0/0 khubd
86 0 0 0 0/0 pdflush
87 0 0 0 0/0 pdflush
88 0 0 0 0/0 kswapd0
89 0 0 0 0/0 aio/0
661 0 0 0 0/0 exec-osm/0
1038 0 0 0 0/0 kjournald
1043 0 0 0 0/0 kjournald
1585 0 0 0 0/0 kjournald
1592 0 0 0 0/0 kjournald
1606 0 0 0 0/0 kjournald
1802 167936 0 1916928 7ffffd80/7ffffc70 portmap
1808 167936 0 2052096 7ffffd70/7ffffb10 rpc.statd
1823 0 0 0 0/0 nfsd
1824 0 0 0 0/0 nfsd
1825 0 0 0 0/0 nfsd
1826 0 0 0 0/0 nfsd
1827 0 0 0 0/0 nfsd
1828 0 0 0 0/0 nfsd
1829 0 0 0 0/0 nfsd
1830 0 0 0 0/0 nfsd
1831 0 0 0 0/0 lockd
1832 0 0 0 0/0 rpciod
1837 188416 0 2097152 7ffffd70/7ffffb90 rpc.mountd
1841 7151616 0 19128320 7ffffd30/7fffee70 sysmgr
2098 0 0 0 0/0 mping-thread
2150 0 0 0 0/0 redun_kthread
2176 0 0 0 0/0 usd_mts_kthread
2205 0 0 0 0/0 ls-notify-mts-t
2300 0 0 0 0/0 sdip-mts-thread
2390 0 0 0 0/0 kfu_fsm-app-137
2391 0 0 0 0/0 kfu_mts-app-137
2430 430080 40454937 19079168 7ffffa60/7ffff8d0 tftpd
2431 5926912 75555392 43229184 7ffffa30/7ffff4a0 syslogd
2432 1290240 56781894 27144192 7ffffa70/7fffe9a0 sdwrapd
2433 5005312 0 37007360 7ffffa70/7fff8040 platform
2490 1208320 29454937 24829952 7ffffa80/7ffff3e0 xbar_driver_usd
2491 696320 29454937 23969792 7ffffa80/7ffff880 babycaesar
2492 630784 39975513 26337280 7ffffa60/7fffdfd0 pfm_dummy
2683 155648 0 1667072 7ffffa50/7ffff980 klogd
2701 5926912 0 26451968 7ffffa30/100e9ac0 syslogd
2703 5926912 0 26451968 7ffffa30/31460940 syslogd
2708 2560000 253125171 31023104 7ffffa80/7ffff220 vshd
2709 3096576 315924364 37728256 7ffffa20/7ffff550 smm
2710 475136 41523609 19632128 7ffff9e0/7fffe710 lmgrd
2711 946176 38772518 25628672 7ffffa50/7ffff600 licmgr
2712 733184 34873344 26464256 7ffffa80/7ffff290 fs-daemon
2713 733184 32611532 24522752 7ffffa50/7fff34a0 feature-mgr
2714 569344 25871788 22515712 7ffffa80/7ffff830 fcfwd
2715 663552 32341196 24154112 7ffffa60/7ffff8a0 confcheck
2716 1323008 40480140 27324416 7ffffa60/7fffea40 capability
2717 2957312 143810022 32104448 7ffffa80/7fffe630 aam
2720 3096576 0 20951040 7ffffa20/1005bb90 smm
2721 3096576 0 20951040 7ffffa20/300ae8d0 smm:active-time
2728 647168 0 3203072 7ffff9b0/7ffff870 cisco
2734 3096576 0 20951040 7ffffa20/31130a10 smm:smm-main-th
2735 3096576 0 20951040 7ffffa20/311b1a10 smm:smm-sysmgr-
2736 7413760 613537971 60370944 7ffffa20/7ffff3a0 clis
2742 1179648 49614822 25194496 7ffffa70/7ffff7e0 wwn
2743 1839104 129926438 29089792 7ffffa70/7ffff620 vsan
2744 901120 57887104 32620544 7ffffaa0/7ffff3f0 ttyd
2745 770048 25331116 22032384 7ffffa70/7ffff630 sysinfo
2746 2142208 65620544 30453760 7ffffa80/7fffe600 span
2747 610304 38664384 25149440 7ffffa80/7ffff5a0 sksd
2748 1294336 38847283 30597120 7ffffa60/7fffe5e0 sensor_usd
2749 1175552 262755468 29007872 7ffffa60/7ffff490 scheduler
2751 1507328 41547968 27885568 7ffffa70/7fffe600 plugin
2752 598016 79956518 22683648 7ffffa60/7ffff1e0 plog_sup
2753 548864 45780556 21618688 7ffffa20/7ffff850 pktmgr_dummy
2754 1777664 314577190 36548608 7ffffa40/7fffee10 ntp
2755 3096576 0 20951040 7ffffa20/312314f0 smm:smm-cli-thr
2758 548864 45780556 21618688 7ffffa10/7ffff840 netstack_dummy
2759 1318912 123200915 27144192 7ffffa50/7fffc0c0 mvsh
2760 548864 29547724 21618688 7ffffa50/7ffff830 mping_server
2761 548864 56888691 21716992 7ffffa70/7ffff770 memmon
2765 548864 45780556 21618688 7ffffa30/7ffff860 ip_dummy
2768 569344 30917427 22859776 7ffffa90/7ffff770 idehsd
2770 1396736 39916940 26988544 7ffffa80/7fffe600 fc2d
2771 1429504 43426803 29609984 7ffffa50/7fffe5f0 evms
2772 1445888 40782016 27287552 7ffffa50/7fffe5e0 evmc
2773 983040 86201280 28495872 7ffffa50/7fffc390 dev_log_sup
2774 1327104 40454937 32010240 7ffff990/7ffff780 dcos-xinetd
2775 1032192 424478067 35885056 7ffffa50/7fffeda0 crdcfg_server
2776 745472 35427532 26963968 7ffffa80/7ffff4c0 core-dmon
2777 782336 116205056 25772032 7ffffa40/7fffd1e0 cimserver
2778 704512 41523609 26259456 7ffffa70/7fffdf10 bios_daemon
2779 1593344 316352396 39649280 7ffffa50/7ffff1a0 ascii-cfg
2780 1138688 53417548 28758016 7ffffa80/7ffff490 acl
2782 11132928 90657318 42491904 7ffffa60/7fffe940 SystemHealth
2783 3182592 40960409 32464896 7ffffa40/7fffe610 securityd
2784 1527808 47962112 38449152 7ffffa50/7fffe810 cert_enroll
2785 983040 35314892 27762688 7ffffa50/7fffe670 aaa
2786 737280 85750720 27951104 7ffffa70/7fffed60 obfl
2787 1581056 150527872 36904960 7ffffa60/7ffff8b0 cdp
2788 2748416 57646969 28827648 7ffffaa0/7ffff960 rib
2790 1794048 68537004 37683200 7ffffa30/7fffe5c0 radius
2791 548864 45780556 21618688 7ffffa20/7ffff850 ipv6_dummy
2795 2916352 86088640 29622272 7ffffa60/7fffe2f0 device-alias
2796 1454080 40655859 27619328 7ffffa80/7fffe840 epp
2797 1720320 93490636 29487104 7ffffa90/7ffff0e0 fcdomain
2798 2342912 57984889 28786688 7ffffa80/7ffff480 fcns
2799 724992 32917913 24805376 7ffffa80/7ffff750 rdl
2800 749568 44948358 25956352 7ffffa80/7ffff680 rlir
2801 1032192 45908051 27099136 7ffffa80/7ffff680 rscn
2805 163840 0 1736704 7ffffd50/7ffffc10 getty
2808 131072 0 2850816 7ffffda0/7fffee50 gettylogin1
2815 6287360 145247308 33492992 7ffffaa0/7ffff0d0 fspf
2816 6287360 0 16715776 7ffffaa0/1005e5b0 fspf
2817 6287360 0 16715776 7ffffaa0/31235e50 fspf
2818 6287360 0 16715776 7ffffaa0/3141ee50 fspf
2819 3231744 66147699 32006144 7ffffa70/7fffe620 flogi
2820 1196032 33323417 25182208 7ffffa50/7ffff490 scsi-target
2824 4317184 57674003 30941184 7ffffa80/7ffff690 fcs
2825 716800 49840102 25391104 7ffffa80/7ffff400 ipfc
2835 745472 37677657 24248320 7ffffa60/7ffff610 arbiter
2841 741376 38195801 24850432 7ffffaa0/7ffff220 fcc
2849 2400256 66120665 30711808 7ffffa60/7fffe3e0 port-channel
2851 2113536 41894899 28876800 7ffffa70/7fffe5e0 ipconf
2852 684032 59533478 24203264 7ffffa70/7ffff700 dstats
2853 9535488 269935564 45834240 7ffffa70/7fffe5b0 cfs
2855 1277952 53309414 29073408 7ffffa60/7fffe510 vrrp-eng
2858 1859584 41489395 28393472 7ffffa70/7fffe590 vrrp-cfg
2859 6873088 146504371 39739392 7ffffa70/7ffff310 zone
2861 11132928 0 25714688 7ffffa60/10136d70 ohms_sup
2862 11132928 0 25714688 7ffffa60/3182be40 ohms_sup
2949 10649600 112592409 66752512 7ffffa10/7fffe400 snmpd
2957 1540096 146513382 33624064 7ffffa40/7ffff810 callhome
2995 1478656 39410483 31109120 7ffffa60/7fffea70 xbar_client
3001 2437120 69761190 33779712 7ffffa80/7fffe420 xbar
3003 1576960 55535180 30617600 7ffffa60/7fffe610 ethport
3004 2068480 41939955 28798976 7ffffa60/7fffe620 port-resources
3008 6291456 94369932 40140800 7ffffa70/7fffe3e0 port
3012 1384448 71245363 25346048 7ffffa80/7ffff540 fdmi
3014 2387968 68724902 33247232 7ffffa70/7fffd1c0 module
3015 1294336 86214796 28717056 7ffffa70/7ffff6f0 qos
3022 5296128 120000108 35024896 7ffffa40/7fffe5c0 fc-redirect
3023 15167488 220089971 53428224 7ffffa60/7fff5470 ivr
3029 10719232 89436300 40353792 7ffffa70/7fffe7b0 ExceptionLog
3030 880640 41881382 28151808 7ffffa70/7ffff390 bootvar
3032 1159168 34877849 26988544 7ffffa80/7fffea20 tcap
3033 782336 40430579 26882048 7ffffa80/7ffff410 vni
3034 10719232 0 23576576 7ffffa70/100a9a80 expl
3035 10719232 0 23576576 7ffffa70/31831940 expl
3036 802816 66696044 25755648 7ffffa60/7ffff430 fc-tunnel
3037 630784 44074272 25059328 7ffffa90/7ffff970 ipacl
3107 163840 0 1736704 7ffffd40/7ffffc00 getty
3265 7413760 0 43593728 7ffffa20/104bf530 clis
3266 7413760 0 43593728 7ffffa20/310b34f0 clis:clis-cli-t
3267 7413760 0 43593728 7ffffa20/311356f0 clis:clis-nvdb-
3268 7413760 0 43593728 7ffffa20/311b6960 clis:clis-seria
3359 0 0 0 0/0 wdpunch_thread
13802 1028096 0 10395648 7ffffa30/7ffff360 dcos_sshd
13805 1527808 0 35438592 7ffffd80/7fffa770 vsh
13893 196608 0 3731456 7ffffc90/7ffffa20 more
13894 1527808 0 35598336 7ffffd80/7fffa1f0 vsh
13895 1015808 0 2695168 7ffffa00/7fffef80 ps
30844 548864 45780556 21618688 7ffffa20/7ffff850 tcpudp_dummy
30845 622592 37474905 24068096 7ffffa90/7ffff110 mcast
30846 704512 34129920 25780224 7ffffa60/7ffff6b0 fscm
30847 700416 40454937 19349504 7ffffa60/7fffeae0 fcanalyzer
30889 466944 0 2785280 7ffffa00/7ffff690 dhcpd
30894 700416 0 2572288 7ffffa60/7fffec20 rpcapd
30898 0 0 0 0/0 lmreread
30906 1122304 0 15020032 7ffffee0/7ffffd30 ntpd
30919 450560 0 2215936 7ffffea0/7fffb4a0 thttpd
30920 10649600 0 49975296 7ffffa10/105aaac0 snmpd
30921 10649600 0 49975296 7ffffa10/310b5bd0 snmpd
31038 0 0 0 0/0 mts-sync-thr
All processes: MemAlloc = 371924992
Please Advise
Thanks
Chetan.R
03-15-2011 11:17 AM
Look like ivr and fcns process acting.
3023 102917 0 0 71.5% ivr
2798 59896 0 0 15.5% fcns
You can try to do "system switchover" to refresh these process or log a support SR
03-20-2011 10:13 PM
Customer removed an IVR zone after which the CPU utilization isn't high anymore. However do you still recommend to do a system switchover and then add the IVR zone and check for performance.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide