Hi, I am trying to troubloeshoot the the High cpu utilization of N7K. The switch is connected in a POC setup and traffic load is very less. It is running with NXOS 6.0.4. For past 1 month I have observed the CPU utilization goes upto 80% without any traffic/load. I tried to capture the o/p of "show process cpu sort" through EEM script. while analyzing the o/p I can not determine the process which is contributing Th eutilization of user or kernel. for example: In Below out put 25.2% user, 31.0% kernel, 43.8% idle Neither any process shown below, is using 25.2% or 31%, nor the aggrigation of all processes utilizaion is 56.2%. (I have 2 VDCs including default) Please help me how to determine the process consuming cpu. PID   Runtime(ms) Invoked   uSecs 1Sec   Process ----- ----------- -------- ----- ------ ----------- 3296   120582933   8003274 15066   6.7% sac_usd 2924     85729099 13281994   6454   2.8% platform 3588     13699639 11361018   1205   0.9% udld    1       42870   1269054     33   0.0% init    2           3       308     12   0.0% kthreadd    3         1062   167018     6   0.0% migration/0    4       462583 133573370     3   0.0% ksoftirqd/0    5       91988   955848     96   0.0% watchdog/0    6         777   110442     7   0.0% migration/1    7       325174 91256362     3   0.0% ksoftirqd/1    8         3025   952785     3   0.0% watchdog/1    9       36939   6700241     5   0.0% events/0    10       30877   6396271     4   0.0% events/1    11           0       10     19   0.0% khelper    12         170       740   230   0.0% kblockd/0    13         180       731   246   0.0% kblockd/1    14            0         2     0   0.0% kacpid    15           0         2     0   0.0% kacpi_notify    16           28         8   3530   0.0% ksuspend_usbd    17           0         6     3   0.0% khubd    18       31730   778523     40   0.0% pdflush    19       20043   569273     35   0.0% pdflush    20           0         2     3   0.0% kswapd0    21           0         2     3   0.0% aio/0    22           0         2     2   0.0% aio/1    23           0         2     2  0.0% nfsiod    24           0         2     2   0.0% SerrLogKthread    25       63668   279821   227   0.0% kide/0    26       601880   1871286   321   0.0% kide/1    29           0         4     10   0.0% rpciod/0    30           0        17     9   0.0% rpciod/1    31       14621   2521982     5   0.0% kirqd 652           89       22   4070   0.0% jffs2_gcd_mtd2 760       43958   208586   210   0.0% kjournald 765       31116   208347   149   0.0% kjournald 886       31349   208114   150   0.0% kjournald 1988       36029   237342   151   0.0% kjournald 2233           1       30     40   0.0% portmap 2242           3         5   696   0.0% rpc.statd 2257           0         2     11   0.0% lockd 2258           8     1728     4   0.0% nfsd 2259           8     1727     4   0.0% nfsd 2260           8     1727     4   0.0% nfsd 2261           9     1727     5   0.0% nfsd 2262           8     1728     4   0.0% nfsd 2263           8     1728     4   0.0% nfsd 2264           8     1729     4   0.0% nfsd 2265           8     1726     4   0.0% nfsd 2270           0         1   132   0.0% rpc.mountd 2290     2138348 13880127   154   0.0% sysmgr 2643           0         1     25   0.0% mping-thread 2718         7036   206917     34   0.0% redun_kthread 2920           43       224   194   0.0% xinetd 2921           50       479   104   0.0% tftpd 2922     1885755 308484299    6   0.0% syslogd 2923         181     1271   142   0.0% sdwrapd 2931           0         7     45   0.0% usd_mts_kthread 2944       16901   3100548     5   0.0% ls-notify-mts-t 3027     3660723   2618895   1397   0.0% cmond 3028        2900       625   4640   0.0% pfm_dummy 3035       19653   780705     25   0.0% vshd 3036           62       17   3669   0.0% smm 3037       53404   1650665     32   0.0% session-mgr 3038       94080   6211601     15   0.0% psshelper 3039         6963   103812     67   0.0% lmgrd 3040       65621   1441087     45   0.0% licmgr 3041         5362   139249     38   0.0% fs-daemon 3042       11122   520200     21   0.0% feature-mgr 3043         206       689   299   0.0% confcheck 3044       25743   826190     31   0.0% capability 3045       94664   6211930     15   0.0% psshelper_gsvc 3053       33425   207353   161   0.0% cisco 3056       40788   638596     63   0.0% clis 3057       59461   1636992     36   0.0% port-profile 3059       49041   2072373     23   0.0% xmlma 3060       53345   1696294     31   0.0% vmm 3061       48536   1719085     28   0.0% vdc_mgr 3062       43610   3106747     14   0.0% ttyd 3063       10612   628063     16   0.0% sysinfo 3064         293       486   603   0.0% sksd 3066         3262   123522     26   0.0% res_mgr 3067       134609   6231106     21   0.0% plugin 3068       467226   1380087   338   0.0% plog_sup 3069        128       15   8544   0.0% npacl 3070       18252   520815     35   0.0% mvsh 3071           26       75   347   0.0% mping_server 3072       159427   1250958   127   0.0% idehsd 3073       16671   522586     31   0.0% evms 3074        13941   397828     35   0.0% evmc 3075     49911293 16123404   3095   0.0% diagmgr 3076       45315   502617     90   0.0% crdcfg_server 3077         331     1017   325   0.0% core-dmon 3078       171639   1761631     97   0.0% cmpproxy 3079       497384   4187027   118   0.0% clk_mgr 3080       11388   621260     18   0.0% bloggerd 3081       22013   1242095     17   0.0% bios_daemon 3082       18112   777689     23   0.0% ascii-cfg 3083       111428   6210101     17   0.0% securityd 3084       98387   6211128     15   0.0% cert_enroll 3085       94987   6214320     15   0.0% aaa 3088       18690   777305     24   0.0% obfl 3090           85       28   3046   0.0% l3vm 3091         127        34   3752   0.0% urib 3093     33075502   6237681   5302   0.0% diagclient 3110           74       34   2196   0.0% adjmgr 3111           97       28   3481   0.0% u6rib 3120       32642   241081   135   0.0% aclmgr 3122         109       76   1446   0.0% arp 3124           91       20   4566   0.0% icmpv6 3161       33124   217673   152   0.0% kjournald 3169     2245769 64712179     34   0.0% mts-sync-thr 3242           3       87     38   0.0% dhcpd 3245      53250   1299891     40   0.0% cardclient 3246       409624   8403407     48   0.0% device_test 3247       55546   577909     96   0.0% xbar 3249     1302033   7717061   168   0.0% module 3254         208       398   523   0.0% ExceptionLog 3255       50295   3108982     16   0.0% bootvar 3256       27871   765502     36   0.0% ifmgr 3259         208       94   2218   0.0% netstack 3276       176505   6215803     28   0.0% radius 3279     1714777   4982359   344   0.0% cfs 3281           22       41   542   0.0% ip_dummy 3282           33       42   791   0.0% ipv6_dummy 3283       51426   738103     69   0.0% ntp 3284       54365   1727529     31   0.0% otm 3285           23       43   543  0.0% pktmgr_dummy 3286     1246677   4547384   274   0.0% snmpd 3287           23       44   544   0.0% tcpudp_dummy 3289       340737   3019215   112   0.0% cdp 3290           83       260   320   0.0% dcos-xinetd 3292       281988 11890167     23   0.0% ntpd 3293     11903804 14652688   812   0.0% R2D2_usd 3294       292360   3153368     92   0.0% arbiter 3295     1505005   3724037   404   0.0% babycaesar 3297     41721542 12143605   3435   0.0% sensor 3298     2892940   4943312   585   0.0% xbar_driver_usd 3317       115183   6208629     18   0.0% callhome 3358     54287184 13950490   3891   0.0% oc_usd 3359     3092574 15907454   194   0.0% metropolis_usd 3372           0         2     4   0.0% kauditd 3471       22801   1240441     18   0.0% sctpt_rx_thr 3472           0         1     5   0.0% sctpt_tx_thr 3508       25446   826409     30   0.0% u2 3509       28225   206525   136   0.0% spm 3510       55102   1725410    31   0.0% sal 3511         329     2197   149   0.0% rpm 3512       568064   2990164   189   0.0% pltfm_config 3513       55064   1715291     32   0.0% pixmc 3514       68105   1715304     39   0.0% pixm 3515       366788   1522226    240   0.0% pfstat 3516         475     2030   234   0.0% mrib 3517       53274   1720790     30   0.0% monitorc 3518       65252   1572542     41   0.0% mcm 3519         260     1059   245   0.0% m6rib 3522     3000740   7577947    395   0.0% l2pt 3524       28816   829700     34   0.0% eltm 3526       305549   416012   734   0.0% copp 3527       74173   1897493     39   0.0% acllog 3532       144421   1631647     88   0.0% mfdm 3533       63805   1638615     38   0.0% monitor 3534       19748   404008     48   0.0% m2rib 3536       105922   1906312     55   0.0% ufdm 3553         166       73   2286   0.0% igmp 3558         170       123   1384   0.0% mcastfwd 3559       30802   783951    39   0.0% eth_port_channel 3560       23603   689775     34   0.0% vlan_mgr 3569         3192   110681     28   0.0% eth_dstats 3570       249041   593924   419   0.0% ipqosmgr 3571       838078   3413023   245   0.0% lacp 3577      19102   203449     93   0.0% dhcp_snoop 3578     2855782 12674401   225   0.0% diag_port_lb 3579       39560   794428     49   0.0% ethpm 3580     2084026   4060385   513   0.0% l2fm 3581       46520   907379     51   0.0% nfm 3582         9639   249890     38   0.0% vtp 3589     1260301 17011436     74   0.0% vpc 3590     11393886 107080504   106   0.0% stp 3591       17574   366700     47   0.0% interface-vlan 3607       25561   826111     30   0.0% port_client 3609     17407279 10686606   1628   0.0% statsclient 3610       23619   650313     36   0.0% xbar_client 3627       55993   6205583     9   0.0% wdpunch_thread 3705           5       19   289   0.0% gettylogin1 3731           1        4   410   0.0% getty 4496           37       80   466   0.0% login 4591           8       60   135   0.0% klogd 4615       213619   8202090     26   0.0% ntpd 8405         1236     5666   218   0.0% vsh 12066           3        2   1535   0.0% vsh 12067           99         8 12416   0.0% telnet 12068         358     4626     77   0.0% dcos-telnet 12069         257     2980     86   0.0% in.dcos-telnetd 12086           35       42   839   0.0% login 12099        340       815   418   0.0% vsh 12673           3         3   1082   0.0% vsh 12696           2         4   578   0.0% vdclogin 12699           15         2   7789   0.0% login 12700         222       909   244   0.0% vsh 13059          3         7   510   0.0% mvsh 13060         179       35   5118   0.0% vsh 13096           9       210     43   0.0% vsh 13099           3         3   1197   0.0% mvsh 13100         136       45   3033   0.0% vsh 13103           8        9   987   0.0% tacacsd 13106           2         3   712   0.0% vsh 13107           39         3 13146   0.0% ps 17076       260337   6217630     41   0.0% tacacs CPU util :   25.2% user,   31.0% kernel,   43.8% idle Please note that only processes from the requested vdc are shown above ... View more

          Hi all !! Recently we have migrated to LMS 4.2.3 before 1 month. In 4.2 I observed for any operationally down interface, duplicate events are appearing with new time stamp periodically. As per my knowledge an "Operationally down" event reappears with new time stamp in below two cases. 1) administrator cleared the event but the the interface status is still down. 2) the event is in active state for last 31 days. In my case 1st concept is working fine, I.e, If I am clearing th eevent it reappears after 30 min. But even if I am not clearing the event I am see the same event reappears with new timestamp. herewith I have attached the fault history. Is there some basic concept I am missing? ... View more

Hi All, Following is my switch configuration and PFC running modes. Due to some harware problem my 3rd module failed and remain in power down state. I have replaced the module and got recovered from the problem. However before replacing teh module I found the other access switches uplinked to this core switch's 4th and 12 th module alsolost connectivity. all my existing connectivities are only on 3rd 4th and 12th module. At access switch end I got logs of uplink port reset for that same time. I am wondering if 3rd module failed then how the connectivity of other module also got reset!! Please  help me. show platform hardware capacity --- System Resources PFC operating mode: PFC3B Supervisor redundancy mode: administratively sso, operationally sso Switching resources: Module   Part number               Series     CEF mode                        3       WS-X6724-SFP             CEF720           CEF                        4       WS-X6724-SFP             CEF720         dCEF                       7       VS-S720-10G           supervisor           CEF                        8       VS-S720-10G           supervisor           CEF                        9       WS-X6148-GE-TX           classic           CEF                        10       WS-X6748-GE-TX           CEF720         dCEF                        11       WS-X6748-GE-TX           CEF720         dCEF                        12       WS-X6724-SFP             CEF720         dCEF ----------------------------------------------------------------------------------------------- Mod Sub-Module                 Model             Serial       Hw     Status ---- --------------------------- ------------------ ----------- ------- ------- 3 Centralized Forwarding Card WS-F6700-CFC       SAL1006D3DB 2.0   PwrDown 4 Distributed Forwarding Card WS-F6700-DFC3B     SAL124592QB 4.7   Ok 7 Policy Feature Card 3       VS-F6K-PFC3C       SAL1411CZTS 1.1   Ok 7 MSFC3 Daughterboard         VS-F6K-MSFC3       SAL1411CXUB 5.0   Ok 8 Policy Feature Card 3       VS-F6K-PFC3C       SAL1519DBJZ 1.1   Ok 8 MSFC3 Daughterboard         VS-F6K-MSFC3       SAL1518CTH9 5.1   Ok 10 Distributed Forwarding Card WS-F6700-DFC3C     SAL1407B26X 1.4   Ok 11 Distributed Forwarding Card WS-F6700-DFC3B     SAD09120C27 2.4   Ok 12 Distributed Forwarding Card WS-F6700-DFC3B     SAD095001D6 4.2   Ok ---------------------------------------------------------------------------- ------------------------------------------------------ Feb 13 05:40:30 IST: %PM_SCP-SP-1-LCP_FW_ERR: System resetting module 3 to recover from error: Linecard received system exception. Errcode = Feb 13 05:40:30 IST: %OIR-SP-3-PWRCYCLE: Card in module 3, is being power-cycled 'Off (Module Reset due to exception or user request)' Feb 13 05:40:30 IST: %C6KPWR-SP-4-DISABLED: power to module in slot 3 set Off (Module Reset due to exception or user request) ... View more

Hi, I have cisco works 3.2 installed on windows 2003 server service pack 1 with java 1.6.0_39 i am facing problem of tomcat service is terminating unexpectedly. my cisco works installed properly, it is running fine but randomly this service goes down as seen from event viewer and then my browser start showing me service temporarily unavailable, after restarting the services, things got normal. can anybody tell why it is being so ??? stdout.logs are attached. rgds ... View more

I had one nortel and one cisco switch. Two diffrent links are coming from two diffrent ISP`s. I have configured etherchannel between nortel and cisco as metroethernet. I want to replace nortel with cisco. When i tried to do so, i am getting RTO between ehterchannel at the same time when i configure point to point it is working fine. Can anyone suggest what might be the problem?????Configuration is checked. Pls suggest ASAP???? ... View more

Hi, I have configured user level NAR in my ACS 1113 SE running with 4.2.0.124(No patch). The routers and switches which are permitted in NAR are  accessble as per configuration and the devices which are not configured are not accessble. I mean to say the NAR is working fine for Routers and switches but for ASA it is not working. Regardless of teh ASA is added in NAR or not I am able to access it useing the acs local users. Following are the configurations I have done in ASA firewall. aaa-server ACSVPN protocol radius max-failed-attempts 2 aaa-server ACSVPN (DMZ_INTERFACE) host 172.X.X.10 timeout 30 key cisco aaa-server ACSVPN (DMZ_INTERFACE) host 172.X.X.16 timeout 30 key cisco radius-common-pw gcisco aaa authentication enable console ACSVPN LOCAL aaa authentication ssh console ACSVPN LOCAL aaa authentication http console ACSVPN LOCAL I can access the ASA without adding it's Ip to NAR. Please help me to understand what can be the possible causes. In success log I cam see teh message "all filteres passed" status. ... View more

Hello Dear, One of my customer has raised a new requirement for implementation of short sequence number format support in PPP multilink header for Cisco MWR 2941 E1/T1 serial interface, whereas router is supporting long sequence number format. here is the output of "debug ppp negotiation" command:- Currently in the MWR debugging logs we can see that by default MWR is sending long sequence header format as below *Mar 13 01:32:55.438: Se0/2:0 LCP: O CONFREQ [REQsent] id 238 len 25 *Mar 13 01:32:55.438: Se0/2:0 LCP:    MagicNumber 0x26CDF693 (0x050626CDF693) *Mar 13 01:32:55.438: Se0/2:0 LCP:    MRRU 1500 (0x110405DC) *Mar 13 01:32:55.438: Se0/2:0 LCP:    EndpointDisc 2 16.16.16.11 (0x1307021010100B) *Mar 13 01:32:55.438: Se0/2:0 LCP:    MultilinkHdrFmt seq long classes 2 (0x1B040202) While as per the requirement  PPP multilink header should support short sequence. MWR configuration: controller E1 0/2 framing NO-CRC4 clock source line channel-group 0 timeslots 1-31 interface Multilink2 ip address 16.16.16.11 255.255.255.224 no cdp enable ppp multilink ppp multilink group 2 ppp multilink endpoint ip 16.16.16.11 ppp multilink mrru local 1500 ppp multilink multiclass interface Serial0/2:0 no ip address encapsulation ppp no cdp enable ppp multilink ppp multilink group 2 ppp multilink mrru local 1500 So will you please suggest me how to proceed for solution? Regards Sandeep ... View more

Hi, As per the below link, In LMS 4.0 only network topology L2 services are supported for "7600-SIP-200" module, with  cisco 7600 series routers. My setup has cisco 7609-S series routers and the service module "7600-SIP-200" installed in it. Following are the sub modules installed in the SIP-200 card. 1) SPA-1XCHSTM1/OC3 2) SPA-4XOC3-POS My doubt is will DFM manage the interfaces of this router? will I get DFM event when th POS interface of this routers goes down? for information this is a managed device in DFM and the POS interfaces managed state is set to true. However I am not getting alerts when teh POS interfaces are going down. Even I have created a seperate group for those interfaces, applied thresholds for the interfaces and set the priority in interface group. I am still not getting any "operationally down" alert for any of my POS interface. Regards, Satya. ... View more