ASR9000 empty output from "show process cpu location */*/*"

[ibragimovs]

I have an ASR9000 box with following cards in it:

IOS XR Software version is: 6.4.2 (32bit)

The problem is when I type "show process cpu location 0/0/CPU0" the output is empty and when I use 0/1/CPU0, the box shows this cpu utilization for this location. Card type is the same (A9K-MOD400-TR). I have multiple ASR9000 boxes with the same behaviour. What could be the problem?

admin show hw-module fpd location all

[tkarnani · Accepted solution]

this happens on our lab routers as well once in a while, restart wdsysmon wait a minute or two and see if it comes back

thanks

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvr23317/?rfs=iqvred

RP/0/RSP1/CPU0:ASR9910-A#
RP/0/RSP1/CPU0:ASR9910-A#
RP/0/RSP1/CPU0:ASR9910-A#show processes cpu location 0/1/cPU0
Thu Oct 31 11:47:36.233 UTC
RP/0/RSP1/CPU0:ASR9910-A#show processes cpu location 0/1/cPU0
Thu Oct 31 11:47:37.273 UTC
RP/0/RSP1/CPU0:ASR9910-A#show processes cpu location 0/1/cPU0
Thu Oct 31 11:47:37.727 UTC
RP/0/RSP1/CPU0:ASR9910-A#show processes cpu location 0/1/cPU0
Thu Oct 31 11:47:38.286 UTC
RP/0/RSP1/CPU0:ASR9910-A#
RP/0/RSP1/CPU0:ASR9910-A#process restart wdsysmon location 0/1/cPU0
Thu Oct 31 11:47:47.087 UTC
RP/0/RSP1/CPU0:ASR9910-A#show processes cpu location 0/1/cPU0
Thu Oct 31 11:47:48.884 UTC
RP/0/RSP1/CPU0:ASR9910-A#show processes cpu location 0/1/cPU0
Thu Oct 31 11:47:50.208 UTC
RP/0/RSP1/CPU0:ASR9910-A#show processes cpu location 0/1/cPU0
Thu Oct 31 11:47:50.863 UTC
RP/0/RSP1/CPU0:ASR9910-A#show processes cpu location 0/1/cPU0
Thu Oct 31 11:47:51.439 UTC
RP/0/RSP1/CPU0:ASR9910-A#show processes cpu location 0/1/cPU0
Thu Oct 31 11:47:52.019 UTC
RP/0/RSP1/CPU0:ASR9910-A#show processes cpu location 0/1/cPU0
Thu Oct 31 11:47:52.503 UTC
RP/0/RSP1/CPU0:ASR9910-A#show processes cpu location 0/1/cPU0
Thu Oct 31 11:47:53.052 UTC
RP/0/RSP1/CPU0:ASR9910-A#show processes cpu location 0/1/cPU0
Thu Oct 31 11:47:53.656 UTC
RP/0/RSP1/CPU0:ASR9910-A#show processes cpu location 0/1/cPU0
Thu Oct 31 11:47:54.069 UTC
RP/0/RSP1/CPU0:ASR9910-A#show processes cpu location 0/1/cPU0
Thu Oct 31 11:47:54.682 UTC
RP/0/RSP1/CPU0:ASR9910-A#show process wdsysmon location 0/1/CPU0
Thu Oct 31 11:48:05.107 UTC
                  Job Id: 414
                     PID: 294967
            Process name: wdsysmon
         Executable path: /iosxr-infra-6.6.2/sbin/wdsysmon
              Instance #: 1
              Version ID: 00.00.0000
                 Respawn: ON
           Respawn count: 2
            Last started: Thu Oct 31 11:47:47 2019
           Process state: Run (last exit status : 1)
           Package state: Normal
                    core: COPY
               Max. core: 0
                   Level: 40
               Mandatory: ON
           MaintModeProc: ON
               Placement: None
            startup_path: /pkg/startup/wdsysmon.startup
            memory limit: 10240
                   Ready: 0.050s
        Process cpu time: 0.062 user, 0.011 kernel, 0.073 total
JID   TID CPU Stack pri state        TimeInState    HR:MM:SS:MSEC   NAME
414    1    3  164K  10 Receive        0:00:17:0859    0:00:00:0022 wdsysmon
414    2    5  164K  20 Sem            0:00:17:0887    0:00:00:0000 wdsysmon
414    3    3  164K  10 Receive        0:00:16:0841    0:00:00:0011 wdsysmon
414    4    5  164K  10 Receive        0:00:17:0887    0:00:00:0000 wdsysmon
414    5    3  164K  10 Receive        0:00:10:0425    0:00:00:0019 wdsysmon
414    6    0  164K  10 Receive        0:00:17:0889    0:00:00:0000 wdsysmon
414    7    5  164K  19 Receive        0:00:00:0086    0:00:00:0001 wdsysmon
414    8    5  164K  10 Receive        0:00:17:0889    0:00:00:0000 wdsysmon
414    9    3  164K  62 Sem            0:00:17:0891    0:00:00:0000 wdsysmon
414    10   0  164K  10 Receive        0:00:02:0889    0:00:00:0000 wdsysmon
414    11   4  164K  10 Nanosleep      0:00:01:0879    0:00:00:0000 wdsysmon
414    12   4  164K  10 Receive        0:00:01:0879    0:00:00:0003 wdsysmon
414    13   2  164K  10 Receive        0:00:17:0893    0:00:00:0000 wdsysmon
414    14   3  164K  10 Receive        0:00:17:0882    0:00:00:0000 wdsysmon
414    15   4  164K  10 Nanosleep      0:00:17:0873    0:00:00:0000 wdsysmon
414    16   5  164K  10 Receive        0:00:08:0816    0:00:00:0000 wdsysmon
414    17   4  164K  10 Receive        0:00:14:0746    0:00:00:0010 wdsysmon
414    18   3  164K  10 Receive        0:00:16:0833    0:00:00:0000 wdsysmon
-------------------------------------------------------------------------------
RP/0/RSP1/CPU0:ASR9910-A#show processes cpu location 0/1/cPU0
Thu Oct 31 11:48:08.673 UTC
RP/0/RSP1/CPU0:ASR9910-A#show processes cpu location 0/1/cPU0
Thu Oct 31 11:48:09.343 UTC
RP/0/RSP1/CPU0:ASR9910-A#show processes cpu location 0/1/cPU0
Thu Oct 31 11:48:46.495 UTC

CPU utilization for one minute: 2%; five minutes: 1%; fifteen minutes: 1%

PID    1Min    5Min    15Min Process
1        1%      1%       1% kernel
8195     0%      0%       0% dllmgr
12290    0%      0%       0% wd-critical-mon
12295    0%      0%       0% a9kpreinit
12296    0%      0%       0% pkgfs
12297    0%      0%       0% devc-ser8250.th
12298    0%      0%       0% pkgfs
12299    0%      0%       0% devc-pty
12300    0%      0%       0% random
12301    0%      0%       0% pipe
12302    0%      0%       0% mqueue
12303    0%      0%       0% mq
12306    0%      0%       0% shmwin_svr
12308    0%      0%       0% pciesvr
28688    0%      0%       0% nvram
28689    0%      0%       0% mediasvr
28691    0%      0%       0% dumper
28693    0%      0%       0% obflmgr
28694    0%      0%       0% devb-ahci
28695    0%      0%       0% syslogd_helper
28696    0%      0%       0% ahci-mon
28697    0%      0%       0% syslog_dev
28698    0%      0%       0% ksh
65564    0%      0%       0% epm_switch_lc
69659    0%      0%       0% vkg_dmac_svr
69662    0%      0%       0% spp
69663    0%      0%       0% qad_server
69664    0%      0%       0% packet
69665    0%      0%       0% lda_server
69667    0%      0%       0% eth_server
73762    0%      0%       0% attachd
73764    0%      0%       0% qnet
73765    0%      0%       0% mbi-hello
73768    0%      0%       0% insthelper
73769    0%      0%       0% attach_server
77866    0%      0%       0% procmgr
86045    0%      0%       0% sysmgr
98308    0%      0%       0% qsm
98309    0%      0%       0% imdr
98347    0%      0%       0% ltrace_server
98348    0%      0%       0% ltrace_sync
98350    0%      0%       0% obfl_pd_driver
98352    0%      0%       0% psa
102449   0%      0%       0% sld
106543   0%      0%       0% sysmgr
106548   0%      0%       0% top_procs
122931   0%      0%       0% zllc
147462   0%      0%       0% sysdb_svr_local
147501   0%      0%       0% enf_broker
147506   0%      0%       0% alphadisplay
147509   0%      0%       0% lrd
147510   0%      0%       0% meminfo_svr
147512   0%      0%       0% fab_si
147513   0%      0%       0% showd_server
147514   0%      0%       0% sysmgr_show_proc_all_edm
147515   0%      0%       0% dumper_config
147516   0%      0%       0% obfl_config
151614   0%      0%       0% canb-server-lc
155711   0%      0%       0% gsp
155712   0%      0%       0% io-pkt-v4
155713   0%      0%       0% aipc_cleaner
155716   0%      0%       0% fab_vqi_alloc
155717   0%      0%       0% fialc
159811   0%      0%       0% mgid_prgm
159814   0%      0%       0% obfl_agent-lc
159815   0%      0%       0% prm_verifier
159816   0%      0%       0% rdsfs_svr
159817   0%      0%       0% pfm_node_lc
159818   0%      0%       0% sysdb_mc
159819   0%      0%       0% bundlemgr_checker
159820   0%      0%       0% licagent
159821   0%      0%       0% prm_ssmh
159822   0%      0%       0% fab_arb
159823   0%      0%       0% fab_xbar
159824   0%      0%       0% inv_agent
159825   0%      0%       0% prm_server_to
159827   0%      0%       0% cerrno_server
159828   0%      0%       0% online_diag_lc
159829   0%      0%       0% online_diag_svr
159830   0%      0%       0% uidb_server
159831   0%      0%       0% envmon_lc
159832   0%      0%       0% cfgmgr-lc
159834   0%      0%       0% subdb_svr
159835   0%      0%       0% wd-stat-publisher
180285   0%      0%       0% ifmgr
180290   0%      0%       0% netio
180306   0%      0%       0% fwd_driver_partner
180313   0%      0%       0% mempool_edm
180316   0%      0%       0% nto_misc_shprocmem_edm
180317   0%      0%       0% pm
180319   0%      0%       0% rsi_agent
180320   0%      0%       0% sint_ma
184414   0%      0%       0% sync_agent
184417   0%      0%       0% timing_drvr_util_proc
184418   0%      0%       0% mpls_io_ea
184419   0%      0%       0% cluster_dlm_checker
184420   0%      0%       0% cluster_dlm_intfmon
184421   0%      0%       0% cluster_dlm_procmon
184422   0%      0%       0% aib
184423   0%      0%       0% bundlemgr_adj
184424   0%      0%       0% statsd_server
184425   0%      0%       0% ipv4_mfwd_partner
184426   0%      0%       0% ipv6_mfwd_partner
184427   0%      0%       0% ipv4_io
184428   0%      0%       0% ipv6_nd
184429   0%      0%       0% fib_mgr
184430   0%      0%       0% inline_service_proc
184431   0%      0%       0% ipv4_ma
184432   0%      0%       0% ipv6_ea
184433   0%      0%       0% ipv6_io
184434   0%      0%       0% pifibm_server_lc
184435   0%      0%       0% procfind
184436   0%      0%       0% ipv6_ma
184437   0%      0%       0% bfd_agent
184438   0%      0%       0% debug_d
184439   0%      0%       0% ether_ctrl_msg_client
184440   0%      0%       0% fsyncmgr
184441   0%      0%       0% iedged
184443   0%      0%       0% timezone_notify
184444   0%      0%       0% ixdb_gc
192551   0%      0%       0% cdm_rs
192634   0%      0%       0% udp
204925   0%      0%       0% accounting_ma
204926   0%      0%       0% daps
204927   0%      0%       0% flowtrap
204929   0%      0%       0% l2fib_mgr
204930   0%      0%       0% vkg_l2fib_evpn_mac
204931   0%      0%       0% vkg_l2fib_mac_cache
204932   0%      0%       0% mpls_fwd_show_proxy
209024   0%      0%       0% statsd_manager_l
237702   0%      0%       0% mpls_io
237704   0%      0%       0% ntpdc
241799   0%      0%       0% iphc_ma
241801   0%      0%       0% nfma
241802   0%      0%       0% session_mon
241803   0%      0%       0% clns
241804   0%      0%       0% arp
241805   0%      0%       0% fhrp_output
241806   0%      0%       0% l2snoop
241807   0%      0%       0% bundlemgr_local
245904   0%      0%       0% x86rmon_fpd_agent
245906   0%      0%       0% showd_lc
245907   0%      0%       0% eem_ed_sysmgr
258181   0%      0%       0% canb_upg_agt
258193   0%      0%       0% ctrlio
258197   0%      0%       0% icpe_sdep
262292   0%      0%       0% imaedm_server
262294   0%      0%       0% ipv6nd_lite
262295   0%      0%       0% lacp_lite
262296   0%      0%       0% lamptest-lc
262297   0%      0%       0% lc_fpd_upgrade
262298   0%      0%       0% li_ea
262299   0%      0%       0% macsec_ea
262300   0%      0%       0% netio_debug_partner
262302   0%      0%       0% pbr_ea
262303   0%      0%       0% pbr_ma
266397   0%      0%       0% pcds_daemon
266400   0%      0%       0% pfilter_ma
266401   0%      0%       0% pkt_trace_agent
266402   0%      0%       0% pm_ma
266404   0%      0%       0% qos_ma
270501   0%      0%       0% slarp_lite
270502   0%      0%       0% spio_ea
270503   0%      0%       0% spio_ma
270504   0%      0%       0% ssm_process
270505   0%      0%       0% svi_adj_mgr
270506   0%      0%       0% tom_fpd_fpga_agent
270507   0%      0%       0% x86lc_fpd_agent
274595   0%      0%       0% radiusd
274604   0%      0%       0% span_ea
274605   0%      0%       0% vic_1_0
274606   0%      0%       0% ether_caps_partner
274607   0%      0%       0% ether_sock
274608   0%      0%       0% vlan_ea
274609   0%      0%       0% cdp
274610   0%      0%       0% vic_0_0
274611   0%      0%       0% l2vpn_checker
274612   0%      0%       0% vlan_ma
274613   0%      0%       0% bundlemgr_ea
274614   0%      0%       0% pfilter_ea
274615   0%      0%       0% qos_ma_ea
274616   0%      0%       0% nfea
274617   0%      0%       0% nfsvr
274618   0%      0%       0% nf_producer
290854   0%      0%       0% macsec_mka
294967   0%      0%       0% wdsysmon
RP/0/RSP1/CPU0:ASR9910-A#