08-12-2021 07:55 AM
Hello everyone
Our both 24X10 TR Line cards got powered off because of following error messages and the only way we could bring them up was rebooting ASR9904 router manually . I do not know the reason if it is a power problem or a bug or something else. We did not have any electricity problem at all.
I want to know what this type of problem is and how to fix it for the future
ce_switch_srv[54]: %PLATFORM-CE_SWITCH-6-UPDN : Interface 3 (NC_3) is down
ce_switch_srv[54]: %PLATFORM-CE_SWITCH-6-UPDN : Interface 4 (NC_4) is down
ce_switch_srv[54]: %PLATFORM-CE_SWITCH-6-UPDN : Interface 3 (NC_3) is down
ce_switch_srv[54]: %PLATFORM-CE_SWITCH-6-UPDN : Interface 4 (NC_4) is down
shelfmgr[407]: %PLATFORM-SHELFMGR-1-NO_POWER_AVAIL : Node 0/0/CPU0 is kept in unpowered state because of insufficient power.
shelfmgr[407]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/0/CPU0 A9K-24x10GE-TR state:UNPOWERED
shelfmgr[407]: %PLATFORM-SHELFMGR-1-NO_POWER_AVAIL : Node 0/1/CPU0 is kept in unpowered state because of insufficient power.
pfm_node_rp[362]: %PLATFORM-CROSSBAR-1-SERDES_ERROR_LNK1 : Set|fab_xbar[213109]|0x101702f|XBAR_1_Slot_0
pfm_node_rp[362]: %PLATFORM-CROSSBAR-1-SERDES_ERROR_LNK1 :Set|fab_xbar[213109]|0x101702e|XBAR_1_Slot_3
pfm_node_rp[362]: %PLATFORM-CROSSBAR-1-SERDES_ERROR_LNK2 : Set|fab_xbar[213109]|0x101702a|XBAR_1_Slot_3
pfm_node_rp[362]: %PLATFORM-CROSSBAR-1-SERDES_ERROR_LNK2 : Set|fab_xbar[213109]|0x1017027|XBAR_1_Slot_0
pfm_node_rp[362]: %PLATFORM-CROSSBAR-1-SERDES_ERROR_LNK0 : Set|fab_xbar[213109]|0x101700e|XBAR_0_Slot_3
pfm_node_rp[362]: %PLATFORM-CROSSBAR-1-SERDES_ERROR_LNK0 : Set|fab_xbar[213109]|0x101700b|XBAR_0_Slot_0
pfm_node_rp[362]: %PLATFORM-CROSSBAR-1-SERDES_ERROR_LNK1 : Set|fab_xbar[213109]|0x1017007|XBAR_0_Slot_0
pfm_node_rp[362]: %PLATFORM-CROSSBAR-1-SERDES_ERROR_LNK0 : Set|fab_xbar[213109]|0x1017030|XBAR_1_Slot_3
pfm_node_rp[362]: %PLATFORM-CROSSBAR-1-SERDES_ERROR_LNK0 : Clear|fab_xbar[213109]|0x101700b|XBAR_0_Slot_0
pfm_node_rp[362]: %PLATFORM-CROSSBAR-1-SERDES_ERROR_LNK1 : Clear|fab_xbar[213109]|0x1017007|XBAR_0_Slot_0
pfm_node_rp[362]: %PLATFORM-CROSSBAR-1-SERDES_ERROR_LNK0 : Clear|fab_xbar[213109]|0x1017030|XBAR_1_Slot_3
pfm_node_rp[362]: %PLATFORM-CROSSBAR-1-SERDES_ERROR_LNK1 : Clear|fab_xbar[213109]|0x101702f|XBAR_1_Slot_0
pfm_node_rp[362]: %PLATFORM-CROSSBAR-1-SERDES_ERROR_LNK1 : Clear|fab_xbar[213109]|0x101702e|XBAR_1_Slot_3
pfm_node_rp[362]: %PLATFORM-CROSSBAR-1-SERDES_ERROR_LNK0 : Clear|fab_xbar[213109]|0x101700e|XBAR_0_Slot_3
Thank you for your help and kindness
08-12-2021 08:12 AM
Hi,
can we check
admin
show environment power
this would tell us if the system is providing enough power for the line cards.
%PLATFORM-CROSSBAR-1-SERDES_ERROR_LNK1
this tells us there are errors on the fabric links from the LC towards the RSP, i would re-seat this card to see if these errors clear " %PLATFORM-CROSSBAR-1-SERDES_ERROR_LNK1" if they do not, it would be best to open a TAC case.
possibly we may need to re-place the card if its still not clearing.
08-12-2021 08:19 AM
Thank you for your response
This the result for power:
(admin)#show environment power-supply
Thu Aug 12 10:19:57.689 EDT
R/S/I Modules Capacity Status
(W)
0/PS0/M0/*
host PM 3000 Ok
0/PS0/M1/*
host PM 3000 Ok
0/PS0/M2/*
host PM 3000 Ok
0/PS0/M3/*
host PM 3000 Ok
R/S/I Power Supply Voltage Current
(W) (V) (A)
0/PS0/M0/* 431.3 55.3 7.8
0/PS0/M1/* 447.9 55.3 8.1
0/PS0/M2/* 447.9 55.3 8.1
0/PS0/M3/* 436.8 55.3 7.9
--------------
Total: 1763.9
R/S/I Power Draw Voltage Current
(W) (V) (A)
0/FT0/* 380 **
--------------
Total: 380.0
R/S/I Power Draw Voltage Current
(W) (V) (A)
0/RSP0/* 231.0 55.0 4.2
0/RSP1/* 209.7 55.2 3.8
--------------
Total: 440.7
R/S/I Power Draw Voltage Current
(W) (V) (A)
0/0/* 697.2 54.9 12.7
0/1/* 708.2 54.9 12.9
--------------
Total: 1405.4
** Indicates worst case power draw (No power measuring sensor)
Chassis Power Draw for rack 0: 1763.9 (W)
Power Budget Summary for Rack 0
--------------------------------
Power Shelves Type: AC
Power Budget Strict Mode: Disabled
Total Power Capacity: 12000W
Usable Power Capacity: 9000W
N+1 Supply Failure Protected Capacity: 9000W
Worst Case Power Used: 2780W
Slot Max Watts
---- ---------
0/0/CPU0 850
0/RSP0/CPU0 350
0/RSP1/CPU0 350
0/1/CPU0 850
0/FT0/SP 380
Worst Case Power Available: 6220W
N+1 Supply Protected Capacity Available: 6220W
08-13-2021 04:24 AM
the router seems to have enough power now.
did we check the output during the time of the issue?
thanks
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: