Buy or Renew
Buy or Renew
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Start a conversation
5259
Views
0
Helpful
5
Replies

Errors on A9K-48X10GE-1G-TR after FPD upgrade running 6.5.3

Go to solution
nesttech1
Level 1
Level 1

‎11-28-2019 10:26 AM

We started seeing these errors below after doing a FPD upgrade on the A9K-48X10GE-1G-TR module running IOS XR 6.5.3 and with RSP880's.  Before the FPD upgrade the module booted up without these error messages and only indicated that FPD upgrade was required.  These errors only show up while the module is booting.  Once it is in the "IOS XR RUN" state, it looks normal and we do not see any more error messages.

Please let me know if these errors are significant any may affect the operation of this module.

Any feedback is appreciated.

 

RP/0/RSP0/CPU0:Nov 26 15:19:15.195 UTC: canb-server[153]: %PLATFORM-CANB_SERVER-3-ERROR_INFO : Error from CBC in slot 0/1/CPU0, error msg Device=INTEL 82599, inst=0, errCode=EOBC Niantic-Switch Link Down

RP/0/RSP1/CPU0:Nov 26 15:19:15.200 UTC: canb-server[153]: %PLATFORM-CANB_SERVER-3-ERROR_INFO : Error from CBC in slot 0/1/CPU0, error msg Device=INTEL 82599, inst=0, errCode=EOBC Niantic-Switch Link Down

RP/0/RSP1/CPU0:Nov 26 15:19:25.386 UTC: canb-server[153]: %PLATFORM-CANB_SERVER-3-ERROR_INFO : Error from CBC in slot 0/1/CPU0, error msg Device=INTEL 82599, inst=0, errCode=EOBC Niantic-Switch Link Down

RP/0/RSP0/CPU0:Nov 26 15:19:25.393 UTC: canb-server[153]: %PLATFORM-CANB_SERVER-3-ERROR_INFO : Error from CBC in slot 0/1/CPU0, error msg Device=INTEL 82599, inst=0, errCode=EOBC Niantic-Switch Link Down

RP/0/RSP0/CPU0:Nov 26 15:19:42.083 UTC: canb-server[153]: %PLATFORM-CANB_SERVER-3-ERROR_INFO : Error from CBC in slot 0/1/CPU0, error msg Device=INTEL 82599, inst=0, errCode=EOBC Niantic-Switch Link Down

RP/0/RSP1/CPU0:Nov 26 15:19:42.087 UTC: canb-server[153]: %PLATFORM-CANB_SERVER-3-ERROR_INFO : Error from CBC in slot 0/1/CPU0, error msg Device=INTEL 82599, inst=0, errCode=EOBC Niantic-Switch Link Down

1 person had this problem
Labels:
0 Helpful
2 Accepted Solutions

Accepted Solutions

Go to solution
tkarnani
Cisco Employee
Cisco Employee
In response to nesttech1

‎12-01-2019 05:09 AM

These are the links we are interested in

48   |            Up   |  VLAN_EOBC_0  |  LC_EOBC_0_0

51   |            Up   |  VLAN_EOBC_0  |  LC_EOBC_1_0

As long as they come up, you should be fine

View solution in original post

0 Helpful

Go to solution
xthuijs
Cisco Employee
Cisco Employee
In response to nesttech1

‎12-01-2019 05:11 AM

hi!

if the card booted up correcty, and is functioning then this is likely not something to worry about (but always a nuisance when you get scary messages) and is likely described with: CSCvn69012

 

the thing is that during boot, the link status is incorrectly interpreted and reported down.

this is cosmetic if the card boots up correctly. if the eobc (ether out of band, used for rp<>lc communications for image download, lc programming etc) was truly down

then heartbeats would fail and the card would be reloading constantly and eventually end up in-reset.

 

the root cause is a rommon operation if I am not mistaken. so you could update all fpd's but skip the rommon for the other card possibly.

 

cheers!

xander

View solution in original post

0 Helpful
5 Replies 5

Go to solution
tkarnani
Cisco Employee
Cisco Employee

‎11-29-2019 05:58 AM - edited ‎11-29-2019 05:59 AM

Good Day,

 

those error messages are alerting you that the EOBC links are down. these links are the ethernet out of band links that connect the LC to the RSP, to push code/updates etc

 

this happens on a normal reload. here is output from our lab.

as long as the links come up there is nothing to worry about.

status of all of these links can be checked with the command below.

 

show controllers epm-switch port-mapping location 0/RSP0/CPU0

you will see "LC_EOBC_X_X" for the links connected to the LC

 

 

RP/0/RSP0/CPU0:ASR9904-B#hw-module location 0/1/CPU0 reload
Fri Nov 29 13:51:14.328 UTC
WARNING: This will take the requested node out of service.
Do you wish to continue?[confirm(y/n)]y
RP/0/RSP0/CPU0:ASR9904-B#term mon
Fri Nov 29 13:51:16.837 UTC
RP/0/RSP0/CPU0:ASR9904-B#RP/0/RSP0/CPU0:Nov 29 13:51:17.508 UTC: canb-server[156                                                                             ]: %PLATFORM-CANB_SERVER-7-CBC_PRE_RESET_NOTIFICATION : Node 0/1/CPU0 , Power Cy                                                                             cle (0x05000000)
RP/0/RSP1/CPU0:Nov 29 13:51:17.512 UTC: canb-server[156]: %PLATFORM-CANB_SERVER-                                                                             7-CBC_PRE_RESET_NOTIFICATION : Node 0/1/CPU0 , Power Cycle (0x05000000)
RP/0/RSP0/CPU0:Nov 29 13:51:20.520 UTC: canb-server[156]: %PLATFORM-CANB_SERVER-7-CBC_POST_RESET_NOTIFICATION : Node 0/1/CPU0 , Po                           wer Cycle (0x05000000)
RP/0/RSP1/CPU0:Nov 29 13:51:20.522 UTC: canb-server[156]: %PLATFORM-CANB_SERVER-7-CBC_POST_RESET_NOTIFICATION : Node 0/1/CPU0 , Po                           wer Cycle (0x05000000)
RP/0/RSP1/CPU0:Nov 29 13:51:20.918 UTC: pfm_node_rp[381]: %PLATFORM-CARD-3-CPI_NOT_READY : Set|rspfpga_server[36897]|0x1026003|CPI                            not ready signal detected on card 0/1/CPU0
RP/0/RSP0/CPU0:Nov 29 13:51:22.162 UTC: shelfmgr[436]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/1/CPU0 A9K-48X10GE-1G-TR state:B                           RINGDOWN
RP/0/RSP0/CPU0:Nov 29 13:51:22.182 UTC: shelfmgr[436]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/1/CPU0 A9K-48X10GE-1G-TR state:R                           OMMON
RP/0/RSP0/CPU0:Nov 29 13:51:22.185 UTC: epm_switch_rp[60]: %PLATFORM-EPM_SWITCH-6-EOBC_LINK_DOWN : EOBC port 51 link down towards                            LC
RP/0/RSP0/CPU0:Nov 29 13:51:22.185 UTC: invmgr[270]: %PLATFORM-INV-6-NODE_STATE_CHANGE : Node: 0/1/CPU0, state: BRINGDOWN
RP/0/RSP1/CPU0:Nov 29 13:51:22.193 UTC: epm_switch_rp[60]: %PLATFORM-EPM_SWITCH-6-EOBC_LINK_DOWN : EOBC port 51 link down towards                            LC
RP/0/RSP0/CPU0:Nov 29 13:51:24.700 UTC: pfm_node_rp[381]: %PLATFORM-CARD-3-CPI_NOT_READY : Set|rspfpga_server[36897]|0x1026003|CPI not ready signal detected on card 0/1/CPU0
RP/0/RSP0/CPU0:Nov 29 13:51:34.700 UTC: pfm_node_rp[381]: %PLATFORM-CARD-3-CPI_NOT_READY : Clear|rspfpga_server[36897]|0x1026003|CPI not ready signal detected on card 0/1/CPU0
RP/0/RSP1/CPU0:Nov 29 13:51:40.920 UTC: pfm_node_rp[381]: %PLATFORM-CARD-3-CPI_NOT_READY : Clear|rspfpga_server[36897]|0x1026003|CPI not ready signal detected on card 0/1/CPU0
RP/0/RSP0/CPU0:Nov 29 13:53:14.370 UTC: canb-server[156]: %PLATFORM-CANB_SERVER-7-CBC_POST_RESET_NOTIFICATION : Node 0/1/CPU0 , Auto Reset (0x0d000000)
RP/0/RSP1/CPU0:Nov 29 13:53:14.374 UTC: canb-server[156]: %PLATFORM-CANB_SERVER-7-CBC_POST_RESET_NOTIFICATION : Node 0/1/CPU0 , Auto Reset (0x0d000000)
RP/0/RSP0/CPU0:Nov 29 13:53:17.718 UTC: canb-server[156]: %PLATFORM-CANB_SERVER-3-ERROR_INFO : Error from CBC in slot 0/1/CPU0, error msg Device=INTEL 82599, inst=0, errCode=EOBC Niantic-Switch Link Down
RP/0/RSP1/CPU0:Nov 29 13:53:17.721 UTC: canb-server[156]: %PLATFORM-CANB_SERVER-3-ERROR_INFO : Error from CBC in slot 0/1/CPU0, error msg Device=INTEL 82599, inst=0, errCode=EOBC Niantic-Switch Link Down
RP/0/RSP0/CPU0:Nov 29 13:53:21.347 UTC: epm_switch_rp[60]: %PLATFORM-EPM_SWITCH-6-EOBC_LINK_DOWN_CLEARED : EOBC port 51 critical link down cleared
RP/0/RSP1/CPU0:Nov 29 13:53:27.908 UTC: canb-server[156]: %PLATFORM-CANB_SERVER-3-ERROR_INFO : Error from CBC in slot 0/1/CPU0, error msg Device=INTEL 82599, inst=0, errCode=EOBC Niantic-Switch Link Down
RP/0/RSP0/CPU0:Nov 29 13:53:27.916 UTC: canb-server[156]: %PLATFORM-CANB_SERVER-3-ERROR_INFO : Error from CBC in slot 0/1/CPU0, error msg Device=INTEL 82599, inst=0, errCode=EOBC Niantic-Switch Link Down
RP/0/RSP0/CPU0:Nov 29 13:53:34.382 UTC: shelfmgr[436]: %PLATFORM-SHELFMGR_HAL-6-BOOT_REQ_RECEIVED : Boot Request from 0/1/CPU0, RomMon Version: 18.27
RP/0/RSP0/CPU0:Nov 29 13:53:34.383 UTC: shelfmgr[436]: %PLATFORM-MBIMGR-7-IMAGE_VALIDATED : Remote location 0/1/CPU0: : MBI tftp:/disk0/asr9k-os-mbi-6.4.2.CSCvj68649-1.0.0/lc/0x3C0266/mbiasr9k-lc-x86e.vm validated
RP/0/RSP0/CPU0:Nov 29 13:53:34.386 UTC: shelfmgr[436]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/1/CPU0 A9K-48X10GE-1G-TR state:MBI-BOOTING
RP/0/RSP1/CPU0:Nov 29 13:53:44.609 UTC: canb-server[156]: %PLATFORM-CANB_SERVER-3-ERROR_INFO : Error from CBC in slot 0/1/CPU0, error msg Device=INTEL 82599, inst=0, errCode=EOBC Niantic-Switch Link Down
RP/0/RSP0/CPU0:Nov 29 13:53:44.618 UTC: canb-server[156]: %PLATFORM-CANB_SERVER-3-ERROR_INFO : Error from CBC in slot 0/1/CPU0, error msg Device=INTEL 82599, inst=0, errCode=EOBC Niantic-Switch Link Down

 

0 Helpful

Go to solution
nesttech1
Level 1
Level 1
In response to tkarnani

‎11-29-2019 10:51 AM

Thank you very much for your reply and simulating this condition.

Below you will find the output of the command that you have suggested.  Let me know if this looks normal.

show controllers epm-switch port-mapping location 0/RSP0/CPU0 

RP/0/RSP0/CPU0:ios#show controllers epm-switch port-mapping location 0/RSP0/CP$

Fri Nov 29 11:06:39.903 UTC

   Port     |    Link Status  |      Vlan     | Connected to

------------|-----------------|---------------|---------------

        0   |          Down   |  VLAN_EOBC_0  |  OFFLOAD_0_0

        1   |          Down   |  VLAN_EOBC_0  |  OFFLOAD_1_0

        2   |          Down   | VLAN_DEFAULT  |  OFFLOAD_2_0

        3   |          Down   | VLAN_DEFAULT  |  OFFLOAD_3_0

        9   |            Up   |    VLAN_PUNT  |  NIANTIC_0_0

       16   |            Up   |    VLAN_PUNT  |  FPGA_6_0

       17   |            Up   |    VLAN_PUNT  |  FPGA_5_0

       18   |            Up   |    VLAN_PUNT  |  FPGA_7_0

       19   |            Up   |    VLAN_PUNT  |  FPGA_4_0

       23   |            Up   | VLAN_10XGE_SWITCH  |  10XGE SWITCH_0_0

       24   |            Up   |  VLAN_EOBC_0  |  NIANTIC_1_1

       25   |            Up   |  VLAN_EOBC_1  |  NIANTIC_1_0

       36   |            Up   |    VLAN_PUNT  |  FPGA_0_0

       37   |            Up   |    VLAN_PUNT  |  FPGA_1_0

       38   |            Up   |    VLAN_PUNT  |  FPGA_2_0

       39   |            Up   |    VLAN_PUNT  |  FPGA_3_0

       41   |            Up   | VLAN_DEFAULT  |  NIANTIC_2_0

       48   |            Up   |  VLAN_EOBC_0  |  LC_EOBC_0_0

       51   |            Up   |  VLAN_EOBC_0  |  LC_EOBC_1_0

       54   |            Up   |  VLAN_EOBC_0  |  PEER_RP_0_0

       55   |            Up   |  VLAN_EOBC_1  |  PEER_RP_0_1

----------------------------------------------------------

We have 2 of these A9K-48X10GE-1G-TR modules installed in the ASR-9904, and we have not upgraded the FPDs on the second unit yet.  Both modules boot into the "IOS XR RUN" state, but the unit which was not upgraded does not show any of these messages while booting

%PLATFORM-CANB_SERVER-3-ERROR_INFO : Error from CBC in slot 0/1/CPU0, error msg Device=INTEL 82599, inst=0, errCode=EOBC Niantic-Switch Link Down

Here is the output of the "show hw-module fpd location all" command

 

RP/0/RSP0/CPU0:ios(admin)#sh hw-module fpd loc all

Tue Nov 26 15:27:47.106 UTC

 

===================================== ==========================================

                                      Existing Field Programmable Devices

                                      ==========================================

                                        HW                       Current SW Upg/

Location     Card Type                Version Type Subtype Inst   Version   Dng?

============ ======================== ======= ==== ======= ==== =========== ====

0/RSP0/CPU0  A9K-RSP880-SE              1.0   lc   cbc     0      34.39     No

                                              lc   rommon  0      10.65     No

                                              lc   fpga2   0       0.66     No

                                              lc   fsbl    0       1.109    No

                                              lc   lnxfw   0       1.109    No

                                              lc   fpga3   0       0.16     No

                                              lc   fpga4   0       0.16     No

                                              lc   fpga5   0       0.12     No

                                              lc   fpga6   0       0.08     No

--------------------------------------------------------------------------------

0/FT0/SP     ASR-9904-FAN               1.0   ft   cbc     7      31.06     No

--------------------------------------------------------------------------------

0/BPID0/SP   ASR-9904-BPID2             1.0   bp   cbc     11      7.105    No

--------------------------------------------------------------------------------

0/PS0/M0/SP  PWR-3KW-AC-V2              1.0   pm   fpga14  13      3.18^    No

                                              pm   fpga15  13      3.06^    No

                                              pm   fpga16  13      3.12^    No

--------------------------------------------------------------------------------

0/PS0/M1/SP  PWR-3KW-AC-V2              1.0   pm   fpga14  14      3.18^    No

                                              pm   fpga15  14      3.06^    No

                                              pm   fpga16  14      3.12^    No

--------------------------------------------------------------------------------

0/PS0/M2/SP  PWR-3KW-AC-V2              1.0   pm   fpga14  15      3.18^    No

                                              pm   fpga15  15      3.06^    No

                                              pm   fpga16  15      3.12^    No

--------------------------------------------------------------------------------

0/PS0/M3/SP  PWR-3KW-AC-V2              1.0   pm   fpga14  16      3.18^    No

                                              pm   fpga15  16      3.06^    No

                                              pm   fpga16  16      3.12^    No

--------------------------------------------------------------------------------

0/RSP1/CPU0  A9K-RSP880-SE              1.0   lc   cbc     0      34.39     No

                                              lc   rommon  0      10.65     No

                                              lc   fpga2   0       0.66     No

                                              lc   fsbl    0       1.109    No

                                              lc   lnxfw   0       1.109    No

                                              lc   fpga3   0       0.16     No

                                              lc   fpga4   0       0.16     No

                                              lc   fpga5   0       0.12     No

                                              lc   fpga6   0       0.08     No

--------------------------------------------------------------------------------

0/0/CPU0     A9K-48X10GE-1G-TR          1.0   lc   cbc     0      47.03     No

                                              lc   rommon  0      18.24     Yes

                                              lc   fpga2   0       1.87     Yes

                                              lc   fsbl    0       1.104    Yes

                                              lc   lnxfw   0       1.104    Yes

                                              lc   fpga3   0       1.00     No

                                              lc   fpga4   0       1.09     Yes

--------------------------------------------------------------------------------

0/0/CPU0     A9K-48X10GE-1G-TR          1.0   lc   fpga3   1       1.00     No

--------------------------------------------------------------------------------

0/1/CPU0     A9K-48X10GE-1G-TR          1.0   lc   cbc     0      47.03     No

                                              lc   rommon  0      18.27     No

                                              lc   fpga2   0       1.88     No

                                              lc   fsbl    0       1.110    No

                                              lc   lnxfw   0       1.110    No

                                              lc   fpga3   0       1.00     No

                                              lc   fpga4   0       1.11     No

--------------------------------------------------------------------------------

0/1/CPU0     A9K-48X10GE-1G-TR          1.0   lc   fpga3   1       1.00     No

--------------------------------------------------------------------------------

Thank you again for any feedback that you can provide on this issue.

 

0 Helpful

Go to solution
tkarnani
Cisco Employee
Cisco Employee
In response to nesttech1

‎12-01-2019 05:09 AM

These are the links we are interested in

48   |            Up   |  VLAN_EOBC_0  |  LC_EOBC_0_0

51   |            Up   |  VLAN_EOBC_0  |  LC_EOBC_1_0

As long as they come up, you should be fine

0 Helpful

Go to solution
nesttech1
Level 1
Level 1
In response to tkarnani

‎12-01-2019 09:49 AM

Thank you!
0 Helpful

Go to solution
xthuijs
Cisco Employee
Cisco Employee
In response to nesttech1

‎12-01-2019 05:11 AM

hi!

if the card booted up correcty, and is functioning then this is likely not something to worry about (but always a nuisance when you get scary messages) and is likely described with: CSCvn69012

 

the thing is that during boot, the link status is incorrectly interpreted and reported down.

this is cosmetic if the card boots up correctly. if the eobc (ether out of band, used for rp<>lc communications for image download, lc programming etc) was truly down

then heartbeats would fail and the card would be reloading constantly and eventually end up in-reset.

 

the root cause is a rommon operation if I am not mistaken. so you could update all fpd's but skip the rommon for the other card possibly.

 

cheers!

xander

0 Helpful
Customers Also Viewed These Support Documents