ASR9010 fpd upgrade issue

[mjgarate]

Hi, do you know how to recover a line card from rommon?

We insert 2 linecards A9K2T-20GE-B in an ASR9010 with version 4.2.3 and noticed  that they needed a fpd upgrade, however after the fpd upgrade and the linecards reload, some of the LC did'nt came up and their state are flapping in MBI-BOOTING to MBI-RUNNING – BRINGDOWN and ROMMON.

We’ve installed two A9K2T-20GE-B on each chassis, however in some of them didn’t fail any LC, in others fail only one, and in others both of them.

We are trying restarting all the chassis and downgrading the software version to 4.2.1 but the problem persists.

Thanks

[smilstea · Accepted solution]

Hi,

There are a few reasons why the cards could be having this issue. Without seeing error messages it is difficult to determine what the issue is.

Given that the cards previously booted and that an FPD upgrade occured, before the reboot and card failing, I would say check the FPD trace and 'show log' for FPD information around the time the event occured.

However, because the card is getting to MBI-RUN this sounds like an install issue of some sort. This could be package corruption, sync issue, XR-RUN heartbeat never occurs, or another issue.

The most helpful right now will likely be 'show log' as that should detail the FPD operation and the cards rebooting, along with any other error messages.

I would also recommend collecting the shelfmgr status on the card as that will provide some details as to why it last went down.

'admin show fpd trace'
'admin show log'
'show shelfmgr status loc <>'

It is likely that a TAC case will need to be opened as looking into these types of problems can involve collecting a lot of data and troubleshooting often requires TAC assistance.

Thanks,

Sam Milstead

CSE - XR TAC