Sadananda Phadke
Cisco Employee
Cisco Employee
  • Introduction:

The goal of this document is to provide a recommended procedure to deploy SMU for

CSCuz14087 A9K-2x100GE-SE cards with capability field 2 Fails to boot after FPD upg

CSCva07986 A9K-2x100GE-SE :new FPDs imux-1.4, emux-1.4 and cgmac-41

CSCvb23010 533/534 FPD SMU for A9K-2x100GE-SE/TR and A9K-1x100GESE/TR

CSCvb31695 Intermittent FPD upgrade failure when upgrading FPD's on multiple cards

In the event this procedure is not followed, there is a possibility of below set of line cards fail to boot.

Following are the affected Line cards:

  •             A9K-2x100GE-SE
  •            A9K-2x100GE-TR
  •            A9K-1x100GE-SE
  •            A9K-1x100GE-TR

Note: Additionally only above cards with specific internal revision are affected. More information is below.

Note: Tomahawk/NP5 based cards are not subject to this write up.

Some of the SMU's may not be posted yet. But all are currently under testing.

  • Who should read this document

If you have a Typhoon based 100G card as per PID list above

If you have configured auto FPD in the admin config

If you are upgrading from a 5.1.1+ release to XR 5.3.3 SP4

  • Issue in detail:

There are three versions of above 100Gig Enhanced Ethernet line cards in the field. Though there is no difference in functionality of these line cards, cards with capability value 00, 02 and 03 have different version FPD’s. This is due to the fact that previous version cards (02 and 00) have older version ASIC. These cards have been differentiated using “Capability value” as shown below.

Following command shows card version (See the first Capability field)

 RP/0/RP0/CPU0:asr9k-1#admin show diag 0/1/CPU0 eeprom-info | include Cap
Thu Aug 28 22:24:44.460
Capabilities : 00 <== NOT affected by this issue
Capabilities : 10 <== Not important for this issue

RP/0/RP0/CPU0:asr9k-1#admin show diag 0/2/CPU0 eeprom-info | include Cap
Thu Aug 28 22:28:44.460
Capabilities : 02 <== affected by this issue
Capabilities : 10 Not important for this issue

RP/0/RP0/CPU0:MT02#admin show diag 0/3/CPU0 eeprom-info | include Cap
Thu Aug 28 22:34:44.460 UT
Capabilities : 03 <== NOT affected by this issue
Capabilities : 10

 

Due to the issue documented in CSCuz14087, 02 capability line cards gets loaded with FPD image of 03 capabilities line cards. Hence after FPD upgrade, LC’s with capability 02 may not boot.

This issue is not applicable to “Capability 03” and “capability 00” line cards as correct version FPD is getting loaded.

  • Solution in detail

  When “fpd auto-upgrade” command is configured and router is reloaded, target pies and SMU’s will be loaded after completion of FPD upgrade. Since changes from CSCuz14087 not yet taken effect, affected LC's not identified correctly. Hence this will result in loading of wrong FPD’s on affected line cards.

  Manual upgrade of FPD's, after router is upgraded to target release along with CSCuz14087 will fix this issue.

  • SMU upgrade procedure for 5.3.2

1. Disable auto FPD upgrade in admin plane. Example output:
RP/1/RSP1/CPU0:asr9k-1#admin config
RP/1/RSP1/CPU0:asr9k-1(admin-config)#no fpd auto-upgrade
RP/1/RSP1/CPU0:asr9k-1(admin-config)#
RP/1/RSP1/CPU0:asr9k-1(admin-config)#commit

2. Install and activate CSCvb31695, CSCva12633, CSCuz14087 & CSCva07986 SMU’s together. Since all are reload SMU’s, it is important to activate all the SMU’s together to avoid multiple reloads.
RP/1/RSP1/CPU0:asr9k-1#admin install add source <> <SMU’s>
3. Router will be reloaded as above SMU’s are reload

4. After the reload, using below command ensure CSCvb31695, CSCva12633, CSCuz14087 and CSCva07986 SMU's are active:
RP/1/RSP1/CPU0:asr9k-1#admin show install active summary

5. Check if FPD upgrade is required using below command:
Admin show hw-module fpd location all

6. If FPD upgrade is needed, upgrade all the FPD’s using below command:
Admin upgrade hw-module fpd all location all

7. Once FPD upgrade is successful, reload the router
Note: If if number of LCs/FC/RP's needs to be reloaded within acceptable limit, specific hardware can be reset instead of full router reload.
But in some cases, router reload might be easier from operational point of view.
  • SMU upgrade procedure for 5.3.3 and 5.2.4

1. Disable auto FPD upgrade in admin plane. Example output:
RP/1/RSP1/CPU0:asr9k-1#admin config
RP/1/RSP1/CPU0:asr9k-1(admin-config)#no fpd auto-upgrade
RP/1/RSP1/CPU0:asr9k-1(admin-config)#
RP/1/RSP1/CPU0:asr9k-1(admin-config)#commit

2. Install and activate CSCva12633, CSCuz14087 & CSCvb23010 (Supercedes CSCva07986) and CSCvb31695 SMU’s together. In 5.3.3 and 5.2.4, CSCva12633 and CSCuz14087 are reload SMU's and CSCvb23010 is a process restart
RP/1/RSP1/CPU0:asr9k-1#admin install add source <> <SMU’s>

3. Router will be reloaded as above SMU’s are reload

4. After the reload, using below command ensure CSCvb31695, CSCva12633, CSCuz14087 and CSCvb23010 SMU’s are active:
RP/1/RSP1/CPU0:asr9k-1#admin show install active summary

5. Check if FPD upgrade is required using below command:
RP/1/RSP1/CPU0:asr9k-1#Admin show hw-module fpd location all

6. If FPD upgrade is needed, upgrade all the FPD’s using below command:
RP/1/RSP1/CPU0:asr9k-1#Admin upgrade hw-module fpd all location all

7. Once FPD upgrade is successful reload the router.

8. NOTE: If these SMU’s are included in SP(service pack), auto fpd needs to be disabled before installing the SP.
  • Upgrade from 423, 511,512,513,514,520, 521, 523, 525, 530, 531 release to 5.3.2,5.3.3,5.3.4, 6.x release

1.Disable auto FPD upgrade in admin plane. if below step is not performed some of the affected LC's may not boot.
Example output:
RP/1/RSP1/CPU0:asr9k-1#admin config
RP/1/RSP1/CPU0:asr9k-1(admin-config)#no fpd auto-upgrade
RP/1/RSP1/CPU0:asr9k-1(admin-config)#
RP/1/RSP1/CPU0:asr9k-1(admin-config)#commit

2. Upgrade the router to target release with following SMU's:
1. Target release 532:
a) CSCva07986
b) CSCuz14087 <=== Mandatory SMU
c) CSCva12633
d) CSCvb31695 [Not yet posted. Under test]

2. Target release 533 or 5.2.4
a). CSCvb23010
b). CSCuz14087 <=== Mandatory SMU
c). CSCva12633
d). CSCvb31695 [Not yet posted. Under test]

3. After router upgrade, manually upgrade the FPD's.
1. If CSCvb31695 SMU is activated, upgrade the FPD's using below command:
admin upgrade hw-module fpd all location all
2. If CSCvb31695 SMU is not activate, upgrade the FPD's using below command:
admin upgrade hw-module fpd all location <one LC at a time>

4. Reload the router to activate the FPD's.

  • Upgrading to 5.3.4, 6.x release

1.Disable auto FPD upgrade in admin plane. if below step is not performed some of the affected LC's may not boot.
Example output:
RP/1/RSP1/CPU0:asr9k-1#admin config
RP/1/RSP1/CPU0:asr9k-1(admin-config)#no fpd auto-upgrade
RP/1/RSP1/CPU0:asr9k-1(admin-config)#
RP/1/RSP1/CPU0:asr9k-1(admin-config)#commit
2. SMU for CSCuz14087 is not required as release itself has this fix incorporated.
3. Manually upgrade the FPD's and reload the router.