Manually upgrade ROMMON Catalyst 9500 X16

[lzampetti]

I have two stand-alone C9500 X16 routers that need to be upgraded from Amsterdam 17.3.3 to Bengaluru 17.6.3. 

It states that this process requires that both the primary and golden SPI flash devices be upgraded manually and that this can be done either before or after the IOS upgrade. 

It gives me these commands to do so. 

Primary - upgrade rom-monitor capsule primary switch 

Golden - upgrade rom-monitor capsule golden switch

Since it says I can do this prior to the upgrade I can just walk up to the switches and run each command and I will be moved from the 17.3.2r bootloader to 17.6.1r just like that and ready to perform the IOS upgrade?

I can't test this before hand because this is specific to our C9500 X16 switches and I don't have a test so some clarification would be nice before I install the IOS upgrade and hope it boots correctly. 

Any help is appreciated. 

[balaji.bandi]

yes you can upgrade ROMMON, before upgrade check you may have latest.

After the ROMMON is upgraded, it will take effect on the next reload. If you go back to an older release after this, the ROMMON is not downgraded. The updated ROMMON supports all previous releases.

https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst9500/software/release/17-6/release_notes/ol-17-6-9500.html#concept_ycv_jdf_3mb__table_9500_xph_gj1_1lb

[lzampetti]

Just to be clear. In order to update the ROMMON and successfully install Bengaluru all I need to do is run those two commands before I install the IOS upgrade. I don't have to do anything else to update the ROMMON.  I am sorry to ask but I don't understand how ROMMON is updating without me providing an update package or something. 

[bmerrill]

This is what I experienced on a 40x which also does not automatically update ROMMON as part of the IOS upgrade.   We have the 40x in a Virtual Stackwise pair.    AFTER IOS is upgraded (in our case, we went to 17.09.04a), I ran "upgrade rom-monitor capsule primary switch active R0".  This asked for confirmation and then immediately rebooted only the Active switch in the pair to update the ROMMON.  The new ROMMON must come from the currently installed IOS package.  Once the updated switch was back in the stack in Standby Ready mode, I ran the upgrade command again and the 2nd switch was updated.  I know these are old posts but since these are the ones I saw when searching for the correct method, and I didn't see anyone else including these steps, I wanted to include them.

[Leo Laohoo]

Upgrading the Golden Capsule is on a per-individual-chassis basis.  It is very time-consuming, particularly on large stacks that requires TWO (2) reboots.  

[bmerrill]

I did not upgrade the Golden capsule, just the Primary capsule.  It also was per switch individual basis and did take around 15 minutes each.  In our case, since it was a stackwise pair with everything redundantly connected, we could do the update with no downtime to attached systems.

[Leo Laohoo]

ROMMON (aka micro-code) and Golden Capsule are two separate things.  

Only on switches, ROMMON gets upgraded automatically.  During the upgrade from 17.3.X to 17.6.X, connect a console cable and there will be a microcode upgrade as soon as the switch reloads.

[lzampetti]

I connected to the console for the upgrade and stayed connected through the reload. The FPGA was upgraded in the process and so was the IOS but the ROMMON was not. I was able to boot the new OS (Bengaluru 17.6.3) but I am still at 17.3.2r when I think should be at 17.6.1r, at least according to the documentation. Since the ROMMON was not upgraded how do I go about it now?

[Leo Laohoo]

Please provide some explanation the obsession to upgrade the ROMMON?

[lzampetti]

I thought it was a necessary part of upgrading the IOS when a new one is available.
I upgraded several other switches where the ROMMON was upgraded automatically.
The release notes for the 9500 series and Bengaluru (Pg. 17) state that there is a new ROMMON available and that it would need to be installed manually using the upgrade-rommon capsule primary switch command.
I have found that the upgrade was not required to update the IOS and boot successfully. Had the documentation not mentioned a new ROMMON or that it would need to be upgraded manually I wouldn't have thought to question my ability to upgrade the IOS without it.

[Leo Laohoo]

---

@lzampetti wrote:
The release notes for the 9500 series and Bengaluru (Pg. 17) state that there is a new ROMMON available and that it would need to be installed manually using the upgrade-rommon capsule primary switch command.

---

That is not correct at all.  

ROMMON (aka micro-code) and Golden Capsule are two separate things.  

It is true then if there is a new ROMMON version, Catalyst switches will automatically upgrade the ROMMON/micro-code, however, there is no operational need to upgrade the Golden Capsule unless you want to risk bricking the switch.  

[lzampetti]

I thought it was a fair question having read this. 

On the C9500-12Q, C9500-16X, C9500-24Q, C9500-40X models of the series, you must manually
upgrade the ROMMON in the primary SPI flash device, if a new version is applicable, and the release
you are upgrading from is Cisco IOS XE Gibraltar 16.12.1 or a later release. (So if you upgrade from
Cisco IOS XE Gibraltar 16.11.1 for example, a manual upgrade does not apply; the ROMMON is
automatically updated, if applicable). Enter theupgrade rom-monitor capsuleprimary switchcommand
in privileged EXEC mode.

[Leo Laohoo]

Ok, let me spell it out in plain black-n-white: 

1. Cisco is no longer testing all the firmware they release.  Cisco relies mainly on paying customers as testers.  
2. Like the ROMMON, Golden Capsule is a partition of the switch that can only be accessed via TAC-generated token. 

Take Points 1 and Points 2 together and the result is this: 

If Cisco developers release a Golden Capsule with a bug in it (remember, Point #1), the switch is bricked.  TAC will not be able to fix it.  The dead switch except to RMA it.

Personally, I do not recommend anyone do any Golden Capsule Upgrade because: 

• It does not make any operational difference if the Golden Capsule is not upgraded. There is no operational benefit upgrading Golden Capsule.  
• There is a strong chance just doing a Golden Capsule Upgrade will cause a permanent/irreversable and irrepairable damage to the switch with a buggy firmware.  

[gdspa]

Do you mean we should not trust Cisco release notes?

I have to upgrade a 9500-16x from 16.12.4 to 17.9.3, I have

ROM: IOS-XE ROMMON
BOOTLDR: System Bootstrap, Version 16.12.2r, RELEASE SOFTWARE (P)

Should I upgrade rommon manually before upgrading firmware? If yes, I read in RN that I need to use these commands

upgrade rom-monitor capsule primary switch

upgrade rom-monitor capsule golden switch

Not clear if internet access is needed.

[Leo Laohoo]

---

@gdspa wrote:
Should I upgrade rommon manually before upgrading firmware?

---

Please read my previous responses.  

Tell me where did I say the ROMMON (aka micro-code) can be upgraded manually.