Upgrading IOS XE - Certificate Error

[pschulz]

In an environment with multiple routers of the ISR 4000 series (ISR 4451-X specifically), some are running IOS XE 16.6.9 and some are running the old version 3.16. Since we have the IOS binary for 16.6.9, I want to load this image onto all routers so they all run the same version.

This is traditional Cisco licensing, so normally this should just work by copying the new binary and change the

boot system

config command.

However, when reloading such a router, it comes up with the error that it cannot verify the digital signature of the IOS image: 

	Signature verification failed for key# 2
	Signature verification failed for key# 3
	Failed to validate digital signature
	RSA Signed REVOCATION Image Signature Verification Failed.
	Package Load Test Latency : 6940 msec
	
	Unsigned package found,  aborting ...
	boot: error executing "boot bootflash:isr4400-universalk9.16.09.07.SPA.bin.bin"
	autoboot: boot failed, restarting...

Do I need to copy additional files, such as new certificates, or is that device irrevocably tied to that version by Cisco?

[Flavio Miranda]

Hi @pschulz 

 Similar issue in another post refers to rommon upgrade as solution

https://community.cisco.com/t5/network-management/isr-4400-firmware-upgrade/td-p/3766072

[Joseph W. Doherty]

BTW, if you don't have a maintenance contract for the switches you wish to upgrade, copying a newer IOS from another switch would be software piracy; which most businesses don't want any part of.

[pschulz]

Quite correct on the maintenance contract with current Cisco licensing. 

Has this actually always been the case - downloads of software used to be free, years ago, with traditional licensing, and I cannot find Cisco terms which explicitly forbid an update for that type of old binary - after all, the licenses are permanent.

Certainly, not true today, but do you have a document showing it also is true fo the classic licensing binaries on older models?

[Joseph W. Doherty]

"Has this actually always been the case - downloads of software used to be free, years ago, with traditional licensing, and I cannot find Cisco terms which explicitly forbid an update for that type of old binary - after all, the licenses are permanent."

Good questions.  Hard to find answers for pre-/on- device licensing. 

Since in your case, since 4551-Xs are involved, and current, you should be able to contact Cisco to determine if you're legally entitled to move from 3.16 to 16.6.9 without an active maintenance contract.   If the answer is yes, Cisco might even provide you access to the software (but, of course, without a maintenance contract, no support/advice in applying the update - or any help with post update issues).

If you contact Cisco, you'll know for sure where you stand legally, moving from 3.16 to 16.6.9, otherwise you might unknowingly put your business into software piracy jeopardy.

Just for chuckles, I tried to download 16.6.9 and got:

So the download software appears to want to service contract.

[pschulz]

Yes, since some years, Cisco has walled off any download so that we must log in and have an active contract. And if you do log in with such a contract it gets you to agree to terms which indicate that this is the only legal way to do it and Cisco may charge you if you don't comply. 

When older software contains security vulnerabilities classified as high or critical, Cisco TAC might on a case-to-case basis allow you to download a newer image which does not contain the bug.

This is per the Cisco security policy documented here

https://sec.cloudapps.cisco.com/security/center/resources/security_vulnerability_policy.html#ssu 

where they state

"As a special customer service, and to improve the overall security of the internet, Cisco may offer customers free software updates to address high-severity security problems. The decision to provide free software updates is made on a case-by-case basis. Refer to the Cisco security publication for details. Free software updates will typically be limited to Critical- and High-severity vulnerabilities.

If Cisco has offered a free software update to address a specific issue, noncontract customers who are eligible for the update may obtain it by contacting the Cisco TAC using any of the methods described in the General Security-Related Queries section of this document."

This does not solve the question whether it's legal or not to install random images pulled off other routers - but if Cisco does agree to update based on a vulnerability, they give you a download which definitely is valid and legal.

[Joseph W. Doherty]

"This does not solve the question whether it's legal or not to install random images pulled off other routers . . ."

Agreed!

Which is also why I suggested you contact Cisco for "their" understanding of "their" licensing.  (As, I believe, they would be the plaintiff in a software piracy case.)

Of course, possibly you and Cisco might have different interpretations of "their" licensing.  Then you could fight it out in court.

But if they agree, that the original license holder is entitled to software updates (including what they consider a major revision, i.e. version number change), you should be able to update your switches.

Another alternative is to read the Cisco licenses that came with, and/or applies to, your 3.16 and/or 16.6.9 installations.  It might address the update question, but if not, I don't know what the legal "default" would be (although I presume the legal default would be you don't have update entitlement).

I don't work for Cisco, never have.  Any place I worked at with Cisco platforms, either had active maintenance for a particular/specific platform, or not.  If the former, mute question.  If the latter, the general consensus was, we could not update the software (even in the good old days when Cisco allowed us to download software and devices didn't have any license management on them).  (One place I worked at [BTW, not a Mom and Pop - actually one of the largest international software development companies], had lots of ISRs without a service contract, I did obtain newer IOS for some of them, due to major security flaws in what they were running.  Same place, I asked, why no Cisco maintenance on these?  They felt it was way too expensive for its benefit.  [This because they had an extensive, knowledgeable network staff, which didn't need TAC help.  Also back in the good old days when Cisco devices, literally would run for years. even decades, without needing a reload.]  I noted you're running Enterprise feature sets, but only using IPBase features.  If you downgrade the feature set, maintenance would be a [lot] less.  They did and it was.)

[pschulz]

Cannot find any specific license for a given IOS version.

The router itself points to the software terms at 

http://www.cisco.com/c/en/us/about/legal/cloud-and-software/software-terms.html. 

The link "View our Cisco End User License Agreement" shows the end-user license. None of the other docs on that site point to specific router models or IOS versions, so I must assume, in present time, we are bound by the above general end-user-agreement. I am not a lawyer, but I would guess this is the case.

There it states 

2.4 Upgrades or Additional Copies of Software. 
You may only use Upgrades or additional copies of the Software beyond 
Your license Entitlement if You have:
(a) acquired such rights under a support agreement covering the Software; or
(b) purchased the right to use Upgrades or additional copies separately

“Upgrades” means all updates, upgrades, bug fixes, error corrections, enhancements and 
other modifications to the Software

“Entitlement” means the specific metrics, duration, and quantity of Cisco Technology You 
commit to acquire from an Approved Source through individual acquisitions or Your participation 
in a Cisco buying program.

“Approved Source” means Cisco or a Cisco Partner

So for a router with a "traditional" license which came with the router, the "entitlement" is likely to mean this software applies to this specific device. 

No updates are then allowed, unless one has a service contract, or purchases updates from Cisco partners.

Again, I am not a lawyer, but if I can see it this way as a layman, I am sure Cisco is able to get any court to see it that way, if it comes to that.

[Joseph W. Doherty]

Okay, so I believe you too now believe, possibly, doing what you want to do in your OP might be considered software piracy, correct?

One option I pursued for another company, that wanted to update their IOS software, was not to buy into a maintenance contract, but to just purchase a current version's license.

Also, BTW, I believe (at least in the past), when you have a license for version X, you may run any prior version, with same or lower features.

[pschulz]

Thank you for clarifiying this - I definitely have a clearer picture now for dealing with this type of older devices, or older software.

BTW - for the current case, I did reach out to Cisco TAC and they sent me update software. That's all exactly in-line with the rules.

[Joseph W. Doherty]

"BTW - for the current case, I did reach out to Cisco TAC and they sent me update software. That's all exactly in-line with the rules."

Fantastic!

[M02@rt37 · Accepted solution]

Hello @pschulz,

Do you check the ROMmon compatibility ?

When you upgrade from

Cisco IOS XE 3.x to 16.x image

you should first upgrade the rommon release to the

16.7(5r) rommon

release. 

If not check here: https://www.cisco.com/c/en/us/td/docs/routers/access/4400/software/configuration/guide/isr4400swcfg/bm_isr_4400_sw_config_guide_chapter_0101.html#Cisco_Concept.dita_8829af23-df9e-4314-89d3-ab7e5a4e243a