Router freeze for long time not boot - after IOS upgrade

[robert-gorajski]

Hello!

I have a C1111-8P router. Unfortunately I can't update the IOS image. For the time being I have temporarily set gi0/0/1 as a DHCP client. Then I downloaded the IOS image from my TFTP server and calculated the MD5 sum.

Directory of bootflash:/

   11  drwx            16384  Apr 14 2021 08:13:07 +00:00  lost+found
163201  drwx             4096  Jan 16 2025 17:57:21 +00:00  .prst_sync
   12  -rw-        457963732  Apr 14 2021 08:21:21 +00:00  c1100-universalk9_ias_npe.16.09.05.SPA.bin
114241  drwx             4096  Jan 16 2025 17:56:07 +00:00  .installer
89761  drwx             4096  Apr 14 2021 08:27:24 +00:00  core
65281  drwx             4096  Apr 14 2021 08:26:02 +00:00  .rollback_timer
73441  drwx             4096  Apr 14 2021 08:26:03 +00:00  gs_script
106081  drwx            73728  Jan 16 2025 18:41:25 +00:00  tracelogs
122401  drwx             4096  Jul 10 2021 13:27:30 +00:00  .dbpersist
24481  -rw-               30  Jan 16 2025 17:57:21 +00:00  throughput_monitor_params
81601  drwx             4096  Apr 14 2021 08:27:02 +00:00  onep
24482  -rw-               35  Jan 16 2025 12:44:44 +00:00  pnp-tech-time
24483  -rw-            66041  Jan 16 2025 12:44:54 +00:00  pnp-tech-discovery-summary
24484  -rw-            10220  Jan 16 2025 17:57:31 +00:00  vlan.dat
24485  -rw-              567  Jul 11 2021 06:26:35 +00:00  1625984510647sh_tech.txt
24486  -rw-              567  Jul 11 2021 06:27:22 +00:00  1625984554263sh_tech.txt
24487  -rw-        701111540  Jan 16 2025 18:34:27 +00:00  c1100-universalk9.17.12.04a.SPA.bin
130561  drwx             4096  Jan 16 2025 14:50:11 +00:00  SHARED-IOX
24488  -rwx                0  Jan 16 2025 14:50:11 +00:00  mode_event_log
138721  drwx             4096  Jan 16 2025 14:50:11 +00:00  pcap

2908606464 bytes total (1578356736 bytes free)

Configuration:

Router(config)#boot system flash c1100-universalk9.17.12.04a.SPA.bin

Router#show bootvar
BOOT variable = c1100-universalk9_ias_npe.16.09.05.SPA.bin,1;c1100-universalk9.17.12.04a.SPA.bin,1;

Router(config)#no boot system flash c1100-universalk9_ias_npe.16.09.05.SPA.bin


Router#show bootvar
BOOT variable = c1100-universalk9.17.12.04a.SPA.bin,1;
CONFIG_FILE variable does not exist
BOOTLDR variable does not exist
Configuration register is 0x2102

Standby not ready to show bootvar
Router(config)#do wr
Building configuration...

[OK]
Router(config)#
*Jan 16 18:50:31.493: %SYS-2-PRIVCFG_ENCRYPT: Successfully encrypted private config file


Router(config)#reload

Reload:

System Bootstrap, Version 16.9(1r), RELEASE SOFTWARE
Copyright (c) 1994-2018  by cisco Systems, Inc.


Current image running: Boot ROM0

Last reset cause: LocalSoft
C1111-8P platform with 4194304 Kbytes of main memory


........

Located c1100-universalk9.17.12.04a.SPA.bin
####################################################
.
.
.
Package header rev 3 structure detected
IsoSize = 646684672
Calculating SHA-1 hash...Validate package: SHA-1 hash:
        calculated C8FC40FC:3F5B9694:390E1398:ACD90582:1CB4E2E6
        expected   C8FC40FC:3F5B9694:390E1398:ACD90582:1CB4E2E6

RSA Signed RELEASE Image Signature Verification Successful
Image validated

Router freeze for long time, and show error:

System Bootstrap, Version 16.9(1r), RELEASE SOFTWARE
Copyright (c) 1994-2018  by cisco Systems, Inc.

Current image running: Boot ROM0

Last reset cause: Unrecoverable Error
C1111-8P platform with 4194304 Kbytes of main memory

After rollback to universalk9_ias_npe.16.09.05.SPA.bin with rommon. The router starts without any errors.
Can I ask for help in solving this problem?

[Leo Laohoo · Accepted solution]

---

@robert-gorajski wrote: 

System Bootstrap, Version 16.9(1r), RELEASE SOFTWARE

---

I suspect this is the problem -- ROMMON version 16.9(1r). 

This is going to be a complicated answer.  

Refer to Cisco ISR1000 ROMmon Compatibility Matrix (Table 3) and it specifically states that the router needs to be on ROMMON version 17.5(1r) if upgrading to 17.5.X 17.6.X or 17.9.X. 

Finally, it is not possible to perform a direct upgrade, from 16.X.X, to 17.12.X (and later) without going through an intermediate version like 17.6.X or 17.9.X.  This information or fact is undocumented.  The reason why is because of ROMMON version 17.5(1r), which is not a downloadable file.  

ROMMON version 17.5(1r) is incorporated into 17.6.X and 17.9.X only.  It is not incorporated in 17.12.X nor 17.15.X and will not be incorporated in any later versions other than 17.6.X and 17.9.X.  And like before, this fact is not documented anywhere else -- Even Cisco TAC was kept in the dark by this information.  

To upgrade to Cisco IOS XE Dublin 17.12.x, follow these steps:

1. If you are on a device that is running software version between Cisco IOS XE 16.x to Cisco IOS XE 17.4.x, upgrade to any IOS XE image between Cisco IOS XE 17.5.x to Cisco IOS XE 17.10.x.

2. After performing step a, upgrade to Cisco IOS XE 17.12.x.

3. For devices that are running on software version Cisco IOS XE 17.5.x or later, you can upgrade to Cisco IOS XE 17.12.x directly.

---

So here is the plan: 

1.  Force the router to boot 16.9.5.  

2.  Read this:  Cisco ISR & ASR 1k Routers: IOS-XE/Firmware Upgrade (Install Mode)

3.  Upgrade the ROMMON to version 17.12(1r) but DO NOT REBOOT the router.

4.  Upgrade to router firmware to either 17.6.X or 17.9.X.  Reboot

5.  Upgrade to router firmware to 17.12.X & reboot. 

[mrmadgig]

Hello,

Thank you for this, however not working for me at all.

I cannot find  version 17.12(1r)  so I tried without it

I do have 6.12 s

Do need to start with a different image? I have Fuji

sh version
Cisco IOS XE Software, Version 16.09.01
Cisco IOS Software [Fuji], ISR Software (ARMV8EB_LINUX_IOSD-UNIVERSALK9_IAS-M), Version 16.9.1, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2018 by Cisco Systems, Inc.
Compiled Tue 17-Jul-18 17:13 by mcpre

Slot CPLD Version Firmware Version
--------- ------------------- ---------------------------------------
0 17100501 16.12(1r)
R0 17100501 16.12(1r)
F0 17100501 16.12(1r)

I did get the GOTCHA

Copying package files
    c1100-firmware_c1100_gfast.17.06.01a.SPA.pkg
    c1100-firmware_c1100_shdsl.17.06.01a.SPA.pkg
    c1100-firmware_c1100_vadsl.17.06.01a.SPA.pkg
    c1100-firmware_p_1t.17.06.01a.SPA.pkg
    c1100-mono-universalk9.17.06.01a.SPA.pkg
    c1100-rpboot.17.06.01a.SPA.pkg
  WARNING: A different version of provisioning file packages.conf already exists in bootflash:
  WARNING: The provisioning file from the expanded bundle will be saved as
  WARNING: bootflash:c1100-universalk9.17.06.01a.SPA.conf
... package files copied
SUCCESS: Finished expanding all-in-one software package.
Oct  1 15:26:41.464 %INSTALL-5-OPERATION_COMPLETED_INFO: R0/0: packtool: Completed expand package bootflash:c1100-universalk9.17.06.01a.SPA.bin

Checked bootflash: and it doesn't matchup with what you have here or I cannot see it

#dir bootflash:
Directory of bootflash:/

   11  drwx            16384  Sep 30 2025 18:24:58 +00:00  lost+found
 8161  drwx             4096   Oct 1 2025 15:39:13 +00:00  .installer
32641  drwx             4096  Sep 30 2025 18:44:25 +00:00  core
163201  drwx             4096   Oct 1 2025 15:40:40 +00:00  .prst_sync
130561  drwx             4096  Sep 30 2025 18:43:03 +00:00  .rollback_timer
155041  drwx             4096  Sep 30 2025 18:43:05 +00:00  gs_script
57121  drwx            65536   Oct 1 2025 15:41:04 +00:00  tracelogs
65281  drwx             4096  Sep 30 2025 18:47:55 +00:00  .dbpersist
   12  -rw-               30   Oct 1 2025 15:40:41 +00:00  throughput_monitor_params
73441  drwx             4096  Sep 30 2025 18:44:21 +00:00  onep
   13  -rw-        458170120  Sep 30 2025 18:46:57 +00:00  c1100-universalk9_ias.16.09.01.SPA.bin
   14  -rw-               35  Sep 30 2025 18:49:24 +00:00  pnp-tech-time
   15  -rw-            57805  Sep 30 2025 18:49:26 +00:00  pnp-tech-discovery-summary
   16  -rw-             8051  Sep 30 2025 22:45:23 +00:00  running-config
   17  -rw-             4949   Oct 1 2025 00:31:41 +00:00  lash:c1100-universalk9_ias.16.08.01.SPA.bin
   18  -rw-             4919   Oct 1 2025 14:00:16 +00:00  packages.conf
   19  -rw-        705753752   Oct 1 2025 15:22:56 +00:00  c1100-universalk9.17.06.01a.SPA.bin
   20  -rw-              272   Oct 1 2025 14:33:19 +00:00  mode_event_log
89763  -rw-         13579316   Oct 1 2025 15:25:26 +00:00  c1100-firmware_c1100_gfast.17.06.01a.SPA.pkg
89764  -rw-         11527220   Oct 1 2025 15:25:26 +00:00  c1100-firmware_c1100_shdsl.17.06.01a.SPA.pkg
89765  -rw-          5592116   Oct 1 2025 15:25:26 +00:00  c1100-firmware_c1100_vadsl.17.06.01a.SPA.pkg
89766  -rw-          8451116   Oct 1 2025 15:25:26 +00:00  c1100-firmware_p_1t.17.06.01a.SPA.pkg
89767  -rw-        666993728   Oct 1 2025 15:25:30 +00:00  c1100-mono-universalk9.17.06.01a.SPA.pkg
89768  -rw-         57839132   Oct 1 2025 15:26:40 +00:00  c1100-rpboot.17.06.01a.SPA.pkg
89762  -rw-             7017   Oct 1 2025 15:26:40 +00:00  c1100-universalk9.17.06.01a.SPA.conf

#sh bootvar
BOOT variable = bootflash:packages.conf,12;
CONFIG_FILE variable =
BOOTLDR variable does not exist
Configuration register is 0x2102

Standby not ready to show bootvar

Continued to pour over instructions and looked at this as instructed.

#dir bootflash:.conf
%Error opening bootflash:/.conf (No such file or directory)

Last reset cause: LocalSoft
C1111-4P platform with 4194304 Kbytes of main memory


........

Located packages.conf
#

Unsupported package header version (1634562163)
Failed to boot file bootflash:packages.conf

........

autoboot: boot failed, restarting...

Resetting .......


Rom image verified correctly


System Bootstrap, Version 16.12(1r), RELEASE SOFTWARE
Copyright (c) 1994-2019  by cisco Systems, Inc.


Current image running: Boot ROM1

Last reset cause: LocalSoft
C1111-4P platform with 4194304 Kbytes of main memory


........

Located packages.conf
#

Unsupported package header version (1634562163)
Failed to boot file bootflash:packages.conf

........

autoboot: boot failed, restarting...

Resetting .......


Rom image verified correctly


System Bootstrap, Version 16.12(1r), RELEASE SOFTWARE
Copyright (c) 1994-2019  by cisco Systems, Inc.


Current image running: Boot ROM1

Last reset cause: LocalSoft
C1111-4P platform with 4194304 Kbytes of main memory


........

Located packages.conf
#

Unsupported package header version (1634562163)
Failed to boot file bootflash:packages.conf

........

autoboot: boot failed, restarting...

Resetting .......


Rom image verified correctly


System Bootstrap, Version 16.12(1r), RELEASE SOFTWARE
Copyright (c) 1994-2019  by cisco Systems, Inc.


Current image running: Boot ROM1

Last reset cause: LocalSoft
C1111-4P platform with 4194304 Kbytes of main memory


........

Located packages.conf
#

Unsupported package header version (1634562163)
Failed to boot file bootflash:packages.conf

.▒Rom image verified correctly


System Bootstrap, Version 16.12(1r), RELEASE SOFTWARE
Copyright (c) 1994-2019  by cisco Systems, Inc.


Current image running: Boot ROM1

Last reset cause: PowerOn
C1111-4P platform with 4194304 Kbytes of main memory

I can always boot is from romon> and then set the boot to an existing bootable image.

I cannot however get past anything else

Any advise on what I am missing here would be special as I have been in a personal loop with this for a day.

Thank you again for posting some help.

Joseph

[mrmadgig]

Well, Well,

Of course it was my mess. I saw my error after posting sorry. I didn't rename the package file and I must have looked at that  thousand times. I was concentrating on the date and not the file itself.

I got it now and thank you again 

[liviu.gheorghe]

Is there a particular reason for using IOS 17.12.4a? If there is, I would suggest upgrading first the ROMMON version to 17.5(1r) as per Cisco recommendation: https://www.cisco.com/c/en/us/td/docs/routers/access/isr1100/software/configuration/xe-17/isr1100-sw-config-xe-17/Configuring_ROMMON.html

and then try again with IOS 17.12.4a

If there isn't a good reason for IOS 17.12.4a, can you try booting the router with IOS 17.9.5e? I'm using some C1111 with this image and don't have any issues.

HTH

[julioms]

You must update the IOS with an intermediate version so that it updates the BIOS and with this last update you will be able to update the required IOS.

[Jens Albrecht]

In case of the Cisco 1100 Integrated Services Routers has published an updated ROMMON version.

This is somewhat surprising as all documentation I have seen states that starting with 17.3 the ROMMON upgrades will be part of the IOS-XE upgrade process. I can only guess that Cisco noticed that this causes a lot of problems if customers directly jump from really old 16.x versions to the latest ones.

The Software Download Center contains a new ROMMON version 17.15(1r) that can be downloaded here:

https://software.cisco.com/download/home/286315006/type/282046486/release/17.15(1r)

According to the documentation the latest ROMMON version is 17.5(1r) so the label 17.15.(1r) might just be a typo.

 HTH!

[Leo Laohoo]

---

@Jens Albrecht wrote:
In case of the Cisco 1100 Integrated Services Routers has published an updated ROMMON version.

---

Yes and no. 

Yes, Cisco has published ROMMON version 17.15(1r) but it is not for the mainstream 1100.  This ROMMON is specific only to "ISR 1100(X) 4G/6G".