F0 remains in init stat on C8500 router

[yacine feniche]

Hi all,

I have half a dozen routers, one of them boots continuously cause of what could be a hardware problem.
I tried with IOS versions 17.12.4 and 17.12.4b aned Rommon 17.10(1r) and 17.15(1r)
Other routeurs wich dons'nt face the probleme are in 17.12.4b and Rommon 14.10(1r)
Any help is welcome

Thanks

Router#sh platform
Chassis type: C8500-20X6C

Slot Type State Insert time (ago)
--------- ------------------- --------------------- -----------------
0 C8500-20X6C ok 00:01:40
0/0 20xSFP+ ok 00:01:03
0/1 6xQSFP ok 00:01:03
R0 C8500-20X6C ok, active 00:01:40
F0 C8500-20X6C init, active 00:01:40
P0 PWR-CH1-1100WAC ok 00:01:27
P1 PWR-CH1-1100WAC ok 00:01:27
P2 PWR-CH1-1100WAC ok 00:01:26
P3 PWR-CH1-1100WAC ok 00:01:26
P4 C8500-FAN-3R ok 00:01:28
P5 C8500-FAN-3R ok 00:01:28
P6 C8500-FAN-3R ok 00:01:28

Slot CPLD Version Firmware Version
--------- ------------------- ---------------------------------------
0 22081716 17.10(1r)
R0 22110916 17.10(1r)
F0 22110916 17.10(1r)

here are some logs

*Mar 19 12:14:24.376: %IOSXE-1-PLATFORM: R0/0: kernel: QFP0.0: Fatal Fault: SW reported: Userspace fault
*Mar 19 12:14:24.376: %IOSXE-1-PLATFORM: R0/0: kernel: QFP0.1: Fatal Fault: SW reported: Userspace fault
*Mar 19 12:14:24.376: %IOSXE-1-PLATFORM: R0/0: kernel: QFP0.2: Fatal Fault: SW reported: Userspace fault
*Mar 19 12:14:24.373: %CPPDRV-3-FATAL_INIT_ID: F0/0: cpp_driver: CPP0: (0x1) Failed to initialize init device drivers(s) - Operation not permitted -Traceback= 1#d5476962c2ba277876fb0ae17ed0a8c0 errmsg:7CB3DA855000+D7A :585CA1055000+7520 :585CA1055000+CEE5 cpp_drv_cmn:7CB3DCC6E000+24A88 binos:7CB3DAC72000+121AE :585CA1055000+9A50 cpp_common_os:7CB3DBAC0000+25069 evlib:7CB3DAC57000+8E56 evlib:7CB3DAC57000+9C60 :585CA1055000+8684 :585CA1055000+8482 :585CA1055000+7F84 c:7CB3D944D000+27ED0 :585CA1055000+61EA
*Mar 19 12:14:24.373: %CPPHA-3-INITFAIL: F0/0: cpp_ha_top_level_server: CPP 0 initialization failed - startup init (0x1)
*Mar 19 12:14:24.373: %CPPHA-3-INITFAIL: F0/0: cpp_ha_top_level_server: CPP 0 initialization failed - start CPP (0x1)
*Mar 19 12:14:24.373: %CPPHA-3-FAULT: F0/0: cpp_ha_top_level_server: CPP:0.0 desc:Platform Collection Error: Message STARTUP_INIT to client cpp_driver0 failed with error [Operation not permitted] det:HA class:CLIENT_SW sev:FATAL id:2 cppstate:RUNNING res:UNKNOWN flags:0x0 cdmflags:0x0
....
*Mar 19 13:10:37.647: %CMRP-3-CHASSIS_MONITOR_READY_TIME_EXCEEDED: R0/0: cmand: Reloading F0 because it has failed to become ready for packet processing

[Richard Burts]

I believe that this part of the output is significant "Traceback= 1#d5476962c2ba277876fb0ae17ed0a8c0". Traceback is ALWAYS a sign of a software problem. Try a different version of software.

[balaji.bandi]

May be bug :

https://bst.cisco.com/quickview/bug/CSCvt44674

there is 17.12.5 release try that one, still issue raise TAC case

[Richard Burts]

BB

Thanks. This reinforces my suggestion that it is a software problem and a different version of software would be the solution.

[yacine feniche]

thank you @Richard Burts and @balaji.bandi 
cause of the cyclic reboot, I didn't have enough time to load an other IOS file between two reloads.

The router went to RMA 

Yacine

[Leo Laohoo]

---

@yacine feniche wrote:

 0 22081716 17.10(1r)
R0 22110916 17.10(1r)
F0 22110916 17.10(1r)

---

ROMMON is version 17.10(1r) and the plan is to upgrade to firmware version 17.12.X?

Direct upgrade, from 17.3.X (and earlier), to 17.12.X (and later) is not allowed/supported.  Failure to heed this warning and the router will go into a boot-crash-loop.  And this fact or information is NOT DOCUMENTED by Cisco.

The reason why is because 17.12.X (and later) requires the router must be on ROMMON version 17.5(1r).  And ROMMON version 17.5(1r) is NOT available for download.  If the router upgrades first to 17.6.X or 17.9.X, the ROMMON gets automatically upgraded to the mandatory 17.5(1r).  

(And, as an insult to customers, the geniuses behind this decision did not document this anywhere else nor told anyone outside of the software development team and kept Cisco TAC, DNAC SME on SWIM in the dark.)

Therefore, the solution is to upgrade first to 17.6.X or 17.9.X before making the final jump to 17.12.X (or later).

EDIT: 

8500 ROMMON version 17.15(1r) is now available for download (18 June 2025).  

[Rich R]

> 8500 ROMMON version 17.15(1r) is now available for download (18 June 2025).  

But they still managed to incorrectly link the release notes to an old ISR 4400 upgrade guide!!!! 
And no longer allow feedback to report such errors <sigh>

[Leo Laohoo]

Just tested 8500 ROMMON version 17.15(1r) on a C8500-12X and the router rejected the ROMMON.  ROMMON upgrade works with 17.11(1r).  

It may look like the Software Developers did not test 17.15(1r).  Again.