CSCwa34149 - Cat9300 device restarts

AlinatechHO · ‎07-14-2023

Hi, the switch when restart show this message all time

Symptom: Cat9300 keeps restarting with the following error messages Oct 18 12:21:59.774: %PMAN-3-PROCHOLDDOWN: R0/0: The process cmand has been helddown (rc 134) Oct 18 12:21:59.876: %PMAN-0-PROCFAILCRIT: R0/0: pvp: A critical process cmand has failed (rc 134) Oct 18 12:21:59.933: %PMAN-3-RELOAD_RP: R0/0: pvp: Reloading: Switch will be reloaded Chassis 0 reloading, reason - Non participant detected Oct 18 12:22:00.653: %PMAN-5-EXITACTION: F0/0: pvp: Process manager is exiting: reload fp action requested Oct 18 12:22:02.152: %PMAN-5-EXITACTION:vp: Process manager is exiting: Critical process cmand fault on rp_0_0 (rc=134) Oct 18 12:22:05.338: %PMAN-3-PROCESS_NOTIFICATION: R0/0: pvp: System report /crashinfo/RP_0-system-report_local_20211018-122202-Universal.tar.gz (size: 12210 KB) generated

Leo Laohoo · ‎07-14-2023

Post the complete output to the following commands:

sh version
dir crashinfo:

And be aware the logs were written back in October.

JohnButterworthJr · ‎12-28-2023

I have virtually the exact same issue - the only difference being the dates/time are current. The entire event sequence (starts to boot, gets these errors, tries again) repeats 3 times, then puts me in ROMMON mode. The commands you suggested are not available in that mode. This is on a Cisco 9300-24P. System Bootstrap is 17.3.2r and IOS is 17.3.4. Any other suggestions?

Leo Laohoo · ‎12-28-2023

@JohnButterworthJr wrote:

I have virtually the exact same issue - the only difference being the dates/time are current. The entire event sequence (starts to boot, gets these errors, tries again) repeats 3 times, then puts me in ROMMON mode. The commands you suggested are not available in that mode. This is on a Cisco 9300-24P. System Bootstrap is 17.3.2r and IOS is 17.3.4. Any other suggestions?

Console into the switch and reboot.

Post the entire boot-up process.

Applmh · ‎04-29-2024

Initializing Hardware...

System Bootstrap, Version 17.9.2r, RELEASE SOFTWARE (P)
Compiled Thu 08/17/2023 14:22:55.52 by rel

Current ROMMON image : Primary
Last reset cause : SoftwareReload
C9300L-24T-4G platform with 8388608 Kbytes of main memory

switch: boot
boot: attempting to boot from [flash:packages.conf]
boot: reading file packages.conf
#
################################################################################
################################################################################
################################################################################
################################################################################
################################################################################
#####################

Apr 29 12:41:17.356: %INSTALL-5-INSTALL_AUTO_ABORT: R0/0: rollback_timer: Instal
l auto abort timer expired: initiating abort
Apr 29 12:41:20.147: %INSTALL-5-INSTALL_START_INFO: R0/0: install_engine: Starte
d install abort
Apr 29 12:41:39.578: %PMAN-3-PROCHOLDDOWN: R0/0: The process cmand has been held
down (rc 134)
Apr 29 12:41:41.161: %PMAN-0-PROCFAILCRIT: R0/0: pvp: A critical process cmand h
as failed (rc 134)
Apr 29 12:41:41.235: %PMAN-3-RELOAD_RP: R0/0: pvp: Reloading: Switch will be rel
oaded

Chassis 0 reloading, reason - Non participant detected
Apr 29 12:41:42.110: %PMAN-5-EXITACTION: F0/0: pvp: Process manager is exiting:
reload fp action requested
Apr 29 12:41:43.446: %PMAN-5-EXITACTIONApr 29 12:41:44.737: %PMAN-3-PROCESS_NOTI
FICATION: R0/0: pvp: System report /crashinfo/system-report_local_20240429-12414
3-Universal.tar.gz (size: 2097 KB) generated

Initializing Hardware...

System Bootstrap, Version 17.9.2r, RELEASE SOFTWARE (P)
Compiled Thu 08/17/2023 14:22:55.52 by rel

Current ROMMON image : Primary
Last reset cause : SoftwareReload
C9300L-24T-4G platform with 8388608 Kbytes of main memory

boot: attempting to boot from [flash:packages.conf]
boot: reading file packages.conf
#
################################################################################
################################################################################
################################################################################
################################################################################
################################################################################
#####################

Apr 29 12:42:56.004: %ISSU-5-ISSU_STATE_INVALID: R0/0: issu_boottime: ISSU store
d state found invalid and cleaned up: ISSU in-progress lock found
Apr 29 12:43:20.640: %PMAN-3-PROCHOLDDOWN: R0/0: The process cmand has been held
down (rc 134)
Apr 29 12:43:21.678: %PMAN-0-PROCFAILCRIT: R0/0: pvp: A critical process cmand h
as failed (rc 134)
Apr 29 12:43:21.756: %PMAN-3-RELOAD_RP: R0/0: pvp: Reloading: Switch will be rel
oaded

Chassis 0 reloading, reason - NonApr 29 12:43:22.628: %PMAN-5-EXITACTION: F0/0:
pvp: Process manager is exiting: reload fp action requested
Apr 29 12:43:23.976: %PMAN-5-EXITACTION: R0/0Apr 29 12:43:25.147: %PMAN-3-PROCES
S_NOTIFICATION: R0/0: pvp: System report /crashinfo/system-report_local_20240429
-124324-Universal.tar.gz (size: 2088 KB) generated

Initializing Hardware...

System Bootstrap, Version 17.9.2r, RELEASE SOFTWARE (P)
Compiled Thu 08/17/2023 14:22:55.52 by rel

Current ROMMON image : Primary
Last reset cause : SoftwareReload
C9300L-24T-4G platform with 8388608 Kbytes of main memory

boot: attempting to boot from [flash:packages.conf]
boot: reading file packages.conf
#
################################################################################
################################################################################
################################################################################
################################################################################
################################################################################
#####################

Apr 29 12:45:00.268: %PMAN-3-PROCHOLDDOWN: R0/0: The process cmand has been held
down (rc 134)
Apr 29 12:45:01.344: %PMAN-0-PROCFAILCRIT: R0/0: pvp: A critical process cmand h
as failed (rc 134)
Apr 29 12:45:01.429: %PMAN-3-RELOAD_RP: R0/0: pvp: Reloading: Switch will be rel
oaded

Chassis 0 reloading, reason - Non participant detected
Apr 29 12:45:02.269: %PMAN-5-EXITACTION: F0/0: pvp: Process manager is exiting:
reload fp action requested
Apr 29 12:45:03.677: %PMAN-5-EXITACTION: R0Apr 29 12:45:04.840: %PMAN-3-PROCESS_
NOTIFICATION: R0/0: pvp: System report /crashinfo/system-report_local_20240429-1
24504-Universal.tar.gz (size: 2088 KB) generated

Initializing Hardware...
boot: attempting to boot from [flash:packages.conf]
boot: reading file packages.conf
#
################################################################################
################################################################################
################################################################################
################################################################################
################################################################################
#####################

Leo Laohoo · ‎04-29-2024

@Applmh wrote:

Apr 29 12:42:56.004: %ISSU-5-ISSU_STATE_INVALID: R0/0: issu_boottime: ISSU stored state found invalid and cleaned up: ISSU in-progress lock found

Did someone attempted to upgrade the firmware of the switch using ISSU?