cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
406
Views
0
Helpful
5
Replies

C3650 keep rebooting Rc 134 error code

Essam Ahmed
Level 1
Level 1

here Putty record , I have two switch same issue .

Booting...
Interface GE 0 link down***ERROR: PHY link is down
The "IP_ADDR" environment variable is not set.

Getting rest of image
Reading full image into memory...Check base package header ...: done = 16384
Getting rest of image
Reading full image into memory....done
Reading full base package into memory...: done = 22301472
Bundle Image
--------------------------------------
Kernel Address : 0x53778384
Kernel Size : 0x34e9e1/3467745
Initramfs Address : 0x53ac6d65
Initramfs Size : 0x119d5bb/18470331
Compression Format: mzip

Bootable image at @ ram:0x53778384
Bootable image segment 0 address range [0x81100000, 0x81b8adc0] is in range [0x80180000, 0x90000000].
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@boot_system: 623
Loading Linux kernel with entry point 0x816902d0 ...
Bootloader: Done loading app on core_mask: 0xf

### Launching Linux Kernel (flags = 0x5)

 

Front-end Microcode IMG MGR: found 1 microcode images for 1 device.
Image for front-end 0: /tmp/microcode_update/front_end/fe_type_8_1

Front-end Microcode IMG MGR: Preparing to program device microcode...
Front-end Microcode IMG MGR: Preparing to program device[0]...86376 bytes.
Front-end Microcode IMG MGR: Programming device 0...
% Front-end Microcode IMG MGR: address 0, Could not enable sys_poll


% Front-end Microcode IMG MGR: unable to program device: 0
% Front-end Microcode IMG MGR: malformed microcode cache

Both links down, not waiting for other switches
Switch number is 1
Jul 3 11:56:01.546 R0/0: %PMAN-3-PROCHOLDDOWN: The process platform_mgr has been helddown (rc 134)

Restricted Rights Legend

Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.

cisco Systems, Inc.
170 West Tasman Drive
San Jose, California 95134-1706

 

Cisco IOS Software [Denali], Catalyst L3 Switch Software (CAT3K_CAA-UNIVERSALK9-M), Version 16.3.5b, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2017 by Cisco Systems, Inc.
Compiled Thu 02-Nov-17 11:07 by mcpre

 

Cisco IOS-XE software, Copyright (c) 2005-2017 by cisco Systems, Inc.
All rights reserved. Certain components of Cisco IOS-XE software are
licensed under the GNU General Public License ("GPL") Version 2.0. The
software code licensed under GPL Version 2.0 is free software that comes
with ABSOLUTELY NO WARRANTY. You can redistribute and/or modify such
GPL code under the terms of GPL Version 2.0. For more details, see the
documentation or "License Notice" file accompanying the IOS-XE software,
or the applicable URL provided on the flyer accompanying the IOS-XE
software.

 


FIPS: Flash Key Check : Begin
FIPS: Flash Key Check : End, Not Found, FIPS Mode Not Enabled

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco WS-C3650-24PD (MIPS) processor (revision U0) with 865070K/6147K bytes of memory.
Processor board ID FDO2318F0GC
2048K bytes of non-volatile configuration memory.
4194304K bytes of physical memory.
252000K bytes of Crash Files at crashinfo:.
1611414K bytes of Flash at flash:.
0K bytes of at webui:.

Base Ethernet MAC Address : 2c:4f:52:62:22:00
Motherboard Assembly Number : 73-15894-07
Motherboard Serial Number : FDO23171VNT
Model Revision Number : U0
Motherboard Revision Number : A0
Model Number : WS-C3650-24PD
System Serial Number : FDO2318F0GC

 


Press RETURN to get started!


*Jul 3 11:56:38.257: %SMART_LIC-6-AGENT_READY: Smart Agent for Licensing is initialized
*Jul 3 11:56:42.821: dev_pluggable_optics_selftest attribute table internally inconsistent @ 0x125

*Jul 3 11:56:43.363: %SMI-5-CLIENT: Smart Install Client feature is enabled. It is recommended to disable the Smart Install feature when it is not actively used. To disable feature execute 'no vstack' in configuration mode
*Jul 3 11:56:43.461: %NBAR-6-CACHE_SYNC_INFO: Cache synchronization. Initialized.
*Jul 3 11:56:46.798: %SPANTREE-5-EXTENDED_SYSID: Extended SysId enabled for type vlan
*Jul 3 11:56:47.294: %LINK-3-UPDOWN: Interface Lsmpi18/0/3, changed state to up
*Jul 3 11:56:47.295: %LINK-3-UPDOWN: Interface EOBC18/0/1, changed state to up
*Jul 3 11:56:47.295: %LINK-3-UPDOWN: Interface GigabitEthernet0/0, changed state to down
*Jul 3 11:56:47.295: %LINK-3-UPDOWN: Interface LIIN18/0/2, changed state to up
*Jul 3 11:56:47.647: %PNP-6-PNP_DISCOVERY_STOPPED: PnP Discovery stopped (Startup Config Present)
Chassis 1 reloading, reason - Reload command

*Jul 3 11:56:47.714: %HMANRP-6-HMAN_IOS_CHANNEL_INFO: HMAN-IOS channel event for switch 1: EMP_RELAY: Channel UP!
*Jul 3 11:56:47.745: %HMANRP-6-EMP_NO_ELECTION_INFO: Could not elect active EMP switch, setting emp active switch to 0: EMP_RELAY: Could not elect switch with mgmt port UP
*Jul 3 11:56:30.791: %STACKMGR-1-STACK_LINK_CHANGE:Switch 1 R0/0: stack_mgr: Stack port 1 on switch 1 is nocable
*Jul 3 11:56:30.792: %STACKMGR-1-STACK_LINK_CHANGE:Switch 1 R0/0: stack_mgr: Stack port 2 on switch 1 is down
*Jul 3 11:56:30.792: %STACKMGR-1-STACK_LINK_CHANGE:Switch 1 R0/0: stack_mgr: Stack port 2 on switch 1 is nocable
*Jul 3 11:56:30.792: %STACKMGR-6-SWITCH_ADDED:Switch 1 R0/0: stack_mgr: Switch 1 has been added to the stack.
*Jul 3 11:56:30.792: %STACKMGR-6-SWITCH_ADDED:Switch 1 R0/0: stack_mgr: Switch 1 has been added to the stack.
*Jul 3 11:56:30.792: %STACKMGR-6-SWITCH_ADDED:Switch 1 R0/0: stack_mgr: Switch 1 has been added to the stack.
*Jul 3 11:56:48.399: %LINEPROTO-5-UPDOWN: Line protocol on Interface Lsmpi18/0/3, changed state to up
*Jul 3 11:56:48.400: %LINEPROTO-5-UPDOWN: Line protocol on Interface EOBC18/0/1, changed state to up
*Jul 3 11:56:48.400: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/0, changed state to down
*Jul 3 11:56:48.401: %LINEPROTO-5-UPDOWN: Line protocol on Interface LIIN18/0/2, changed state to up
*Jul 3 11:56:50.509: %IOSXE_MGMTVRF-6-CREATE_SUCCESS_INFO: Management vrf Mgmt-vrf created with ID 1, ipv4 table-id 0x1, ipv6 table-id 0x1E000001
*Jul 3 11:56:51.924: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan1, changed state to down
*Jul 3 11:56:57.049: %SYS-6-STARTUP_CONFIG_IGNORED: System startup configuration is ignored based on the configuration register setting.
*Jul 3 11:56:59.791: %LINK-3-UPDOWN: Interface Vlan1, changed state to up
*Jul 3 11:57:00.538: %HMANRP-6-EMP_NO_ELECTION_INFO: Could not elect active EMP switch, setting emp active switch to 0: EMP_RELAY: Could not elect switch with mgmt port UP
*Jul 3 11:57:02.322: %SYS-5-RESTART: System restarted --
Cisco IOS Software [Denali], Catalyst L3 Switch Software (CAT3K_CAA-UNIVERSALK9-M), Version 16.3.5b, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2017 by Cisco Systems, Inc.
Compiled Thu 02-Nov-17 11:07 by mcpre
*Jul 3 11:57:07.928: %PMAN-0-PROCFAILCRIT:Switch 1 R0/0: pvp.sh: A critical process platform_mgr has failed (rc 134)
*Jul 3 11:57:08.534: %PMAN-3-RELOAD_SYSTEM:Switch 1 R0/0: pvp.sh: Reloading: Peer switch is not standby ready. System will be reloaded
*Jul 3 11:57:08.564: %STACKMGR-1-RELOAD:Switch 1 R0/0: stack_mgr: Reloading due to reason Reload command Jul 3 11:57:18.501 R0/0: %PMAN-5-EXITACTION: Process manager is exiting: reload fp action requested

*Jul 3 11:57:18.538: %PMAN-5-EXITACTION:Switch 1 R0/0: pvp.sh: Process manager is exiting: reload fp action requestedJul 3 11:57:34.535 R0/0: %PMAN-5-EXITACTION: Process manager is exiting: rp processes exit with reload switch code
Jul 3 11:57:52.470 R0/0: %PMAN-3-PROCESS_NOTIFICATION: System report /crashinfo/system-report_1_20250703-115743-UTC.tar.gz (size: 5739 KB) generated

octeon_wdt: WDT device closed unexpectedly. WDT will not stop!
reboot: Restarting system

 

5 Replies 5

Essam Ahmed
Level 1
Level 1

I have installed last IOS 16.12.13   but no way same issue.

pleas your advice 

 

Enes Simnica
Level 5
Level 5

hello. The continuous reboot cycle with "platform_mgr has failed (rc 134)" typically indicates either corrupted firmware or hardware issues. For ur ws-C3650-24pd/ switches u need to try and boot the golden bundle image in rommon mode, should be like this: #flash_init and then : boot flash:cat3k_caa-universalk9.SSA.03.07.05.E.152-3.E5.bin

hoping this works, but if it doesnt, u'll need to first completely reinstall IOS via rommon/ using usb ortftp. 2. replace the flash drive if corruption persists, and check power supply stabillity....

So yes,/ dev_pluggable_optics_selftest error, suggests possible hardware faults. 

Try those, and let me know the outcome...

hope it helps G

 

-Enes

more Cisco?!
more Gym?!



If this post solved your problem, kindly mark it as Accepted Solution. Much appreciated!

Enes Simnica
Level 5
Level 5

and also G, I just saw that someone had the same issue back on 2022: check the link, and hope it helps:

https://community.cisco.com/t5/switching/3650-switches-boot-in-a-loop-after-renumbering/m-p/4554722#M518683 

 

more Cisco?!
more Gym?!



If this post solved your problem, kindly mark it as Accepted Solution. Much appreciated!

Jens Albrecht
Spotlight
Spotlight

Hello @Essam Ahmed,

based on your output the root cause for the crash loop might be a failed microcode upgrade:

Front-end Microcode IMG MGR: found 1 microcode images for 1 device.
Image for front-end 0: /tmp/microcode_update/front_end/fe_type_8_1

Front-end Microcode IMG MGR: Preparing to program device microcode...
Front-end Microcode IMG MGR: Preparing to program device[0]...86376 bytes.
Front-end Microcode IMG MGR: Programming device 0...
% Front-end Microcode IMG MGR: address 0, Could not enable sys_poll


% Front-end Microcode IMG MGR: unable to program device: 0
% Front-end Microcode IMG MGR: malformed microcode cache


You can try a full power cycle to clear transient microcode issues.

A reload or warm boot is NOT enough!
You need to unplug all power cables, wait a few seconds and then recable and turn on the switch again.

If this does not help, then you need to open a TAC case and attach the crash report that was generated.

HTH!

Leo Laohoo
Hall of Fame
Hall of Fame

@Essam Ahmed wrote:
% Front-end Microcode IMG MGR: unable to program device: 0
% Front-end Microcode IMG MGR: malformed microcode cache

Contact Cisco TAC and organize and RMA. 

It looks like someone interrupted the switch during a micro-code upgrade and corrupted the sector beyond repair.