3560 Won't Boot

[cpratt001]

Hi guys,

I'm hoping someone may be able to provide some assistance here. My 3560 will no longer boot. It passes all the boot checks and it gets to Model --> Version etc. and it just hangs there. I've left it for over an hour and it just doesn't boot. I have formatted the flash and copied across a working IOS from a working same model switch I have however it fails at the same point. The MODE button is still functional, I can press it and it cycles through stat, duplex, speed, and PoE. See below. 

flashfs[1]: Initialization complete....done Initializing flashfs.

Checking for Bootloader upgrade.. not needed
POST: CPU MIC register Tests : Begin
POST: CPU MIC register Tests : End, Status Passed

POST: PortASIC Memory Tests : Begin
POST: PortASIC Memory Tests : End, Status Passed

POST: CPU MIC interface Loopback Tests : Begin
POST: CPU MIC interface Loopback Tests : End, Status Passed

POST: PortASIC RingLoopback Tests : Begin
POST: PortASIC RingLoopback Tests : End, Status Passed

POST: Inline Power Controller Tests : Begin
POST: Inline Power Controller Tests : End, Status Passed

POST: PortASIC CAM Subsystem Tests : Begin
POST: PortASIC CAM Subsystem Tests : End, Status Passed

POST: PortASIC Port Loopback Tests : Begin
POST: PortASIC Port Loopback Tests : End, Status Passed

Waiting for Port download...Complete

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco WS-C3560-48PS (PowerPC405) processor (revision H0) with 131072K bytes of memory.
Processor board ID CAT0123A1B0
Last reset from power-on
1 Virtual Ethernet interface
48 FastEthernet interfaces
4 Gigabit Ethernet interfaces
The password-recovery mechanism is enabled.

512K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address : 00:00:00:00:00:00
Motherboard assembly number : 73-9298-02
Power supply part number : 341-0029-03
Motherboard serial number : CAT1234567Q
Power supply serial number : LIT1234567T
Model revision number : H0
Motherboard revision number : A0
Model number : WS-C3560-48PS-S
System serial number : CAT0123A1B0
SFP Module assembly part number : 73-7757-02
SFP Module revision Number : A0
SFP Module serial number : CAT12345AFV
Top Assembly Part Number : 800-24787-02
Top Assembly Revision Number : B0
Version ID : N/A
Hardware Board Revision Number : 0x0C

Switch Ports Model SW Version SW Image
------ ----- ----- ---------- ----------
* 1 52 WS-C3560-48PS 12.2(55)SE10 C3560-IPSERVICESK9-M

This is starting to look like a hardware issue to me but I'd love to hear any other suggestions out there.

Thanks in advance

[Leo Laohoo]

Try a different computer or review the configuration/settings of the serial port.

[cpratt001]

Thanks,

I have tried a different computer and both Putty and Hyperterminal to no avail. If I plug in a network cable, I get no activity on that port either which makes me think the switch hasn't completely loaded it's config.

[Peter Paluch]

Hi,

Hmm, the base ethernet MAC address is indicated as all zeros. That suggests that some vital data unique to this switch must have become corrupted somehow. Unfortunately, I am not sure how to restore them.

Can you enter the bootloader for this switch (power it off, then on while holding the MODE button for 10-15 seconds) and then capture the output of the set command please and post it here?

Best regards,
Peter

[cpratt001]

Hi Peter, please see below:

switch: set
BOOT=flash:/c3560-ipservicesk9-mz.122-55.SE10.bin
CLEI_CODE_NUMBER=CNMWF00ARA
MAC_ADDR=00:12:7F:0C:75:00
MODEL_NUM=WS-C3560-48PS-S
MODEL_REVISION_NUM=H0
MOTHERBOARD_ASSEMBLY_NUM=73-9298-02
MOTHERBOARD_REVISION_NUM=A0
MOTHERBOARD_SERIAL_NUM=CAT0843038Q
POWER_SUPPLY_PART_NUM=341-0029-03
POWER_SUPPLY_SERIAL_NUM=LIT0842061T
SDM_TEMPLATE_ID=0
SFP_MODULE_BOARD_ASSEMBLY_PART_NUM=73-7757-02
SFP_MODULE_BOARD_REVISION_NUM=A0
SFP_MODULE_BOARD_SERIAL_NUM=CAT08430AFV
SWITCH_NUMBER=1
SWITCH_PRIORITY=1
SYSTEM_SERIAL_NUM=CAT0843K1D0
TAN_NUM=800-24787-02
TAN_REVISION_NUMBER=B0
VERSION_ID=N/A

Sorry, all details are shown correctly at time of boot, the serial numbers and mac address details were changed in the original post. My bad.

[Peter Paluch]

Hi,

Sorry, all details are shown correctly at time of boot, the serial numbers and mac address details were changed in the original post.

Oh - so the MAC address shows up correctly in the output during the boot, right?

You wrote earlier that you have formatted the flash memory - did you really do that, or did you just delete the IOS image and downloaded it anew? The reason I'm asking is that I would like to know if the switch is currently booting with no configuration files present. On this Catalyst switch type, there's no NVRAM, and all configuration is stored in the flash, specifically:

flash:config.text (this is the startup-config)
flash:private-config.text (RSA keys, etc.)
flash:multiple-fs (other files appearing in the nvram: filesystem)
flash:vlan.dat (VLAN database and VTP parameters)

If you don't mind, I would suggest deleting these files using the bootloader delete command, and then trying to boot the IOS.

Best regards,
Peter

[cpratt001]

That's correct, it shows up correctly in the output during boot.

I did a format through the boot loader -- switch:  format flash:

I then copied a working ios image over from another switch via xmodem. The only item on the flash is the ios.bin file:

switch: dir flash:
Directory of flash:/

2 -rwx 12749440 <date> c3560-ipservicesk9-mz.122-55.SE10.bin

3248128 bytes available (12750848 bytes used)

[Peter Paluch]

Hi,

Well... just one more thing to try that comes to my mind, and it's actually what Leo suggested.

These switches have a boot parameters filesystem that is stored yet in another location, and it seems to keep the console speed stored there. It is possible that it remembers a different console speed than 9600 bps, and when IOS reads it and changes the console speed, it gives off an impression that the console got frozen.

So what I suggest is booting the IOS fully, and when it appears to stall, try changing your COM port speeds on your PC all the way through 1200, 2400, 4800, 9600, 19200, 38400, 57600 up to 115200 bps, always hitting Enter a number of times to see if the switch responds. If yes, we're good.

Best regards,
Peter

[cpratt001]

Thanks for your suggestions. I'll try again tomorrow and advise. 

[cpratt001]

unfortunately I receive Variable ALL" not set, or is readonly. when I run unset ALL

[Leo Laohoo]

These switches have a boot parameters filesystem that is stored yet in another location

I think the command is "unset ALL".  NOTE:  "ALL" must be entered with in all CAPs.  

[cpratt001]

Booted the IOS and cycled through each COM port speed and none responded to Enter or any other keystrokes.

[Peter Paluch]

Hi,

Well, we only have one blind shot left. Assuming that the problem is indeed caused by the console speed being somehow distorted which confuses IOS, we can try to reinit the boot parameters filesystem with the by-the-book settings.

Please have your switch reloaded and break the boot process so that you enter the bootloader prompt. Then issue the following commands:

set BAUD 9600
set_bs bs: rw
set_param
reset

This configuration sets the BAUD variable controlling the console speed to 9600 bps. Then the boot parameters filesystem is set to read-write mode and the variable are stored into it. Afterwards, the switch is reloaded.

Unfortunately, if this does not help then personally, I am out of options, and the only thing I can suggest is having the switch replaced.

Best regards,
Peter

[cpratt001]

Thanks for your help Peter but it still stalled at the same spot.