Catalyst 4506 - System returned to ROM by abort at PC 0x0

[burleyman]

I had a Catalyst 4506 switch reload itself this weekend and had the error "System returned to ROM by abort at PC 0x0"

I am running  cat4000-i9s.mz.122-25.EWA6

Attached is the output of the SHOW PLATFORM CRASHDUMP

I am looking for some direction as to what may have happened.

Thanks,

Mike

[IAN WHITMORE]

If you've got a CCO Login you can go here:

http://www.cisco.com/en/US/support/tsd_most_requested_tools.html

Click on output interpreter and paste a #sh version and your stack dump.

That will tell you what went wrong...if not on the same page you can look for a bug for your IOS and failing that, open a TAC case. They will surely tell you what went wrong and probably advise to update the IOS.

HTH,

Ian

[IAN WHITMORE]

Found this:

CSCsi17158 Bug Details

Multiple invalid ssh attempts crashes switch

Symptoms: Devices running Cisco IOS may reload with the  error message "System returned to ROM by abort at PC 0x0" when  processing SSHv2 sessions. A switch crashes. We have a script running  that will continuously ssh-v2 into the 3560 then close the session  normally. If the vty line that is being used by SSHv2 sessions to  the device is cleared while the SSH session is being processed, the next  time an ssh into the device is done, the device will crash. Conditions:  This problem is platform independent, but it has been seen on Cisco  Catalyst 3560, Cisco Catalyst 3750 and Cisco Catalyst 4948 series  switches. The issue is specific to SSH version 2, and its seen only  when the box is under brute force attack. This crash is not seen under  normal conditions. Workaround: There are mitigations to  this vulnerability: For Cisco IOS, the SSH server can be disabled by  applying the command crypto key zeroize rsa while in  configuration mode. The SSH server is enabled automatically upon generating  an RSA key pair. Zeroing the RSA keys is the only way to completely  disable the SSH server. Access to the SSH server on Cisco IOS  may also be disabled via removing SSH as a valid transport protocol.  This can be done by reapplying the transport input command with  'ssh' removed from the list of permitted transports on VTY lines while  in configuration mode. For example: line vty 0 4 transport  input telnet                                                  end If SSH server functionality is desired, access to the server can  be restricted to specific source IP addresses or blocked entirely  using Access Control Lists (ACLs) on the VTY lines as shown in the following  URL: http://www.cisco.com/en/US/docs/switches/lan/catalyst2950/software/release/12.1_9_ea1/configur ation/guide/swacl.html#xtocid14 More information on configuring ACLs can be found on the Cisco  public website: http://www.cisco.com/warp/public/707/confaccesslists.html

Among affected versions:

12.2(25)EWA               
12.2(25)EWA1               
12.2(25)EWA2               
12.2(25)EWA3               
12.2(25)EWA4               
12.2(25)EWA5               
12.2(25)EWA6               
12.2(25)EWA7               
12.2(25)EWA8               
12.2(25)EWA9               
12.2(25)EWA10               
12.2(25)EWA11

Also, although it's not exactly your system you might be interested in this:

http://www.cisco.com/en/US/products/hw/switches/ps708/products_tech_note09186a008072c406.shtml

HTH,

Ian

[burleyman]

Thanks Ian,

I did already see that bug but we are not running SSH.

I did forget about the output intrepeter so I will check that out and see what it says.

Mike

[burleyman]

Well the output interpreter is not giving me anything. Any "special" Cisco people help out this the crash dump?

Mike

[Leo Laohoo]

ErrDr1=00 ErrDr2=00 ErrStsCpu=72 ErrStsPci=00
BusErrAddr=D00010A0
Single bit ECC count 0

Hi Mike,

How's your CCNP review?

I don't see anything fishy except the three lines.  So I guess if it's not too much of an ask, load a newer IOS into the switch but don't reload it.  The next time the switch crashes, it will load the newer IOS.

PS:  Dude, I've created a thread just for your poetry. 

[burleyman]

Thanks Leo and Ian.

We have decieded to go with updating the IOS.

Mike

[burleyman]

Leo,

Studying is going good, learning lots but lots more to go.

I saw the thread...

I will do what I can

because Leo, you are the man!

Mike

[Leo Laohoo]

lol Mike

[Michael Simon]

This crash was caused by a kind of party error.

If it happens more than once you should open a TAC case to get an RMA.  A single event is what we typically see and is not necessarily a hardware failure.

This is likely a Single Event Upset (SEU) or a soft parity error.

There are two kinds of parity errors. 

Soft parity errors, by far the most common kind, occur only once.  They are caused by a transient condition and are not related to any hardware or software fault.

Hard parity errors are caused by a hardware fault.  Hard parity errors by definition happen more than once. 

For soft parity errors there is no action required but to be aware you've had one and recognize a hard parity error should there be a second event. 

[burleyman]

Thanks for your responce.

If it does happen again what would get RMA'ed...the chassis or the Sup...or something else?

Mike

[Michael Simon]

Hi Mike,

It would require an RMA of the SUP.

........Mike Simon