cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
3981
Views
0
Helpful
7
Replies

ISR 2911 crashing saying "%Software-forced reload"

yuehua zhu
Level 1
Level 1

ISR 2911 crashing saying "%Software-forced reload"

Hi,

I have an ISR 2911.  Every now and then, it crashes saying "Possible software fault".  and the system Constantly reload

The following information is presented under the CONSOL

%Software-forced reload

: Breakpoint exception, CPU signal 23, PC = 0x219CC00C

--------------------------------------------------------------------

   Possible software fault. Upon reccurence, please collect

   crashinfo, "show tech" and contact Cisco Technical Support.

--------------------------------------------------------------------

-Traceback= 0x219C1D6Cz 0x219C0248z 0x2115BF1Cz 0x219C6F6Cz 0x219C01E4z 0x2115EB0Cz

$0 : 00000000 00000000, AT : 00000000 28600000, v0 : 00000000 287EB3B8, v1 : 00000000 00000000

a0 : 00000000 00000001, a1 : 00000000 2640DAF3, a2 : 00000000 00000000, a3 : 00000000 00000000

t0 : 00000000 00009F00, t1 : 00000000 50009FE1, t2 : 00000000 219D1B28, t3 : 00000000 27EC0000

t4 : 00000000 28600000, t5 : 00000000 00000000, t6 : 00000000 28600000, t7 : 00000000 00000C28

s0 : 00000000 00000000, s1 : 00000000 27EB0000, s2 : 00000000 00000200, s3 : FFFFFFFF 80010700

s4 : 00000000 0000002E, s5 : 00000000 28A6A9E4, s6 : 00000000 00000000, s7 : FFFFFFFF 80010700

t8 : 00000000 00000C40, t9 : 00000000 00800118, k0 : 00000000 00000000, k1 : 00000000 00000000

gp : 00000000 28603FE0, sp : 00000000 28FB5C18, s8 : 00000000 00000000, ra : 00000000 219CA4E8

EPC  : 219CC00C, ErrorEPC : 80412BEC, SREG     : 50009FE3

MDLO : 00000000, MDHI     : 00000002, BadVaddr : 00000000

CacheErr_Icache : 0x007F7F00 - 000003F8, CacheErr_Dcache : 0x00000000 - 00000D60

TagLo_Icache : 0x00000000 - 21164F83, DataLo_Icache : 0x02518025 - 00007821

DataHi_Icache: 0x00000000 - 00000000

TagLo_Dcache : 0x00000000 - 285FC903, TagHi_Dcache : 0x00000000 - 085FC101

DataLo_Dcache: 0x00000001 - 2114AA84, DataHi_Dcache : 0x00000000 - 00000010

TEXT_START : 0x2100AB78

DATA_START : 0x259362A0

Cause 00000024 (Code 0x9): Breakpoint exception

: Breakpoint exception, CPU signal 23, PC = 0x219CC00C

--------------------------------------------------------------------

   Possible software fault. Upon reccurence, please collect

   crashinfo, "show tech" and contact Cisco Technical Support.

--------------------------------------------------------------------

Anyone ever encountered such a problem?

Thanks,

1 Accepted Solution

Accepted Solutions

Hello,

I agree with Paolo - if you have tried different IOS versions and each of them behaves in the same way then I am afraid the hardware may be faulty.

As a last resort, try pulling out all added interfaces, modules, AIMs, PVDMs etc. from your router, leaving only a barebone router. Then try booting it up. Perhaps the problem is limited to a particular module you could then try to isolate.

If this does not help, and using tftpdnld -r to download and boot an IOS from TFTP without writing it into FLASH does not help either (try either 15.2(4)M4 or 15.3(3)M) then I am afraid we are out of luck, and you should RMA the device.

Just curious, was there any incident you could trace back to when the problem started appearing? Was there a power surge, or a power brownout/blackout, anything unusual that happened around the time the router became unable to boot? When did you first notice these problems and what preceded them?

Best regards,

Peter

View solution in original post

7 Replies 7

Peter Paluch
Cisco Employee
Cisco Employee

Hi,

This suggests at a bug in IOS, or to a hardware fault - most probably in the RAM. Let's hope it is the IOS.

What IOS version are you currently running? Can you afford to upgrade to a different version? Avoid the 'T' versions if possible.

Also, can you post the complete configuration and the crashinfo file that was stored into FLASH when your router reloaded? Use the advanced editor when responding here - it allows attaching files.

Best regards,

Peter

i Have been re-replaced two IOS, and neither is "T" version, the phenomenon is still the same

Hi,

Would you mind posting the complete configuration and the crashinfo files?

Best regards,

Peter

it continuously reboot, i can not enter configuration mode, so i have no configuration

All tips are as follows

System Bootstrap, Version 15.0(1r)M16, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 2012 by cisco Systems, Inc.

Total memory size = 512 MB - On-board = 512 MB, DIMM0 = 0 MB

CISCO2911/K9 platform with 524288 Kbytes of main memory

Main memory is configured to 72/-1(On-board/DIMM0) bit mode with ECC enabled

Readonly ROMMON initialized

program load complete, entry point: 0x80803000, size: 0x1b340

program load complete, entry point: 0x80803000, size: 0x1b340

IOS Image Load Test

___________________

Digitally Signed Release Software

program load complete, entry point: 0x81000000, size: 0x3bc25d0

Self decompressing the image : ################################################################################################################################################################################################################################################################################################################################################################################ [OK]

Smart Init is enabled

smart init is sizing iomem

                 TYPE      MEMORY_REQ

    Onboard devices &

         buffer pools      0x0228F000

-----------------------------------------------

               TOTAL:      0x0228F000

Rounded IOMEM up to: 36Mb.

Using 7 percent iomem. [36Mb/512Mb]

              Restricted Rights Legend

Use, duplication, or disclosure by the Government is

subject to restrictions as set forth in subparagraph

(c) of the Commercial Computer Software - Restricted

Rights clause at FAR sec. 52.227-19 and subparagraph

(c) (1) (ii) of the Rights in Technical Data and Computer

Software clause at DFARS sec. 252.227-7013.

           cisco Systems, Inc.

           170 West Tasman Drive

           San Jose, California 95134-1706

Cisco IOS Software, C2900 Software (C2900-UNIVERSALK9-M), Version 15.0(1)M3, RELEASE SOFTWARE (fc2)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2010 by Cisco Systems, Inc.

Compiled Sun 18-Jul-10 03:32 by prod_rel_team

Image text-base: 0x2100AB78, data-base: 0x259362A0

%Software-forced reload

: Breakpoint exception, CPU signal 23, PC = 0x219CC00C

--------------------------------------------------------------------

   Possible software fault. Upon reccurence, please collect

   crashinfo, "show tech" and contact Cisco Technical Support.

--------------------------------------------------------------------

-Traceback= 0x219C1D6Cz 0x219C0248z 0x2115BF1Cz 0x219C6F6Cz 0x219C01E4z 0x2115EB0Cz

$0 : 00000000 00000000, AT : 00000000 28600000, v0 : 00000000 287EB3B8, v1 : 00000000 00000000

a0 : 00000000 00000001, a1 : 00000000 2640DAF3, a2 : 00000000 00000000, a3 : 00000000 00000000

t0 : 00000000 00009F00, t1 : 00000000 50009FE1, t2 : 00000000 219D1B28, t3 : 00000000 27EC0000

t4 : 00000000 28600000, t5 : 00000000 00000000, t6 : 00000000 28600000, t7 : 00000000 00000C28

s0 : 00000000 00000000, s1 : 00000000 27EB0000, s2 : 00000000 00000200, s3 : FFFFFFFF 80010700

s4 : 00000000 0000002E, s5 : 00000000 28A6A9E4, s6 : 00000000 00000000, s7 : FFFFFFFF 80010700

t8 : 00000000 00000C40, t9 : 00000000 00800118, k0 : 00000000 00000000, k1 : 00000000 00000000

gp : 00000000 28603FE0, sp : 00000000 28FB5C18, s8 : 00000000 00000000, ra : 00000000 219CA4E8

EPC  : 219CC00C, ErrorEPC : 80412BEC, SREG     : 50009FE3

MDLO : 00000000, MDHI     : 00000002, BadVaddr : 00000000

CacheErr_Icache : 0x007F7F00 - 000003F8, CacheErr_Dcache : 0x00000000 - 00000D60

TagLo_Icache : 0x00000000 - 21164F83, DataLo_Icache : 0x02518025 - 00007821

DataHi_Icache: 0x00000000 - 00000000

TagLo_Dcache : 0x00000000 - 285FC903, TagHi_Dcache : 0x00000000 - 085FC101

DataLo_Dcache: 0x00000001 - 2114AA84, DataHi_Dcache : 0x00000000 - 00000010

TEXT_START : 0x2100AB78

DATA_START : 0x259362A0

Cause 00000024 (Code 0x9): Breakpoint exception

: Breakpoint exception, CPU signal 23, PC = 0x219CC00C

--------------------------------------------------------------------

   Possible software fault. Upon reccurence, please collect

   crashinfo, "show tech" and contact Cisco Technical Support.

--------------------------------------------------------------------

-Traceback= 0x219C1D6Cz 0x219C0248z 0x2115BF1Cz 0x219C6F6Cz 0x219C01E4z 0x2115EB0Cz

$0 : 00000000 00000000, AT : 00000000 28600000, v0 : 00000000 287EB3B8, v1 : 00000000 00000000

a0 : 00000000 00000001, a1 : 00000000 2640DAF3, a2 : 00000000 00000000, a3 : 00000000 00000000

t0 : 00000000 00009F00, t1 : 00000000 50009FE1, t2 : 00000000 219D1B28, t3 : 00000000 27EC0000

t4 : 00000000 28600000, t5 : 00000000 00000000, t6 : 00000000 28600000, t7 : 00000000 00000C28

s0 : 00000000 00000000, s1 : 00000000 27EB0000, s2 : 00000000 00000200, s3 : FFFFFFFF 80010700

s4 : 00000000 0000002E, s5 : 00000000 28A6A9E4, s6 : 00000000 00000000, s7 : FFFFFFFF 80010700

t8 : 00000000 00000C40, t9 : 00000000 00800118, k0 : 00000000 00000000, k1 : 00000000 00000000

gp : 00000000 28603FE0, sp : 00000000 28FB5C18, s8 : 00000000 00000000, ra : 00000000 219CA4E8

EPC  : 219CC00C, ErrorEPC : 80412BEC, SREG     : 50009FE3

MDLO : 00000000, MDHI     : 00000002, BadVaddr : 00000000

CacheErr_Icache : 0x007F7F00 - 000003F8, CacheErr_Dcache : 0x00000000 - 00000D60

TagLo_Icache : 0x00000000 - 21164F83, DataLo_Icache : 0x02518025 - 00007821

DataHi_Icache: 0x00000000 - 00000000

TagLo_Dcache : 0x00000000 - 285FC903, TagHi_Dcache : 0x00000000 - 085FC101

DataLo_Dcache: 0x00000001 - 2114AA84, DataHi_Dcache : 0x00000000 - 00000010

TEXT_START : 0x2100AB78

DATA_START : 0x259362A0

Cause 00000024 (Code 0x9): Breakpoint exception

: Breakpoint exception, CPU signal 23, PC = 0x219CC00C

--------------------------------------------------------------------

   Possible software fault. Upon reccurence, please collect

   crashinfo, "show tech" and contact Cisco Technical Support.

--------------------------------------------------------------------

-Traceback= 0x219C1D6Cz 0x219C0248z 0x2115BF1Cz 0x219C6F6Cz 0x219C01E4z 0x2115EB0Cz

$0 : 00000000 00000000, AT : 00000000 28600000, v0 : 00000000 287EB3B8, v1 : 00000000 00000000

a0 : 00000000 00000001, a1 : 00000000 2640DAF3, a2 : 00000000 00000000, a3 : 00000000 00000000

t0 : 00000000 00009F00, t1 : 00000000 50009FE1, t2 : 00000000 219D1B28, t3 : 00000000 27EC0000

t4 : 00000000 28600000, t5 : 00000000 00000000, t6 : 00000000 28600000, t7 : 00000000 00000C28

s0 : 00000000 00000000, s1 : 00000000 27EB0000, s2 : 00000000 00000200, s3 : FFFFFFFF 80010700

s4 : 00000000 0000002E, s5 : 00000000 28A6A9E4, s6 : 00000000 00000000, s7 : FFFFFFFF 80010700

t8 : 00000000 00000C40, t9 : 00000000 00800118, k0 : 00000000 00000000, k1 : 00000000 00000000

gp : 00000000 28603FE0, sp : 00000000 28FB5C18, s8 : 00000000 00000000, ra : 00000000 219CA4E8

EPC  : 219CC00C, ErrorEPC : 80412BEC, SREG     : 50009FE3

MDLO : 00000000, MDHI     : 00000002, BadVaddr : 00000000

CacheErr_Icache : 0x007F7F00 - 000003F8, CacheErr_Dcache : 0x00000000 - 00000D60

TagLo_Icache : 0x00000000 - 21164F83, DataLo_Icache : 0x02518025 - 00007821

DataHi_Icache: 0x00000000 - 00000000

TagLo_Dcache : 0x00000000 - 285FC903, TagHi_Dcache : 0x00000000 - 085FC101

DataLo_Dcache: 0x00000001 - 2114AA84, DataHi_Dcache : 0x00000000 - 00000010

TEXT_START : 0x2100AB78

DATA_START : 0x259362A0

Cause 00000024 (Code 0x9): Breakpoint exception

-Traceback= 0x219C1D6Cz 0x219C0248z 0x2115BF1Cz 0x219C6F6Cz 0x219C01E4z 0x2115EB0Cz

The IOS you are using is quite old, please update from rommon unsing tftpdnld command.

Anyway, from address present the failure, it is possible that this is an hardware fault requiring device replacement.

Hello,

I agree with Paolo - if you have tried different IOS versions and each of them behaves in the same way then I am afraid the hardware may be faulty.

As a last resort, try pulling out all added interfaces, modules, AIMs, PVDMs etc. from your router, leaving only a barebone router. Then try booting it up. Perhaps the problem is limited to a particular module you could then try to isolate.

If this does not help, and using tftpdnld -r to download and boot an IOS from TFTP without writing it into FLASH does not help either (try either 15.2(4)M4 or 15.3(3)M) then I am afraid we are out of luck, and you should RMA the device.

Just curious, was there any incident you could trace back to when the problem started appearing? Was there a power surge, or a power brownout/blackout, anything unusual that happened around the time the router became unable to boot? When did you first notice these problems and what preceded them?

Best regards,

Peter

This equipment is the customer's, customer can't provide the events before the failure

i will try using tftpdnld to download an IOS later

as you say,may be it's hardware fault

Thank you very much for help

Review Cisco Networking for a $25 gift card