Re: 3825 Router crashing

ortiz.d · ‎06-30-2011

I pulled the 64MB compact flash card and I was able to get it to Rommon. I then tried to boot from a USB stick, but I get the same error. Any ideas what it can be?

rommon 30 > boot usbflash0:c3825-adventerprisek9-mz.151-4.M1.bin
program load complete, entry point: 0x8000f000, size: 0x3d1a0

program load complete, entry point: 0x80010000, size: 0x42af6bc
Self decompressing the image : ########################################################################################################################################################################################################################################################################################################################################################################################################################################## [OK]

Smart Init is enabled
smart init is sizing iomem
ID            MEMORY_REQ                 TYPE
00042A          0X010D78F3 C3825 motherboard
                0X00288860 Onboard PVDM2 SIMM
                0X0027250A Onboard VPN
                0X000021B8 Onboard USB
                0X00660670 public buffer pools
                0X0078F000 public particle pools
-----------------------------------------------
TOTAL:          0X023C3E85

If any of the above Memory Requirements are
"UNKNOWN", you may be using an unsupported
configuration or there is a software problem and
system operation may be compromised.
Rounded IOMEM up to: 36Mb.
Using 7 percent iomem. [36Mb/512Mb]

Restricted Rights Legend

Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.

           cisco Systems, Inc.
           170 West Tasman Drive
           San Jose, California 95134-1706

Cisco IOS Software, 3800 Software (C3825-ADVENTERPRISEK9-M), Version 15.1(4)M1, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2011 by Cisco Systems, Inc.
Compiled Tue 14-Jun-11 20:35 by prod_rel_team

%ALIGN-1-FATAL: Illegal access to a low address 18:13:31 UTC Thu Jun 30 2011
addr=0x0, pc=0x607006A8z , ra=0x6070069Cz , sp=0x70201D70

18:13:32 UTC Thu Jun 30 2011: TLB (store) exception, CPU signal 10, PC = 0x6070E908

--------------------------------------------------------------------
Possible software fault. Upon reccurence, please collect
crashinfo, "show tech" and contact Cisco Technical Support.
--------------------------------------------------------------------

-Traceback= 0x607006A8z 0x607009F4z 0x6070135Cz 0x607026D4z 0x6089745Cz 0x60899308z 0x608999ACz 0x60888FD4z 0x6089144Cz 0x60885B3Cz 0x602A0788z 0x63EFF2E8z 0x63EFF3D0z 0x61E5BB90z 0x61E5BE6Cz 0x63E96E80z
$0 : 00000000 00000000, AT : 00000000 68340000, v0 : 00000000 00000001, v1 : 00000000 00000001
a0 : 00000000 705AC510, a1 : 00000000 658E0000, a2 : 00000000 00000000, a3 : 00000000 608C30D4
t0 : 00000000 00000001, t1 : 00000000 00000000, t2 : 00000000 00000000, t3 : 00000000 00000042
t4 : 00000000 00000000, t5 : FFFFFFFF C0000000, t6 : 00000000 691B1B1C, t7 : 00000000 00000315
s0 : 00000000 00000000, s1 : 00000000 705AC510, s2 : 00000000 69601200, s3 : 00000000 70201DD0
s4 : 00000000 00000005, s5 : 00000000 00000001, s6 : 00000000 00000000, s7 : 00000000 00000001
t8 : 00000000 00000000, t9 : 00000000 60EFA8FC, k0 : 00000000 30408001, k1 : FFFFFFFF B0020000
gp : 00000000 68349700, sp : 00000000 70201D70, s8 : 00000000 000005DD, ra : 00000000 6070E8FC
EPC : 6070E908, ErrorEPC : BFD85CFC, SREG : 3400E103
MDLO : 00000061, MDHI : 00004A61, BadVaddr : 00000000
TEXT_START : 0x6001EB30
DATA_START : 0x65116260
Cause 0000080C (Code 0x3): TLB (store) exception

18:13:32 UTC Thu Jun 30 2011: TLB (store) exception, CPU signal 10, PC = 0x6070E908

--------------------------------------------------------------------
Possible software fault. Upon reccurence, please collect
crashinfo, "show tech" and contact Cisco Technical Support.
--------------------------------------------------------------------

-Traceback= 0x607006A8z 0x607009F4z 0x6070135Cz 0x607026D4z 0x6089745Cz 0x60899308z 0x608999ACz 0x60888FD4z 0x6089144Cz 0x60885B3Cz 0x602A0788z 0x63EFF2E8z 0x63EFF3D0z 0x61E5BB90z 0x61E5BE6Cz 0x63E96E80z
$0 : 00000000 00000000, AT : 00000000 68340000, v0 : 00000000 00000001, v1 : 00000000 00000001
a0 : 00000000 705AC510, a1 : 00000000 658E0000, a2 : 00000000 00000000, a3 : 00000000 608C30D4
t0 : 00000000 00000001, t1 : 00000000 00000000, t2 : 00000000 00000000, t3 : 00000000 00000042
t4 : 00000000 00000000, t5 : FFFFFFFF C0000000, t6 : 00000000 691B1B1C, t7 : 00000000 00000315
s0 : 00000000 00000000, s1 : 00000000 705AC510, s2 : 00000000 69601200, s3 : 00000000 70201DD0
s4 : 00000000 00000005, s5 : 00000000 00000001, s6 : 00000000 00000000, s7 : 00000000 00000001
t8 : 00000000 00000000, t9 : 00000000 60EFA8FC, k0 : 00000000 30408001, k1 : FFFFFFFF B0020000
gp : 00000000 68349700, sp : 00000000 70201D70, s8 : 00000000 000005DD, ra : 00000000 6070E8FC
EPC : 6070E908, ErrorEPC : BFD85CFC, SREG : 3400E103
MDLO : 00000061, MDHI : 00004A61, BadVaddr : 00000000
TEXT_START : 0x6001EB30
DATA_START : 0x65116260
Cause 0000080C (Code 0x3): TLB (store) exception
File flash:crashinfo_20110630-181332-UTC Device Error :No such device
File flash:crashinfo_20110630-181332-UTC Device Error :No such device
File flash:crashinfo_20110630-181332-UTC Device Error :No such device

18:13:32 UTC Thu Jun 30 2011: TLB (store) exception, CPU signal 10, PC = 0x6070E908

--------------------------------------------------------------------
Possible software fault. Upon reccurence, please collect
crashinfo, "show tech" and contact Cisco Technical Support.
--------------------------------------------------------------------

-Traceback= 0x607006A8z 0x607009F4z 0x6070135Cz 0x607026D4z 0x6089745Cz 0x60899308z 0x608999ACz 0x60888FD4z 0x6089144Cz 0x60885B3Cz 0x602A0788z 0x63EFF2E8z 0x63EFF3D0z 0x61E5BB90z 0x61E5BE6Cz 0x63E96E80z
$0 : 00000000 00000000, AT : 00000000 68340000, v0 : 00000000 00000001, v1 : 00000000 00000001
a0 : 00000000 705AC510, a1 : 00000000 658E0000, a2 : 00000000 00000000, a3 : 00000000 608C30D4
t0 : 00000000 00000001, t1 : 00000000 00000000, t2 : 00000000 00000000, t3 : 00000000 00000042
t4 : 00000000 00000000, t5 : FFFFFFFF C0000000, t6 : 00000000 691B1B1C, t7 : 00000000 00000315
s0 : 00000000 00000000, s1 : 00000000 705AC510, s2 : 00000000 69601200, s3 : 00000000 70201DD0
s4 : 00000000 00000005, s5 : 00000000 00000001, s6 : 00000000 00000000, s7 : 00000000 00000001
t8 : 00000000 00000000, t9 : 00000000 60EFA8FC, k0 : 00000000 30408001, k1 : FFFFFFFF B0020000
gp : 00000000 68349700, sp : 00000000 70201D70, s8 : 00000000 000005DD, ra : 00000000 6070E8FC
EPC : 6070E908, ErrorEPC : BFD85CFC, SREG : 3400E103
MDLO : 00000061, MDHI : 00004A61, BadVaddr : 00000000
TEXT_START : 0x6001EB30
DATA_START : 0x65116260
Cause 0000080C (Code 0x3): TLB (store) exception

-Traceback= 0x607006A8z 0x607009F4z 0x6070135Cz 0x607026D4z 0x6089745Cz 0x60899308z 0x608999ACz 0x60888FD4z 0x6089144Cz 0x60885B3Cz 0x602A0788z 0x63EFF2E8z 0x63EFF3D0z 0x61E5BB90z 0x61E5BE6Cz 0x63E96E80z
File flash:crashinfo_20110630-181332-UTC Device Error :No such device
File flash:crashinfo_20110630-181332-UTC Device Error :No such device
File flash:crashinfo_20110630-181332-UTC Device Error :No such device

=== Flushing messages (18:13:33 UTC Thu Jun 30 2011) ===

Queued messages:
*Jun 30 18:13:33.867: %SYS-3-LOGGER_FLUSHING: System pausing to ensure console debugging output.

*Jun 30 18:13:31.827: fpga fw verification false
*Jun 30 18:13:31.827: FPGA failed to dw properly
*Jun 30 18:13:32.475: %LINK-5-CHANGED: Interface GigabitEthernet0/0, changed state to reset
*Jun 30 18:13:32.475: %LINK-5-CHANGED: Interface GigabitEthernet0/1, changed state to reset
*Jun 30 18:13:32.475: %LINEPROTO-5-UPDOWN: Line protocol on Interface VoIP-Null0, changed state to up
L2 ECC Counter = 0000000000000000
Memory and IO Err Counter = 0000000000000000

Bus Watcher Counters
cor_l2cache_data_ecc_count = 0
bad_l2cache_data_ecc_count = 0
cor_l2cache_tag_ecc_count = 0
bad_l2cache_tag_ecc_count = 0
cor_memory_data_ecc_count = 0
bad_memory_data_ecc_count = 0
bus_errors = 0

         CPU temp is 48
         Temp1 temp is 26
         Temp2 temp is 25

         Internal voltage is 3300
         Remote voltage1 is 5210
         Remote voltage2 is 2525
         Remote voltage3 is 1215

No warm reboot Storage
*** System received a Bus Error exception ***
signal= 0xa, code= 0x80c, context= 0x68998f34
PC = 0x6070e908, SP = 0x70201d70, RA = 0x6070e8fc
Cause Reg = 0x0000080c, Status Reg = 0x3400e103
rommon 31 >
rommon 31 >

Leo Laohoo · ‎06-30-2011

TLB (store) exception, CPU signal 10

It's an IOS bug. Please contact TAC.

ortiz.d · ‎07-01-2011

Are you sure? I just tried to load the latest IOS-

c3825-adventerprisek9-mz.151-4.M1.bin

Can you recommend a better one?

ortiz.d · ‎07-01-2011

I just tried a different IOS image and I get the same. Could it be bad memory?

rommon 6 > boot usbflash0:c3825-spservicesk9-mz.124-25e.bin
program load complete, entry point: 0x8000f000, size: 0x3d1a0

program load complete, entry point: 0x80010000, size: 0x20f45d8
Self decompressing the image : ############################################################################################################################################################################################### [OK]

Smart Init is enabled
smart init is sizing iomem
ID            MEMORY_REQ                 TYPE
00042A          0X010D78F3 C3825 motherboard
                0X00288860 DSP SIMM
                0X000021B8 Onboard USB
                0X00660670 public buffer pools
                0X0078F000 public particle pools
TOTAL:          0X0215197B

If any of the above Memory Requirements are
"UNKNOWN", you may be using an unsupported
configuration or there is a software problem and
system operation may be compromised.
Rounded IOMEM up to: 34Mb.
Using 6 percent iomem. [34Mb/512Mb]

Restricted Rights Legend

Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.

           cisco Systems, Inc.
           170 West Tasman Drive
           San Jose, California 95134-1706

Cisco IOS Software, 3800 Software (C3825-SPSERVICESK9-M), Version 12.4(25e), RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2011 by Cisco Systems, Inc.
Compiled Wed 16-Mar-11 21:15 by prod_rel_team

%ALIGN-1-FATAL: Illegal access to a low address 20:56:56 UTC Fri Jul 1 2011
addr=0x0, pc=0x605406CC , ra=0x605406C0 , sp=0x7004A3A8

20:56:56 UTC Fri Jul 1 2011: TLB (store) exception, CPU signal 10, PC = 0x605406CC

--------------------------------------------------------------------
Possible software fault. Upon reccurence, please collect
crashinfo, "show tech" and contact Cisco Technical Support.
--------------------------------------------------------------------

-Traceback= 0x605406CC 0x60540F94 0x605425AC 0x60682250 0x60684000 0x60684628 0x60673D8C 0x6067C344 0x60670BDC 0x60274B24 0x621E5618 0x6117DBDC 0x6117DEAC 0x621BE55C 0x621BE540
$0 : 00000000, AT : 64200000, v0 : 00000001, v1 : 00000001
a0 : 701FB46C, a1 : 62C00000, a2 : 00000000, a3 : 6069D0C0
t0 : FFFFFFFF, t1 : 00000000, t2 : 00000000, t3 : 00000042
t4 : 00004200, t5 : 00000000, t6 : 646B39E0, t7 : 641A0000
s0 : 00000000, s1 : 701FB46C, s2 : 6492BC14, s3 : 7004A408
s4 : 00000005, s5 : 00000001, s6 : 00000000, s7 : 00000001
t8 : 643D933C, t9 : 63CE0000, k0 : 30408001, k1 : B0020000
gp : 6420AAB0, sp : 7004A3A8, s8 : 000005DD, ra : 605406C0
EPC : 605406CC, ErrorEPC : BFD85CFC, SREG : 3400E103
MDLO : 00000548, MDHI : 00000000, BadVaddr : 00000000
DATA_START : 0x6267ECB0
Cause 0000080C (Code 0x3): TLB (store) exception

File flash:crashinfo_20110701-205656 Device Error :No such device
File flash:crashinfo_20110701-205656 Device Error :No such device
File flash:crashinfo_20110701-205656 Device Error :No such device

20:56:56 UTC Fri Jul 1 2011: TLB (store) exception, CPU signal 10, PC = 0x605406CC

--------------------------------------------------------------------
Possible software fault. Upon reccurence, please collect
crashinfo, "show tech" and contact Cisco Technical Support.
--------------------------------------------------------------------

-Traceback= 0x605406CC 0x60540F94 0x605425AC 0x60682250 0x60684000 0x60684628 0x60673D8C 0x6067C344 0x60670BDC 0x60274B24 0x621E5618 0x6117DBDC 0x6117DEAC 0x621BE55C 0x621BE540
$0 : 00000000, AT : 64200000, v0 : 00000001, v1 : 00000001
a0 : 701FB46C, a1 : 62C00000, a2 : 00000000, a3 : 6069D0C0
t0 : FFFFFFFF, t1 : 00000000, t2 : 00000000, t3 : 00000042
t4 : 00004200, t5 : 00000000, t6 : 646B39E0, t7 : 641A0000
s0 : 00000000, s1 : 701FB46C, s2 : 6492BC14, s3 : 7004A408
s4 : 00000005, s5 : 00000001, s6 : 00000000, s7 : 00000001
t8 : 643D933C, t9 : 63CE0000, k0 : 30408001, k1 : B0020000
gp : 6420AAB0, sp : 7004A3A8, s8 : 000005DD, ra : 605406C0
EPC : 605406CC, ErrorEPC : BFD85CFC, SREG : 3400E103
MDLO : 00000548, MDHI : 00000000, BadVaddr : 00000000
DATA_START : 0x6267ECB0
Cause 0000080C (Code 0x3): TLB (store) exception

-Traceback= 0x605406CC 0x60540F94 0x605425AC 0x60682250 0x60684000 0x60684628 0x60673D8C 0x6067C344 0x60670BDC 0x60274B24 0x621E5618 0x6117DBDC 0x6117DEAC 0x621BE55C 0x621BE540

=== Flushing messages (20:56:56 UTC Fri Jul 1 2011) ===

Queued messages:
*Jul 1 20:56:56.611: %SYS-3-LOGGER_FLUSHING: System pausing to ensure console debugging output.

*Jul 1 20:56:56.343: fpga fw verification false
*Jul 1 20:56:56.343: FPGA failed to dw properly
L2 ECC Counter = 0000000000000000
Memory and IO Err Counter = 0000000000000000

Bus Watcher Counters
cor_l2cache_data_ecc_count = 0
bad_l2cache_data_ecc_count = 0
cor_l2cache_tag_ecc_count = 0
bad_l2cache_tag_ecc_count = 0
cor_memory_data_ecc_count = 0
bad_memory_data_ecc_count = 0
bus_errors = 0

         CPU temp is 48
         Temp1 temp is 27
         Temp2 temp is 25

         Internal voltage is 3300
         Remote voltage1 is 5210
         Remote voltage2 is 2525
         Remote voltage3 is 1215

No warm reboot Storage
*** System received a Bus Error exception ***
signal= 0xa, code= 0x80c, context= 0x6450fccc
PC = 0x605406cc, SP = 0x7004a3a8, RA = 0x605406c0
Cause Reg = 0x0000080c, Status Reg = 0x3400e103
rommon 7 >

Reza Sharifi · ‎07-01-2011

Did the router work fine before you pulled the compaq flash card? If there is enough space you can load IOS 15 from the same compaq flash. IOS 15 is about 45MB.

Also, you may have a bad usb port and not every usb works with Cisco routers.

have a look at this document for supported usb devices.

Cisco routers support only Cisco USB flash memory modules and the Aladdin USB eToken Pro key.

http://www.cisco.com/en/US/docs/routers/access/2800/hardware/notes/mithril_ps5855_TSD_Products_Installation_Guide_Chapter.html

HTH

Reza

Leo Laohoo · ‎07-01-2011

Hmmm ... Two different IOS and you get software crashes. That's very, very odd. Interesting ...

Ok, if you have any network modules plugged in then REMOVE each and every one of them. Try that.

If you still get crashes then use the CF instead of USB. I'm seriously considering Reza's opininion that the USB flash could be faulty or the USB port and by booting via CF will eliminate that out of the equation.

Check the file checksum of your IOS. Could be corrupted IOS.

paolo bevilacqua · ‎07-02-2011

Can very well be hardware fault. Try replacing memory module if still fails open a TAC case for replacement.

ortiz.d · ‎07-09-2011

Okay. So I pulled a WIC card and I was able to boot from USB. I had to boot from USB. I then reformatted the flash card and copied the IOS from the USB to flash. It worked. Then I got more history. I power surge from Verizon must nave messed up the card and flash. It now boots fine.

Sent from Cisco Technical Support iPhone App