Router hanging problem

[shijuuu]

HI ,

Facing route hanging issue at this point as shown below....any clue why ?

thanks

Smart Init is enabled
smart init is sizing iomem
  ID            MEMORY_REQ                 TYPE
0003E7          0X003DA000 C2811 Mainboard
                0X00263F50 Onboard VPN
                0X000021B8 Onboard USB
                0X002C29F0 public buffer pools
                0X00211000 public particle pools
TOTAL:          0X00B13AF8

If any of the above Memory Requirements are
"UNKNOWN", you may be using an unsupported
configuration or there is a software problem and
system operation may be compromised.
Rounded IOMEM up to: 12Mb.
Using 4 percent iomem. [12Mb/256Mb]
SDRAM ECC Error,
err addr - 0x09436A07, err data_l 0x01000000,err data_h 0x00200000,
rcv ecc - 0x00000000, calc ecc 0x21000000, err cnt 0x65000000
r0  = FFFFFFFF r1  = FFFFFFFF r2  = 0        r3  = 43060000 r4  = 0       
r5  = 38       r6  = 0        r7  = 4022FB9C r8  = 0        r9  = 20      
r10 = FFFFFFFF r11 = FEFDFFFF r12 = 0        r13 = 1000000  r14 = 0       
r15 = F4240    r16 = 0        r17 = C100     r18 = 0        r19 = 3400C101
r20 = 0        r21 = 40083590 r22 = FFFFFFFF r23 = FFFF00FF r24 = 0       
r25 = 40083570 r26 = 0        r27 = 4390EA24 r28 = 0        r29 = 4390EA20
r30 = 0        r31 = 4390EA1C r32 = FFFFFFFF r33 = FFFFFFFF r34 = FFFFFFFF
r35 = FFFFFFFF r36 = FFFFFFFF r37 = FFFFFFFF r38 = FFFFFFFF r39 = FFFFFFFF
r40 = FFFFFFFF r41 = FFFFFFFF r42 = FFFFFFFF r43 = FFFFFFFF r44 = FFFFFFFF
r45 = FFFFFFFF r46 = FFFFFFFF r47 = FFFFFFFF r48 = 0        r49 = D0D0D0D 
r50 = 0        r51 = 0        r52 = 0        r53 = 3040A801 r54 = FFFFFFFF
r55 = A000F000 r56 = FFFFFFFF r57 = FFFFFFFF r58 = FFFFFFFF r59 = 8000EDB0
r60 = FFFFFFFF r61 = FFFFFFFF r62 = 0        r63 = 40090CCC
sreg     = 3400C103 mdlo_hi    = 0        mdlo        = 3E8    
mdhi_hi  = 0        mdhi       = 0        badvaddr_hi = FFFFFFFF
badvaddr = FFFFFFFF cause      = FFFFFFFF epc_hi      = 0      
epc      = 40083054 err_epc_hi = FFFFFFFF err_epc     = FFFFFFFF

%ERR-1-FATAL: Fatal error interrupt, reloading
err_stat=0x0

TLB (load or instruction fetch) exception, CPU signal 10, PC = 0x0

-Traceback=
$0 : 00000000, AT : 43060000, v0 : 00000000, v1 : 0000000E
a0 : 0000002A, a1 : 00000000, a2 : 43912FA8, a3 : 42D30000
t0 : 00000028, t1 : 34008001, t2 : 34008000, t3 : FFFF00FF
t4 : 40083570, t5 : 4390EA24, t6 : 4390EA20, t7 : 4390EA1C
s0 : 00001000, s1 : 00000000, s2 : 433E4D80, s3 : 00000000
s4 : 8000EDC0, s5 : 43912FA8, s6 : 8000C8E0, s7 : 8000C8D8
t8 : 0D0D0D0D, t9 : 00000000, k0 : 00000020, k1 : 00000000
gp : 430659A0, sp : 43912E58, s8 : 00000001, ra : 4165B89C
EPC  : 00000000, ErrorEPC : 20690184, SREG     : 34008003
MDLO : 00000000, MDHI     : 0000000F, BadVaddr : 00000000
Cause 00000008 (Code 0x2): TLB (load or instruction fetch) exception

TLB (load or instruction fetch) exception, CPU signal 10, PC = 0x0

-Traceback=
$0 : 00000000, AT : 43060000, v0 : 00000000, v1 : 00000040
a0 : 42CD0250, a1 : FFFFFFFF, a2 : 42CD01DC, a3 : FFFFFFFF
t0 : 00000000, t1 : 00000000, t2 : 00000000, t3 : 00000000
t4 : 00000000, t5 : 00000000, t6 : 0000000F, t7 : 00000000
s0 : 42CD0128, s1 : 42CD0250, s2 : 42CD01DC, s3 : 00000000
s4 : 00000000, s5 : FFFFFFFF, s6 : 7FFFFFFF, s7 : 0000000A
t8 : 0D0D0D0D, t9 : 00000000, k0 : 431EA2B0, k1 : 00000028
gp : 430659A0, sp : 42CD0100, s8 : FFFFFFFF, ra : 40ABA2BC
EPC  : 00000000, ErrorEPC : 20690184, SREG     : 34008003
MDLO : 00000000, MDHI     : 00000008, BadVaddr : 00000000
Cause 00000008 (Code 0x2): TLB (load or instruction fetch) exception

Nested write_crashinfo call (2 times)

TLB (load or instruction fetch) exception, CPU signal 10, PC = 0x0

-Traceback=
$0 : 00000000, AT : 43060000, v0 : 433D0000, v1 : 00000000
a0 : 00000002, a1 : 00008000, a2 : 00000000, a3 : 42D30000
t0 : 00000038, t1 : 34008001, t2 : 34008000, t3 : FFFF00FF
t4 : 40083570, t5 : 00000008, t6 : 00000000, t7 : 00000000
s0 : 0000000A, s1 : 00000008, s2 : 0000000A, s3 : 43320000
s4 : 00000008, s5 : FFFFFFFF, s6 : 00000038, s7 : 431F0000
t8 : 0D0D0D0D, t9 : 00000000, k0 : 431EA2B0, k1 : 00000038
gp : 430659A0, sp : 42CD0028, s8 : 42CD0000, ra : 4007EEA0
EPC  : 00000000, ErrorEPC : 20690184, SREG     : 34008003
MDLO : 00000000, MDHI     : 00000002, BadVaddr : 00000000
Cause 00000008 (Code 0x2): TLB (load or instruction fetch) exception

Nested write_crashinfo call (3 times)

TLB (load or instruction fetch) exception, CPU signal 10, PC = 0x0

-Traceback=
$0 : 00000000, AT : 43060000, v0 : 433D0000, v1 : 00000000
a0 : 00000002, a1 : 00008000, a2 : 00000000, a3 : 42D30000
t0 : 00000038, t1 : 34008001, t2 : 34008000, t3 : FFFF00FF
t4 : 40083570, t5 : 00000008, t6 : 00000000, t7 : 00000000
s0 : 0000000A, s1 : 00000008, s2 : 0000000A, s3 : 43320000
s4 : 00000008, s5 : FFFFFFFF, s6 : 00000038, s7 : 431F0000
t8 : 0D0D0D0D, t9 : 00000000, k0 : 431EA2B0, k1 : 00000038
gp : 430659A0, sp : 42CD0028, s8 : 42CD0000, ra : 4007EEA0
EPC  : 00000000, ErrorEPC : 20690184, SREG     : 34008003
MDLO : 00000000, MDHI     : 00000002, BadVaddr : 00000000
Cause 00000008 (Code 0x2): TLB (load or instruction fetch) exception

-Traceback=

[andtoth · Accepted solution]

Hi,

"SDRAM ECC Error" message means there was a problem with the memory on the router, probably it's a parity error. If it happened only once, it can be considered a one-time issue. If it happened multiple times, you need to replace the memory module in the router. If that does not help, probably the router needs a replacement.

To read more about parity errors, please refer to the following link:

http://www.cisco.com/en/US/products/hw/routers/ps341/products_tech_note09186a0080094793.shtml#softvshard

Andras

[shijuuu]

hi thanks a lot.......this is repeating issue surprisingly we have three routers with same condition.

thanks.

[andtoth]

I would check the environment to see if the devices are grounded properly and also make sure there's no electric generator or similar equipment nearby. You might also try physically reseating the memory modules to see if it helps.

[shijuuu]

now I got it there is an electric  generator nearby....... this was installed in a restricted  area completely running on generator power.I have to make sure I never been to the site physically.

thanks for sharing valuable information.

[ccie4297]

One more solution for those who  may also encounter this problem... I receieved the exact same error  output and all I had to do was to reseat the RAM chips in our 2811 and  voila, the problem is solved.

    Cheers,

        Ken