Cisco Router 7600 continiously rebooting issue

[rajafurqan2011]

Hi,  i have cisco 7606 and issue is router rebooted in 4 to 5 time in a day.  details given below and crashfile also attached. 

Router#show version
Cisco IOS Software, c7600rsp72043_rp Software (c7600rsp72043_rp-ADVENTERPRISEK9-M), Version 15.0(1)S1, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2010 by Cisco Systems, Inc.
Compiled Wed 29-Sep-10 02:28 by prod_rel_team

ROM: System Bootstrap, Version 12.2(33r)SRE, RELEASE SOFTWARE (fc1)
BOOTLDR: Cisco IOS Software, c7600rsp72043_rp Software (c7600rsp72043_rp-ADVENTERPRISEK9-M), Version 15.0(1)S1, RELEASE SOFTWARE (fc1)

Router uptime is 10 hours, 12 minutes
Uptime for this control processor is 10 hours, 11 minutes
System returned to ROM by s/w reset (SP by error - a Software forced crash, PC 0x8361EFC)
System image file is "disk0:c7600rsp72043-adventerprisek9-mz.150-1.S1.bin"
Last reload type: Normal Reload

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

Cisco CISCO7606 (M8500) processor (revision 1.0) with 1900544K/131072K bytes of memory.
Processor board ID TBA06180199
BASEBOARD: RSP720
CPU: MPC8548_E, Version: 2.1, (0x80390021)
CORE: E500, Version: 2.2, (0x80210022)
CPU:1200MHz, CCB:400MHz, DDR:200MHz,
L1: D-cache 32 kB enabled
I-cache 32 kB enabled

Last reset from power-on
1 SIP-400 controller (10 GigabitEthernet).
1 Virtual Ethernet interface
12 Gigabit Ethernet interfaces
3964K bytes of non-volatile configuration memory.

500472K bytes of Internal ATA PCMCIA card (Sector size 512 bytes).
Configuration register is 0x2102

[rajafurqan2011]

*Jun  8 07:55:54.403: %SYS-SP-5-RESTART: System restarted --    


in Crashfile 

[Diana Karolina Rojas]

Hello,

Can you provide the "show track" from your device? adittionally do you have another RSP to test?

Please do not forget to rate useful post.

Best Regards,

[rajafurqan2011]

nothing found in "show track" 

NAS-7606-QML#show track
NAS-7606-QML#
NAS-7606-QML#
NAS-7606-QML#
NAS-7606-QML#show track
NAS-7606-QML#
NAS-7606-QML#

No, i dnt have another RSP

[Diana Karolina Rojas]

You hace various tracebacks and errors in your output one of those is: Unexpected exception to CPU vector 1500

I do not know in detail what this error means. but from what I was investigating these kinds of messages can be a hardware or software problem. You can do the next like littles workarounds:

Check the proper ventilation of the router and make sure it is not overheating. If there are any modules installed into the router, try removing and reseating them. This includes internal modules, RAM, Compact Flash cards, anything that can be removed and plugged back. 

If all of that does not work then Upgrade you IOS version.

If even that does not work I'm afraid you'll have to open a case in the TAC, maybe you will need an RMA.

Please do not forget to rate useful post.

Best Regards,

[rajafurqan2011]

Currently i am using ios "disk0:c7600rsp72043-adventerprisek9-mz.150-1.S1.bin" 

and i am using this router as BRAS and only found "pppoe enable group global" command in above mention IOS.

i upgraded following ios, but in that ios i dn't found this command "pppoe enable group global"

and without this command users can't b connected.

c7600rsp72043-adventerprisek9-mz.152-4.S6.bin

c7600rsp72043-adventerprisek9-mz.154-3.S6.bin

c7600rsp72043-adventerprisek9-mz.155-3.S5.bin

[Diana Karolina Rojas]

On IOS 15.2(2)S 'pppoe enable group' is still supported so you can try this, from 15.2(4)S the support from this feature is deprecated, that is why you could not configure it.

Please do not forget to rate useful post.

Best Regards,

[rajafurqan2011]

I Upgraded IOS last Night

From "disk0:c7600rsp72043-adventerprisek9-mz.150-1.S1.bin" 

to "c7600rsp72043-adventerprisek9-mz.152-2.S2.bin"

But today its again rebooted.

QML#show version
Cisco IOS Software, c7600rsp72043_rp Software (c7600rsp72043_rp-ADVENTERPRISEK9-M), Version 15.2(2)S2, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2012 by Cisco Systems, Inc.
Compiled Tue 07-Aug-12 10:08 by prod_rel_team

ROM: System Bootstrap, Version 12.2(33r)SRE, RELEASE SOFTWARE (fc1)
BOOTLDR: Cisco IOS Software, c7600rsp72043_rp Software (c7600rsp72043_rp-ADVENTERPRISEK9-M), Version 15.2(2)S2, RELEASE SOFTWARE (fc1)

QML uptime is 2 hours, 38 minutes
Uptime for this control processor is 2 hours, 38 minutes
System returned to ROM by s/w reset (SP by error - a Software forced crash, PC 0x83D5B4C)
System image file is "disk1:c7600rsp72043-adventerprisek9-mz.152-2.S2.bin"
Last reload type: Normal Reload
Last reload reason: power-on

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

Cisco CISCO7606 (M8500) processor (revision 1.0) with 1900544K/131072K bytes of memory.
Processor board ID TBA06180199
BASEBOARD: RSP720
CPU: MPC8548_E, Version: 2.1, (0x80390021)
CORE: E500, Version: 2.2, (0x80210022)
CPU:1200MHz, CCB:400MHz, DDR:200MHz,
L1: D-cache 32 kB enabled
I-cache 32 kB enabled

Last reset from power-on
1 SIP-400 controller (10 GigabitEthernet).
1 Virtual Ethernet interface
12 Gigabit Ethernet interfaces
3964K bytes of non-volatile configuration memory.

500472K bytes of Internal ATA PCMCIA card (Sector size 512 bytes).
Configuration register is 0x2102

[Leo Laohoo]

---

@rajafurqan2011 wrote:

c7600rsp72043-adventerprisek9-mz.152-2.S2.bin

---

Whao.  You went from one very old IOS to another very old IOS (2015).  
Post the crashinfo file for the latest crash.

[rajafurqan2011]

i also tried some other IOS but the issue is "pppoe enable group global" command not working on some other ios. Can you tell me the latest ios with supported command "pppoe enable group global"

crashfile

*Jun 8 21:56:02.407: SP: SP: Currently running ROMMON from S (Gold) regionCisco 7600 (M8500) processor with 917504K/65536K bytes of memory.
Processor board ID TBA06180199
BASEBOARD: RSP720
CPU: MPC8548_E, Version: 2.1, (0x80390021)
CORE: E500, Version: 2.2, (0x80210022)
CPU:1200MHz, CCB:400MHz, DDR:200MHz,
L1: D-cache 32 kB enabled
I-cache 32 kB enabled

Last reset from s/w reset
2 Gigabit Ethernet interfaces
3964K bytes of non-volatile configuration memory.

500472K bytes of Internal ATA PCMCIA card (Sector size 512 bytes).
*Jun 8 21:56:09.711: %SYS-SP-5-RESTART: System restarted --
Cisco IOS Software, c7600rsp72043_sp Software (c7600rsp72043_sp-ADVENTERPRISEK9-M), Version 15.0(1)S1, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2010 by Cisco Systems, Inc.
Compiled Wed 29-Sep-10 02:44 by prod_rel_team
*Jun 8 21:56:11.791: %OIR-SP-6-INSPS: Power supply inserted in slot 1
*Jun 8 21:56:11.791: %C7600_PWR-SP-4-PSOK: power supply 1 turned on.
*Jun 8 21:56:11.855: %OIR-SP-6-INSPS: Power supply inserted in slot 2
*Jun 8 21:56:11.855: %C7600_PWR-SP-4-PSOK: power supply 2 turned on.
*Jun 8 21:56:11.863: %C7600_PWR-SP-4-PSREDUNDANTBOTHSUPPLY: in power-redundancy mode, system is operating on both power supplies.
*Jun 8 21:56:13.359: %C7600_ENV-SP-4-FANHIOUTPUT: Version 2 high-output fan-tray is in effect
*Jun 8 21:56:17.703: %FABRIC-SP-5-CLEAR_BLOCK: Clear block option is off for the fabric in slot 6.
*Jun 8 21:56:17.783: %FABRIC-SP-5-FABRIC_MODULE_ACTIVE: The Switch Fabric Module in slot 6 became active.
*Jun 8 21:56:19.003: %DIAG-SP-6-RUN_MINIMUM: Module 6: Running Minimal Diagnostics...
*Jun 8 21:56:23.271: %DIAG-SP-6-DIAG_OK: Module 6: Passed Online Diagnostics
*Jun 8 21:56:23.799: %OIR-SP-6-INSCARD: Card inserted in slot 6, interfaces are now online
*Jun 8 21:57:11.433: %DIAG-SP-6-RUN_MINIMUM: Module 3: Running Minimal Diagnostics...
*Jun 8 21:57:21.235: %DIAG-SP-6-DIAG_OK: Module 3: Passed Online Diagnostics
*Jun 8 21:57:22.443: %OIR-SP-6-INSCARD: Card inserted in slot 3, interfaces are now online
*Jun 8 21:57:24.131: %DIAG-SP-6-RUN_MINIMUM: Module 3/0: Running Minimal Diagnostics...
*Jun 8 21:57:24.731: %DIAG-SP-6-DIAG_OK: Module 3/0: Passed Online Diagnostics
*Jun 8 21:57:25.451: %DIAG-SP-6-RUN_MINIMUM: Module 3/1: Running Minimal Diagnostics...
*Jun 8 21:57:25.463: %DIAG-SP-6-DIAG_OK: Module 3/1: Passed Online Diagnostics
*Jun 8 22:29:00.559: %EARL_L2_ASIC-SP-4-DBUS_HDR_ERR: EARL L2 ASIC #0: Dbus Hdr. Error occurred. Ctrl1 0xB08D0E3D
*Jun 8 22:38:15.055: SP: SP detected RP/Draco rebooting, resetting SP

%Software-forced reload

03:38:15 PST Sat Jun 9 2018: Unexpected exception to CPU: vector 1500, PC = 0x8361EFC , LR = 0x8361E90

-Traceback= 0x8361EFCz 0x8361E90z 0x864C7C0z 0x865AE68z 0x864FC30z 0x8363A84z 0x835DCF8z

CPU Register Context:
MSR = 0x00029200 CR = 0x20000022 CTR = 0x08D37B9C XER = 0x00000000
R0 = 0x08361E90 R1 = 0x1146E9F0 R2 = 0xFFFCFFFC R3 = 0x109DAE3C
R4 = 0x1146E980 R5 = 0x00000000 R6 = 0x00003327 R7 = 0xFEC20030
R8 = 0x00000005 R9 = 0x00000000 R10 = 0xFEC2002C R11 = 0xFEC20000
R12 = 0x0000001D R13 = 0x04044000 R14 = 0x0864FB50 R15 = 0x00000000
R16 = 0x00000000 R17 = 0x00000000 R18 = 0x00000000 R19 = 0x00000000
R20 = 0x00000000 R21 = 0x00000000 R22 = 0x00000000 R23 = 0x00000000
R24 = 0x00000000 R25 = 0x00000001 R26 = 0x00000000 R27 = 0x10324828
R28 = 0x00000001 R29 = 0x00000005 R30 = 0x00000006 R31 = 0x00000000

=== Start of Crashinfo Collection (03:38:15 PST Sat Jun 9 2018) ===

For image:
Cisco IOS Software, c7600rsp72043_sp Software (c7600rsp72043_sp-ADVENTERPRISEK9-M), Version 15.0(1)S1, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2010 by Cisco Systems, Inc.
Compiled Wed 29-Sep-10 02:44 by prod_rel_team

========= Show Alignment =============================

No spurious memory references have been recorded.

No alignment data has been recorded.

========= Malloc and Free Traces=====================

MallocFree Trace: ixmallocfree=0x8 ptr=0xE825BB8
E825B78: 12CBADCC 6000003C 12CBACE0 4000005E 12CBACE0 8339EE8 12CBACE0 833A5C0
E825B98: 12CBADCC 6000003C 12CBACE0 4000005E 12CBACE0 8339EE8 12CBACE0 833A5C0
E825BB8: 12CBADCC 6000003C 12CBACE0 4000005E 12CBACE0 8339EE8 12CBACE0 833A5C0
E825BD8: 12CBADCC 6000003C 12CBACE0 4000005E 12CBACE0 8339EE8 12CBACE0 833A5C0
E825BF8: 12CBADCC 6000003C 12CBACE0 4000005E 12CBACE0 8339EE8 12CBACE0 833A5C0
E825C18: 12CBADCC 6000003C 12CBACE0 4000005E 12CBACE0 8339EE8 12CBACE0 833A5C0
E825C38: 12CBADCC 6000003C 12CBACE0 4000005E 12CBACE0 8339EE8 12CBACE0 833A5C0
E825C58: 12CBADCC 6000003C 12CBACE0 4000005E 12CBACE0 8339EE8 12CBACE0 833A5C0
E825C78: 12CBADCC 6000003C 12CBACE0 4000005E 12CBACE0 8339EE8 12CBACE0 833A5C0
E825C98: 12CBADCC 6000003C 12CBACE0 4000005E 12CBACE0 8339EE8 12CBACE0 833A5C0
E825CB8: 12CBADCC 6000003C 12CBACE0 4000005E 12CBACE0 8339EE8 12CBACE0 833A5C0
E825CD8: 12CBADCC 6000003C 12CBACE0 4000005E 12CBACE0 8339EE8 12CBACE0 833A5C0
E825CF8: 12CBADCC 6000003C 12CBACE0 4000005E 12CBACE0 8339EE8 12CBACE0 833A5C0
E825D18: 12CBADCC 6000003C 12CBACE0 4000005E 12CBACE0 8339EE8 12CBACE0 833A5C0
E825D38: 12CBADCC 6000003C 12CBACE0 4000005E 12CBACE0 8339EE8 12CBACE0 833A5C0
E825D58: 12CBADCC 6000003C 12CBACE0 4000005E 12CBACE0 8339EE8 12CBACE0 833A5C0

ChunkMallocFree Trace: ixchunkmallocfree=0xF ptr=0xE8255CC
E825518: 859C75C 858A228 12963050 859C75C 858A3BC 12963050 859C7B4 858A50C
E825538: 129638CC 859C7B4 858A434 129638CC 859C75C 858A228 12963050 859C75C
E825558: 858A3BC 12963050 859C7B4 858A50C 129638CC 859C7B4 858A434 129638CC
E825578: 859C75C 858A228 12963050 859C75C 858A3BC 12963050 859C7B4 858A50C
E825598: 129638CC 859C7B4 858A434 129638CC 859C7B4 858A434 129638A4 859C75C
E8255B8: 858A228 12963050 859C704 8591298 1296184C 83711AC 8373270 10B64ED4
E8255D8: 83711AC 8373270 10B64ED4 83711AC 837317C 10B64ED4 83711AC 8373270
E8255F8: 10B64ED4 83711AC 837317C 10B64ED4 836A12C 836BB10 10B60488 859C7B4
E825618: 858A434 129638CC 859C7B4 858A50C 129638CC 859C6AC 85959F4 1296111C
E825638: 859C6AC 85959F4 12961140 859C6AC 85959F4 12961164 859C7B4 858A434
E825658: 129638CC 859C7B4 858A50C 129638CC 859C6AC 8595AEC 1296111C 859C6AC
E825678: 8595AEC 12961140 859C6AC 8595AEC 12961164 836A12C 836BC08 10B60488
E825698: 83711AC 8373270 10B64ED4 83711AC 837317C 10B64ED4 859C7B4 858A434
E8256B8: 129638CC 859C75C 858A228 12963050 859C75C 858A3BC 12963050 859C7B4
E8256D8: 858A50C 129638CC 83711AC 8373270 10B64ED4 83711AC 837317C 10B64ED4
E8256F8: 859C7B4 858A434 129638CC 859C75C 858A228 12963050 859C75C 858A3BC
E825718: 12963050 859C7B4 858A50C 129638CC 836A12C 836BB10 10B60488 859C7B4
E825738: 858A434 129638CC 859C75C 858A228 12963050 859C75C 858A3BC 12963050
E825758: 859C7B4 858A50C 129638CC 859C6AC 85959F4 12961164 859C6AC 85959F4
E825778: 12961140 859C6AC 85959F4 1296111C 859C7B4 858A434 129638CC 859C75C
E825798: 858A228 12963050 859C75C 858A3BC 12963050 859C7B4 858A50C 129638CC
E8257B8: 859C7B4 858A434 129638CC 859C75C 858A228 12963050 859C75C 858A3BC
E8257D8: 12963050 859C7B4 858A50C 129638CC 859C6AC 8595AEC 12961164 859C6AC
E8257F8: 8595AEC 12961140 859C6AC 8595AEC 1296111C 859C7B4 858A434 129638CC
========== Stack Trace========

-Traceback= 0x8361EFCz 0x8361E90z 0x864C7C0z 0x865AE68z 0x864FC30z 0x8363A84z 0x835DCF8z

========= Context ======================

c7600rsp72043_sp Software (c7600rsp72043_sp-ADVENTERPRISEK9-M), Version 15.0(1)S1, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Compiled Wed 29-Sep-10 02:44 by prod_rel_team
Signal = 23 Vector = 0x1500 Uptime = 00:43:21

CPU Register Context:
PC = 0x08361EFC MSR = 0x00029200 CR = 0x20000022 LR = 0x08361E90
CTR = 0x08D37B9C XER = 0x00000000 DAR = 0x00000000 DSISR = 0x41000000
DEC = 0x00007FB2 TBU = 0x0000001E TBL = 0x1A57104F IMMR = 0x00000000
R0 = 0x08361E90 R1 = 0x1146E9F0 R2 = 0xFFFCFFFC R3 = 0x109DAE3C
R4 = 0x1146E980 R5 = 0x00000000 R6 = 0x00003327 R7 = 0xFEC20030
R8 = 0x00000005 R9 = 0x00000000 R10 = 0xFEC2002C R11 = 0xFEC20000
R12 = 0x0000001D R13 = 0x04044000 R14 = 0x0864FB50 R15 = 0x00000000
R16 = 0x00000000 R17 = 0x00000000 R18 = 0x00000000 R19 = 0x00000000
R20 = 0x00000000 R21 = 0x00000000 R22 = 0x00000000 R23 = 0x00000000
R24 = 0x00000000 R25 = 0x00000001 R26 = 0x00000000 R27 = 0x10324828
R28 = 0x00000001 R29 = 0x00000005 R30 = 0x00000006 R31 = 0x00000000

========= Stack Dump ==========================

Stack Frame Pointer in Context is 0x1146E9F0, at process level

1146E5F0: FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
1146E610: FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
1146E630: FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
1146E650: FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
1146E670: FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
1146E690: FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
1146E6B0: FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
1146E6D0: FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
1146E6F0: 1146E700 FFFFFFFF 1 1146E718 1146E710 81AE79C 5 9F41588
1146E710: 1146E738 9AF3300 3535FFFF 5 23 9 FB89C20 9F415F8
1146E730: 1146E748 114CBEC4 1146E748 A 1 23 1146E7D8 8CD29E0
1146E750: 114CBEC4 5 0 23 A 1146E788 A 114CBEC4
1146E770: 1146E888 8CD0D38 1146E7A8 1146E8D4 0 10919F54 0 3C05B7D8
1146E790: 5A FCFCCAC 1146E7C8 116FAAAC 0 10919F54 1146E7C8 3C05B7D8
1146E7B0: 5A FCFCCAC 129169A4 100 FCFCCAC 116FA4D0 1146E7F0 8F38990
1146E7D0: 1146E7D8 3C05B7D8 5A FCFCCAC 1146E7F0 116FA4D0 116FA4D0 116FA4D0
1146E7F0: 1146E808 8F38F38 1146E808 0 1146E828 3C05B7D8 1146E828 82B1A48
1146E810: 1 0 1146E828 0 4 1ABC5DEC 1146E848 833944C
1146E830: 1146E840 1A510E24 1 0 1146E848 835978C 1146E868 8339C68
1146E850: 1 0 0 4 1146E950 D33B860 1146E870 833B238
1146E870: 1146E878 8EAD4B8 1146E898 8359A94 1146E898 0 0 2774C8
1146E890: 1146E8A0 2774C8 1146E8A8 8359538 1146E8A8 1ABDEC4C 1146E9D0 833BB2C
1146E8B0: 0 1 0 0 0 0 0 0
1146E8D0: 0 0 1146E8F8 0 0 0 0 0
1146E8F0: 1146E900 1 1146E900 1 1146E9D0 8D37984 0 0
1146E910: 6 4C 12AD4100 1 FE480000 0 0 0
1146E930: 0 0 0 0 0 0 0 0
1146E950: 0 0 0 0 0 0 0 0
1146E970: 7000018 1146E9D8 1146E908 0 7E2 6 9 3
1146E990: 26 F 5F 6 A0 4650 1 5
1146E9B0: DEC583D7 1887F030 1146E9D0 833C0BC 1 5 1146E9E0 C1A0000
1146E9D0: 1146E9E0 5 D319610 0 1146E9F0 8361988 6 0
1146E9F0: 1146EA08 8361E90 864B890 5 6 12B856C4 1146EA78 864B89C

--------- show heartbeat settings --------
heartbeat interval = 2
heartbeat warning interval = 30
heartbeat fatal count = 5
heartbeat interrupt interval = 30
heartbeat interrupt max = 5
ignore rx = no
ignore tx = no

--------- show heartbeat --------
Out of buffers = 0
EOBC tx failures = 0
Negative clock updates = 0
Fatal errors = 0
Last heartbeat sched = 1844
Last heartbeat sent = 1844

Peer 6/0 [online]:
Heartbeats received = 0
Config msgs received = 0
Clock cfg msgs received = 0
Unknown msgs received = 0
Not heard warnings = 0
Current rx warnings = 0 (max 5)

Peer 6/1 [online]:
Heartbeats received = 1270
Config msgs received = 1
Clock cfg msgs received = 1
Unknown msgs received = 0
Not heard warnings = 0
Current rx warnings = 0 (max 5)

Scheduler history [SCP Download Listen 6%/0% (00:00:00.1632331375084 14%/1%)] [SCP Download Listen 00:00:00.172042494436] [REP LSL Hello PP Process 00:00:00.172042494436] [SCP Download Listen 00:00:00.189222363620]

------Fabman Error-----
Error Name # of Reported Err Info Time Elapsed (FPOE 4 fab_id:0)
LC SYNC 0 0
LC CRC 0 0
LC HBEAT 0 0
LC DDRSYNC 0 0
LC LINK PERSIST 0 0
FAB SYNC 0 0
FAB BUFFER 0 0
FAB TMEOUT 0 0
FAB LINK PERSIS 0 0

Error Name # of Reported Err Info Time Elapsed (FPOE 4 fab_id:1)
LC SYNC 0 0
LC CRC 0 0
LC HBEAT 0 0
LC DDRSYNC 0 0
LC LINK PERSIST 0 0
FAB SYNC 0 0
FAB BUFFER 0 0
FAB TMEOUT 0 0
FAB LINK PERSIS 0 0

--------- show buffers --------

Buffer elements:
1110 in free list (500 max allowed)
249464 hits, 0 misses, 618 created

Public buffer pools:
Small buffers, 104 bytes (total 1024, permanent 1024):
1023 in free list (128 min, 2048 max allowed)
128658 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Medium buffers, 256 bytes (total 3000, permanent 3000):
2998 in free list (64 min, 3000 max allowed)
3562 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Middle buffers, 600 bytes (total 512, permanent 512):
511 in free list (64 min, 1024 max allowed)
12755 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Big buffers, 1536 bytes (total 1000, permanent 1000):
998 in free list (64 min, 1000 max allowed)
119206 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
VeryBig buffers, 4520 bytes (total 10, permanent 10):
9 in free list (0 min, 300 max allowed)
459 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Large buffers, 9240 bytes (total 8, permanent 8):
8 in free list (0 min, 10 max allowed)
9 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Huge buffers, 18024 bytes (total 2, permanent 2):
2 in free list (0 min, 4 max allowed)
12 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)

Interface buffer pools:
CF Small buffers, 104 bytes (total 100, permanent 100):
100 in free list (100 min, 200 max allowed)
0 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
SCP Private Buffer buffers, 212 bytes (total 2, permanent 2):
2 in free list (2 min, 4 max allowed)
0 hits, 0 fallbacks, 0 trims, 0 created
0 failures (0 no memory)
CF Middle buffers, 600 bytes (total 100, permanent 100):
100 in free list (100 min, 200 max allowed)
0 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
DMA-2 buffers, 1536 bytes (total 1024, permanent 1024):
512 in free list (0 min, 1024 max allowed)
512 hits, 0 misses
512 max cache size, 512 in cache
0 hits in cache, 0 misses in cache
CF Big buffers, 1536 bytes (total 25, permanent 25):
25 in free list (25 min, 50 max allowed)
0 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
EOBC0/0 buffers, 1686 bytes (total 6144, permanent 6144):
3072 in free list (0 min, 6144 max allowed)
3072 hits, 0 fallbacks
3072 max cache size, 2560 in cache
29271 hits in cache, 0 misses in cache
EOBC0/2 buffers, 1686 bytes (total 6144, permanent 6144):
3072 in free list (0 min, 6144 max allowed)
3072 hits, 0 fallbacks
3072 max cache size, 2816 in cache
83025 hits in cache, 0 misses in cache
IPC buffers, 4096 bytes (total 1008, permanent 1008):
937 in free list (224 min, 2240 max allowed)
69210 hits, 0 fallbacks, 0 trims, 0 created
0 failures (0 no memory)
CF VeryBig buffers, 4520 bytes (total 2, permanent 2):
2 in free list (2 min, 4 max allowed)
0 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
CF Large buffers, 5024 bytes (total 1, permanent 1):
1 in free list (1 min, 2 max allowed)
0 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Private Huge IPC buffers, 18024 bytes (total 2, permanent 2):
2 in free list (1 min, 4 max allowed)
0 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Private Huge buffers, 65280 bytes (total 2, permanent 2):
2 in free list (1 min, 4 max allowed)
1 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)

Header pools:

Particle Clones:
1024 clones, 0 hits, 0 misses

Public particle pools:
Normal buffers, 1532 bytes (total 1024, permanent 1024):
1024 in free list (512 min, 2048 max allowed)
0 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
F/S buffers, 1590 bytes (total 512, permanent 512):
0 in free list (0 min, 512 max allowed)
512 hits, 0 misses
512 max cache size, 512 in cache
0 hits in cache, 0 misses in cache

Private particle pools:
IBC0/0 buffers, 648 bytes (total 1024, permanent 1024):
0 in free list (0 min, 1024 max allowed)
1024 hits, 0 fallbacks
1024 max cache size, 512 in cache
4060 hits in cache, 0 misses in cache

Processor Memory: free 404827224
IO Memory: free 29581216

--------- show proc --------

CPU utilization for five seconds: 6%/0%; one minute: 6%; five minutes: 6%
PID QTy PC Runtime (ms) Invoked uSecs Stacks TTY Process
1 Cwe 82D3304 0 4 0 5216/6000 0 Chunk Manager
2 Csp 83592E8 4 508 7 2276/3000 0 Load Meter
3 Mwe 960F3A0 0 237 011748/12000 0 CFC stats broker
4 Mwe 8633C70 4 223 17 5760/6000 0 Retransmission o
5 Mwe 8631D18 0 3 0 4752/6000 0 IPC ISSU Dispatc
6 Mwe 8CB3B14 0 1 0 5744/6000 0 PF Redun ICC Req
7 Lst 8330104 7436 1562 4760 5220/6000 0 Check heaps
8 Cwe 8322B50 0 1 0 5732/6000 0 Pool Manager
9 Mwe 8322A38 0 1 0 5768/6000 0 DiscardQ Backgro
10 Mst 80AFC20 0 2 0 5396/6000 0 Timers
11 Mwe 80F7858 0 1 0 2772/3000 0 WATCH_AFS
12 Mwe 800D07C 0 2 0 5404/6000 0 Serial Backgroun
13 Lwe 8095EEC 0 1 0 5768/6000 0 AAA_SERVER_DEADT
14 Mwe 819CAA0 0 1 011736/12000 0 Policy Manager
15 Mwe 81E3588 0 1 023660/24000 0 Crash writer
16 Mwe 81E3EB0 0 1 0 5536/6000 0 Exception contro
17 Mwe 82808E8 0 1 0 5760/6000 0 RO Notify Timers
18 Mwe 8293CB8 0 1 0 2756/3000 0 RMI RM Notify Wa
19 Mwe 85EAEA8 0 516 0 5404/6000 0 IPC Event Notifi
20 Mwe 8613EA0 4 2499 1 4736/6000 0 IPC Mcast Pendin
21 Mwe 85F8E38 0 44 0 5776/6000 0 IPC Dynamic Cach
22 Mwe 86092A4 0 514 0 5760/6000 0 IPC Service NonC
23 Mwe 85F9488 0 1 0 5776/6000 0 IPC Zone Manager
24 Mwe 85F8E90 0 2499 0 5000/6000 0 IPC Periodic Tim
25 Mwe 85F8CA8 0 2499 0 5756/6000 0 IPC Deferred Por
26 Mwe 85F8F60 0 1 0 5780/6000 0 IPC Process leve
27 Mwe 85F915C 32 636 50 3648/6000 0 IPC Seat Manager
28 Mwe 85E1A0C 0 47 0 5740/6000 0 IPC Check Queue
29 Mwe 85F931C 4 121 33 4576/6000 0 IPC Seat RX Cont
30 Mwe 85F93E0 0 2 0 5596/6000 0 IPC Seat TX Cont
31 Mwe 85F96DC 4 265 15 4928/6000 0 IPC Keep Alive M
32 Hsi 85FDF50 4 514 7 5772/6000 0 IPC Loadometer
33 Mst 8638F78 852 110 7745 4112/6000 0 PrstVbl
34 Mwe 8CAFBA8 1416 2582 548 3500/6000 0 PF_Init Process
35 Mwe 91C58D0 0 74 0 5344/6000 0 ARP Input
36 Mwe 91C08A8 0 2672 0 5620/6000 0 ARP Background
37 Mwe 94F3C6C 0 2 0 5396/6000 0 DDR Timers
38 Mwe 8017FDC 0 1 0 5764/6000 0 SERIAL A'detect
39 Msp 812E5C0 0 2525 0 5444/6000 0 GraphIt
40 Mwe 8B3BAFC 8 68 117 4368/6000 0 fabm_ch_sync:2
41 Mwe 8CF9864 0 1 0 5788/6000 0 Reset Process
42 Hwe 869D15C 0 3 0 5548/6000 0 RPC pf_media_lay
43 Mwe 8CE1144 8 13 615 4896/6000 0 client_entity_se
44 Mwe 8CA110C 60 441 136 4300/6000 0 RF Slave Main Th
45 Mwe 951199C 0 2 011416/12000 0 Dialer event
46 Cwe 833FBAC 0 1 0 5776/6000 0 Critical Bkgnd
47 Mwe 820CD24 0 585 011044/12000 0 Net Background
48 Mwe 820CF24 0 3 011288/12000 0 IDB Work
49 Lwe 817D790 0 30 010700/12000 0 Logger
50 Mwe 80B92BC 4 2514 1 5432/6000 0 TTY Background
51 Hwe 8648D9C 4 115 34 5116/6000 0 SCP Multicast
52 Mwe 961D568 4 2802 1 4816/6000 0 CEF LC Stats
53 H* 0 280 22238 12 4412/6000 0 SCP Download Lis
54 Mwe 865C91C 0 1 0 5764/6000 0 LCP Bootupg proc
55 Hwe 869EA48 0 119 0 4548/6000 0 RPC Sync
56 Hwe 869D15C 0 1 0 5720/6000 0 RPC rpc-master
57 Mwe 87D5CE0 0 2 0 5592/6000 0 DFC Command Task
58 Hwe 8219170 0 1 0 5764/6000 0 Net Input
59 Mwe 8B13A88 0 7 0 5000/6000 0 Module oper stat

[Leo Laohoo]

---

@rajafurqan2011 wrote:

EARL_L2_ASIC-SP-4-DBUS_HDR_ERR: EARL L2 ASIC

---

I don't like the look of this error message.  Please post the complete output to the command "sh module".

[rajafurqan2011]

QML#show module
Mod Ports Card Type Model Serial No.
--- ----- -------------------------------------- ------------------ -----------
3 0 4-subslot SPA Interface Processor-400 7600-SIP-400 JAE1121J4HK
6 2 Route Switch Processor 720 (Active) RSP720-3CXL-GE SAL16031CDN

Mod MAC addresses Hw Fw Sw Status
--- ---------------------------------- ----- ------------- ------------ -------
3 001b.2a3d.8d50 to 001b.2a3d.8dcf 2.4 15.2(2)S2 15.2(2)S2 Ok
6 2894.0fd1.0270 to 2894.0fd1.0273 5.13 12.2(33r)SR 15.2(2)S Ok

Mod Sub-Module Model Serial Hw Status
---- --------------------------- ------------------ ----------- ------- -------
3/0 5xGE SPA SPA-5X1GE-V2 JAE1221IQYC 1.2 Ok
3/1 5xGE SPA SPA-5X1GE-V2 JAE1222JQ1D 1.2 Ok
6 Policy Feature Card 3 7600-PFC3CXL SAL16052KVU 1.1 Ok
6 C7600 MSFC4 Daughterboard 7600-MSFC4 SAL16031ALR 5.0 Ok

Mod Online Diag Status
---- -------------------
3 Pass
3/0 Pass
3/1 Pass
6 Pass

[rajafurqan2011]

Followings files attached. 

1. logs on Console

2. Show Module

3. Show version 

[Leo Laohoo]

Raise a TAC Case. A software bug is crashing the RP.

[rajafurqan2011]

A software bug is crashing the RP.

can you more elaborate it?