Hi,
My GSR12410 Active PRP-2 crashed causing all services to resart. The Standby PRP-2 module took the control. Please suggest the root cause.
IOS: 12.0(33)S7
Loggs: soon after PRP module crashed:
++++++++++++++++++++++++++++++++++++++++++
000057: Mar 31 17:47:25.995 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback0, changed state to up
000058: Mar 31 17:47:26.179 PST: %SYS-5-CONFIG_I: Configured from memory by console
000059: Mar 31 17:47:26.211 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback1005, changed state to up
000060: Mar 31 17:47:26.211 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback1605, changed state to up
000061: Mar 31 17:47:26.211 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback6005, changed state to up
000062: Mar 31 17:47:26.211 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback7001, changed state to up
000063: Mar 31 17:47:26.211 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback7002, changed state to up
000064: Mar 31 17:47:26.211 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback8001, changed state to up
000065: Mar 31 17:47:26.211 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback18005, changed state to up
000066: Mar 31 17:47:26.315 PST: %FPD_MGMT-3-INCOMP_IMG_VER: Incompatible CHOPPER PROM FP (FPD ID=1) image version detected for PRP-2 card in slot 8. Detected version = 2.149, minimum required version = 2.151. Current HW version = 10242.6.
000067: Mar 31 17:47:26.315 PST: %FPD_MGMT-4-UPGRADE_REQUIRED: FPD Image Upgrade is required for PRP-2 card in slot 8
000068: Mar 31 17:47:26.351 PST: %SYS-5-RESTART: System restarted --
Cisco Internetwork Operating System Software
IOS (tm) GS Software (C12KPRP-K4P-M), Version 12.0(33)S7, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2010 by cisco Systems, Inc.
Compiled Mon 28-Jun-10 07:37 by debajpal
000069: Mar 31 17:47:26.351 PST: %SNMP-5-COLDSTART: SNMP agent on host LLR314-PE01 is undergoing a cold start
000070: Mar 31 17:47:26.423 PST: %SYS-6-BOOTTIME: Time taken to reboot after reload = 23385017 seconds
000071: Mar 31 17:47:26.643 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface POS1/0/3, changed state to down
000072: Mar 31 17:47:27.619 PST: %LINK-5-CHANGED: Interface POS1/0/1, changed state to administratively down
000073: Mar 31 17:47:27.683 PST: %LINK-5-CHANGED: Interface POS1/0/2, changed state to administratively down
000074: Mar 31 17:47:27.683 PST: %LINK-5-CHANGED: Interface POS1/0/4, changed state to administratively down
000075: Mar 31 17:47:27.683 PST: %LINK-5-CHANGED: Interface POS1/0/5, changed state to administratively down
000076: Mar 31 17:47:27.795 PST: %LINK-5-CHANGED: Interface Ethernet0, changed state to administratively down
000077: Mar 31 17:47:27.815 PST: %LINK-5-CHANGED: Interface Ethernet1, changed state to administratively down
000078: Mar 31 17:47:27.919 PST: %SSH-5-ENABLED: SSH 1.99 has been enabled
000079: Mar 31 17:47:28.618 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface POS1/0/1, changed state to down
000080: Mar 31 17:47:28.678 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface POS1/0/2, changed state to down
000081: Mar 31 17:47:28.682 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface POS1/0/4, changed state to down
000082: Mar 31 17:47:28.682 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface POS1/0/5, changed state to down
000083: Mar 31 17:47:28.794 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Ethernet0, changed state to down
000084: Mar 31 17:47:28.814 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Ethernet1, changed state to down
000085: Mar 31 17:47:29.266 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Ethernet2, changed state to down
000086: Mar 31 17:47:33.356 PST: %MBUS-6-FABCONFIG: Switch Cards 0x7F (bitmask)
Primary Clock is CSC_1
Fabric Clock is Redundant
Bandwidth Mode : 10Gbps Bandwidth
SLOT 1:00:01:42: Setting LEDs
SLOT 1:00:01:42: %SYS-5-RESTART: System restarted --
Cisco Internetwork Operating System Software
IOS (tm) GS Software (GLC1-LC-M), Version 12.0(33)S7, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2010 by cisco Systems, Inc.
Compiled Mon 28-Jun-10 07:21 by debajpal
SLOT 1:00:01:42: %LC-3-LAST_CRASH: LAST Shutdown Info: Reload from RP - interrupt:
SLOT 0:00:01:45: %SYS-5-RESTART: System restarted --
Cisco Internetwork Operating System Software
IOS (tm) GS Software (GLC1-LC-M), Version 12.0(33)S7, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2010 by cisco Systems, Inc.
Compiled Mon 28-Jun-10 07:21 by debajpal
SLOT 0:00:01:45: Setting LEDs
SLOT 0:Mar 31 17:49:16.255 PST: %LC-3-LAST_CRASH: LAST Shutdown Info: Reload from RP - interrupt:
00:00:03: %MBUS-6-FIA_CONFIG: Switch Cards 0x1F (bit mask); Primary Clock CSC_1
00:00:03: %MBUS-6-RP_STATUS: RP in Slot 9 Mode = MBUS Standby
000087: Mar 31 17:49:28.089 PST: %RP-6-REDMODE: Operating redundancy mode is RPR
000088: Mar 31 17:49:32.439 PST: %SONET-4-ALARM: POS1/0/0: SLOS
000089: Mar 31 17:49:33.330 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface POS1/0/0, changed state to down
000090: Mar 31 17:49:35.317 PST: %RPGE-6-LINK_STATUS_DOWN: Interface GigabitEthernet0/0/1: Detected RX Link Status Down
000091: Mar 31 17:49:35.321 PST: %RPGE-6-AUTONEG_STATE: Interface GigabitEthernet0/0/1: Link OK - autonegotiation complete
000092: Mar 31 17:49:35.329 PST: %SPA_OIR-6-ONLINECARD: SPA (SPA-2X1GE) online in subslot 0/0
000093: Mar 31 17:49:35.329 PST: %RPGE-6-AUTONEG_STATE: Interface GigabitEthernet0/0/1: Link OK - autonegotiation complete
000094: Mar 31 17:49:35.789 PST: %RPGE-6-LINK_STATUS_DOWN: Interface GigabitEthernet0/1/0: Detected RX Link Status Down
000095: Mar 31 17:49:35.789 PST: %RPGE-6-AUTONEG_STATE: Interface GigabitEthernet0/1/0: Link OK - autonegotiation complete
000096: Mar 31 17:49:35.885 PST: %RPGE-6-AUTONEG_STATE: Interface GigabitEthernet0/1/0: Link OK - autonegotiation complete
000097: Mar 31 17:49:36.157 PST: %RP-3-RESTART: Linecard packet buffers recarved
000098: Mar 31 17:49:36.189 PST: %SPA_OIR-6-ONLINECARD: SPA (SPA-8XOC3-POS) online in subslot 1/0
000099: Mar 31 17:49:36.273 PST: %RPGE-6-LINK_STATUS_DOWN: Interface GigabitEthernet0/1/1: Detected RX Link Status Down
000100: Mar 31 17:49:36.281 PST: %SPA_OIR-6-ONLINECARD: SPA (SPA-2X1GE) online in subslot 0/1
000101: Mar 31 17:49:36.281 PST: %RPGE-6-AUTONEG_STATE: Interface GigabitEthernet0/1/1: Link OK - autonegotiation complete
000102: Mar 31 17:49:36.281 PST: %RPGE-6-AUTONEG_STATE: Interface GigabitEthernet0/1/1: Link OK - autonegotiation complete
000103: Mar 31 17:49:36.293 PST: %RPGE-6-LINK_STATUS_DOWN: Interface GigabitEthernet0/2/0: Detected RX Link Status Down
000104: Mar 31 17:49:36.293 PST: %RPGE-6-AUTONEG_STATE: Interface GigabitEthernet0/2/0: Link OK - autonegotiation complete
000105: Mar 31 17:49:36.297 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/0/1, changed state to down
000106: Mar 31 17:49:36.305 PST: %SPA_OIR-6-ONLINECARD: SPA (SPA-2X1GE) online in subslot 0/2
000107: Mar 31 17:49:36.309 PST: %RPGE-6-LINK_STATUS_DOWN: Interface GigabitEthernet0/2/1: Detected RX Link Status Down
000108: Mar 31 17:49:36.309 PST: %RPGE-6-AUTONEG_STATE: Interface GigabitEthernet0/2/1: Link OK - autonegotiation complete
000109: Mar 31 17:49:36.317 PST: %SONET-4-ALARM: POS1/0/3: LRDI
000110: Mar 31 17:49:36.321 PST: %RPGE-6-AUTONEG_STATE: Interface GigabitEthernet0/2/0: Link OK - autonegotiation complete
000111: Mar 31 17:49:36.337 PST: %RPGE-6-AUTONEG_STATE: Interface GigabitEthernet0/2/1: Link OK - autonegotiation complete
000112: Mar 31 17:49:36.373 PST: %RPGE-6-LINE_STATE: Interface GigabitEthernet0/0/0: Line state OK
000113: Mar 31 17:49:36.377 PST: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/0, changed state to up
000114: Mar 31 17:49:36.445 PST: %SONET-4-ALARM: POS1/0/6: LRDI
000115: Mar 31 17:49:36.449 PST: %LINK-3-UPDOWN: Interface POS1/0/6, changed state to up
000116: Mar 31 17:49:36.565 PST: %SONET-4-ALARM: POS1/0/7: LRDI
000117: Mar 31 17:49:36.569 PST: %LINK-3-UPDOWN: Interface POS1/0/7, changed state to up
000118: Mar 31 17:49:36.589 PST: %RPGE-6-AUTONEG_STATE: Interface GigabitEthernet1/1/0: Link OK - autonegotiation complete
000119: Mar 31 17:49:36.593 PST: %RPGE-6-AUTONEG_STATE: Interface GigabitEthernet1/1/0: Link OK - autonegotiation complete
000120: Mar 31 17:49:36.601 PST: %RPGE-6-AUTONEG_STATE: Interface GigabitEthernet1/1/1: Link OK - autonegotiation complete
000121: Mar 31 17:49:36.605 PST: %SPA_OIR-6-ONLINECARD: SPA (SPA-2X1GE-V2) online in subslot 1/1
000122: Mar 31 17:49:36.609 PST: %RPGE-6-AUTONEG_STATE: Interface GigabitEthernet1/1/1: Link OK - autonegotiation complete
000123: Mar 31 17:49:36.625 PST: %RPGE-6-AUTONEG_STATE: Interface GigabitEthernet1/1/0: Link OK - autonegotiation complete
000124: Mar 31 17:49:36.641 PST: %RPGE-6-AUTONEG_STATE: Interface GigabitEthernet1/1/1: Link OK - autonegotiation complete
000125: Mar 31 17:49:36.788 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/1/0, changed state to down
000126: Mar 31 17:49:37.192 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface POS1/0/6, changed state to up
+++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++++++++++++++++++++++++++++++++++
Please suggest the wayout and root cause.
