6500 error Log

nibinrodrigues · ‎10-30-2014

Hi,

Am getting continuous logs from my 6500 mentioning line card failed/disabled etc. Strange thing here is that the slots mentioned in the logs are empty. Kindly advice !!!!

#############################################################################################################

No Inactive Message Discriminator.

Console logging: level debugging, 386671 messages logged, xml disabled,
filtering disabled
Monitor logging: level debugging, 147 messages logged, xml disabled,
filtering disabled
Buffer logging: level debugging, 386702 messages logged, xml disabled,
filtering disabled
Exception Logging: size (4096 bytes)
Count and timestamp logging messages: disabled
Persistent logging: disabled

No active filter modules.

ESM: 0 messages dropped

Trap logging: level informational, 386624 message lines logged

Log Buffer (8192 bytes):
failed, card will be disabled.
Oct 29 05:27:13.345: %OIR-SP-3-UNKNOWN: Unknown card in slot 7, card is being disabled
Oct 29 05:27:13.842: %OIR-SP-6-REMCARD: Card removed from slot 3, interfaces disabled
Oct 29 05:27:13.850: %OIR-SP-6-REMCARD: Card removed from slot 4, interfaces disabled
Oct 29 05:27:13.858: %OIR-SP-6-REMCARD: Card removed from slot 7, interfaces disabled
Oct 29 05:27:13.862: %OIR-SP-6-REMCARD: Card removed from slot 8, interfaces disabled
Oct 29 05:27:13.866: %OIR-SP-6-REMCARD: Card removed from slot 9, interfaces disabled
Oct 29 05:27:25.389: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 3 failed,card will be disabled.
Oct 29 05:27:25.389: %OIR-SP-3-UNKNOWN: Unknown card in slot 3, card is being disabled
Oct 29 05:27:25.897: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 4 failed,card will be disabled.
Oct 29 05:27:25.897: %OIR-SP-3-UNKNOWN: Unknown card in slot 4, card is being disabled
Oct 29 05:27:26.397: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 7 failed, card will be disabled.
Oct 29 05:27:26.397: %OIR-SP-3-UNKNOWN: Unknown card in slot 7, card is being disabled
Oct 29 05:27:26.901: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 8 failed, card will be disabled.
Oct 29 05:27:26.901: %OIR-SP-3-UNKNOWN: Unknown card in slot 8, card is being disabled
Oct 29 05:27:27.401: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 9 failed, card will be disabled.
Oct 29 05:27:27.401: %OIR-SP-3-UNKNOWN: Unknown card in slot 9, card is being disabled
Oct 29 05:27:33.413: %OIR-SP-6-REMCARD: Card removed from slot 3, interfaces disabled
Oct 29 05:27:33.417: %OIR-SP-6-REMCARD: Card removed from slot 4, interfaces disabled
Oct 29 05:27:33.421: %OIR-SP-6-REMCARD: Card removed from slot 7, interfaces disabled
Oct 29 05:27:33.429: %OIR-SP-6-REMCARD: Card removed from slot 8, interfaces disabled
Oct 29 05:27:33.433: %OIR-SP-6-REMCARD: Card removed from slot 9, interfaces disabled
Oct 29 05:27:44.965: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 3 failed, card will be disabled.
Oct 29 05:27:44.965: %OIR-SP-3-UNKNOWN: Unknown card in slot 3, card is being disabled
Oct 29 05:27:45.469: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 4 failed, card will be disabled.
Oct 29 05:27:45.469: %OIR-SP-3-UNKNOWN: Unknown card in slot 4, card is being disabled
Oct 29 05:27:45.973: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 7 failed, card will be disabled.
Oct 29 05:27:45.973: %OIR-SP-3-UNKNOWN: Unknown card in slot 7, card is being disabled
Oct 29 05:27:46.477: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 8 failed, card will be disabled.
Oct 29 05:27:46.477: %OIR-SP-3-UNKNOWN: Unknown card in slot 8, card is being disabled
Oct 29 05:27:46.977: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 9 failed, card will be disabled.
Oct 29 05:27:46.977: %OIR-SP-3-UNKNOWN: Unknown card in slot 9, card is being disabled
Oct 29 05:27:48.814: %OIR-SP-6-REMCARD: Card removed from slot 3, interfaces disabled
Oct 29 05:27:48.822: %OIR-SP-6-REMCARD: Card removed from slot 4, interfaces disabled
Oct 29 05:27:48.826: %OIR-SP-6-REMCARD: Card removed from slot 7, interfaces disabled
Oct 29 05:27:48.834: %OIR-SP-6-REMCARD: Card removed from slot 8, interfaces disabled
Oct 29 05:27:48.838: %OIR-SP-6-REMCARD: Card removed from slot 9, interfaces disabled
Oct 29 05:32:17.373: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 3 failed, card will be disabled.
Oct 29 05:32:17.373: %OIR-SP-3-UNKNOWN: Unknown card in slot 3, card is being disabled
Oct 29 05:32:17.373: %PMB-SP-4-NOTSUPPORTED: Unsupported/old card at slot 4
Oct 29 05:32:17.373: %OIR-SP-3-UNKNOWN: Unknown card in slot 4, card is being disabled
Oct 29 05:32:17.373: %PMB-SP-4-NOTSUPPORTED: Unsupported/old card at slot 7
Oct 29 05:32:17.377: %OIR-SP-3-UNKNOWN: Unknown card in slot 7, card is being disabled
Oct 29 05:32:17.377: %PMB-SP-4-NOTSUPPORTED: Unsupported/old card at slot 8
Oct 29 05:32:17.377: %OIR-SP-3-UNKNOWN: Unknown card in slot 8, card is being disabled
Oct 29 05:32:17.377: %PMB-SP-4-NOTSUPPORTED: Unsupported/old card at slot 9
Oct 29 05:32:17.389: %OIR-SP-3-UNKNOWN: Unknown card in slot 9, card is being disabled
Oct 29 05:32:17.865: %OIR-SP-6-REMCARD: Card removed from slot 3, interfaces disabled
Oct 29 05:32:17.873: %OIR-SP-6-REMCARD: Card removed from slot 4, interfaces disabled
Oct 29 05:32:17.877: %OIR-SP-6-REMCARD: Card removed from slot 7, interfaces disabled
Oct 29 05:32:17.881: %OIR-SP-6-REMCARD: Card removed from slot 8, interfaces disabled
Oct 29 05:32:17.889: %OIR-SP-6-REMCARD: Card removed from slot 9, interfaces disabled
Oct 29 05:32:29.413: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 3 failed, card will be disabled.
Oct 29 05:32:29.417: %OIR-SP-3-UNKNOWN: Unknown card in slot 3, card is being disabled
Oct 29 05:32:29.917: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 4 failed, card will be disabled.
Oct 29 05:32:29.941: %OIR-SP-3-UNKNOWN: Unknown card in slot 4, card is being disabled
Oct 29 05:32:30.441: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 7 failed, card will be disabled.
Oct 29 05:32:30.441: %OIR-SP-3-UNKNOWN: Unknown card in slot 7, card is being disabled
Oct 29 05:32:30.948: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 8 failed, card will be disabled.
Oct 29 05:32:30.948: %OIR-SP-3-UNKNOWN: Unknown card in slot 8, card is being disabled
Oct 29 05:32:31.448: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 9 failed, card will be disabled.
Oct 29 05:32:31.448: %OIR-SP-3-UNKNOWN: Unknown card in slot 9, card is being disabled
Oct 29 05:39:36.596: %OIR-SP-6-REMCARD: Card removed from slot 7, interfaces disabled
Oct 29 05:39:38.604: %OIR-SP-6-REMCARD: Card removed from slot 4, interfaces disabled
Oct 29 05:39:38.608: %OIR-SP-6-REMCARD: Card removed from slot 8, interfaces disabled
Oct 29 05:39:38.612: %OIR-SP-6-REMCARD: Card removed from slot 9, interfaces disabled
Oct 29 05:39:42.621: %OIR-SP-6-REMCARD: Card removed from slot 3, interfaces disabled
Oct 29 06:13:41.982: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 3 failed, card will be disabled.
Oct 29 06:13:41.982: %OIR-SP-3-UNKNOWN: Unknown card in slot 3, card is being disabled
Oct 29 06:13:42.482: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 7 failed, card will be disabled.
Oct 29 06:13:42.482: %OIR-SP-3-UNKNOWN: Unknown card in slot 7, card is being disabled
Oct 29 06:13:42.986: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 9 failed, card will be disabled.
Oct 29 06:13:42.986: %OIR-SP-3-UNKNOWN: Unknown card in slot 9, card is being disabled
Oct 29 06:13:44.489: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 4 failed, card will be disabled.
Oct 29 06:13:44.489: %OIR-SP-3-UNKNOWN: Unknown card in slot 4, card is being disabled
Oct 29 06:13:44.989: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 8 failed, card will be disabled.
Oct 29 06:13:44.993: %OIR-SP-3-UNKNOWN: Unknown card in slot 8, card is being disabled
Oct 29 06:13:47.001: %OIR-SP-6-REMCARD: Card removed from slot 7, interfaces disabled
Oct 29 06:13:47.005: %OIR-SP-6-REMCARD: Card removed from slot 8, interfaces disabled
Oct 29 06:13:47.009: %OIR-SP-6-REMCARD: Card removed from slot 9, interfaces disabled
Oct 29 06:13:58.533: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 7 failed, card will be disabled.
Oct 29 06:13:58.533: %OIR-SP-3-UNKNOWN: Unknown card in slot 7, card is being disabled
Oct 29 06:13:59.033: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 8 failed, card will be disabled.
Oct 29 06:13:59.033: %OIR-SP-3-UNKNOWN: Unknown card in slot 8, card is being disabled
Oct 29 06:13:59.534: %PMB-SP-4-CANTPWROFF: Attempts to power down slot 9 failed, card will be disabled.
Oct 29 06:13:59.534: %OIR-SP-3-UNKNOWN: Unknown card in slot 9, card is being disabled
Oct 29 06:15:15.782: %OIR-SP-6-REMCARD: Card removed from slot 3, interfaces disabled
Oct 29 06:15:15.786: %OIR-SP-6-REMCARD: Card removed from slot 4, interfaces disabled
Oct 29 06:15:15.790: %OIR-SP-6-REMCARD: Card removed from slot 7, interfaces disabled
Oct 29 06:15:15.794: %OIR-SP-6-REMCARD: Card removed from slot 8, interfaces disabled
Oct 29 06:15:15.802: %OIR-SP-6-REMCARD: Card removed from slot 9, interfaces disabled
ATCBmer-COREcs01#
ATCBmer-COREcs01#
ATCBmer-COREcs01#sh mod
ATCBmer-COREcs01#sh mod
Mod Ports Card Type Model Serial No.
--- ----- -------------------------------------- ------------------ -----------
1 8 CEF720 8 port 10GE with DFC WS-X6708-10GE SAL1339ZFS3
2 8 CEF720 8 port 10GE with DFC WS-X6708-10GE SAL1339ZCF3
5 5 Supervisor Engine 720 10GE (Active) VS-S720-10G SAL1545UTYY
6 5 Supervisor Engine 720 10GE (Hot) VS-S720-10G SAL1338Z46U

Mod MAC addresses Hw Fw Sw Status
--- ---------------------------------- ------ ------------ ------------ -------
1 0025.84f0.f570 to 0025.84f0.f577 2.1 12.2(18r)S1 12.2(33)SXI3 Ok
2 001f.6ca4.0e40 to 001f.6ca4.0e47 2.1 12.2(18r)S1 12.2(33)SXI3 Ok
5 e05f.b947.55fd to e05f.b947.5604 4.1 8.5(4) 12.2(33)SXI3 Ok
6 0023.33ab.a7c0 to 0023.33ab.a7c7 3.1 8.5(3) 12.2(33)SXI3 Ok

Mod Sub-Module Model Serial Hw Status
---- --------------------------- ------------------ ----------- ------- -------
1 Distributed Forwarding Card WS-F6700-DFC3C SAL1340ZNXW 1.4 Ok
2 Distributed Forwarding Card WS-F6700-DFC3C SAL1340ZPT5 1.4 Ok
5 Policy Feature Card 3 VS-F6K-PFC3C SAL1546V07U 1.1 Ok
5 MSFC3 Daughterboard VS-F6K-MSFC3 SAL1626F8E8 5.1 Ok
6 Policy Feature Card 3 VS-F6K-PFC3C SAL1338Z5L3 1.1 Ok
6 MSFC3 Daughterboard VS-F6K-MSFC3 SAL1339ZHKC 2.0 Ok

Mod Online Diag Status
---- -------------------
1 Pass
2 Pass
5 Pass

Thanks and Regards

Nibin Rodrigues

InayathUlla Sharieff · ‎10-30-2014

Try doing the sup failover and if that doesnt help try rebooting it once and let me know.

Regards

Inayath