%ONLINE-SP-6-REGN_TIMER: multiple line cards down after config change

Sahmad1983 · ‎05-29-2013

Hi,

I was wondering if someone could help.

I recently changed the entire configuration on two 6509 switches, and the fibre modules on slot 7 and 8 have gone down on BOTH switches. It would be easy to say that its a hardware fault but I cannot understand how module 7 and 8 can go down on both switches. Initially I thought it maybe due to IOS bug 22-33.SXI2a.bin" so I upgraded it to s72033-advipservicesk9_wan-mz.122-18.SXF12a.bin but it still down.

These are the actions I have carrried out to try resolve the issue:

- Reverted back to orginal configs but no luck

- Tried to use "power enable" command but the status light on both modules seems to turn red and then go off again.

- I have also tried to put one of the fibre modules into a different slot but no joy

- Upgraded the IOS from 22-33.SXI2a. to 122-18.SXF12a

I am running out of ideas.. can anyone help ? I am sure that hardware fault cannot occur on BOTH modules 7 & 8 on BOTH switches after config change.

The error in the log shows that modules "failed to bring online because of registration timer event". I understand this means that it is unable to download the image within the allocated time.. maybe I could manually copy the image onto the line card if that is possible ?? any ideas ?

Daughterboard WS-SUP720 SAL135063V2 4.0 Ok

7 Centralized Forwarding Card WS-F6700-CFC SAL1419HKJK 4.1 PwrDown

8 Centralized Forwarding Card WS-F6700-CFC SAL1419HAG0 4.1 PwrDown

Mod Hw Fw Sw Status

--- ---------------------------------- ------ ------------ ------------ -------

7 5475.d016.075c to 5475.d016.075f 3.1 Unknown Unknown PwrDown

8 5475.d0bb.7404 to 5475.d0bb.7407 3.1 Unknown Unknown PwrDown

System image file is "disk0:s72033-advipservicesk9_wan-mz.122-18.SXF12a.bin"

BOOT variable = disk0:s72033-advipservicesk9_wan-mz.122-18.SXF12a.bin,12;sup-bootdisk,1;
CONFIG_FILE variable =
BOOTLDR variable =
Configuration register is 0x2102

Standby is not present.

May 28 17:25:10.352: %C6KPWR-SP-4-DISABLED: power to module in slot 7 set off (Module Failed SCP dnld)

May 28 17:25:10.452: %ONLINE-SP-6-REGN_TIMER: Module 8, Proc. 0. Failed to bring online because of registration timer event

sm(cygnus_oir_bay slot8), running yes, state wait_til_online

Last transition recorded: (disabled)-> disabled (disabled)-> disabled2 (restart)-> wait_til_disabled (timer)-> may_be_occupied (timer)-> occupied (known)-> can_power_on (yes_power)-> powered_on (real_power_on)-> check_power_on (timer)-> check_power_on (power_on_ok)-> wait_til_online

May 28 17:25:10.452: %C6KPWR-SP-4-DISABLED: power to module in slot 8 set off (Module Failed SCP dnld)

May 29 11:41:16.601: %SYS-5-CONFIG_I: Configured from console by console

Reza Sharifi · ‎05-29-2013

Hi,

Going from SXI to SXF is downgrade and not upgrade. If you want to upgrade then the options would be to go to higher releases of SXI ie SXI3, 4, 5 or possibly SXJ.,

Can you post the model number for your Sup and also the fiber line cards?

HTH

Sahmad1983 · ‎05-29-2013

Hi Reza,

Thanks for quick response.

This is a development test lab we are working on and we changed the IOS to match the production. We only downgraded the IOS on one of the switches and have left the other switch as it is.

the supervisor model

WS-SUP720-3B

fibre model

WS-X6704-10GE

Reza Sharifi · ‎05-29-2013

Hi,

Your Sup supports X6704. SXI2 is very buggy. So, can you upgrade to SXI4 or a later version and try again?

HTH

Sahmad1983 · ‎05-29-2013

We are using SXF12a on our production network and its working fine. We are trying to mirror it to production which is why I changed the config to match one of our sites but it seems to have all gone wrong. All the modules were up and running untill I changed the config.

Gabriel Hill · ‎05-29-2013

Hello Sufraz,

How is your VSL link setup? Is it purely on your SUP module, or does it connect to some of your 10G modules on other line cards as well?

Have you tried resetting the module on the 6500? hq-module # reset ?

Thanks,
Gabriel

Sahmad1983 · ‎05-29-2013

Hi Gabriel

We have 2 port-channels set up between 6509s.. one port channel on gig interfaces and another port channel on 10G interfaces. We also have 10G port-channel to UCM.

I have tried hw-module reset after enabling power but goes off again

Regards,

Surfraz

Sahmad1983 · ‎05-31-2013

Would anyone know if the line cards are maybe failing due to Embedded Event Manager configuration ? The person who configured this lab has left the company so I am not sure what the EEM configurations actually mean.

The reason i am suspecting the EEM configuration is because the error in the log seems to suggest the module failed due to registration timer event.

May 30 14:47:00.366: %ONLINE-SP-6-REGN_TIMER: Module 8, Proc. 0. Failed to bring online because of registration timer event

Just trying to check all the possibilities as I have tried everything else,

event manager directory user policy "disk0:/fcp_eem"

event manager directory user repository tftp://10.11.x.x/xxx/

event manager scheduler script thread class default number 10

no event manager policy Mandatory.go_switchbus.tcl type system

event manager policy rp_test.tcl type user

!

event manager detector rpc ssh acl 60 event manager directory user policy "disk0:/fcp_eem"
event manager directory user repository tftp://10.11.216.145/FCPD/
event manager scheduler script thread class default number 10
no event manager policy Mandatory.go_switchbus.tcl type system
event manager policy rp_test.tcl type user
!
event manager detector rpc ssh acl 60

mts.by · ‎02-10-2015

Hi

I had similar problem.
But there was no software upgrade or config changes. According to logs my VLS link went down and one of switch has rebooted. After this line card WS-X6704-10GE didn't come up and I saw message

%ONLINE-SW2_SPSTBY-6-REGN_TIMER: Module 7, Proc. 0. Failed to bring online because of registration timer event

I have reinsert card, reset by command hw-module. But nothing hepls with this issue.

Then I have found cisco doc about similar problem. As a workaround cisco recommended to change DFC or/and memory module on this line card. I had no additional DFC, but memory card was available. After changing memory card on line card the problem was solved!

Here is cisco doc about this issue:
http://www.cisco.com/c/en/us/support/docs/field-notices/637/fn63743.html