cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
793
Views
0
Helpful
0
Replies
david.mitchell
Beginner

CSM FT Heartbeat / Keepalives failing

Hi All,

I have two 6509s with a single CSM in each set up for FT.  No changes have been made to the configuration recently but I have noticed the following EMs occuring in the logs.  Both logs are from the same period of time so you can see what is happening on one CSM and compare it to the same time stamp on the other.

This problem is reoccuring every couple of days

As far as I can see everything looks good in terms of hardware and config, any help would be greately appreciated

Please note I only have remote access to the switches and the Software version on both CSM is 4.2(2)

Switch uptime is not greater that 828 days and I can see that of the 58,152 heartbeats the Active Master has sent, only 58,056 have been recieved by the Backup/Secondary

I have looked into this for the last 4 hours and the only assumption I can make is that the L2 Etherchannel beteen the Cores is getting swamped with traffic and causing the keep alives to fail, but I don't know how to prove this.

               

Core 1A

Feb  7 14:49:23.130 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: LRP: no ACK from standby.. standby may be down

Feb  7 14:49:23.730 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: LRP: no ACK from standby.. standby may be down

Feb  7 14:49:26.130 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: LRP: no ACK from standby.. standby may be down

Feb  7 14:49:35.730 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: LRP: no ACK from standby.. standby may be down

Feb  7 14:50:04.691 UTC: %SEC-6-IPACCESSLOGRL: access-list logging rate-limited or missed 210 packets

Feb  7 14:50:14.131 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: LRP: no ACK from standby.. standby may be down

Feb  7 14:51:04.715 UTC: %SEC-6-IPACCESSLOGRL: access-list logging rate-limited or missed 1 packet

Feb  7 14:52:04.739 UTC: %SEC-6-IPACCESSLOGRL: access-list logging rate-limited or missed 3 packets

Feb  7 14:52:08.451 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: LRP: no ACK from standby.. standby may be down

Feb  7 14:52:09.051 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: LRP: no ACK from standby.. standby may be down

Feb  7 14:52:11.447 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: LRP: no ACK from standby.. standby may be down

Feb  7 14:52:21.047 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: LRP: no ACK from standby.. standby may be down

Feb  7 14:52:59.452 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: LRP: no ACK from standby.. standby may be down

Feb  7 14:53:04.764 UTC: %SEC-6-IPACCESSLOGRL: access-list logging rate-limited or missed 224 packets

Feb  7 14:53:37.808 UTC: %C6KPWR-SP-4-DISABLED: power to module in slot 1 set off (Module not responding to Keep Alive polling)

Feb  7 14:55:16.820 UTC: %MLS_RATE-4-DISABLING: The Layer2 Rate Limiters have been disabled.

Feb  7 14:55:15.800 UTC: %DIAG-SP-6-RUN_MINIMUM: Module 1: Running Minimal Diagnostics...

Feb  7 14:55:16.468 UTC: %DIAG-SP-6-DIAG_OK: Module 1: Passed Online Diagnostics

Feb  7 14:55:18.988 UTC: %OIR-SP-6-INSCARD: Card inserted in slot 1, interfaces are now online

Feb  7 14:55:20.356 UTC: %CSM_SLB-6-REDUNDANCY_INFO: Module 1 FT info: State Transition Init -> Standby

Feb  7 14:55:20.700 UTC: %CSM_SLB-6-REDUNDANCY_INFO: Module 1 FT info: State Transition Standby -> Active

Feb  7 14:55:20.864 UTC: %CSM_SLB-6-INFO: Module 1 info: config reload completed successfully.

Feb  7 14:55:23.896 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: LRP: no ACK from standby.. standby may be down

Feb  7 14:55:24.496 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: LRP: no ACK from standby.. standby may be down

Feb  7 14:55:26.896 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: LRP: no ACK from standby.. standby may be down

Feb  7 14:55:36.496 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: LRP: no ACK from standby.. standby may be down

Feb  7 14:56:14.905 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: LRP: no ACK from standby.. standby may be down

COR-1A#show mod contentswitchingModule 1 ft

FT group 1, vlan 4

This box is active

priority 20, heartbeat 1, failover 3, preemption is on

Core 1B

Feb  7 14:49:44.463 UTC: %C6KPWR-SP-4-DISABLED: power to module in slot 1 set off (Module not responding to Keep Alive polling)

Feb  7 14:51:22.467 UTC: %DIAG-SP-6-RUN_MINIMUM: Module 1: Running Minimal Diagnostics...

Feb  7 14:51:24.495 UTC: %MLS_RATE-4-DISABLING: The Layer2 Rate Limiters have been disabled.

Feb  7 14:51:24.156 UTC: %DIAG-SP-6-DIAG_OK: Module 1: Passed Online Diagnostics

Feb  7 14:51:26.657 UTC: %OIR-SP-6-INSCARD: Card inserted in slot 1, interfaces are now online

Feb  7 14:51:28.511 UTC: %CSM_SLB-6-INFO: Module 1 info: config reload completed successfully.

Feb  7 14:51:28.567 UTC: %CSM_SLB-6-REDUNDANCY_INFO: Module 1 FT info: State Transition Init -> Standby

Feb  7 14:52:31.748 UTC: %C6KPWR-SP-4-DISABLED: power to module in slot 1 set off (Module not responding to Keep Alive polling)

Feb  7 14:54:10.782 UTC: %MLS_RATE-4-DISABLING: The Layer2 Rate Limiters have been disabled.

Feb  7 14:54:09.762 UTC: %DIAG-SP-6-RUN_MINIMUM: Module 1: Running Minimal Diagnostics...

Feb  7 14:54:10.554 UTC: %DIAG-SP-6-DIAG_OK: Module 1: Passed Online Diagnostics

Feb  7 14:54:12.870 UTC: %OIR-SP-6-INSCARD: Card inserted in slot 1, interfaces are now online

Feb  7 14:54:14.794 UTC: %CSM_SLB-6-INFO: Module 1 info: config reload completed successfully.

Feb  7 14:55:04.094 UTC: %CSM_SLB-6-REDUNDANCY_INFO: Module 1 FT info: State Transition Elect -> Active

Feb  7 14:55:07.102 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: LRP: no ACK from standby.. standby may be down

Feb  7 14:55:07.698 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: LRP: no ACK from standby.. standby may be down

Feb  7 14:55:10.098 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: LRP: no ACK from standby.. standby may be down

Feb  7 14:55:19.698 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: LRP: no ACK from standby.. standby may be down

Feb  7 14:55:20.502 UTC: %CSM_SLB-4-REDUNDANCY_WARN: Module 1 FT warning: FT configuration might be out of sync.

Feb  7 14:55:20.702 UTC: %CSM_SLB-6-REDUNDANCY_INFO: Module 1 FT info: State Transition Active -> Standby

Feb  7 14:55:45.043 UTC: %C6KPWR-SP-4-DISABLED: power to module in slot 1 set off (Module not responding to Keep Alive polling)

Feb  7 14:57:24.076 UTC: %MLS_RATE-4-DISABLING: The Layer2 Rate Limiters have been disabled.

Feb  7 14:57:23.048 UTC: %DIAG-SP-6-RUN_MINIMUM: Module 1: Running Minimal Diagnostics...

Feb  7 14:57:23.740 UTC: %DIAG-SP-6-DIAG_OK: Module 1: Passed Online Diagnostics

Feb  7 14:57:26.164 UTC: %OIR-SP-6-INSCARD: Card inserted in slot 1, interfaces are now online

Feb  7 14:57:27.736 UTC: %CSM_SLB-6-REDUNDANCY_INFO: Module 1 FT info: State Transition Init -> Standby

Feb  7 14:57:28.104 UTC: %CSM_SLB-6-INFO: Module 1 info: config reload completed successfully.

COR-1B#show module contentSwitchingModule 1 ft

FT group 1, vlan 4

This box is in standby state

priority 10, heartbeat 1, failover 3, preemption is on

Thanks in advance

David

0 REPLIES 0