cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Announcements
1247
Views
0
Helpful
6
Replies
Highlighted
omz Rising star
Rising star

HA standby WLC keeps reloading - 8.5.135.0

Hi, 

The standby WLC in HA keeps reloading. What does IPC timeout mean? Is the standby unable to reach the primary?

(W854_01-Standby) >show logging last-reset

!!!Message and Trap Logs from Previous Reset

--------------------------------------------- Last Reboot MsgLog & Traplog ---------------------------------------------
Sys Name: W854_01
Model: AIR-CT8540-K9
Version: 8.5.135.0
Primary Boot Image: 8.5.135.0 (default) (active)
Backup Boot Image: 8.5.131.0
LastReset Reason: Planned Reset
Timestamp: Wed Aug 29 11:07:04 2018
SystemUpTime: 0 days 7 hrs 37 mins 1 secs

-------------------------------------------------------MsgLog Dump ------------------------------------------------------------

*rmgrMain: Aug 29 11:07:03.869: %RMGR-0-RED_HA_RELOAD: [SS]rmgr_utils.c:263 System is rebooting, reason: IPC timeout has occurred multiple times, category:Peer reload req
*rmgrMain: Aug 29 11:07:03.869: %RMGR-3-RED_HA_KA_STATS: [SS]rmgr_main.c:711 Keep-alive stats: peer RP KA loss count 0, peer RMI KA received count 26620
*rmgrMain: Aug 29 11:07:03.869: %RMGR-3-RED_HA_GW_STATS: [SS]rmgr_main.c:710 Default gateway stats: ping loss count 0, ping received count 1

Thanks for any help!

 

Everyone's tags (2)
1 ACCEPTED SOLUTION

Accepted Solutions
omz Rising star
Rising star

Re: HA standby WLC keeps reloading - 8.5.135.0

  • IPC - communication between the two boxes before IOSd boots up, transport of HA control messages (such as role selection, keep-alives and so on) and provide the transport for Interprocess Communication (IPC) between the two boxes.
  • Upgrade to 8.5.140.0 did not stop the IPC timeouts.
  • Packet captures on WLC RP ports showed missing packets.
  • Intermediate Nexus switch was affected by CSCvj28133 and had Incrementing Rcv-Err and IntMacRx-Er Errors on uplinks to WLCs.
  • Issue was resolved by upgrading the Nexus switches.
6 REPLIES 6
VIP Mentor

Re: HA standby WLC keeps reloading - 8.5.135.0

 

I think you are hitting a bug related to IPC timeout...

 

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvi25532/?rfs=iqvred

Recommandation: dwongrade the software to 8.5(131.0) and try again.

 

Regards

Dont forget to rate helpful posts

omz Rising star
Rising star

Re: HA standby WLC keeps reloading - 8.5.135.0

Hi Sandeep

Thanks for the response.

The WLC is running 8.5.135.0.

CSCvi25532 show up in Resolved Caveats for 8.5.131.0

CSCvk20484 show up in Resolved Caveats for 8.5.135.0 - IPC timeout and tracebacks reported on Cisco 8540 HA pair running 8.5.131.0 (8.5MR3)

 

You suggest downgrading to 8.5.131.0

 

Thanks 

 

VIP Mentor

Re: HA standby WLC keeps reloading - 8.5.135.0

My suggestion is to downgrade to 8.5.131.0 and give a try.

 

if even after the downgrade you have the same issue then contact to Cisco TAC.

 

Regards

Dont forget to arte helpful posts 

Beginner

Re: HA standby WLC keeps reloading - 8.5.135.0

Hello,

 

We have been seeing this very problem with a 8540 HA pair, running 8.5.135.0. Cisco are struggling to find a resolution or link to any known bugs - any resolution detail would be much appreciated.

 

Thanks

Phil

Rising star

Re: HA standby WLC keeps reloading - 8.5.135.0

This sounds like a bug, its very similar to CSCvi25532 and CSCvi25532, as already mentioned

Even tho they say they are fixed in 8.5.131.0. I have seen many times where its fixed in one release and then back in the next.

 

My recommendation if you can would be to raise a TAC case and get confirmation of the issue, and confirm what version to go to.

The TAC current 8.5 recommended version is 8.5.140.0 if you want to try upgrading and seeing how it goes if you don't have a support agreement

 

For future reference if your looking at software upgrades bookmark the TAC AIROS recommended software.

https://www.cisco.com/c/en/us/support/docs/wireless/wireless-lan-controller-software/200046-tac-recommended-aireos.html#anc12

*****Help out other by using the rating system and marking answered questions as "Answered"*****
*** Please rate helpful posts ***
omz Rising star
Rising star

Re: HA standby WLC keeps reloading - 8.5.135.0

  • IPC - communication between the two boxes before IOSd boots up, transport of HA control messages (such as role selection, keep-alives and so on) and provide the transport for Interprocess Communication (IPC) between the two boxes.
  • Upgrade to 8.5.140.0 did not stop the IPC timeouts.
  • Packet captures on WLC RP ports showed missing packets.
  • Intermediate Nexus switch was affected by CSCvj28133 and had Incrementing Rcv-Err and IntMacRx-Er Errors on uplinks to WLCs.
  • Issue was resolved by upgrading the Nexus switches.
CreatePlease to create content
Content for Community-Ad
August's Community Spotlight Awards