Wireless controller 5520 stuck in maintenance mode.

muzecisco · ‎02-22-2020

Hi Team

I have my primary WLC 5520 (software 8.10.105.0) stuck in MAINTENANCE. I have remotely tried reset system with out any success.

The software version is the same as standby HA. I can ping the RM and RP ip address from both Primary to standby and vice versa.

On the standby WLC(currently active), reason for switchover is Default gateway is not reachable. I tested pings and they are all OK.

May you kindly provide leads on what could be happening and how to solve this issue.

(Cisco Controller) >show redundancy sum
Redundancy Mode = SSO ENABLED
Local State = MAINTENANCE
Peer State = UNKNOWN - Communication Down
Unit = Primary
Unit ID = 08:96:
Redundancy State = Non Redundant
Mobility MAC = 08:96:
Redundancy Port = UP
Link Encryption = ENABLED
Maintenance Mode = Enabled
Maintenance cause= Too many reboots

I have also just checked the reboot history and i see the below message.

========================================
Reset reason : active default gateway is not reachable ,(Version: 8.10.105.0)
Reset time : Thu Feb 20 15:40:38 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1cb1a34 0x1ca455a 0x1ca6a36 0x1c9c73e 0x1c31827 0x7dff 0xe898d
========================================

--More-- or (q)uit
Reset reason : Certificate transfer from Active to Standby failed ,(Version: 8.10.105.0)
Reset time : Thu Feb 20 15:49:01 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1cb1a34 0x1ca7284 0x1ca6a36 0x1c9c9a3 0x1c31827 0x7dff 0xe898d
========================================
Reset reason : Certificate transfer from Active to Standby failed ,(Version: 8.10.105.0)
Reset time : Thu Feb 20 15:55:05 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1cb1a34 0x1ca7284 0x1ca6a36 0x1c9c9a3 0x1c31827 0x7dff 0xe898d
========================================
Reset reason : Certificate transfer from Active to Standby failed ,(Version: 8.10.105.0)
Reset time : Thu Feb 20 16:03:26 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1cb1a34 0x1ca7284 0x1ca6a36 0x1c9c9a3 0x1c31827 0x7dff 0xe898d
========================================
Reset reason : Certificate transfer from Active to Standby failed ,(Version: 8.10.105.0)
Reset time : Thu Feb 20 16:09:28 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1cb1a34 0x1ca7284 0x1ca6a36 0x1c9c9a3 0x1c31827 0x7dff 0xe898d
========================================
Reset reason : User Initiated reset ,(Version: 8.10.105.0)
Reset time : Fri Feb 21 23:03:13 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1c1b616 0x1fb4c81 0x23b0f09 0x1a50741 0x1a29e19 0x1a6b598 0x1a5af01 0x1a4c4a8 0x1a46d1a 0x1a70aaf 0x1c31827 0x7dff 0xe898d
========================================
Reset reason : Certificate transfer from Active to Standby failed ,(Version: 8.10.105.0)
Reset time : Fri Feb 21 23:11:33 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1cb1a34 0x1ca7284 0x1ca6a36 0x1c9c9a3 0x1c31827 0x7dff 0xe898d

Reset reason : Certificate transfer from Active to Standby failed ,(Version: 8.10.105.0)
Reset time : Fri Feb 21 23:17:35 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1cb1a34 0x1ca7284 0x1ca6a36 0x1c9c9a3 0x1c31827 0x7dff 0xe898d
========================================
Reset reason : Certificate transfer from Active to Standby failed ,(Version: 8.10.105.0)
Reset time : Fri Feb 21 23:25:57 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1cb1a34 0x1ca7284 0x1ca6a36 0x1c9c9a3 0x1c31827 0x7dff 0xe898d
========================================
Reset reason : Certificate transfer from Active to Standby failed ,(Version: 8.10.105.0)
Reset time : Fri Feb 21 23:32:00 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1cb1a34 0x1ca7284 0x1ca6a36 0x1c9c9a3 0x1c31827 0x7dff 0xe898d

Scott Fella · ‎02-22-2020

I’m going to assume that the code versions are the same? What you can do is disconnect the RP port and the connection to the network on the primary, since it’s down. Console in and reload the unit and see if it comes up as primary. Don’t connect the unit to the network, you just want to see if it detects the other unit down and comes up.

-Scott
*** Please rate helpful posts ***

Leo Laohoo · ‎02-22-2020

@muzecisco wrote:

Peer State = UNKNOWN - Communication Down

RP is connected using copper cable. Try replacing that first.

muzecisco · ‎02-22-2020

Thanks i will try this tomorrow.

From reboot history see below some informative logs i got.

========================================
Reset reason : active default gateway is not reachable ,(Version: 8.10.105.0)
Reset time : Thu Feb 20 15:40:38 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1cb1a34 0x1ca455a 0x1ca6a36 0x1c9c73e 0x1c31827 0x7dff 0xe898d
========================================

--More-- or (q)uit
Reset reason : Certificate transfer from Active to Standby failed ,(Version: 8.10.105.0)
Reset time : Thu Feb 20 15:49:01 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1cb1a34 0x1ca7284 0x1ca6a36 0x1c9c9a3 0x1c31827 0x7dff 0xe898d
========================================
Reset reason : Certificate transfer from Active to Standby failed ,(Version: 8.10.105.0)
Reset time : Thu Feb 20 15:55:05 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1cb1a34 0x1ca7284 0x1ca6a36 0x1c9c9a3 0x1c31827 0x7dff 0xe898d
========================================
Reset reason : Certificate transfer from Active to Standby failed ,(Version: 8.10.105.0)
Reset time : Thu Feb 20 16:03:26 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1cb1a34 0x1ca7284 0x1ca6a36 0x1c9c9a3 0x1c31827 0x7dff 0xe898d
========================================
Reset reason : Certificate transfer from Active to Standby failed ,(Version: 8.10.105.0)
Reset time : Thu Feb 20 16:09:28 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1cb1a34 0x1ca7284 0x1ca6a36 0x1c9c9a3 0x1c31827 0x7dff 0xe898d
========================================
Reset reason : User Initiated reset ,(Version: 8.10.105.0)
Reset time : Fri Feb 21 23:03:13 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1c1b616 0x1fb4c81 0x23b0f09 0x1a50741 0x1a29e19 0x1a6b598 0x1a5af01 0x1a4c4a8 0x1a46d1a 0x1a70aaf 0x1c31827 0x7dff 0xe898d
========================================
Reset reason : Certificate transfer from Active to Standby failed ,(Version: 8.10.105.0)
Reset time : Fri Feb 21 23:11:33 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1cb1a34 0x1ca7284 0x1ca6a36 0x1c9c9a3 0x1c31827 0x7dff 0xe898d

--More-- or (q)uit
========================================
Reset reason : Certificate transfer from Active to Standby failed ,(Version: 8.10.105.0)
Reset time : Fri Feb 21 23:17:35 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1cb1a34 0x1ca7284 0x1ca6a36 0x1c9c9a3 0x1c31827 0x7dff 0xe898d
========================================
Reset reason : Certificate transfer from Active to Standby failed ,(Version: 8.10.105.0)
Reset time : Fri Feb 21 23:25:57 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1cb1a34 0x1ca7284 0x1ca6a36 0x1c9c9a3 0x1c31827 0x7dff 0xe898d
========================================
Reset reason : Certificate transfer from Active to Standby failed ,(Version: 8.10.105.0)
Reset time : Fri Feb 21 23:32:00 2020
Trace back : 0x8e7b60 0x1c1a4ba 0x1cb1a34 0x1ca7284 0x1ca6a36 0x1c9c9a3 0x1c31827 0x7dff 0xe898d