WLC Redundancy has failed. Looking to restore.

[Mogwai]

Hello, over the weekend it seems that our Cisco 5520's that are in an HA, went down & came in in a non-HA state. 

(CEN-5520) >show redundancy summary
Redundancy Mode = SSO ENABLED
Local State = ACTIVE
Peer State = UNKNOWN - Communication Down
Unit = Primary
Unit ID = 00:A2:EE:9C:69:AD
Redundancy State = Non Redundant
Mobility MAC = 00:A2:EE:9C:69:AD
Redundancy Port = UP
BulkSync Status = Pending

[Mark Elsen]

  - Check if all network connections are solid on the controller, especially RP to RP ,

   M.

[Mogwai]

Yes, all RP's are connected & passing traffic. One the Secondary unit, I do see the following error. 

(Secondary) >show redundancy summary
            Redundancy Mode = SSO ENABLED 
                Local State = MAINTENANCE 
                 Peer State = UNKNOWN - Communication Down 
                       Unit = Secondary
                    Unit ID = 00:B7:71:8B:04:2F
           Redundancy State = Non Redundant
               Mobility MAC = 00:A2:EE:9C:69:AD
               Redundancy Port  = UP
               Link Encryption = ENABLED
Maintenance Mode = Enabled
Maintenance cause= Peer RP is not reachable or Hardware model/Software version mismatch

I've checked the versioning of both the primary & secondary and both match. Although, the Bootloader & Emergency versions are different between the Primary & Secondary. Do those need to match as well?

[Mark Elsen]

  -   Both  Bootloader & Emergency versions  need to match!

  M.

[Mogwai]

So, I believe I see the issue I just checked & I as mistaken & one of the RP ports isn't passing traffic. If I replace the possibly bad SFP & the interface comes back up, would there be downtime for the WLC's to enter back into an HA state or would downtime be required?

[Mark Elsen]

  - Depends , start by issuing the command show redundancy summary again, when you replace the bad SFP ,

  M.

[Mark Elsen]

 - Here you have a number of other commands which can provide insights :
show redundancy summary
show redundancy infra statistics
show redundancy transport statistics
show redundancy keepalive statistics
show redundancy gw-reachability statistics
show redundancy config-sync statistics
show redundancy ap-sync statistics
show redundancy client-sync statistics

 M.

[Leo Laohoo]

Power down the secondary WLC and replace the faulty SFP.  

Power on.