Replacing a failed WLC

[mumbles202]

Had a vWLC running that had about 25 APs connected to it.  Had a failure of a datastore and didn't have a backup of the unit or the configuration.  I was able to get a new device deployed and configured and able to get APs joined to the controller if I factory reset them, but otherwise I see them in the AP Join as trying to join but they never do.  The new WLC has the same ip as the old and APs and WLC are on the same L2 segment so I thought L2 Discovery would handle this, but I added an option 43 for good measure as well.  If i do a debug on the console I see a ton of DTLS errors; will the new APs try to join another controller if they're unable to join the old controller and but learn controller ip via option 43? 

I can see why it would be locked out for security reasons and would force you to go the route of migrating to a new controller ip and changing the Primary WLC for an AP on the old controller; just in a pinch when that controller is offline what is the approach?  telnet/ssh weren't enabled on the APs on the old WLC so it seems like ladder and a reset is the only solution?

[johnd2310]

Hi,

What errors are you seeing on the WLC? is the new controller same software version as the old?

Thanks

John

[Rich R]

This is clearly documented in the vWLC deployment guide: https://www.cisco.com/c/en/us/td/docs/wireless/technology/mesh/8-2/b_Virtual_Wireless_LAN_Controller_Deployment_Guide_8-2.html?referring_site=RE&pos=1&page=https://www.cisco.com/c/en/us/support/wireless/virtual-wireless-controller/series.html#concept...

Note: When an AP moves from one vWLC to another, it may refuse to join the second vWLC. It occurs when the server hardware fails, or a new instance of vWLCs are created. It is recommended to implement server mirroring scheme at the VMware level such as vMotion or some orchestrator. It is highly recommended to retain a snapshot of the VM instance, one from the mobility domain to which access points have joined previously. Then use the snapshot to start the vWLC instance. Access points then join the vWLC. This method can be also be used for priming access points instead of a physical controller.

More info: https://bst.cloudapps.cisco.com/bugsearch/bug/CSCva69352
https://www.cisco.com/c/en/us/td/docs/wireless/controller/8-4/config-guide/b_cg84/high_availability.html#vwlc-and-nplus1-ha

So you should probably get your ladder out ...

[Scott Fella]

Just like what rrudling mentioned... now at least you know what you need to do in the future.  Backups, snapshots and enable ssh.  With the newer 9800-CL, you don't have that issue, it is one of the reasons I never like using the vWLC and would rather prefer the appliance.  Heck, you probably can find a 3504 for cheap to replace the vWLC until you refresh your access points.  just keep that in mind.

[jagan.chowdam]

Can you get console access to one of your APs and post the console output once you power cycle the AP.

CJ