Mobility Control And Data Plan Down (C9800)

[Flambo]

Hi,

Currently have a couple of C9800 controllers in a LAB environment for a POC. They both at this time connect to the same switch but on different subnet's so no firewall to consider.

Each WLC can ping each other, yet I am struggling to bring up the mobility tunnel.

Here is controller 1 details:

And the same for the other controller:

Have been following this guide:

https://www.cisco.com/c/en/us/support/docs/wireless/catalyst-9800-series-wireless-controllers/213913-building-mobility-tunnels-on-catalyst-98.html

That last screenshot in comparison to the other controller is missing the first line. This was there and still didn't work, think it disappeared after a reboot - so thinking it has nothing to do with issue.

Have additionally tried creating a new mobility group (as per URL) and put different MAC on each controller and pointed the peer to that MAC and WLC IP.

I couldn't find any specific debug command that I could use on these controllers, or any detail in the log as to what the problem might be.

Could be a config issue, hence throwing it out their.

Appreciate any assistance in advance.

Thanks

[Scott Fella]

You shouldn’t have to change the MAC address for one. Since it’s in the lab, why don’t you put it on the same subnet and test that scenario first. If you get that working then you at least know that the configuration is good. Then move it to the other subnet and test again. I have mine with mobility to other 9800’s and AireOS controllers on different subnets and it’s working. I’m just testing also.

[Flambo]

Thanks for posting a reply.

I tried putting them on the same network, but made no difference.

Run a compare on the configs and noticed quite a bit of difference on one of them, looks like the startup wizard didn't complete properly.

I deleted the WLC and built a new one, and since there seems to be some success.

I see messages display desperately that the data and then the control path was up.

Oddly when I run the 'show wireless mobility summery' its showing the Control Path Down.

So almost there, just not sure why the control path is showing down now.

Anymore ideas, show commands, debug commands I could use to diagnose the issue?

Many thanks

[Flambo]

Notice on the other controller the following error:
%DTLS_TRACE)MSG-3-EWC_DTLS_ERR: Chasis 1 R0/0: mobilityd: DTLS Error, session:192.168.0.222[16666], Certificate validation failed

Cert verify Error, SCC validation failure, Reason: certificate is not yet valid.

Guess that is the reason for the control path being down but not sure how to correct. Will keep digging.

Thanks

[Scott Fella]

Make sure the time is correct either using NTP or manually setting the time. Also make sure you have your trustpoint added. This is required for all CL controllers.

[neil_titchener]

I have a setup with a HA cluster of C9800-CL (Virtual) devices and two stand-alone C9800-L devices. I'm running 17.4.1 code so the tunnels are CAPWAP.  My experience has been that once you configure the tunnels (vis the GUI), which are reasonably simple to configure they seem to error out every time.  By rebooting the devices either singularly or sometimes both together the tunnels come up.  It's not a great solution but that's the only way I've got them to work.

Just a note.  The beginner badge refers to how often I use this forum, I've been a network engineer for over twenty years

[Scott Fella]

We understand the badge system, but having input is a great thing to help out other and the community, so thanks for posting!

I have never had to bounce controllers to get tunnels up, so that would be bad as I can see that the tunnels would go down in the future.  Probably a bug I would think if that was the workaround.  I too have 9800-CL's and 9800-L's and have tunnels up with no issues at all along with a 5520.  The OP's is using default settings, in which I have always configured prior to setting up any mobility.  The group name and the multicast ip is a standard for me before I start adding other controllers.

Curious to see if the OP was able to get the tunnels up or not.

[neil_titchener]

Hi Scot,

Thanks for your comments.  I haven't configured multicast in the tunnel configuration; my understanding was it wasn't required.  We have upgraded the physical and virtual WLCs three times and have had the same issue with all software versions (virtual and physical running same IOS).  During the build phase of the project the virtual devices have been re-created several times (by the previous engineer) and the same issue persists.  Our experience of the virtual WLCs is that they are quite buggy and CLI configurations are not always displayed correctly in the GUI.  Netflow configuration would be a great example of this; when configured via CLI what is displayed in the GUI does not match the CLI by a long shot. 

[Scott Fella]

Well from my experience, I never had issues with mobility groups with the 9800's I have been testing.  I have also spun up a lot of 9800-CL's for testing and never had issues.  I'm running 17.3 and 17.4 now and also have rebuilt these from scratch.  Make sure you follow the 9800-CL install guide for the hypervisor to make sure you make the changes on your hypervisor that is required also.

[dwrice000]

I was setting up a guest anchor in my DMZ fresh and had the quadruple whammy I needed to fix to make it work.  The things I had to fix which included some of the things mentioned previously were:

-Firewall  (UDP ports 16666 and 16667 allowed between the controllers)

-NTP

-Trustpoint  (wireless config vwlc-ssc...)

-Wireless regulatory domain  (wireless country XX)

Hope this helps.