C9105 AP does not join C9800 WLC

[ygi]

Hello

We are facing a curious issue that I cannot manage to solve, maybe one of you can spot what I did wrong.

Here is the context: 

• 2x C9800-L-C in redundancy mode
• Dozen of C9105AXI and C9105AXW
• WLC located in DataCenter with LAN side using LACP and WMI set on a VLAN interface
• AP discover WLC over DNS
• WLC set with 2 VRF (Management and AP exchange, and no VFR for the Wireless VLAN to the router) for trafic isolation purpose (guest WiFi only)
• Basic configuration so far (Wizard based)
• WLC is in version 17.7.1
• AP are in version 8.10.130.0

The issue we have is weird, all our AP does the Discovery Request and get the Discovery Response, but never pass to the Join steps.

AP can ping the WLC WMI IP.

WLC can ping from WMI IP the AP.

In attachment there is a packet capture screenshot from the WLC point of view for example.

Here is the log from the AP side:

[01/11/2022 15:29:10.6930] DNS resolved CISCO-CAPWAP-CONTROLLER.local.example.com
[01/11/2022 15:29:10.6930] DNS discover IP addr: 10.0.13.2
[01/11/2022 15:29:10.6980] Discovery Request sent to 10.0.13.2, discovery type DNS(3)
[01/11/2022 15:29:10.7010] Discovery Request sent to 255.255.255.255, discovery type UNKNOWN(0)
[01/11/2022 15:29:10.7020]
[01/11/2022 15:29:10.7020] CAPWAP State: Discovery
[01/11/2022 15:29:40.2660]
[01/11/2022 15:29:40.2660] CAPWAP State: Discovery
[01/11/2022 15:29:40.2680] IP DNS query for CISCO-CAPWAP-CONTROLLER.local.example.com
[01/11/2022 15:29:40.2710] DNS resolved CISCO-CAPWAP-CONTROLLER.local.example.com
[01/11/2022 15:29:40.2710] DNS discover IP addr: 10.0.13.2
[01/11/2022 15:29:40.2840] Discovery Request sent to 10.0.13.2, discovery type DNS(3)
[01/11/2022 15:29:40.2910] Discovery Request sent to 255.255.255.255, discovery type UNKNOWN(0)
[01/11/2022 15:30:09.7750]
[01/11/2022 15:30:09.7750] CAPWAP State: Discovery
[01/11/2022 15:30:09.7770] IP DNS query for CISCO-CAPWAP-CONTROLLER.local.example.com
[01/11/2022 15:30:09.7820] DNS resolved CISCO-CAPWAP-CONTROLLER.local.example.com
[01/11/2022 15:30:09.7820] DNS discover IP addr: 10.0.13.2
[01/11/2022 15:30:09.7870] Discovery Request sent to 10.0.13.2, discovery type DNS(3)
[01/11/2022 15:30:09.8300] Discovery Request sent to 255.255.255.255, discovery type UNKNOWN(0)
[01/11/2022 15:30:39.3480]
[01/11/2022 15:30:39.3480] CAPWAP State: Discovery
[01/11/2022 15:30:39.3500] IP DNS query for CISCO-CAPWAP-CONTROLLER.local.example.com
[01/11/2022 15:30:39.3530] DNS resolved CISCO-CAPWAP-CONTROLLER.local.example.com
[01/11/2022 15:30:39.3530] DNS discover IP addr: 10.0.13.2
[01/11/2022 15:30:39.3690] Discovery Request sent to 10.0.13.2, discovery type DNS(3)
[*01/11/2022 15:30:39.3770] Discovery Request sent to 255.255.255.255, discovery type UNKNOWN(0)     

If someone here have an idea of what I did wrong, that could help.

Thanks a lot

[Flavio Miranda · Accepted solution]

Hi

  From the logs, I saw a lot of discovery but didn´t see any capwap request/response.   Also, you could perform a test  using a vlan only, whithout VRF.

[ygi]

AP and WLC are in two different location.

We tried to remove the VRF (but still can't rely on Layer 2 discovery) and the result is the same.

There is no firewall inbetween.

[Leo Laohoo]

---

@ygi wrote:

WLC is in version 17.7.1

---

Is there any operational/critical reason why the WLC is loaded with 17.7.X?

Please check to make sure the APs are not loaded with eWLC firmware.

[ygi]

17.7.1 is just the last one we tested.

It was shipped with a 16 one which is incompatible with 9105 so we updated to 17.6.2 initially and since it didn't changed anything we also tried 17.7.1.

Do you see any issue with that release?

(Once on site today I will check the firmware, but I don't think eWLC works on 9105)

[Leo Laohoo]

Try 17.3.4c.

[ygi]

For whatever reason I can download all version from 17.4+ but not the 17.3 branch. I've opened a request to Cisco support for that

[marce1000 · Accepted solution]

 - Can you have an overall sanity check of the 9800 configuration by using (CLI) show tech wireless and have the output analyzed by : https://cway.cisco.com/tools/WirelessAnalyzer/

 M.

[ygi]

Here is the result.

Is there a real limitation with VRF on this hardware? Not really confortable with the idea of no isolating guest access in a VRF.

Currently the setup is:

- VRF for management

- VRF for ap link

- no VRF for VLAN assigned to WLAN

Does that make sense to you?

[ygi · Accepted solution]

The problem was coming from the VRF. Thanks Flavio and Marce to led us here.

Removing VRF config from the WMI settings solved the issue.

Not sure why it didn't previously when we did other tests.

We will rely on ACL and VLAN to secure the network.

[Flavio Miranda]

Hi

   If you can spend a bit more money, a very good solution is to use Anchor WLC.  You can put the Anchor WLC in a DMZ, maybe you can use VRF on this scenrio if you do not want to use Firewall. 

https://www.cisco.com/c/en/us/support/docs/wireless/catalyst-9800-series-wireless-controllers/213912-configure-mobility-anchor-on-catalyst-98.html#anc12 

[Rich R]

If you go through all the webinars and best practice guides Cisco do not recommend using SVI on 9800 but rather terminate the IP on a router or switch where you're expected to provide whatever security is required (so your separation on the WLC is by vlan).  If you use SVIs then you will need ACLs as you say because they're all in the same global default VRF.

As far as I know the only thing which can be in a separate VRF is the management and even that did not work properly until 17.6.1.

https://www.cisco.com/c/en/us/td/docs/wireless/controller/9800/17-6/config-guide/b_wl_17_6_cg/m_config-wmi.html

I don't think there are any plans to support VRF for wireless functions though.

[ygi]

I didn't see that parts in the guides I've read, and more over:

• the CLI does not provide any alerts when you configure it
• the web UI is 100% made to allow VRF for that use

Also, there is another huge limitation (or bug actually): the WMI interface can only be loopback, GigabitEthernet, or SVI.

You cannot configure WMI on the twoGigabitEthernet, not on the tenGigabitEthernet, and not on the Port-Channel.

The system refuse it.

So when you are in double attachment between your WLC and you distribution stack, you have really limited options:

• loopback and routing table fun
• SVI