DHCP addr. required - Guest Wireless

[WifiMyGuy]

Controller: 9800-80

S/W Version: 17.9.5

Issue: Wireless guests on open SSID are not getting COA

Description: Our wireless infrastructure is made up of 15+ wireless controllers. Right now we have a mix of 9800s, 5520s, 8540s using 2800, 3800, and 9166 access points. Guests across ALL controllers are experiencing an issue where they connect to our guest wifi, are redirected to our splash page, hit "accept", however no COA takes place. During a packet capture i can see the COA being passed to the client, but the controller does nothing. I can see traffic getting past our FW on port 1700, and everything looks good. During this, the client will receive an IP address, but is unreachable. The client is sitting in an IP learn state within the controller.

If i go into our guest policy, and disable the DHCP Addr. Req. setting, everything works. Additionally, when the client is sitting at the splash page, it's IP is reachable. 

I also want to note, there are no SVIs on these controllers, we simply hand everything off L2 to an upstream switch where the SVIs live with IP helpers.

I'd like to keep DHCP Addr. Req. for security, but i don't understand WHY it's causing an issue with clients who are already obtaining DHCP addresses.

Photos below are from when DHCP Addr. Req was ON and client could not COA

[Saikat Nandy]

Can you help me please with the answers- 

What kind of guest solution is this? LWA/EWA/CWA?

Do you have AAA Override configured?

Through CoA are you changing vlan?

[WifiMyGuy]

Cisco ISE is providing the splash page, so CWA. We have AAA override configured on the guest WLAN policy. We are not changing VLANs. The guest policy is set to use a guest vlan group, and gets placed on any of the vlans within the group.

[Saikat Nandy]

I reckon we do not have any evidence to co-relate CoA with DHCP and here is why (based on your answers) - 

If we break the flow of CWA, it happens like  - 

1. Assoc Request - Access Request is sent with MAB.
2. MAB success happens and Assoc Resp is sent.
3. DHCP happens.
4. Followed by DNS, TCP, HTTP
5. Then comes CoA with whatever the types of CoA you are using. ISE send CoA Request and WLC responds back with CoA Response.
6. Assoc Request - Access Request is sent.
7. Access Accept followed by final Assoc Response.

So DHCP and CoA happens at different stages. If You say CoA is not working and config is correct, then could be - 
https://bst.cisco.com/bugsearch/bug/CSCwm54065
https://bst.cisco.com/bugsearch/bug/CSCwk63163
https://bst.cisco.com/bugsearch/bug/CSCwj01157
https://bst.cisco.com/bugsearch/bug/CSCvr73095
https://bst.cisco.com/bugsearch/bug/CSCwi34080

If you say DHCP is not working, then would suggest to use vlans one by one. There could be an issue with one of the vlans, not all.

[Rich R]

- "mix of 9800s, 5520s, 8540s using 2800, 3800, and 9166 access points. Guests across ALL controllers are experiencing an issue"
so it's unlikely that it's a 9800 bug because it's affecting 9800 & AireOS WLCs but worth noting those anyway for sure, and another reason to ensure the software is up to date!

I see 17.12.5 has got gold star marking on the download pages but not yet on the TAC recommended versions, but likely to be soon.

[ammahend]

do you have CoA server key configured, if Yes then follow this guide

https://www.cisco.com/c/en/us/support/docs/wireless/catalyst-9800-series-wireless-controllers/220852-troubleshoot-central-web-authentication.html#toc-hId-1441757191

[Rich R]

1. Was this working before and it has stopped working at some point, or it's never worked?
2. Is it happening all the time (not working at all) or only sometimes?
3. Just to be clear CoA is sent to the controller, not the client.
4. If you're sure the WLC receives the CoA and doesn't act on it then you need client debugs to see why.  On AireOS debug client MAC and aaa.  On 9800 Radioactive Trace on client MAC.  Use the Debug Analyzer (link below) to clean up the debug output and RA trace.
5. If all the WLCs are receiving the CoAs and ignoring them then the most likely cause is incorrect CoA key.  Have you tried re-applying the key on both sides to ensure it's correct?
6. Take note of the TAC recommended releases (links below) - important to keep your software up to date to make sure you have updates for known bug fixes.

[WifiMyGuy]

1. Was this working before and it has stopped working at some point, or it's never worked?

This used to work, started noticing issues when we were replacing legacy AIROS controllers with 9800s. Not saying it's related, just want to note it.
2. Is it happening all the time (not working at all) or only sometimes?

It seems to be 90% of the time, users are experiencing this issue.3. Just to be clear CoA is sent to the controller, not the client.
4. If you're sure the WLC receives the CoA and doesn't act on it then you need client debugs to see why.  On AireOS debug client MAC and aaa.  On 9800 Radioactive Trace on client MAC.  Use the Debug Analyzer (link below) to clean up the debug output and RA trace.

WLC debugs and traces, as well as ISE debugs have been sent to TAC. Waiting for help, but posting here as well for visibility.
5. If all the WLCs are receiving the CoAs and ignoring them then the most likely cause is incorrect CoA key.  Have you tried re-applying the key on both sides to ensure it's correct?

I have not, but can today.
6. Take note of the TAC recommended releases (links below) - important to keep your software up to date to make sure you have updates for known bug fixes.

Because we're a very sensitive service area, we tend to move slower with upgrades, except for "test prod" areas, which have to run the latest to be able to pilot new features, like AnyLocate

[JPavonM]

It is unclear how the "DHCP Required" feature works on different vendors as this is not a 802.11 standard feature but a vendor one. It is only clear that for 802.1X authentications it does not allow traffic to/from the client until the AP does not see a complete DHCP process (to either renew or get a new lease) under a new connection and roaming scenarios.

It may be that the AP needs to see a DHCP packet from the client once it is authorized again after CoA (as the device is forced to disconnect, re-connect, and re-authenticate to the RADIUS again), thus not releasing traffic to the client if that not happen.

I've experienced similar situations with roaming clients that they do not start a DHCP process when a new association is done, and they are stuck with no connection on the destination AP. The solution for me was to not rely on the DHCP required feature, as this is a vendor feature not supported by the 802.11 standard and it may lead to issues with some devices.

[WifiMyGuy]

"The solution for me was to not rely on the DHCP required feature, as this is a vendor feature not supported by the 802.11 standard and it may lead to issues with some devices."

This may be we're headed too. The security of it is nice on guest address space but we may have to just dump it.

[Rich R]

As I've mentioned on other similar discussions we run public WiFi hotspots, so DHCP Required is considered a mandatory security feature which we have enabled across 20,000+ APs (AireOS and 9800) without problems, so I would concentrate on what is actually causing your issue which I think is unlikely to be DHCP required.

> Because we're a very sensitive service area, we tend to move slower with upgrades, except for "test prod" areas, which have to run the latest to be able to pilot new features, like AnyLocate
Well do you see the same problems on those "test prod" areas running the recommended code versions?

[WifiMyGuy · Accepted solution]

ISSUE HAS BEEN RESOLVED:

This was being caused by two things...

1: Within the GUI, NAC was enabled. However, when going into the CLI and viewing the policy, NAC was not enabled. There is a mismatch between what the GUI is reporting and what the CLI is reporting. After enabling NAC within the policy in the CLI, clients would be able to get redirected, but now were getting stuck in a "IP learn" state.

2: The "IP Learn" state was due to the fact that the DHCP addr. was being learned from ARP, and not DHCP. Because DHCP Addr. Required was enabled. this would break things. By disabling IP MAC binding within the policy, the issue was then resolved.

[Rich R]

You'll see plenty of comments from me and others here saying we avoid using the GUI 99% of the time - you should really only trust the config you can see on the CLI.  The GUI has numerous bugs and quirks (and you are using an old version of code).
How were the clients getting their IP addresses?
Did you check the config with Config Analyzer?
Do you have ARP proxy enabled as per the best practice guide?