topic Re: c9800 Flex Connect Post Auth ACL Issue in Wireless

c9800 Flex Connect Post Auth ACL Issue

xxkozxx — Mon, 28 Aug 2023 18:15:02 GMT

I already have a TAC case open on this but I wanted to see if anyone else has run across this...

I am deploying new c9800-40's and have a requirement to do flex connect with CWA. I had this set up with my 5520's and it worked without issue but for some reason I am hitting a wall with the new configs.

I have the SSID set up and the CWA redirect and subsequent auth happens without issue. However when the post auth "internet-only" ACL gets applied on the AP, the client has no network access. The exact same ACL is used on a centrally switch WLAN and works without issue.

I've test this on both 3802i and 9136i AP's and it made no difference.

I have found plenty of documentation on how to set up the redirect (much of it vague) and none seem to actually discuss the post-auth ACL portion of the configuration. Any help would be greatly appreciated.

EDIT: In testing with TAC, we found that the ACL that is downloaded to the AP is blocking the user. It is not adding an exception for the client so the line in the ACL which blocks the 10.x network is effectively blocking the client and access to the gateway. We were able to test this theory by removing the line referencing the 10.x network and the client had access. However, this negates the security control by also allowing the client access to internal resources on a guest network. The same ACL processed by the controller (centrally switched vs. flex connect) works without issue.

Post-Mortem: I worked with my account team, TAC and a wireless engineer to test this both on AIR-OS and on the IOS-XE controllers. The behavior is the same. After much debate, it was determined that the ACL behavior is by design as the AP Operating System does not currently process ACL's in the same manner as switches, routers or wireless controllers. As a result you are left with solutions that require additional configuration and or infrastructure to support it. So, you can leave it centrally switched or add infrastructure and or configuration to accommodate security policy around locally switched networks (i.e. ACL's on upstream switch, firewall gateway, VRF's etc...)

I have requested that a feature request be opened on this issue as I firmly believe that if I can push policy via a NAC or SGT type solution to pretty much any other Cisco Product that the AP should behave in the same manner. Hopefully, this functionality will come in later releases.

Here is the Wireless Configuration:

======== WLAN ======== wlan flex-test 7 flex-test assisted-roaming prediction dot11ax target-waketime dot11ax twt-broadcast-support mac-filtering ise-radius-aaa scan-report association no security ft adaptive no security wpa no security wpa wpa2 no security wpa wpa2 ciphers aes no security wpa akm dot1x security dot1x authentication-list ise-radius-aaa no shutdown ============= FLEX PROFILE ============= wireless profile flex cwa-flex-profile acl-policy PERMIT-ANY acl-policy ACL-INTERNET-ONLY acl-policy ACL-WEBAUTH-REDIRECT central-webauth ip http client proxy 0.0.0.0 0 native-vlan-id 50 vlan-name flex-client-wireless vlan-id 50 =================== FLEX POLICY PROFILE =================== wireless profile policy cwa-portal-flex-policy-profile aaa-override aaa-policy dvn-aaa-policy no accounting-interim accounting-list ise-radius-aaa no central dhcp no central switching no flex umbrella dhcp-dns-option http-tlv-caching ipv4 flow monitor wireless-avc-basic input ipv4 flow monitor wireless-avc-basic output ipv6 flow monitor wireless-avc-basic-ipv6 input ipv6 flow monitor wireless-avc-basic-ipv6 output nac passive-client radius-profiling vlan 50 no shutdown ================ AP JOIN SITE TAG ================ wireless tag site ap-join-flex-site-tag ap-profile flex-test-ap-join-profile flex-profile cwa-flex-profile no local-site =============== AP JOIN PROFILE =============== ap profile flex-test-ap-join-profile country US mgmtuser username [REDACTED] password [REDACTED] secret [REDACTED] ntp ip 0.0.0.0 no oeap link-encryption no oeap local-access no oeap provisioning-ssid preferred-mode ipv4 ssh statistics ap-system-monitoring alarm-enable statistics ap-system-monitoring enable statistics ap-radio-monitoring action radio-reset statistics ap-radio-monitoring alarm-enable statistics ap-radio-monitoring enable syslog host 255.255.255.255 ================== AP THAT IS TAGGED ================== ap 6871.61f2.2a04 policy-tag flex-test-policy-tag rf-tag dvn-campus-rf-tag site-tag ap-join-flex-site-tag

Here are my ACL's:

ip access-list extended ACL-INTERNET-ONLY 10 permit udp any any eq bootps 20 permit udp any any eq bootpc 30 permit udp any any eq domain 40 permit tcp any 172.17.242.0 0.0.1.255 <---ACCESS TO ISE PSN's Post-Auth 50 permit tcp 172.17.242.0 0.0.1.255 any <---ACCESS FROM ISE PSN's Post-Auth 60 deny ip any 192.168.0.0 0.0.255.255 70 deny ip any 172.16.0.0 0.15.255.255 80 deny ip any 10.0.0.0 0.255.255.255 90 permit ip any any ip access-list extended ACL-WEBAUTH-REDIRECT 10 deny ip any 172.17.242.0 0.0.1.255 20 deny ip 172.17.242.0 0.0.1.255 any 30 deny udp any any eq domain 40 deny udp any eq domain any 50 permit tcp any any eq www

I see the client in a run state and in ISE I see a full complete auth against the CWA portal.

MAC Address AP Name Type ID State Protocol Method Role ------------------------------------------------------------------------------------------------------------ 2222.98b7.0b43 AP6871-61F2-2A04 WLAN 7 Run 11ax(5) MAB Local

Here I see the association and auth applied:

WLC1# sh wireless client mac-address 2222.98b7.0b43 detail Client MAC Address : 2222.98b7.0b43 Client MAC Type : Locally Administered Address Client DUID: NA Client IPv4 Address : 10.2.18.188 Client IPv6 Addresses : fe80::41f:24f5:bbf7:850 Client Username : XXXXXXXXXXXXXX AP MAC Address : 6871.6196.0630 AP Name: AP6871-61F2-2A04 AP slot : 1 Client State : Associated Policy Profile : cwa-portal-flex-policy-profile Flex Profile : cwa-flex-profile Wireless LAN Id: 7 WLAN Profile Name: flex-test Wireless LAN Network Name (SSID): flex-test BSSID : 6871.6196.063f Connected For : 150 seconds Protocol : 802.11ax - 5 GHz Channel : 161 Client IIF-ID : xxx Association Id : 2 Authentication Algorithm : Open System Idle state timeout : N/A Session Timeout : 1800 sec (Remaining time: 1594 sec) Session Warning Time : Timer not running Input Policy Name : None Input Policy State : None Input Policy Source : None Output Policy Name : None Output Policy State : None Output Policy Source : None WMM Support : Enabled U-APSD Support : Disabled Fastlane Support : Enabled Client Active State : Active Power Save : OFF Current Rate : 24.0 Supported Rates : 6.0,9.0,12.0,18.0,24.0,36.0,48.0,54.0 AAA QoS Rate Limit Parameters: QoS Average Data Rate Upstream : 0 (kbps) QoS Realtime Average Data Rate Upstream : 0 (kbps) QoS Burst Data Rate Upstream : 0 (kbps) QoS Realtime Burst Data Rate Upstream : 0 (kbps) QoS Average Data Rate Downstream : 0 (kbps) QoS Realtime Average Data Rate Downstream : 0 (kbps) QoS Burst Data Rate Downstream : 0 (kbps) QoS Realtime Burst Data Rate Downstream : 0 (kbps) Mobility: Move Count : 0 Mobility Role : Local Mobility Roam Type : None Mobility Complete Timestamp : 07/11/2023 12:39:47 CDT Client Join Time: Join Time Of Client : 07/11/2023 12:39:47 CDT Client State Servers : None Client ACLs : None Policy Manager State: Run Last Policy Manager State : IP Learn Complete Client Entry Create Time : 343 seconds Policy Type : N/A Encryption Cipher : None Transition Disable Bitmap : 0x00 User Defined (Private) Network : Disabled User Defined (Private) Network Drop Unicast : Disabled Encrypted Traffic Analytics : No Protected Management Frame - 802.11w : No EAP Type : Not Applicable VLAN Override after Webauth : No VLAN : 50 Multicast VLAN : 0 WiFi Direct Capabilities: WiFi Direct Capable : No Central NAT : DISABLED Session Manager: Point of Attachment : capwap_9040000e IIF ID : xxx Authorized : TRUE Session timeout : 1800 Common Session ID: xxx Acct Session ID : xxx Last Tried Aaa Server Details: Server IP : 172.17.243.40 Auth Method Status List Method : MAB SM State : TERMINATE Authen Status : Success Local Policies: Service Template : wlan_svc_cwa-portal-flex-policy-profile (priority 254) VLAN : 50 Absolute-Timer : 1800 Server Policies: Filter-ID : ACL-INTERNET-ONLY Resultant Policies: Filter-ID : ACL-INTERNET-ONLY VLAN : 50 Absolute-Timer : 1800 DNS Snooped IPv4 Addresses : None DNS Snooped IPv6 Addresses : None Client Capabilities CF Pollable : Not implemented CF Poll Request : Not implemented Short Preamble : Not implemented PBCC : Not implemented Channel Agility : Not implemented Listen Interval : 0 Fast BSS Transition Details : Reassociation Timeout : 0 11v BSS Transition : Implemented 11v DMS Capable : No QoS Map Capable : No FlexConnect Data Switching : Local FlexConnect Dhcp Status : Local FlexConnect Authentication : Central Client Statistics: Number of Bytes Received from Client : 299981 Number of Bytes Sent to Client : 2507060 Number of Packets Received from Client : 1695 Number of Packets Sent to Client : 2121 Number of Policy Errors : 0 Radio Signal Strength Indicator : -32 dBm Signal to Noise Ratio : 64 dB Fabric status : Disabled Radio Measurement Enabled Capabilities Capabilities: Link Measurement, Passive Beacon Measurement, Active Beacon Measurement, Table Beacon Measurement, Statistics Measurement, AP Channel Report Client Scan Report Time : Timer not running Client Scan Reports Last Report @: 07/11/2023 12:43:00 Assisted Roaming Neighbor List Nearby AP Statistics: EoGRE : Pending Classification Device Protocol : HTTP Type : 1 115 Data : 73 00000000 00 01 00 6f 4d 6f 7a 69 6c 6c 61 2f 35 2e 30 20 |...oMozilla/5.0 | 00000010 28 69 50 68 6f 6e 65 3b 20 43 50 55 20 69 50 68 |(iPhone; CPU iPh| 00000020 6f 6e 65 20 4f 53 20 31 36 5f 35 5f 31 20 6c 69 |one OS 16_5_1 li| 00000030 6b 65 20 4d 61 63 20 4f 53 20 58 29 20 41 70 70 |ke Mac OS X) App| 00000040 6c 65 57 65 62 4b 69 74 2f 36 30 35 2e 31 2e 31 |leWebKit/605.1.1| 00000050 35 20 28 4b 48 54 4d 4c 2c 20 6c 69 6b 65 20 47 |5 (KHTML, like G| 00000060 65 63 6b 6f 29 20 4d 6f 62 69 6c 65 2f 31 35 45 |ecko) Mobile/15E| 00000070 31 34 38 |148 | Max Client Protocol Capability: Wi-Fi6 (802.11ax) WiFi to Cellular Steering : Not implemented Cellular Capability : N/A Advanced Scheduling Requests Details: Apple Specific Requests(ASR) Capabilities/Statistics: Regular ASR support: DISABLED

And on the AP I can see the ACL being applied to the client but it shows a bunch of drops:

AP6871-61F2-2A04#sh client access-lists post-auth all 2222.98b7.0b43 Post-Auth URL ACLs for Client: 22:22:98:B7:0B:43 IPv4 ACL: ACL-INTERNET-ONLY IPv6 ACL: ACTION URL-LIST Resolved IPs for Client: 22:22:98:B7:0B:43 HIT-COUNT URL ACTION IP-LIST ACL-INTERNET-ONLY rule 0: allow true and ip proto 17 and dst port 67 rule 1: allow true and ip proto 17 and dst port 68 rule 2: allow true and ip proto 17 and dst port 53 rule 3: allow true and dst 172.17.242.0 mask 255.255.254.0 and ip proto 6 rule 4: allow true and src 172.17.242.0 mask 255.255.254.0 and ip proto 6 rule 5: deny true and dst 192.168.0.0 mask 255.255.0.0 rule 6: deny true and dst 172.16.0.0 mask 255.240.0.0 rule 7: deny true and dst 10.0.0.0 mask 255.0.0.0 rule 8: allow true No IPv6 ACL found Acl name Quota Bytes left In bytes Out bytes In pkts Out pkts Drops-in Drops-out ACL-INTERNET-ONLY 0 0 1756 38725 8 374 348 9 CLIENT STATE: FWD WEBAUTH_REQUIRED: FALSE DNS POST AUTH: FALSE PREAUTH ENABLED: FALSE POSTAUTH ENABLED: TRUE

Re: c9800 Flex Connect Post Auth ACL Issue

Mark Elsen — Wed, 12 Jul 2023 15:09:42 GMT

- Have a checkup of the 9800-40's configuration(s) with the CLI command show tech wireless ; feed the output into :
https://cway.cisco.com/wireless-config-analyzer/
To begin with this proceduren is strongly advised

- Make sure that for flexconntect APs , the necessary VLANs corresponding to WLANs are arriving at the APs through trunking

For client debugging use : https://logadvisor.cisco.com/logadvisor/wireless/9800/9800CWA
https://logadvisor.cisco.com/logadvisor/wireless/9800/9800ClientConnectivity
The latter (full client debugs) can be analyzed with : https://cway.cisco.com/tools/WirelessDebugAnalyzer/

Also noted : https://bst.cisco.com/bugsearch/bug/CSCvr58194

Re: c9800 Flex Connect Post Auth ACL Issue

xxkozxx — Wed, 12 Jul 2023 16:14:38 GMT

So question about your statement:

- Make sure that for flexconntect APs , the necessary VLANs corresponding to WLANs are arriving at the APs through trunking

I have only one VLAN for wireless and all my AP's have always been access ports with no specific tags on the AP themselves. Is it really necessary to trunk the AP if you only have one vlan and that native vlan is set in the wireless config?

I should also note that in testing, I changed the ACL to a permit any ACL and that gives network access.

Re: c9800 Flex Connect Post Auth ACL Issue

Mark Elsen — Thu, 13 Jul 2023 05:45:38 GMT

>... Is it really necessary to trunk the AP
- If you have only one WLAN that would not be needed , use the WirelessAnalyzer procedure (first step from my reply ) too !

Re: c9800 Flex Connect Post Auth ACL Issue

xxkozxx — Thu, 13 Jul 2023 15:18:38 GMT

@Mark Elsen

Configuration Analyzer came back with nothing related to a WLAN configuration issue. Doing some additional testing I trimmed the ACL down to just deny RFC1918 addresses and permit everything else. This blocked the system from even reconnecting after CoA.

It's as though the ACL gets applied after CoA but it is not properly processing traffic. This whole issue is related to the post-auth ACL on flex connect. I see good association and CWA is properly auth'ing. I see the post-auth ACL get applied on the AP itself for that client MAC. But once the ACL is applied, the client is dead in the water (with the exception of the DNS, DHCP and ISE PSN allows I have above the RFC1918 denies).

Conversely, in a centrally switched scenario, the ACL works without issue. There seems to be a difference in how the AP is applying the ACL to the client versus how the WLC handles it.

I also have a dot1x wlan that is configured on flex connect and uses a permit any. That wlan has no issues. If I change the CWA wlan to use a permit any, the client has full access. It only seems to have an issue when an ACL is applied that has denies in it. And I can clearly see in the ACL stats, that it is dropping packets both in and out.

Re: c9800 Flex Connect Post Auth ACL Issue

Mark Elsen — Thu, 13 Jul 2023 16:10:33 GMT

- What software version is the controller on and or consider :
https://www.cisco.com/c/en/us/support/docs/wireless/catalyst-9800-series-wireless-controllers/214749-tac-recommended-ios-xe-builds-for-wirele.html
if applicable ,

Re: c9800 Flex Connect Post Auth ACL Issue

xxkozxx — Thu, 13 Jul 2023 16:11:33 GMT

17.9.3

Re: c9800 Flex Connect Post Auth ACL Issue

Mark Elsen — Thu, 13 Jul 2023 16:47:38 GMT

- Looks good (recent) ; for the rest it is out of my capacities ; you may want to engage Cisco on the matter (TAC) ,

Re: c9800 Flex Connect Post Auth ACL Issue

Rich R — Sat, 29 Jul 2023 11:03:49 GMT

Yes I think you need to keep pushing TAC to open a bug for it. They will want to repro it first which should be pretty easy for them to do. Get the bug opened then the BU assign a dev to look at it. They will probably come back and say "feature enhancement" because that seems to be standard procedure for many bugs these days. That's when you quote "feature parity" because it already works fine on AireOS so it's not an enhancement at all - they've just broken it on IOS-XE. It's good that you're already on 17.9.3 because that's effectively the most up to date release (discounting 17.10 and 17.11 because they're limited support releases).

I suspect very few people use post-auth ACL on flex local switching so you're probably the first to discover and report this! Most would put guest traffic in a separate VLAN so the enforcement can be done at the next hop rather than relying on AP ACL. And that might be the workaround you have to consider because a fix won't be available before 17.9.5 at the earliest unless you can persuade them to produce a APSP or SMU for you once they have the fix - which would likely be based on 17.9.4 which should be out soon.

Re: c9800 Flex Connect Post Auth ACL Issue

xxkozxx — Mon, 31 Jul 2023 13:54:04 GMT

Rich,

Thanks for the reply. I am currently doing exactly as you suggested. It appears that the ACL's are improperly applied in this manner. I suspect you are correct in that very few people are using post-auth ACL's in flex connect. However, it's much more scalable than having to set up infrastructure at several locations (such as branch offices). Though adding next-hop infrastructure is an option and a workaround I have considered, it's not the preferred method. For now, I have left that particular WLAN in centrally switched until this issue can be resolved. I will follow-up with my account team and TAC to see what next steps are...

Re: c9800 Flex Connect Post Auth ACL Issue

Rich R — Mon, 31 Jul 2023 14:07:54 GMT

Cool - let us know the bug id once you get it.

Re: c9800 Flex Connect Post Auth ACL Issue

Erich Schommarz — Mon, 19 Feb 2024 17:00:39 GMT

moved to the end

Re: c9800 Flex Connect Post Auth ACL Issue

Erich Schommarz — Mon, 19 Feb 2024 17:00:48 GMT

We have exactly the same issue. I was trying out to find out how the acl / where the acl's are assigned to -> Ingress Traffic, Egress Traffic, to or from Client.

Today I did a lot of testing and for me it seems that the ACL is applied in both directions. So I adapted my ACL to reflect this and now it works for me. Couldn't find any actual documentation except AirOS 8.2. There this behaviour is mentioned. But since 8.2 a lot of things have changed and added with Flexconnect: https://www.cisco.com/c/en/us/td/docs/wireless/controller/8-2/config-guide/b_cg82/b_cg82_chapter_010010110.pdf

Interesting that TAC doesn't even know how things are applied / should work!

My ACL for the moment to block communication internal and allow traffic to Internet:

ip access-list extended BLOCK-INTRAZONE-188
permit icmp any any
permit udp any any eq bootps
permit udp any any eq bootpc
deny ip 10.0.188.0 0.0.1.255 10.0.188.0 0.0.1.255
permit ip any any

Re: c9800 Flex Connect Post Auth ACL Issue

Aomar bahloul — Fri, 24 May 2024 19:12:14 GMT

Hi,

I'm having the exact same issue. My use case is limiting access for some clients to only internet and few internal resources I don't want to create a separate WLAN just for that. Access to internal resources works but Internet access doesn't. I suspect it's because of the "permit ip any any" ACE at the end if the Felxconnect ACL. I do see it replaced by "allow true" statement on the AP client ACL but that doesn't seem to work for returning traffic since I can see outbound traffic reach the upstream firewall.

Were you able to file a Bug with TAC, if so, what's the Bug ID I would like to get updates when it's fixed.

FYI I'm running version 17.9.4 and this still hasn't been fixed.

Thank you.

Aomar.

Re: c9800 Flex Connect Post Auth ACL Issue

Rich R — Sat, 25 May 2024 10:36:30 GMT

@Aomar bahloul - @xxkozxx said above "After much debate, it was determined that the ACL behavior is by design as the AP Operating System" so as far as Cisco is concerned this is not a bug.

@Erich Schommarz confirmed above that it is working (once you understand how it is applied) and gives an example of a working ACL. Have you followed that guidance?

I don't think it's your permit ip any any causing the problem - it's more likely that you've misunderstood how the prior deny statements are being applied to traffic in both directions.

> FYI I'm running version 17.9.4 and this still hasn't been fixed.
Well since it's not considered to be a bug, it will not be "fixed".
If you can persuade your Cisco account team, and have a large enough business justification, you could raise an enhancement request to have the behaviour changed. But realistically I don't think there is much chance of that happening. You might get more mileage out of getting a documentation bug opened to have the behaviour documented and explained fully in the config guides.

Re: c9800 Flex Connect Post Auth ACL Issue

Aomar bahloul — Sat, 25 May 2024 19:01:18 GMT

@Rich R

This is another instance of cisco being consistently inconsistent! It seems like you have a better understanding of the AP ACLs can you tell me how you would configure this ACL to work as an AP ACL:

permit ip any host 10.40.40.200
permit tcp any host 10.40.40.50 eq 445
permit tcp any host 10.30.40.126 eq 443
deny ip any 10.0.0.0 0.255.255.255
deny ip any 192.168.0.0 0.0.255.255
deny ip any 172.16.0.0 0.15.255.255
permit ip any any

Re: c9800 Flex Connect Post Auth ACL Issue

Rich R — Sat, 25 May 2024 23:24:59 GMT

ha ha - didn't you know the Cisco developers' motto is "consistently inconsistent" ? <smile>

Seriously though - as I mentioned above you have forgotten to allow the traffic in both directions!
You haven't told us what your client range is so we can't complete but let's assume your client range is 10.1.1.0/24 then your 4th ACL entry (deny ip any 10.0.0.0 0.255.255.255) will drop the return traffic to your clients - just doing what you tell it to do... So you would the rewrite it like this to allow the traffic in both directions and only deny your client range trying to reach the RFC1918 ranges not the replies to your client range:
permit ip any host 10.40.40.200
permit ip host 10.40.40.200 any
permit tcp any host 10.40.40.50 eq 445
permit tcp host 10.40.40.50 eq 445 any
permit tcp any host 10.30.40.126 eq 443
permit tcp host 10.30.40.126 eq 443 any
deny ip 10.0.0.0 0.255.255.255 10.0.0.0 0.255.255.255
deny ip 10.0.0.0 0.255.255.255 192.168.0.0 0.0.255.255
deny ip 10.0.0.0 0.255.255.255 172.16.0.0 0.15.255.255
permit ip any any

Re: c9800 Flex Connect Post Auth ACL Issue

Aomar bahloul — Tue, 28 May 2024 15:13:58 GMT

@Rich R

On the actual ACL I deployed I did allow traffic on both directions the ACL I provided is a normal ACL not the AP ACL.

The part I didn't figure out was the the deny statements.

Thank you for your help.

Re: c9800 Flex Connect Post Auth ACL Issue

Rich R — Tue, 28 May 2024 16:01:15 GMT

No problem - glad to help. Don't forget to mark as solution if that solved your problem.

Re: c9800 Flex Connect Post Auth ACL Issue

jcatanzaro — Thu, 30 May 2024 17:33:31 GMT

was a bug ID ever assigned to this?