Fabric AP's and VXLAN tunnel to connected switch and MAC addresses?

andrew.butterworth · ‎04-20-2022

We have a SD Access lab consisting of two C9300 control-plane/border nodes connecting to a pair of C3750X switches doing the fusion role. Connected to the C3750X's are the ISE, DNS, DHCP, AD and NTP nodes, plus a C9800 WLC. There are another two C9300 Edge switches connected to the CP/B C9300's and connected to these Edge C9300's are two C9130AX fabric APs.

This was originally built with DNA 1.3 and IOS-XE 16.12.4, however it has been upgraded to DNA 2.2.3.5 and IOS-XE 17.3.5 for the C9300's and C9800 (17.3.5a).

I haven't touched it for a while, however an issue has developed on the Edge switches where the APs are connected. The APs get MAB authenticated by ISE and assigned to the INFRA_VN and register with the C9800 WLC. We have a wireless profile with an Enterprise and a Guest SSID defined. Wireless devices can connect OK - Windows AD hosts use dot1x & PEAP, Guests get redirected to a web page.

On the Edge switch we are seeing authentication failures for the MAC addresses of the wireless devices constantly. The switch see's multiple access sessions on the interface where the AP is connected and I don't think this was the case when I last played with it.

EDGE-1#sho access-session interface tenGigabitEthernet 1/0/13
Interface                MAC Address    Method  Domain  Status Fg  Session ID
--------------------------------------------------------------------------------------------
Te1/0/13                 044f.4c87.53ba N/A     UNKNOWN Unauth      434041640000678C08ECAE68
Te1/0/13                 1416.9de0.1ce4 mab     DATA    Auth        43404164000000F1DB27AB65
Te1/0/13                 68ca.e476.a9f1 N/A     UNKNOWN Unauth      4340416400000107DB3B7D2B
Te1/0/13                 989c.57a6.8101 N/A     UNKNOWN Unauth      4340416400006589083F20C2
Te1/0/13                 d037.45fd.e075 N/A     UNKNOWN Unauth      43404164000000F2DB2A566C
Te1/0/13                 d037.45fe.3b21 N/A     UNKNOWN Unauth      43404164000000F4DB2E6E8E

The AP is the only Authenticated device. The other MACs are the test wireless clients. d037.45.fe.3b21 is the MAC of a 802.1x authenticated client that appears on the INFRA_VN and the Data_VN. I didn't think the AP would bridge any client traffic to the Ethernet interface and it all should be tunneled?

EDGE-1#show mac address-table  | inc 21
1028    d037.45fe.3b21    CP_LEARN    Ac0
2045    d037.45fe.3b21    DYNAMIC     Te1/0/13

Have I misunderstood this?

Flavio Miranda · ‎04-20-2022

If I understood correctly, the WLC is not part of the Fabric?

If that is the case, you are running what cisco call OTT (Cisco Unified Wireless Network Wireless Over the Top). On that case, the Access Point keeps the CAPWAP tunnel for management, but the data plane is distributed using a Virtual Extensible LAN (VXLAN) directly from the fabric-enabled APs.

This could explain why you are seing clients mac address outside the capwap tunnel.

andrew.butterworth · ‎04-20-2022

The C9800 WLC is not inside the fabric (I think this was the recommended topology when we originally built this). The APs are registered with the WLC, however the two wireless SSID's are fabric. My understanding is the fabric AP creates a VXLAN tunnel to the Loopback0 IPv4 address of the connected switch and any data-plane traffic would appear on this tunnel (with the authentication happening over the CAPWAP tunnel).

I'm certain this is new behavior and I didn't see this occur previously and there was only a single wired access-session on the switch interface.

RDG-GND-1#sho access-session interface tenGigabitEthernet 1/0/13  details
            Interface:  TenGigabitEthernet1/0/13
               IIF-ID:  0x17C477EB
          MAC Address:  044f.4c87.53ba
         IPv6 Address:  Unknown
         IPv4 Address:  Unknown
            User-Name:  044f4c8753ba
          Device-type:  Huawei-Device
          Device-name:  HUAWEI TECHNOLOGIES CO.,LTD
               Status:  Unauthorized
               Domain:  UNKNOWN
       Oper host mode:  multi-auth
     Oper control dir:  both
      Session timeout:  N/A
    Common Session ID:  434041640000678C08ECAE68
      Acct Session ID:  Unknown
               Handle:  0x53000015
       Current Policy:  PMAP_DefaultWiredDot1xLowImpactAuth_1X_MAB


Server Policies:


Method status list:
       Method           State
        dot1x           Running

----------------------------------------

            Interface:  TenGigabitEthernet1/0/13
               IIF-ID:  0x14749497
          MAC Address:  1416.9de0.1ce4
         IPv6 Address:  Unknown
         IPv4 Address:  100.65.1.33
            User-Name:  14-16-9D-E0-1C-E4
          Device-type:  Cisco-CAT-LAP
          Device-name:  cisco C9130AXI-E
               Status:  Authorized
               Domain:  DATA
       Oper host mode:  multi-auth
     Oper control dir:  both
      Session timeout:  N/A
  Acct update timeout:  172800s (local), Remaining: 88795s
    Common Session ID:  43404164000000F1DB27AB65
      Acct Session ID:  0x0000000d
               Handle:  0xc5000010
       Current Policy:  PMAP_DefaultWiredDot1xLowImpactAuth_1X_MAB


Local Policies:

Server Policies:
           Vlan Group:  Vlan: 2045


Method status list:
       Method           State
        dot1x           Stopped
          mab           Authc Success

----------------------------------------

            Interface:  TenGigabitEthernet1/0/13
               IIF-ID:  0x12BD8950
          MAC Address:  68ca.e476.a9f1
         IPv6 Address:  Unknown
         IPv4 Address:  10.0.2.128
            User-Name:  68cae476a9f1
          Device-type:  Cisco-Device
          Device-name:  CISCO SYSTEMS, INC
               Status:  Unauthorized
               Domain:  UNKNOWN
       Oper host mode:  multi-auth
     Oper control dir:  both
      Session timeout:  N/A
    Common Session ID:  4340416400000107DB3B7D2B
      Acct Session ID:  Unknown
               Handle:  0x7c000013
       Current Policy:  PMAP_DefaultWiredDot1xLowImpactAuth_1X_MAB


Server Policies:


Method status list:
       Method           State
        dot1x           Running

----------------------------------------

            Interface:  TenGigabitEthernet1/0/13
               IIF-ID:  0x1017C16C
          MAC Address:  989c.57a6.8101
         IPv6 Address:  Unknown
         IPv4 Address:  Unknown
            User-Name:  989c57a68101
          Device-type:  Huawei-Device
          Device-name:  HUAWEI TECHNOLOGIES CO.,LTD
               Status:  Unauthorized
               Domain:  UNKNOWN
       Oper host mode:  multi-auth
     Oper control dir:  both
      Session timeout:  N/A
    Common Session ID:  4340416400006589083F20C2
      Acct Session ID:  Unknown
               Handle:  0x9a000014
       Current Policy:  PMAP_DefaultWiredDot1xLowImpactAuth_1X_MAB


Server Policies:


Method status list:
       Method           State
        dot1x           Running

----------------------------------------

            Interface:  TenGigabitEthernet1/0/13
               IIF-ID:  0x13E1F42F
          MAC Address:  d037.45fd.e075
         IPv6 Address:  Unknown
         IPv4 Address:  Unknown
            User-Name:  d03745fde075
          Device-type:  TP-LINK-Device
          Device-name:  TP-LINK TECHNOLOGIES CO.,LTD.
               Status:  Unauthorized
               Domain:  UNKNOWN
       Oper host mode:  multi-auth
     Oper control dir:  both
      Session timeout:  N/A
    Common Session ID:  43404164000000F2DB2A566C
      Acct Session ID:  Unknown
               Handle:  0x99000011
       Current Policy:  PMAP_DefaultWiredDot1xLowImpactAuth_1X_MAB


Server Policies:


Method status list:
       Method           State
        dot1x           Running

----------------------------------------

            Interface:  TenGigabitEthernet1/0/13
               IIF-ID:  0x1CBF2BFE
          MAC Address:  d037.45fe.3b21
         IPv6 Address:  Unknown
         IPv4 Address:  Unknown
            User-Name:  d03745fe3b21
          Device-type:  TP-LINK-Device
          Device-name:  TP-LINK TECHNOLOGIES CO.,LTD.
               Status:  Unauthorized
               Domain:  UNKNOWN
       Oper host mode:  multi-auth
     Oper control dir:  both
      Session timeout:  N/A
    Common Session ID:  43404164000000F4DB2E6E8E
      Acct Session ID:  Unknown
               Handle:  0x95000012
       Current Policy:  PMAP_DefaultWiredDot1xLowImpactAuth_1X_MAB


Server Policies:


Method status list:
       Method           State
        dot1x           Running

Flavio Miranda · ‎04-20-2022

Well, according to Cisco docs, OTT, which is your case as it seems, i may be wrong, "Data plane is locally switched. Wireless traffic is treated like wired traffic."

For you reference:

Slide 71:

https://www.ciscolive.com/c/dam/r/ciscolive/latam/docs/2018/pdf/BRKEWN-2020.pdf

andrew.butterworth · ‎04-20-2022

No, the WLC is integrated into the fabric, its just physically outside the fabric domain (page 69 on the .pdf link you sent me).

The SSID's are Fibric so I think all the client data plane traffic should be within the VXLAN tunnel to the local switch rather than appearing on the 'wire'.

markus.forrer · ‎04-27-2022

Just facing the same issue after upgrading the WLC to 17.6.3

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCwb68720

Try the workaround mentioned in the bug notice:

remove switchport authentication will avoid the issue

andrew.butterworth · ‎07-26-2022

Fixed in WLC IOS-XE 17.3.5b