Preserving SGTs across a WAN that supports MTU of 1500 only

[Redman1804]

Hi Everyone

Hope you can help me with this query.

Hello

I have a customer for whom we are proposing Software-Defined Access and for whom micro-segmentation will be a key selling point.

One of their remote sites is connected to a WAN that only supports an MTU of 1500 meaning the transit to this location will have to be an IP transit.

A high-level illustration of this connectivity is shown below:

MAIN_SITE: <FE>---<Border>--<Fusion>---(WAN)--<Fusion>---<Border>--<FE> Remote_Site:

The FE will be a Catalyst 9300 whilst everything else will be Catalyst 9500.

Since it won't be possible to implement VXLAN across the WAN due to MTU < 1500 and since the VXLAN header is what is used to transport SGTs, I'd like some guidance on techniques for preserving SGTs end-to-end across this network.

I've found some documentation on SXPs but it's not fully clear on how this should be implemented. Do I need to allow for additional hardware at each location to support the preservation of SGTs between both locations?

[Preston Chilcote]

There is a plan for that!  Best resource I found that explains it is:

https://community.cisco.com/t5/networking-documents/cisco-sd-access-for-distributed-campus-with-cisco-sd-access-as-a/ta-p/3837269#toc-hId--1703369038 specifically the "MTU Settings for Cisco SD-Access for Distributed Campus with Cisco SD- Access as a Transit" section 

In short, you can use "ip tcp adjust-mss" command so that the pre-encapsulated traffic is small enough to allow for the vxlan header.  However, this doesn't help UDP traffic.  Luckily large UDP packets are rare enough that it's usually a sufficient solution.

[Damien Miller]

I would say it's a recipe for disaster. Any network utilizing sgt's is likely also using eap-tls endpoint authentication. In this case it's extremely likely that certs greater than 1500 bytes (2048+ size is the norm now) will be sourced in the return Radius stream from the NAD to ISE. The switch will try to send this packet stream at its configured system mtu, usually 1500+, as such the initial UDP datagram containing the majority of the cert would be discarded.

You see this behavior from ISE as repeated dot1x failures. It's a critical failure in the traffic flow.

This is actually less of an issue with wireless, wlcs usually have a default radius mtu of 1300 bytes, they pre fragment the traffic. Not something I've seen implemented on switches, so still an issue.