IPsec over GRE and VXLAN/EVPN fragmentation support

[writeafreen]

We plan to use IPsec on 9300x and want to know what the best practice is to handle fragmentation. Should we do it on the 9300x at access or leave it for border routers? what fragmentation support is available on 9300x if any? documentation dont say much.

[ammahend]

Fragmentation is not good for any network, instead increase MTU to handle extra overhead.

[writeafreen]

We have jumbo MTU enabled on the internal network but our WAN provider do not allow that. We want to do fragmentation at access possibly 9300x if supported as our IPsec tunnels originate there and traverse WAN. Plus some additional VXLAN overhead which is currently not allowed by our WAN provider beyond 1500B etc. How do you expect us to use IPsec on 9300x if the packet size is large in this case?

[ammahend]

Well I think, If the WAN provider to not support more than 1500B then only option would be to reduce MSS small enough that with over heads your MTU is within 1500, else you might see some network performance issues as result of fragmentation.

[Joseph W. Doherty]

I don't know the features of the 9300, but ideally, on tunnel interface, you set IP MTU small enough that any downstream interface doesn't need to fragment and use IP TCP ADJUST-MSS small enough tunnel doesn't need to fragment TCP traffic (i.e. it fits in tunnel's IP MTU).

[writeafreen]

Thanks @ammahend @Joseph W. Doherty We have used TCP/MSS on few places. But it doesn't solve everything for us. I mean you have to end up reducing payload on the application side and sometimes it is not simple or possible.

With GRE or IPsec over IP and even EVPN supported in 9300x don't you see this a need in general in the enterprise with other customers as these encapsulations add overhead?

[Joseph W. Doherty]

There are other solutions, many (all?) of which depend on what your WAN provider supports.

For instance, as you've already noted, if your WAN provider supported jumbo Ethernet (or other WAN media with a MTU larger than Ethernet's) this wouldn't be problem for you.

Other technologies that extend the frame, that might be used, include MPLS or Q-in-Q or even a VPN Ethernet.

Some other technologies, though you don't "see" fragmentation, might still fragment "under the covers", where you might only notice you cannot always seem to achieve 100% line rate (because some of your bandwidth is being consumed by the fragmentation overhead).  (Ideally WAN vendor doing such, their hardware doesn't also slow because of the additional workload [good chance it won't].)