MTU/MSS confguration for Tunnel link across Encryption Devices

[Kgrevemberg]

Hi,

I'm in need of some expert help. We have an encrypted link that is having some major fragmenting issues.

I'm not an expert on this and want to learn more about it but don't have the luxury of time to fix this issue as its holding up production in a big way.

Here is a basic picture of the setup. Can anyone help with mtu sizes I should set on my encryption devices, physical interfaces, and tunnel interfaces? And also mss size if necessary, I'm still learning this stuff so I'm just not sure.

I can try my best to answer any questions.

Thanks for any and all advice.

[balaji.bandi]

Can you post the exiting configuration, have you tested using extend ping to see what is supported MTU have better ping success ?

[Kgrevemberg]

The configs are on secret devices so it would take some time to get them posted after screening but possible if you need them.

I can answer any questions you would like to ask.

If I send a df-bit ping from one end to the other it fragments after 1424.

Is that what you mean?

Thanks for your reply

[balaji.bandi]

yes that is the way to test or configure as below

ip mtu 1400

ip tcp-adjust mss 1360

[Kgrevemberg]

do you recommend I change the mtu of the cypher/plan text devices?

[Georg Pauwen]

Hello,

Cisco recommends an MTU of 1400 (and MSS of 1360), these values cover most setups. I agree with Balaji that we would need to see your configs first, but if you are using tunnel interfaces, configure:

interface tunnel 0

ip mtu 1400

ip tcp-adjust mss 1360

on both ends.

[Kgrevemberg]

my physical encryption devices have a space for mtu size as well. Do you recommend I change them also? they are at 1500 by default.

[Joseph W. Doherty]

Is the traffic being encrypted twice? (I.e. by your "physical encryption devices" and the routers supporting the tunnel.)

BTW, you may want to enable PMTUD on the tunnel routers.

You might want to also review this Cisco document: https://www.cisco.com/c/en/us/support/docs/ip/generic-routing-encapsulation-gre/25885-pmtud-ipfrag.html

[Kgrevemberg]

The routers do not have ipsec on the tunnels if I understand you correctly. they are just encapsulating.

[Joseph W. Doherty]

In that case, you still want to issues the commands already noted, but the MTU allowance is much, much less if only doing GRE encapsulation (usually just 24 bytes).

[Kgrevemberg]

I have another question. If I'm suspecting my tunnels are fragmenting the traffic in a bad way and interrupting host reach-ability, is there a test I can do to verify that? I know that I can send a df-bit from end-to-end now at 1424, how can I tell if the hosts are trying to go over that?

[Joseph W. Doherty]

Yea, tunnels are likely doing fragmentation, so might also your encapsulation devices (in other than the end hosts).

Generally, if hosts are configured for PMTUD, they will use the MTU of their interface. So, if you know certain hosts are going to have their traffic encrypted, you can either have the encryption device allows for the MTU impact (if it can) or you can reduce the MTU on the transmitting hosts.

[Kgrevemberg]

i've just been made aware of adding PMTUD to the tunnel interface? would that have a positive outcome?

Sorry if I am asking too much or seem ignorant. I'm one of the last members of a skeleton crew here and we are in a bind.

Thanks for all your help thus far.

[Joseph W. Doherty]

Last first, no need to apologize for not knowing. That's what these forums are about. ;)

As to whether adding PMTUD to the tunnel interface will help, that's a depends answer. It helps deal with tunnel packets, themselves, needing to be fragmented along the way. Further, I recall(?) if tunnel receives packets with DF set, it informs sending host that they do need fragmentation.

In your case, I suspect you might be getting double fragmentation. It's possible your encryption device fragments packets, and the the tunnel device (your Cisco router) could fragment them again. Setting the TCP adjust-mss, on your tunnel interface, might resolve all fragmentation. In this case, you would set tunnel's(?) IP MTU to just allow for the tunnel overhead, but the TCP adjust-mss would be set to allow for both the tunnel overhead and encryption overhead (even though the latter isn't done on the router).

Setting the IP MTU down on the host, to allow for both encryption and tunnel overhead would work too. However, it would do it for all traffic, not just for traffic that will be encrypted and will transit the tunnel. I.e. the TCP adjust-mss would only target the latter traffic. Also, however, if you're sending non-TCP traffic, the TCP adjust-mss won't work for that traffic.

You can also downsize the tunnel's IP MTU to allow for both tunnel and encryption overhead, but unless DF is set (unlikely for non-TCP traffic), none TCP traffic won't "know" it's too large and it will be fragmented. So, again, for such traffic, the usual way to avoid its fragmentation is to set the MTU to the smaller size on the sending host.