Voice Site-to-Site MTU Concerns

[gilbert.aispuro1]

Hello beautiful people!

I'm looking to get feedback on how people are utilizing MTU and TCP MSS on their sub-interfaces. 

I'm having sporadic issues with my ShoreTel communication from my Branch sites over to my DC where the PBX is installed. ShoreTel stated that I'm having MTU issues from Branch to DC and say I need to make sure I have at least 1400 MTU for communication.

From SD-WAN router to router, I'm able to ping just fine with packet size of 1400 with the DF-Bit set sourcing from my Voice gateway interface over to my DC PBX.

All my sub-interfaces are set to 1496 and I have a TCP MSS-Adjust of 1370.(This was set by Pro Services on my initial site install, so I just copied that config to others.)

When I ping from my Branch voice appliance over to my DC PBX, I get drops starting at packet size of 1373

Can someone assist please. Thanks!!!

[Joseph W. Doherty]

Generally, VoIP bearer packets are small enough the fragmentation for them is unusual. VoIP control traffic, which unlike the bearer packets, is likely TCP, probably are smallish.

If you're having communication issues, you've eliminated issues dealing with VoIP's latency, jitter and very low packet loss requirements?

BTW, a MTU size of 1496 is unusual, perhaps it's allowing for a MPLS label which hasn't increased the frame size (as it should)? (NB: PPPoE often uses a MTU of 1492.) Also, tcp mss-adjust is generally set to be IP MTU less 40.

[gilbert.aispuro1]

I appreciate the feedback Joseph!

Yes, I've checked all other possible issues and just needed to check the SD-WAN cEdge just in case. 

As for the 1496, I needed to use 4 for the use of Sub-Interfaces, as described in the documentation. 

As for the TCP MSS, should I be able to raise this to 1400 without any issues, since you stated it's usually 40 less the MTU size? My hesitation with this is I have to make sure I account for the SD-WAN IPsec site-to-site tunneling payload as well but I can't find an exact number on how much that payload utilizes anywhere in the documentation. Not sure if there is a command that shows that info. 

Let me know if there is any more info you need from me. 

Thanks! 

[Joseph W. Doherty]

Hmm, what kind of technology for your subinterfaces? Reason I ask, generally VLAN tags are used and they extend the frame size. I.e. you don't need to reduce the standard Ethernet IP MTU of 1500.

IPSec, on the other hand, I recall (?) does reduce the available IP MTU. Size requirements vary a bit whether using tunnel mode or not. (I also recall Cisco VTI IPSec tunnels don't have GRE overhead and can be configured to use, or not, tunnel mode.) I further recall a "safe" allowance is using 1400 for the IP MTU and 1360 for the MSS (both 100 bytes less the normal.)

Some more info on this subject can be found in Cisco's white paper: https://www.cisco.com/c/en/us/support/docs/ip/generic-routing-encapsulation-gre/25885-pmtud-ipfrag.html

[gilbert.aispuro1 · Accepted solution]

Here is what I read, under Configure Subinterfaces and VLANs : 

https://sdwan-docs.cisco.com/Product_Documentation/Software_Features/Release_18.1/02System_and_Interfaces/06Configuring_Network_Interfaces

Ha, that exact same document shows the payload of its tunnels. If this is correct, I should be okay moving TCP MSS to 1400:

On vEdge routers, you must configure the tunnel encapsulation. The encapsulation can be either IPsec or GRE. For IPsec encapsulation, the default MTU is 1442 bytes, and for GRE it is 1468 bytes, These values are a function of overhead required for BFD path MTU discovery, which is enabled by default on all TLOCs. (For more information, see Configuring Control Plane and Data Plane High Availability Parameters.) 

[Joseph W. Doherty]

Ah, didn't realize there's (much) more involved than just IPSec overhead.