Problem with packets size after changing MTU size

[AdrienF]

Hello !

I have an issue with the size of the packets transmitted in my network. I have 2 routers linked through serial interfaces. Behind each router, I have a computer. I'm generating a 3Mb TCP flow with iPerf, from PC1 to PC2. The MTUs of all my interfaces and PCs are at 1500 bytes and it works well. When iPerf is done, I now change the MTU of the router1 (the one after PC1) to 1000 bytes. I then relaunch a 3Mb flow of TCP with iPerf and everything works fine again, the size of my packets are 1000 bytes. When it's done, I switch back the MTU that I modified to 1500 bytes. Then I relaunch the same flow with iPerf and the size of my packets is 1000 bytes. I've checked all my MTUs and my MSS and they are all set to 1500 bytes.

What I understood is that it tries to transmit data to the router1 with a don't fragment flag and the router responds with an ICMP packet that he needs to fragment, because his MTU is lower than the packets from the PC1 (still at 1500). And after (when I increase again my MTU back to 1500), it's never checked and increased because it satisfies the condition of being sent without being fragmented.

I don't know if my problem is clear, don't hesitate if you have questions or stuff. I also have the wireshark captures for further informations.

PS: sorry if it's not in the good category, I'm a beginner, so I might have a bad vocab.

Thanks,

Adrien

[M02@rt37]

Hello @AdrienF,

When you initially set the MTU of router1 to 1000 bytes, the ICMP "need to fragment" messages are sent back to PC1 indicating that the packets need to be fragmented to fit the lower MTU. PC1 receives these messages and adjusts the packet size accordingly to avoid fragmentation, resulting in packets with a size of 1000 bytes.

However, when you switch back the MTU of router1 to 1500 bytes, the ICMP "need to fragment" messages are not triggered because the packets from PC1 are already below the MTU of router1. As a result, PC1 continues to send packets with a size of 1000 bytes.

[AdrienF]

Hello M02@rt37 !

Thanks for your answer and your explanations ! I get it now. I saw the icmp packet saying to the PC1 that he needs to fragment. Do you have any idea of how can I make my packets size increase back to 1500 when I increase my MTU back to 1500?

I've thought that maybe modifiying the don't fragment flag in the packets would change something but I'm not sure at all ..

Best regards,

Adrien

[M02@rt37]

You're welcome @AdrienF,

Restart the network interface on PC1 after setting the MTU back to 1500. This will ensure that any cached information is cleared and the new MTU value is taken into account.

The "don't fragment" flag is used to indicate that a packet should not be fragmented if it exceeds the maximum size allowed along the path. It's generally best to let PMTUD handle the fragmentation and packet sizing automatically.

[Joseph W. Doherty]

"It's generally best to let PMTUD handle the fragmentation and packet sizing automatically."

It's certainly common, but "best", in many cases might be debatable.  ; )

[Joseph W. Doherty]

Possibly, your second test started before your PC's PMTUD timer expired.

[AdrienF]

Afternoon M02@rt37 @Joseph W. Doherty,

Thanks for your help and your time !

About the PMTUD timer, it's kinda weird, because I usually wait 2 to 3 minutes between each test and from what I saw the timer is set to 15 secs.

But I think I understood almost everything now. I have two solutions, disabling PMTUD or reseting my interface every time I change my MTU. I'll look for both solutions, and see which one is the best for my problem, but from what I saw, reseting interface seems better as you said M02@rt37.

Once more, thanks !

Adrien

[Joseph W. Doherty]

15 sec timer, eh?  Hmm, I've read 10 minutes is a common time out, but there's, I believe, no standard, so a host can use whatever value it wants.

[AdrienF]

Hello @Joseph W. Doherty,

I checked on my Linux computer and it was written 15 that's why I thought 15 secs, but yeah it seems more coherent if it is minutes. My bad ... I jumped to conclusions too quickly ..

I'll check what's the best solution in my case !

Thanks a lot for your help.

Adrien