PPPoE MTU & ICMP TBP message

[devinliu1991]

Hi,all

When we deploy PPPoE in our network, the "ip mtu 1942" command is necessary to be typed on the outgoing virtual interface.

To find the reason, I had done such a lab as shown below.

(1)Topology

(2)Configuration

PPPoE was deployed between R2 and R3 with R2 being the PPPoE client and R3 the server.

Static routing was configured on each router to make R1 capable of reaching the loopback interface of R4  4.4.4.4 with the source 1.1.1.1.

The mtu value of the outgoing interface fa1/0 on R2 was 1500 as normal.

(3)Phenomenon

When I issued the command "ping 4.4.4.4 source lo 0 repeat 1 size 1500 df-bit" on R1, the icmp packet was dropped by R2 and I got a "." on R1.

After I adjusted the "ip mtu" value of the dial interface on R2 with 1499, and had the ping test again, I got an "M" which means that R1 had received the ICMP TBP message.

(4)Question

When I leave the mtu value of the dial interface to be default, why the PPPoE router merely drops the packet and won't send the TBP message back?

I have also had a further lab with the same topology but R2, R3, R4 run mpls. When ICMP packets to the loopback interface of R4, which come from R1 and have a size of 1500 bytes, are forwarded by R2, R2 will send back the TBP message because of the label inserting issue.

So, why PPPoE won't but MPLS will ?

[ghostinthenet]

MPLS adds four bytes on top of the 1500-byte packet, giving you a 1504-byte MPLS packet and therefore no need to manage the path MTU. PPPoE, on the other hand, subtracts eight bytes from the 1500-byte packet. This leaves only a 1492-byte packet and introduces the need to manage the MTU along the path.

In order to enable efficient MTU management on these links, add "ip tcp adjust-mss 1452" on the PPPoE interfaces. This will cause the routers to send ICMP packet-too-big messages to sources of packets larger than 1492 bytes that attempt to traverse these links.

[M SEB]

Hi Jody,

I beg to differ on your "ip tcp adjust-mss 1452" explanation. 

This command "ip tcp adjust-mss 1452" will rather "clamp" the MSS value exchanged during the TCP 3 way handshake (tcp datagrams with SYN flag present). 

This means during first SYN sent from client, the MSS value is rewritten from what is was (commonly 1460 if standard ethernet network -> 1500-40). The corollary being the recomputation of tcp checksum. 

What causes the router to send an ICMP packet too big (in IPv6) or ICMP fragmentation needed (IPv4, type 3 code 4) is rather the PATHMTUD mechanism when MTU on path is too low and df bit is set to 1 in ip header. 

[devinliu1991]

Yeah, I agree with M SEB, PMTUD mechanism uses ICMP to adjust the mtu size, and on the other hand, Adjust-mss works like an agent.

The reason why a router would send an ICMP message back to the source is that the payload size of a packet is bigger than the mtu value of the outgoing interface. 

In my opinion, both MPLS and PPPoE need to add extra tag to the frame, on is label with 4 bytes, and the other is "PPPoE header plus PPP header" with 8 bytes.

Why in the MPLS case, a router could find the mtu problem and send back the ICMP message, while in the PPPoE case, the router seems not able ?

Is that because the "PPPoE head plus PPP header" is considered as the layer 2 encapsulation so that the mtu calculation is started with the IP layer

and in the MPLS case, label is calculated in mtu along with IP layer and above

or is that because of the virtual interface used in PPPoE 

[ghostinthenet]

This is what happens when I post while on vacation. I get out of the head space and start mixing things up. :)

Yes, you're right. The MSS clamping can produce similar effects for TCP traffic, but it isn't the same as PMTU discovery. For PMTU discovery to work, you need to turn off ICMP rate limiting for the packet-too-big messages on each router in the path. The routers will automatically send the message when they drop the packet and as long as there's nothing limiting the reply, you should be good.

MPLS uses a 1504-byte Ethernet frame and so the payload remains 1500 bytes, so there's no need to adjust the MTU from 1500. PPPoE uses a 1500-byte Ethernet frame, leaving only 1492 bytes for the payload, necessitating the need for MTU adjustment along the way.

Theoretically, if you ran PPPoE over a 1508-byte Ethernet frame, you could run it with a fixed payload MTU of 1500 bytes, but I've never tried this.

[devinliu1991]

Hi, Jody

By “MPLS uses a 1504-byte Ethernet frame and so the payload remains 1500 bytes” you mean that when I deploy MPLS, the mtu capacity of the “mpls ip” interface will automatically change to fit the length of the payload？because there could be more than one label being added.

[ghostinthenet]

Ideally, the MPLS MTU should be increased by four bytes for each potential label you're adding, so 1504 for MPLS and 1508 or more for MPLS VPNs. If you're not manually configuring the MPLS MTU, it's worth doing a packet capture to see what your hardware is doing.

[devinliu1991]

In the GNS environment ，the MPLS mtu value won't adjust and I have got an ICMP message to notify that the payload size is greater than the outgoing link and of course the packet was dropped.

But in PPPoE environment，the packet was merely dropped without notifying the source.