Solved: BGP Flapping and MTU

sarahr202 · ‎02-17-2015

Hi everybody

I hope all my old pals ( no pun intended :) are doing fine. Today at work I encountered a strange issue, one of customer opened a ticket for BGP flapping.

It was only when I match the MTU of CE on PE router did bgp stabilize I asked of my senior co worker but he did not answer why. A lot of cisco documentation on google did not go into details why mismatched MTU could cause flapping.

I am reaching out to you guys may be someone knows and enlighten me.

Much appreciated

thanks

.

Peter Paluch · ‎02-18-2015

Hi Sarah,

You're welcome as always :)

Regarding the BGP flapping - can you be more specific? What exact kind of flapping was that? Were you getting any BGP notifications, or BGP hold intervals being exceeded? Any more technical information, such as logging messages, would be greatly helpful.

Speaking in an extremely general sense and most probably saying nothing new, MTU mismatches on a common link will cause the router with the smaller MTU to discard all incoming IP packets whose total size exceeds this smaller MTU. For unreliable protocols, this would cause irreparable packet losses. For TCP, the situation is more complex: each router opening a TCP session should advertise a TCP Maximum Segment Size (MSS) derived from its own MTU on the egress interface. The router with the smaller MTU would advertise its MSS equal to its smaller MTU minus 40, telling the other router to send TCP segments that would, after adding TCP and IP headers, be exactly MTU bytes long, so the other router would never send TCP segments that would be oversized. So here, despite the MTU mismatch on the same link, the TCP communication between the two routers should actually work.

Just wondering, what was the exact MTU value you had to configure? Was it significantly different from the default of 1500 bytes?

Best regards,
Peter

View solution in original post

Peter Paluch · ‎02-18-2015

Hi Sarah,

You're welcome as always :)

Regarding the BGP flapping - can you be more specific? What exact kind of flapping was that? Were you getting any BGP notifications, or BGP hold intervals being exceeded? Any more technical information, such as logging messages, would be greatly helpful.

Speaking in an extremely general sense and most probably saying nothing new, MTU mismatches on a common link will cause the router with the smaller MTU to discard all incoming IP packets whose total size exceeds this smaller MTU. For unreliable protocols, this would cause irreparable packet losses. For TCP, the situation is more complex: each router opening a TCP session should advertise a TCP Maximum Segment Size (MSS) derived from its own MTU on the egress interface. The router with the smaller MTU would advertise its MSS equal to its smaller MTU minus 40, telling the other router to send TCP segments that would, after adding TCP and IP headers, be exactly MTU bytes long, so the other router would never send TCP segments that would be oversized. So here, despite the MTU mismatch on the same link, the TCP communication between the two routers should actually work.

Just wondering, what was the exact MTU value you had to configure? Was it significantly different from the default of 1500 bytes?

Best regards,
Peter

sarahr202 · ‎02-19-2015

Hi Peter

Sorry about the late response, so CE was a cisco router with IP MTU 1500, PE was a juniper with default physical MTU 9640, the error message i saw was " Hold timer expired" every few minutes, i set up the MTU on PE IP MTU 1500, BGP stabilized.

I enjoy your detail answers as usual.

Have a great day

Ethan55 · ‎10-27-2020

Hi Sarah,

Do your PE and CE connect directly?

I see that if we connect PE (Juniper) and CE (Cisco) directly then the BGP session will be stable.

Can you provide more detail about your topo and BGP config?

Ethan55 · ‎10-27-2020

Hi Peter,

I don't know why a BGP session can flap when there is a mismatch MTU between 2 BGP speakers.

I can not reproduce this case in the lab.

If I configure MTU of an interface large enough in order to forward BGP messages (Almost BGP messages have a size smaller than 1000 bytes) then the BGP will be flapped or not?