BGP keeps flapping due to high bandwidth utilization.

[NetworkingGeek1]

Hello experts,

There is a router with interface with 10 Mbps bandwidth. It's MPLS link. Because of high bandwidth utilization of that interface on transmit direction, BGP is flapping frequently. Can I apply some QoS on interface to prioritize BGP control plane traffic? And I thought that Cisco has some default control plane packets bandwidth reservation, which should prioritize control plane traffic on the interface, even if QoS is not configured?

Thank you in advance.

[balaji.bandi]

Rather than saturating the link 100% why not apply QoS to use only 9.5 MB or based on the requirement?

[NetworkingGeek1]

Hello @balaji.bandi  There is some QoS in place, but I think BGP is treated there as class-default. And regarding your question, I can't reply, because this as it was designed there. I saw your reply in: https://community.cisco.com/t5/routing/bgp-flapping/td-p/4446108   - Could please clarify where did you get this table from?

Basically, I have 2 questions:

• 1. Doesn't Cisco by default reserve some bandwidth for control plane traffic? Seems that no.
• 2. What can be done in my case? Do I need to configure some special class for BGP in existing QoS?

[balaji.bandi]

it all depends on what router you using and IOS code running.

that is one of the presentations from my docs of the archive.

in the same thread, you can have an example config also.

[Joseph W. Doherty]

"@Peter Paluch your reply will be especially appreciated, since you're my favorite author"

Peter is my favorite author too, although, I suspect, QoS is possibly the only area where I might have a bit more expertise in the subject, even though I'll not be able to express such expertise as well as he could.

"Can I apply some QoS on interface to prioritize BGP control plane traffic?"

Probably.

"And I thought that Cisco has some default control plane packets bandwidth reservation, which should prioritize control plane traffic on the interface, even if QoS is not configured?"

Possibly you are thinking of pak-priority.

The referenced document mentions: "In contrast, the Border Gateway Protocol (BGP) instructs TCP to mark its traffic with IP precedence 6, but does not set pak_priority."

So, if that's the case insuring you are just not lumping BGP into a single FIFO queue (especially on a highly utilized link) is likely a very good idea.

[NetworkingGeek1]

Hello @Joseph W. Doherty   Thank you for the reply. Yes, you're right, I'm not asking about protecting the control plane itself (CoPP).

Since there a lot of traffic and bandwidth is low, BGP packets are getting dropped on the router interface itself and that's why BGP is flapping. My questions are: 1. If Cisco has some default bandwidth reservation for control plane traffic (like BGP). Because it looks it doesn't. 2. What can be done in my case? Should I configure QoS on the Router's interface to give BGP packets priority? I'll check your link.

@Leo Laohoo Thanks for the reply. I don't need upstream provider to support QoS. Because I don't need end-to-end QoS here. BGP packets are getting dropped on Router's WAN interface, I just need to give them priority to leave interface as quick as possible.

[Leo Laohoo]

If the WAN link is 10 Mbps, why not configure the port for 10 Mbps?

[NetworkingGeek1]

Yes, it's really better to set it 10 Mbps, now speed is 100 Mbps, even though real bandwidth is 10 Mbps. But how will it improve this situation? Anyway, there a lot of traffic needs to leave the interface and BGP packets still be treated as FIFO (first in, first out). I don't think changing the speed will help.

[Joseph W. Doherty]

I cannot speak for Leo, why he suggested 10 Mbps, but yes there's a chance it would help.

Why?

Well, by moving your 10 Mbps to your interface, it may, by default, do a better job of buffering the congestion than a downstream bottleneck of which you have no direct visibility into or control of.  Of course, there's also the chance of it making no change or actually making the situation worst.  I.e. possibly worth trying to see the result.

That said, I too will often suggest running an interface at its physical bandwidth if it matches the downstream bottleneck.  The reason, for my suggesting this, QoS works better with a physical interface than with a shaper (which tries to mimic the former).

In your case, if you see the possible situation of upgrading your logical cap above 10 Mbps, but not to 100 Mbps, then you probably would want to shape for the logical cap bandwidth, even though, currently, it's a physical interface "speed".

[Georg Pauwen]

Hello,

I have not followed the entire thread, but I guess the simplest way to proritize BGP would be to match CS6:

class-map match-all BGP

match dscp 6

policy-map BGP_Policy

class BGP

priority level 1

[Joseph W. Doherty]

Matching and treating CS6 marked traffic, would be an appropriate approach, although I wouldn't recommend using an LLQ class for BGP.

As it being the "simplest way", possibly on a device supporting FQ, FQ in class default might be even simpler, i.e. no need for a class-map or non-default class.

[Joseph W. Doherty]

"1. If Cisco has some default bandwidth reservation for control plane traffic (like BGP). Because it looks it doesn't."

Yes, agreed, it appears Cisco does not provide special treatment for BGP.

"2. What can be done in my case? Should I configure QoS on the Router's interface to give BGP packets priority?"

Yes, using QoS above and beyond whatever the device is doing now would, generally, be an appropriate approach.  Unsure you really need, so much, to "prioritize" BGP, possibly the need is more to avoid those packets being dropped.  Either or both usually can be accomplished via QoS.

One wrinkle to your problem, you're using MPLS.  I've never used native MPLS on an interface, and so, off-the-top-of-my-head, don't know whether standard interface QoS works a bit differently when using MPLS.  I "know" MPLS experimental 3 bit field can be used for a QoS setting, but don't recall if an IP packet's ToS, by default, maps into it (perhaps using the IPPrec bits) or whether you need to configure a ToS to MPLS experimental mapping.

Further, in your case, I suspect class-default using FQ (as you mention using a router) might solve the issue, but here too, unsure interface QoS would still see multiple flows or consider all the MPLS tagged frames similar to a GRE tunnel, i.e. only one flow.

If the latter, likely you could still treat your BGP traffic "special" (similar to what @Georg Pauwen posted), but again, we need to confirm who/when you convert IP ToS to MPLS Experimental.

BTW, a very important question, is the 10 Mbps a logical cap or the physical interface bandwidth?  If the latter, we'll need to shape the egress traffic to 10 Mbps too.

Oh, and to be clear, we need to apply an egress interface policy on the device providing the 10 Mbps data stream.

What's the egress platform and its IOS version?

[NetworkingGeek1]

@Joseph W. Doherty  Sorry, I think confused you a little bit, it's not natively MPLS interface. By MPLS I meant that it's connected to PE router, which implements MPLS. So, my router is CE, which is not aware of MPLS.

1. Interface itself is Gigabit, but speed is 100 Mbps there. So, 10 Mbps is a logical cap. So, speed is 100 Mbps, bandwidth is set to 10000 (10 Mbps). "BTW, a very important question, is the 10 Mbps a logical cap or the physical interface bandwidth?  If the latter, we'll need to shape the egress traffic to 10 Mbps too." Maybe you meant not latter, shaping is needed if it's logical cap?

Platform: 2921; IOS: 15.1, it's regular IOS, not XE.

[Joseph W. Doherty]

Ah, a CE router - great, have worked quite a bit with those.

"Maybe you meant not latter, shaping is needed if it's logical cap?"

Correct, worded that backwards.

Understand gig interface running at 100 Mbps.  Ok, next question.  The 10 Mbps cap to applied to all bandwidth from CE to PE or from your CE to another site's CE?  I.e. Do you have multiple CEs that intercommunicate?  If so, I need to understand you topology a bit better, especially how your MPLS SP deals with bandwidth caps in/out of their MPLS cloud.  (In the past I worked with tier one or two MPLS national and international providers.)

PS:

Also are your BGP peers, your MPLS vendor's or your other sites?

[NetworkingGeek1]

Thank you so much for the help. I'll try summarize everything here:

Router: ISR 2921; IOS: 15.1, it's regular IOS, not XE.

Problem: BGP flapping during particular hours, when link is overutilized (probably by some running back up). Interface which are experiencing the overutilization issue is facing Service Provider, so it's WAN interface. Interface itself is Gigabit, but speed set to 100 Mbps there. But actual bandwidth which is given by Service Provider is 10 Mbps. Bandwidth command is set to 10000 (10 Mbps). BGP peering is with Service Provider (as you described - MPLS vendor). There is only one BGP neighbor, which is SP's router. We don't have multiple CEs that intercommunicate. It's our CE connection to SP's PE. Therefore 10 Mbps cap to applied to all bandwidth from CE to PE. Also I want to mention that on the WAN interface we have subinterface with dot1q tag and actually this sub interface is connected to SP (so, most likely SP is EVPN, not MPLS). There is only one subinterface configured on that interface.