ASR9K BFD over Bundle (BoB) limitation

[franklaszlo]

Hi There,

According to the Routing Configuration Guide for Cisco ASR 9000 Series Routers, IOS XR Release 7.5.x and also in a very nice article in this topic it is mentioned that, I quote "BoB does not provide a true Layer 3 check..."

Can someone explain what does this exactly means and in what circumstances should this be considered a limitation ?

Thank you,

Laszlo

[M02@rt37 · Accepted solution]

Hello @franklaszlo 

The statement "BFD over bundle does not provide a true Layer 3 check" refers to the fact that when BFD is configured over a link bundle or port channel (commonly known as a "bundle"), the BFD protocol operates at L2 and may not perform in-depth L3 checks.

To go further,  a "true L3 check" would involve inspecting the L3 (IP) header of packets. This is typically done by routing protocols when making forwarding decisions based on IP addresses.

BFD is a protocol designed for fast detection of failures in the forwarding path between two routers. It operates at a lower layer (usually L3) but can be used in various networking scenarios. When BFD is configured over a link bundle (a group of physical links aggregated into a logical interface), it may not perform extensive L3 checks because link bundles are often associated with L2 technologies, such as EtherChannel or Port Channel.

Note that in scenarios where a link bundle is used for redundancy or increased bandwidth, BFD can help quickly detect link failures and trigger fast convergence. The statement suggests that if your network relies on detailed L3 checks for forwarding decisions, using BFD over a bundle might not provide the same level of L3 visibility as using BFD directly on individual interfaces.

Limitation ?
The limitation here is that, due to the nature of link bundles (which are often associated with L22 technologies), BFD over a bundle might not inspect L3 informaton as thoroughly as some routing protocols do.
If your network design relies on specific L3 checks, you may need to consider alternative solutions or configurations.

[franklaszlo]

So when one says that "BFD over bundle does not provide a true Layer 3 check" then this statement is made by comparing BoB to a routing protocol and not to BLB or BOB-BLB coexistence mode, right? Ultimately this is also true then to any BFD variant I assume, hence while I do understand your reasoning and accept it as a solution, I still find this statement a bit misleading.
Based on above - and unless I am still missing something - I believe this is can't be taken as an argument against BoB, and does not explain why BoB-BLB coexistence mode should be preferred instead.

As I also highlighted in my other reply, I ultimately want to understand what advantage BoB-BLB coexistence provide over BoB ?

[M02@rt37]

@franklaszlo 

I understand your point of view now. The statement "BFD over bundle does not provide a true Layer 3 check" is likely comparing BFD in the context of BoB to traditional routing protocols that perform comprehensive L3 reachability checks.

BFD itself is designed for fast link failure detection and might not provide the same level of thoroughness in L3 reachability validation as some routing protocols. However, this limitation is not unique to BoB; it applies to BFD in general.

The preference for BoB-BLB coexistence mode over simple BoB is usually driven by considerations specific to the behavior of BLB and the desire to mitigate certain drawbacks. It's not about addressing a deficiency in BFD itself but rather about optimizing the behavior of the bundle in the presence of BFD-triggered events.

Great discution !

[Ramblin Tech]

BoB is Cisco's name for RFC 7130 (aka, Micro BFD) and provides integrity checking for the individual links in a LAG bundle. BoB's client is not a routing protocol (eg, BGP, OSPF, static, etc), but the the bundle manager process. When BoB detects a link failure, it notifies the bundle manager process, which then removes the link from the bundle. LACP checks the integrity of individual links as well, but LACP timers are longer than when BoB is offloaded to hardware (ie, runs in an NPU).

By contrast, BFD over Logical Bundle (BLB) treats the entire LAG bundle as a single logical link and does not check the integrity of each individual link. BLB's client is a routing protocol and operates much like ordinary BFD over a physical link. When BLB detects a failure, it notifies the routing protocol client, which can begin its reconvergence on the new topology. This is the L3 functionality that BoB does not provide. BoB and BLB can work together on a LAG bundle to check the integrity of both the individual links in the bundle and the bundle as a whole.

References:

https://community.cisco.com/t5/service-providers-knowledge-base/bfd-support-on-cisco-asr9000/ta-p/3153191

https://community.cisco.com/t5/service-providers-blogs/bfd-over-logical-bundle-blb-implementation-on-ncs5500-platforms/ba-p/3309345

https://xrdocs.io/ncs5500/tutorials/bfd-architecture-on-ncs5500-and-ncs500/#:~:text=BFD%20Feature%20Support,-BFD%20is%20supported&text=Static%2C%20OSPF%2C%20BGP%20and%20IS,are%20supported%20in%20IPv4%20BFD.&text=BFD%20over%20BVI%20is%20supported,damp....

https://xrdocs.io/ncs5500/tutorials/bfd-over-bundle-interfaces-on-ncs5500-and-ncs500/

https://xrdocs.io/ncs5500/tutorials/multipath-and-multihop-bfd-sessions-on-ncs5500-and-ncs500/

https://xrdocs.io/ncs5500/tutorials/coexistence-between-bfd-over-bundle-and-bfd-over-logical-bundle/

[franklaszlo]

BoB cheks the links individually and notifies the bundlemgr, which will in
turn bring down the whole bundle when the number of operational links is
below the set limit, and this can trigger the routing protocol failover.
Actually the bfd microsession does L3 reachability checks to my
understanding.

Therefore, in my view this is not an explanation to the statement, that
"BoB does not provide true L3 checks" and primarily it does not explain
why would BoB-BLB coexistence (inherited or logical mode) be preferred over
simple BoB configuration.

BLB itself has a clear disatvantage, that it would declare the whole bundle
down if the link over which the BFD session was hashed goes down. Thus
BoB-BLB coexistnece has an obvious advantage over a simple BLB.

But what advatage does the BoB-BLB coexistence mode provide over BoB ? Why
is that better, or in what situation is it better than BoB ? The only
reasoning I found is that - once again - "BoB does not provide true L3
checks". But this still sounds a vague statement to me.

[M02@rt37]

@franklaszlo 

The advantage of BoB-BLB coexistence over simple BoB lies in handling BFD failures more gracefully. While BoB relies on BFD for link monitoring, the coexistence mode allows the bundle to better tolerate BFD-related issues.

For example:

- If a link fails and triggers BFD notifications, BoB might bring down the entire bundle, affecting all traffic.

- BoB-BLB, in coexistence mode, provides a more fine-grained control. It doesn't necessarily bring down the entire bundle when BFD reports a failure on a single link. This can prevent unnecessary bundle shutdowns due to transient or BFD-specific issues.

BoB-BLB coexistence is preferred over simple BoB in situations where the network operator wants more control over how BFD-related events impact the bundle. It provides a balance between the advantages of BLB (granular link monitoring) and BoB (efficient link utilization). It's particularly beneficial in scenarios where avoiding unnecessary bundle shutdowns due to transient BFD issues is important.

[Ramblin Tech]

If you want to monitor subinterfaces on LAG bundles with BFD and directly notify routing protocols of failues, then you need BLB, as BoB is not supported on subinterfaces. If you want to monitor individual links on LAG bundles with sub-second failure notification to the bundle manager, then you need BoB, as BLB does not monitor each individual link. BoB will also not directly notify routing protocols of failures, only indirectly if all (or min) links fail and the bundle manager takes the entire logical link out of service (as you said).  If you want both individual physical link and logical subinterface monitoring, then you need BoB + BLB.