How to enable BFD in ACI fabric?

[sarabsin]

Hello,

Could you please advise steps or options that need to be done/enabled to have BFD enabled in fabric?

I understand the support was added in 3.1.1i

Thanks,

[sarabsin]

i did enable BFD ISIS under Fabric -> interface policies-L3 interface-default

however on Spine and leafs the BFD neighbors are showing AdminDown

LAB1SSW0101# show bfd neigh
OurAddr NeighAddr LD/RD RH/RS Holdown(mult) State Int Vrf
10.0.24.66 10.0.24.69 1090519045/0 AdminDown N/A(3) Down Eth1/34.39 overlay-1
10.0.24.66 10.0.24.67 1090519046/0 AdminDown N/A(3) Down Eth1/32.40 overlay-1
10.0.24.66 10.0.24.64 1090519047/0 AdminDown N/A(3) Down Eth1/31.37 overlay-1
10.0.24.66 10.0.24.68 1090519048/0 AdminDown N/A(3) Down Eth1/33.42 overlay-1
LAB1SSW0101# show bfd neigh detail
OurAddr NeighAddr LD/RD RH/RS Holdown(mult) State Int Vrf
10.0.24.66 10.0.24.69 1090519045/0 AdminDown N/A(3) Down Eth1/34.39 overlay-1

Any ideas how can i fix it? please advise.

[sarabsin]

OurAddr NeighAddr LD/RD RH/RS Holdown(mult) State Int Vrf
10.0.24.66 10.0.24.68 1090519048/0 AdminDown N/A(3) Down Eth1/33.42 overlay-1

Session state is Down and not using echo function
Local Diag: 0, Demand mode: 0, Poll bit: 0, Authentication: None
MinTxInt: 0 us, MinRxInt: 0 us, Multiplier: 0
Received MinRxInt: 0 us, Received Multiplier: 0, Received MinEchoRx: 0 us
Holdown (hits): 6000 ms (0), Hello (hits): 2000 ms (0)
Rx Count: 0, Rx Interval (ms) min/max/avg: 0/0/0 last: 0 ms ago
Tx Count: 0, Tx Interval (ms) min/max/avg: 0/0/0 last: 0 ms ago
Registered protocols: isis
Downtime: 0 days 0 hrs 39 mins 17 secs
Last packet: Version: 1 - Diagnostic: 0
State bit: AdminDown - Demand bit: 0
Poll bit: 0 - Final bit: 0
Multiplier: 0 - Length: 24
My Discr.: 0 - Your Discr.: 0
Min tx interval: 0 - Min rx interval: 0
Min Echo interval: 0 - Authentication bit: 0
Hosting LC: 1, Down reason: No Diagnostic, Reason not-hosted: None

Rx and Tx is zero ...BFD is registerd in ISIS though.

[Rick1776]

Please check out this guide.

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/2-x/L3_config/b_Cisco_APIC_Layer_3_Configuration_Guide/b_Cisco_APIC_Layer_3_Configuration_Guide_chapter_0100.html#d1698e4931a1635

[sarabsin]

Hello Rick1776,

I have read the guide .

Is there any specific that you are referring to in guide?

please note i am enabling BFD inside fabric between Spine and Leafs.

I could not find anything related to BFD for Fabric itself in the documentation.

Mostly its for External Routed Protocols.

[Rick1776]

You wouldn't need to configure BFD from the Leafs to Spines as it runs Multi-Protocol BGP with EVPN for the control plane and VxLAN for the data plane. Plus this is all done within the ACI fabric automatically so you don't have to setup and BGP-MP or EVPN manually.

[sarabsin]

Hi,

As per my understanding of topic here is , i maybe wrong but it would good if some expert can confirm this.

IS-IS is is the underlying routing protocol in Fabric that route traffic within Fabric .

MP-BGP in Fabric is used for external routing that runs on top of it (IS-IS).

MP-BGP EVPN is optional in case you want to extend vxlan to other Fabric.

in release 3.1.1 there is option to enable BFD in IS-IS in fabric.I am talking about this option.

BFD detects sub-second failure of leafs or spine whereas IS-IS may take longer to detect (without BFD).

I have attached the screenshot which shows this option under Fabric policies.

[Rick1776]

That still looks like it's for creating a L3 interface not for IS-IS in the pod.

I only have access to a 3.0(K) fabric put go under this menu setting.
Fabric Tab, Fabric Policies, POD Policies, IS-IS default Policy and see if BFD is enabled/disabled from there.

[sarabsin]

Yes BFD is L3 interface feature . Its actually UDP ping packets with specific udp port with self interface ip address as destination sent to neighboring device.

IS-IS registers to BFD , thats why the option says ISIS BFD.

In normal NX-OS config you have to go under the routing protocol process and register it to BFD.

[Rick1776]

Correct. Really need someone from Cisco TAC to chime in here.

[Rick1776]

Your also correct on the BFD support.

If you look at the following release notes.

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/3-x/release_notes/apic_rn_311.html

It also says it's only for the Spine switches.
This release adds support for Bidirectional Forwarding Detection (BFD) on spine switch.

[Robert Burns · Accepted solution]

With Cisco 3.1(x) release, BFD would be supported for ISIS on both Spine and Leaf fabric interfaces.  Spine Switches/LCs must also be EX or later.  To configure it you'll need to enable the Global Config, Interface-level config, and then on the Protocol level (ISIS).

Ex.

BFD IPV4 Global Configuration (No change, same as existing)
apic1# configure
apic1(config)# template bfd ip bfd_ipv4_global_policy
apic1(config-bfd)# [no] echo-address 1.2.3.4
apic1(config-bfd)# [no] slow-timer 2500
apic1(config-bfd)# [no] min-tx 100
apic1(config-bfd)# [no] min-rx 70
apic1(config-bfd)# [no] multiplier 3
apic1(config-bfd)# [no] echo-rx-interval 500
apic1(config-bfd)# exit

Configure spine policy group and Inherit the BFD global policies
apic1# configure
apic1(config)# template spine-policy-group test_spine_policy_group
apic1(config-spine-policy-group)# [no] inherit bfd ip bfd_ipv4_global_policy
apic1(config-spine-policy-group)# [no] inherit bfd ipv6 bfd_ipv6_global_policy
apic1(config-spine-policy-group)# exit

Associate spine policy group onto a spine
apic1# configure
apic1(config)# spine-profile test_spine_profile
apic1(config-spine-profile)# spine-group test_spine_group
apic1(config-spine-group)# spine-policy-group test_spine_policy_group
apic1(config-spine-group)# spine 103-104
apic1(config-leaf-group)# exit

Configure BFD Interface Policy (bfdIfPol) (No change, same as existing)
apic1(config)# spine 103
apic1(config-spine)# template bfd bfdIfPol1 tenant infra
apic1(config-template-bfd-pol)# [no] echo-mode enable
apic1(config-template-bfd-pol)# [no] echo-rx-interval 500
apic1(config-template-bfd-pol)# [no] min-rx 70
apic1(config-template-bfd-pol)# [no] min-tx 100
apic1(config-template-bfd-pol)# [no] multiplier 5
apic1(config-template-bfd-pol)# [no] optimize subinterface

Inherit the BFD interface policy onto a L3 interface with IPv4 address  (No change, same as existing)
apic1(config)# spine 103
apic1(config-spine)# interface ethernet 5/3.4
apic1(config-spine-if)# vrf  member tenant infra vrf overlay-1
apic1(config-spine-if)# bfd ip tenant mode
apic1(config-spine-if)# bfd ip inherit interface-policy bfdPol1
apic1(config-spine-if)# bfd ip authentication keyed-sha1 key 10 key password

BFD ISIS configuration on Spine fabric-interface
apic1(config)# spine 103
apic1(config-spine)# fabric-interface ethernet 5/2
apic1(config-spine-if)# isis bfd enabled
apic1(config-spine-if)# exit
 
BFD ISIS configuration on Leaf fabric-interface
apic1(config)# leaf 101
apic1(config-leaf)# fabric-interface ethernet 1/49
apic1(config-leaf-if)# isis bfd enabled
apic1(config-leaf-if)# exit

Hope this helps.

Robert

[Rick1776]

Didn't realise that they had to be EX switch versions that makes sense.

[sarabsin]

Thanks Robert.

Is Baby spine 9336PQ  supported for BFD?

Thats what i have in my Lab.

[Robert Burns · Accepted solution]

No those aren't supported.  Those are Gen1 spines.  Supported models are detailed in the Layer3 Config Guide:

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/2-x/L3_config/b_Cisco_APIC_Layer_3_Configuration_Guide.pdf

Observe the following BFD guidelines and limitations:
• Starting from APIC Release 3.1(1), BFD between leaf and spine switches is supported on fabric-interfaces
for IS-IS. In addition, BFD feature on spine switch is supported for OSPF and static routes.
• BFD is supported on modular spine switches that have -EX and -FX line cards (or newer versions), and
BFD is also supported on the Nexus 9364C non-modular spine switch (or newer versions).

Robert