not documented BFD error log message

[langobardo]

Hi all,

I tried to find out what below BFD log message means exactly. In the BFD whitepapers and on the cisco.com page I couldn't find any related informations. I guess it could be a hardware related BFD issue?

%BFD_HWO-3-BFD_HWO_ERR_MSGS: % BFD_HWO: DETECT Timer Expired

Does anyone has experience with the meaning of this message? 

Thanks a lot any hints and references.

regards,

davide

[MHM Cisco World]

What is router platform you use ?

MHM

[langobardo]

we are using 6807 with C6800-SUP6T and 15.5(1)SY12. Not real routing platform.

[Giuseppe Larosa]

Hello @langobardo ,

a BFD timer has expired on a BFD session check the logs to see what interface and what neighbor  are affected

A Cisco Cat6800 Sup6T may have some form of hardware assistance or linecard delegation to handle BFD sessions directly on the linecard itself.

Hope to help

Giuseppe

[langobardo]

Hello @Giuseppe Larosa 

thanks for your reply and help is much appreciated.

that is what I first thought it would be. But the suffix of the message : HWO_ERR_MSGS made me curious, that looks to me not like a simple timer expiration message, they look different. Unfortunately the interface is not mentioned in the log.

I paste the full log so you have the full sight of the message:

08:17:46.769 CEST: %BFD_HWO-3-BFD_HWO_ERR_MSGS: % BFD_HWO: DETECT Timer Expired
08:17:46.769 CEST: %BGP-5-NBR_RESET: Neighbor 1y.x.9.242 reset (BFD adjacency down)
08:17:46.769 CEST: %BGP-5-NBR_RESET: Neighbor 1y.x.9.245 reset (BFD adjacency down)
08:17:46.777 CEST: %BGP-5-ADJCHANGE: neighbor 1y.x.9.242 vpn vrf BASE Down BFD adjacency down
08:17:46.777 CEST: %BGP_SESSION-5-ADJCHANGE: neighbor 10.x.9.242 IPv4 Unicast vpn vrf BASE topology base removed from session BFD adjacency down

Normally the BFD session expiration timer looks something like this:

%BFDFSM-6-BFD_SESS_DOWN: BFD-SYSLOG: BFD session ld:1023 handle:1,is going Down Reason: DETECT TIMER EXPIRED

thanks again for your help!

Regards,

davide

[Giuseppe Larosa]

Hello @langobardo ,

Davide we can say the affected BGP neighbor is the following:

08:17:46.769 CEST: %BFD_HWO-3-BFD_HWO_ERR_MSGS: % BFD_HWO: DETECT Timer Expired
08:17:46.769 CEST: %BGP-5-NBR_RESET: Neighbor 1y.x.9.242 reset (BFD adjacency down)

why the message looks like different it can be platform dependent or it can be a software issue.

is the neighbor up again or it is flapping up and down ?

i.e. you had a single event or a series of events in your log buffer ?

Hope to help

Giuseppe

[langobardo]

@Giuseppe Larosa the neighbors are flapping and it shows for each peer the above mentioned events bundle.

we isolated the faulty parts and we are going to replace the faulty hardware now, but I was interested in understanding if BFD message suffix HWO_ERR_MSGS: is indicating that it really relies to a hardware fault. 

this would have cost us less time in troubleshooting :))

[MHM Cisco World]

we isolated the faulty parts and we are going to replace the faulty hardware now <<- use depend on bfd issue to detect faulty parts ?

If Yes please before order new one contact cisco it can easily one internal bug.

Thanks 

MHM

[langobardo]

@MHM Cisco World yes we opened a case now and let's see what the outcome will be.

[MHM Cisco World]

It bug I see many bug for bfd mis timer read and this lead to bfd flapping 

Contact cisco 

MHM

[langobardo]

hello @MHM Cisco World ,

thanks for your answer and hint, so you think this software is buggy which we are running on our device? ok will check this together also with cisco, thanks again. 

Regards,

davide

[vishalbhandari]

The log message %BFD_HWO-3-BFD_HWO_ERR_MSGS: % BFD_HWO: DETECT Timer Expired is related to the Bidirectional Forwarding Detection (BFD) protocol on Cisco devices. While specific documentation on this exact error might not be readily available, I can help you understand the general context and potential causes.

Understanding the Log Message:

• BFD: Bidirectional Forwarding Detection is a network protocol used to detect faults in the path between two forwarding engines, such as routers, switches, or other network devices.
• HWO: This typically refers to "Hardware Offload" or "Hardware Offload Engine," indicating that the BFD process is being offloaded to specialized hardware components in the device for efficiency.
• DETECT Timer Expired: The "Detect Timer" in BFD is a crucial part of the protocol, used to determine how long to wait before declaring a link as down when no BFD packets are received.

Possible Causes:

1. Hardware Issue: Given the mention of "HWO" and "DETECT Timer Expired," it could point to a hardware-related issue, possibly with the hardware offloading engine responsible for BFD processing.
2. Configuration Mismatch: If the BFD configuration (such as the timer settings) between peers is not aligned, it could cause the timer to expire prematurely, leading to this error.
3. Network Congestion or Latency: High latency or congestion on the network could delay BFD packets, causing the timer to expire before a response is received.
4. Software Bug: In some cases, this error might be the result of a software bug, where the BFD implementation does not behave as expected under certain conditions.

Troubleshooting Steps:

1. Check Hardware Health: Use hardware diagnostic commands (like show hardware commands) to ensure that the hardware offloading components are functioning correctly.
2. Review BFD Configuration: Verify the BFD configuration on both ends of the BFD session, ensuring that the timer settings are consistent and appropriate for the network conditions.
3. Monitor Network Performance: Use tools like ping, traceroute, or network monitoring tools to check for high latency or congestion issues.
4. Check for Software Updates: Look for any software updates or patches that might address known BFD-related issues.

Cisco TAC:

If the issue persists or if you suspect it is a hardware-related problem, it might be worth reaching out to Cisco TAC (Technical Assistance Center) for a deeper analysis, especially if this error is impacting network performance.

[langobardo]

hello @vishalbhandari ,

thanks for your action plan, now I know what HWO means

BFD configuration is correct on both ends of the peers. We opened a case at Cisco-TAC so let's see wait for their analisys.

[Giuseppe Larosa]

Hello @langobardo ,

so now we know that HWO = HW offload it is the concept of HW assistance or linecard delegation I have mentioned in my first post .

the affected HW is a linecard  and its DFC is correct ?

Hope to help

Giuseppe

[langobardo]

@Giuseppe Larosa okay sorry may I have overseen this in your post. Yes, indeed it's a DFC equipped line card: C6800-8P40G.

the correct version of the card is: DCEF2T 8 port 40GE / 32 port 10GE with Distributed Forwarding Card C6800-DFC

thanks for your help!