UDLD misbehaviour

[Leonardo Gama]

Hello my friends,

I had some problems on an optical fibre between two 6509 switches and UDLD
kicked in to avoid STP loops, but when the switch tried to recover from the error-disable state,
the link went up, even with optical fibre problems.
This misbehaviour caused a major outage in the network. I couldn't find any known bug for the
current IOS version 12.2933)SXI3.
I worked around the issue keeping the interface in a shutdown state until I
resolved the cabling issue.
Can someone shed some light on the solution?

09:20:24.737: %LINEPROTO-5-UPDOWN: Line protocol on Interface
TenGigabitEthernet2/4/10, changed state to down
09:20:24.757: %LINK-3-UPDOWN: Interface TenGigabitEthernet2/4/10, changed state to
down
09:20:24.994: %PM-SW2_SPSTBY-4-ERR_DISABLE: udld error detected on Te2/4/10,
putting Te2/4/10 in err-disable state
09:20:24.710: %UDLD-SW1_SP-4-UDLD_PORT_DISABLED: UDLD disabled interface Te2/4/10,
aggressive mode failure detected
09:20:24.710: %PM-SW1_SP-4-ERR_DISABLE: udld error detected on Te2/4/10, putting
Te2/4/10 in err-disable state
09:20:25.203: %LINEPROTO-SW1_SP-5-UPDOWN: Line protocol on Interface
TenGigabitEthernet2/4/10, changed state to down
09:20:25.203: %LINK-SW1_SP-3-UPDOWN: Interface TenGigabitEthernet2/4/10, changed
state to down
09:20:55.004: %PM-SW1_SP-4-ERR_RECOVER: Attempting to recover from udld err-disable
state on Te2/4/10
09:20:55.119: %PM-SW2_SPSTBY-4-ERR_RECOVER: Attempting to recover from udld
err-disable state on Te2/4/10
09:20:56.362: %LINK-3-UPDOWN: Interface TenGigabitEthernet2/4/10, changed state to
up
09:20:56.333: %LINK-SW1_SP-3-UPDOWN: Interface TenGigabitEthernet2/4/10, changed
state to up

I will really appreciate any input.

[Peter Paluch · Accepted solution]

Hello,

My way of thinking is that the UDLD can detect unidirectional links successfully only if both sides have heard the UDLD hello packets and have determined that the UDLD neighborhood is up and working. UDLD does not bring a link down just because it does not hear the other peer. It brings a link down only if it stopped hearing the other peer, assuming that it heard it before.

What you have observed is, in my opinion, logical:

1. An unidirectional link condition was created in your network.
2. UDLD reacted and brought the line down. So far so good.
3. Because of the err-disable recovery, some time after this incident, the port was brought back up. However, the unidirectional link condition was not removed.
4. The port did not hear the UDLD packets from the other peer but because it basically came from "down" state to "up" state, it remained waiting for the other party to start speaking UDLD. This is a normal startup for any UDLD-enabled link. However, because of unidirectional link condition, even if the other peer spoke UDLD, the messages could not be heard.
5. The UDLD kept waiting for the other peer, keeping the link up and obviously causing a switching loop in the process just like with any other unidirectional link in a network.

So to me, this does not look like a bug at all. The problem was entirely caused by the automatic err-disable recovery. Unidirectional link errors are not good candidates for automatic recovery. They must first be corrected manually, only then the ports can be safely brought back to operation.

Best regards,

Peter

[Leonardo Gama]

Ok. but the neighbouring switch behaved differently:

Jul  6 09:17:14.774: %UDLD-SW1_SP-4-UDLD_PORT_DISABLED: UDLD disabled interface Te2/3/12, aggressive mode failure detected

Jul  6 09:17:14.774: %PM-SW1_SP-4-ERR_DISABLE: udld error detected on Te2/3/12, putting Te2/3/12 in err-disable state

Jul  6 09:17:15.355: %PM-SW2_SPSTBY-4-ERR_DISABLE: udld error detected on Te2/3/12, putting Te2/3/12 in err-disable state

Jul  6 09:17:45.063: %PM-SW1_SP-4-ERR_RECOVER: Attempting to recover from udld err-disable state on Te2/3/12

Jul  6 09:17:45.191: %PM-SW2_SPSTBY-4-ERR_RECOVER: Attempting to recover from udld err-disable state on Te2/3/12

.Jul  6 10:48:39.440: %SYS-5-CONFIG_I: Configured from console by local on vty0 (172.30.4.31)

It didnt bring up the interface, what I consider the right behaviour, isnt it?

[Leonardo Gama]

Hi Peter,

You are correct, the far side didnt bring the interface up because it didnt have the errdisable recovery for UDLD.

The recommendation is not enable errdisable recovery for UDLD at all.

Thanks much.