07-21-2010 02:54 AM - edited 03-06-2019 12:07 PM
I have a number of 3750G-12S based switches in a stack that connect to a number of 3550 edge switches via fibre. All the ports on the 3750 are set in trunk mode and there are 25 vlans set up that uses VTP for vlan management and uses the default VLAN 1 as the management vlan. The 3550 gig ports are also set to trunk mode with similar configuration. Also I have enabled udld (not aggressive mode) globally at both ends.
The relevant configuration on the 3750 is as follows.
udld enable
spanning-tree mode pvst
spanning-tree extend system-id
spanning-tree uplinkfast
!
vlan internal allocation policy ascending
!
!
interface GigabitEthernet1/0/1
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet1/0/2
switchport trunk encapsulation dot1q
switchport mode trunk
The switches initially establish connectivity and work perfectly for a number minutes upto hours, then all active ports on the 3750 based stack are shut down and the following message is displayed:
Jul 20 21:20:50.178: %UDLD-4-UDLD_PORT_DISABLED: UDLD disabled interface Gi3/0/12, neighbor mismatch detected (RouterBox1-3)
Jul 20 21:20:50.178: %PM-4-ERR_DISABLE: udld error detected on Gi3/0/12, putting Gi3/0/12 in err-disable state (RouterBox1-3)
Jul 20 21:20:50.203: %UDLD-4-UDLD_PORT_DISABLED: UDLD disabled interface Gi2/0/4, neighbor mismatch detected (RouterBox1-2)
Jul 20 21:20:50.203: %PM-4-ERR_DISABLE: udld error detected on Gi2/0/4, putting Gi2/0/4 in err-disable state (RouterBox1-2)
Jul 20 21:20:51.210: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet3/0/12, changed state to down
Jul 20 21:20:51.218: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet2/0/4, changed state to down
Jul 20 21:20:52.258: %LINK-3-UPDOWN: Interface GigabitEthernet2/0/4, changed state to down
Jul 20 21:20:52.325: %LINK-3-UPDOWN: Interface GigabitEthernet3/0/12, changed state to down
The 62.5/125 MM fibre runs are at most 50m in length and the 3550 interface uses the WS-G5484 Cisco GBIC ( 850nm wavelength) and the 3750 uses the GLC-SX-MM Cisco SFP ( 850nm wavelength) and all interfaces are genuine Cisco parts.
Even though the error message "neighbor mismatch detected" might indicate that there may be a physical link incompatibility between both ends of a link, I am reluctant to accept that it what the problem is. As you can see from the logs above all ports shutdown at the same time and if I disable udld altogether the links remain up and traffic flows as normal. Also I understood that when udld was in normal mode that the ports would not go into an err-disable state.
I would certainly appreciate any help on this issue.
07-21-2010 03:34 AM
I think there is missing configuration
,,,
UDLD must configure in 2 end cable point (sw 1 connect to sw 2 , must configure in sw1 and sw2 , globally and under interface)
udld enable
interface GigabitEthernet1/0/1
switchport trunk encapsulation dot1q
switchport mode trunk
udld port aggressive
!
interface GigabitEthernet1/0/2
switchport trunk encapsulation dot1q
switchport mode trunk
udld port aggressive
And do it to all switches and every port connected to 3750 use command (udld port aggressive)
Try and it will be work fine
07-21-2010 07:20 AM
Hello Khaled.
Many thanks for your response. I did exactly as you outlined but unfortunately the links went down approx. 15 mins after configuration.
I enabled udld globally and the config for any 3750 port is as follows:
!
interface GigabitEthernet2/0/1
switchport trunk encapsulation dot1q
switchport mode trunk
udld port aggressive
!
and for the corresponding 3550 gig port is:
interface GigabitEthernet0/1
switchport trunk encapsulation dot1q
switchport mode trunk
udld port aggressive
The 3750 showed the following output:
Jul 21 13:51:41.230: %UDLD-4-UDLD_PORT_DISABLED: UDLD disabled interface Gi2/0/1, neighbor mismatch detected (RouterBox1-2)
Jul 21 13:51:41.230: %PM-4-ERR_DISABLE: udld error detected on Gi2/0/1, putting Gi2/0/1 in err-disable state (RouterBox1-2)
Jul 21 13:51:41.373: %UDLD-4-UDLD_PORT_DISABLED: UDLD disabled interface Gi2/0/6, neighbor mismatch detected (RouterBox1-2)
Jul 21 13:51:41.373: %PM-4-ERR_DISABLE: udld error detected on Gi2/0/6, putting Gi2/0/6 in err-disable state (RouterBox1-2)
Jul 21 13:51:41.465: %UDLD-4-UDLD_PORT_DISABLED: UDLD disabled interface Gi2/0/7, neighbor mismatch detected (RouterBox1-2)
Jul 21 13:51:41.465: %PM-4-ERR_DISABLE: udld error detected on Gi2/0/7, putting Gi2/0/7 in err-disable state (RouterBox1-2)
Jul 21 13:51:42.254: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet2/0/1, changed state to down
Jul 21 13:51:42.380: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet2/0/6, changed state to down
Jul 21 13:51:42.572: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet2/0/7, changed state to down
and one of the 3550 showed this output: (ignore the time as it was not set on switch)
*Mar 2 22:27:21.136: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/1, changed state to down
Kind Regards
07-21-2010 07:41 AM
Can you try to connect console
and do :
3750 :
no udld enable
default interface GigabitEthernet2/0/1
interface GigabitEthernet2/0/1
shutdown
switchport trunk encapsulation dot1q
switchport mode trunk
udld port aggressive
3550
no udld enable
default interface GigabitEthernet0/1
shutdown
switchport trunk encapsulation dot1q
switchport mode trunk
udld port aggressive
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
After complete configuration do # no shutdown
if the problem not solved then use in global mode #udld aggressive
if use command udld aggressive globally thet mean = Enable UDLD protocol in aggressive mode on fiber ports except where locally configured
,,,,,,,,,,,,,
And I have one question is the bpdu guard enable globally ???
07-21-2010 08:15 AM
Hello Khaled
I have configured udld agressive at port level and disabled udld globally as you outlined. I will leave the switches on to see what happens. I will not be around for the next few days to keep this discussion going but I will be back online on Monday next.
Many thanks for your help.
Below is the output of the show spanning-tree summary totals command for the 3550 and the 3750 that shows bpdu guard is disabled.
3550
#show spanning-tree summary totals
Switch is in pvst mode
Root bridge for: none
Extended system ID is enabled
Portfast Default is disabled
PortFast BPDU Guard Default is disabled
Portfast BPDU Filter Default is disabled
Loopguard Default is disabled
EtherChannel misconfig guard is enabled
UplinkFast is enabled
BackboneFast is disabled
Configured Pathcost method used is short
Name Blocking Listening Learning Forwarding STP Active
---------------------- -------- --------- -------- ---------- ----------
24 vlans 0 0 0 48 48
Station update rate set to 150 packets/sec.
UplinkFast statistics
-----------------------
Number of transitions via uplinkFast (all VLANs) : 0
Number of proxy multicast addresses transmitted (all VLANs) : 0
3750
#show spanning-tree summary totals
Switch is in pvst mode
Root bridge for: none
Extended system ID is enabled
Portfast Default is disabled
PortFast BPDU Guard Default is disabled
Portfast BPDU Filter Default is disabled
Loopguard Default is disabled
EtherChannel misconfig guard is enabled
UplinkFast is enabled
Stack port is StackPort1
BackboneFast is disabled
Configured Pathcost method used is short
Name Blocking Listening Learning Forwarding STP Active
---------------------- -------- --------- -------- ---------- ----------
24 vlans 0 0 0 96 96
Station update rate set to 150 packets/sec.
UplinkFast statistics
-----------------------
Number of transitions via uplinkFast (all VLANs) : 0
Number of proxy multicast addresses transmitted (all VLANs) : 1436
07-27-2010 01:57 AM
Hello Khaled,
I tried the two options you suggested: 1) global udld aggressive and 2) local port udld aggressive, but the ports still shutdown after an up time of 5 to 30 minutes. I also tried udld normal at both ends of the link and there too the ports will shut down after a few minutes up time.
I cannot find any reference as to the meaning of "neighbor mismatch detected" in the log line:
Jul 20 21:20:50.178: %UDLD-4-UDLD_PORT_DISABLED: UDLD disabled interface Gi3/0/12, neighbor mismatch detected
Would you know what it refers to?
It doesn't make sense udld is finding the same fault "neighbor mismatch detected" on multiple ports at the same time resulting in those ports being shutdown.
Also I understood that when switches were configured with udld normal (udld enable) that the ports would not shutdown.
Kind Regards and thanks for your help.
07-27-2010 04:32 AM
Supposing your switche runs some flavor of IOS 1.2 , please refer to:
http://www.cisco.com/en/US/docs/ios/12_2/sem2/system/message/emfubr72.html
you will read:
Error Message
%UDLD-4-UDLD_PORT_DISABLED : UDLD disabled interface [chars], [chars] detected
Explanation An interface was disabled. UDLD protocol has detected the condition. The interface and cause of the condition is specified in the error message. This condition is most likely caused by either a failed interface hardware or a cable misconfiguration.
Recommended Action Investigate the causes of the problem by checking the interface hardware and cables. Copy the error message exactly as it appears on the console or in the system log. Issue the show tech-support command to gather data that may help identify the nature of the error. If you cannot determine the nature of the error from the error message text or from the show tech-support command output, contact your Cisco technical support representative and provide the representative with the gathered information
Anyway, you are rigth, UDLD (not agressive) should not disable the interface.
The behavior looks to be a bug.
Besides, I wonder why you need UDLD.
In my view, only the agressive mode is helpfull, in case a STP backup link fails, UDLD will disable the related interface(s).
Most of the time, switches runs CDP and SPT (by default), which provides mush more
information about the neighbors.
07-27-2010 07:01 AM
Hello bonnardopjl,
You are right I don't really need udld as the links are internal within the same building and are very unlikely to get damaged. But I would like to get a solution to the problem I am experiencing as it should not be happening.
I have tried a very old image and the most recent image -- and they both bring down the ports via udld after 5 - 30 minutes.
Also any reference to udld error messages on the Cisco site does not contain any information about what "neighbor mismatch detected" means in the line:
Jul 20 21:20:50.178: %UDLD-4-UDLD_PORT_DISABLED: UDLD disabled interface Gi3/0/12, neighbor mismatch detected.
Kind Regards.
07-27-2010 10:05 AM
I notice you have the issue on members 2 and 3 of the stack, is UDLD working fine on member 1 ?
This could be a clue ...
You could try various flavor of "debug udld".
I tested udld agressive mode between 3750 12.2(46)SE and 4500 12.2(31)SGA10.
I did not have the same message, but:
*May 6 23:50:53.114: %UDLD-4-UDLD_PORT_DISABLED: UDLD disabled interface Gi1/0/3, aggressive mode failure detected
*May 6 23:50:53.114: %PM-4-ERR_DISABLE: udld error detected on Gi1/0/3, putting Gi1/0/3 in err-disable state
*May 6 23:50:53.768: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/3, changed state to down
*May 6 23:50:55.177: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/3, changed state to down
konw, iI can provide you with my knowledge abou UDLD:
UDLD specifications are published within RFC 5171, April 2008, written by M. Foschiano of Cisco Systems.
The UniDirectional Link Detection protocol (often referred to in short as "UDLD") is a lightweight protocol that can be used to detect and disable one-way connections before they create dangerous situations such as Spanning Tree loops or other protocol malfunctions.
For instance, in case of one way communication, Spanning Tree BPDU are not received, so a port that should be blocked will be unlocked, so creating a one-way loop.
Like many other L2 protocols, UDLD uses a specific Destination Address: 01-00-0c-cc-cc-cc, which is a multicast address.
The Source Address is the mac address of the switch module.
IEEE 802.3 SNAP encapsulation is used, which includes Logical Link Control (LLC) 0xAAAA03 and Organizational Unique Identifier (OUI) 0x00000C.
UDLD protocol type is 01-11.
UDLD uses 3 types of PDU: Probe, Echo and Flush. The Probe PDU contains the switch and the interface identification.
For UDLD to operate on a switch, it is necessary that the switch interface and the neighbor switch interface are both UDLD enabled. It is also necessary that the switch has received at least one Probe PDU from the neighbor.
At the beginning, the “Current bidirectional state” remains “Unknown”. When the first Probe PDU is received, the “Current bidirectional state” becomes “Bidirectional”.
UDLD maintains a Neighbor Database which stores in a cache the information contained in the Probe PDU’s from the neighbor switch. When the switch receives a Probe PDU, it caches the information until the age time expires. If the switch receives a new Probe PDU before an older cache entry ages, the switch replaces the older entry with the new one.
When the last cache entry is cleared, UDLD in aggressive mode will disable the related interface.
The whole cache is erased when the interface is disabled (also if UDLD is disabled). A Flush PDU is sent to the neighbor, so it will erase corresponding entries in its cache.
07-27-2010 03:01 PM
I just want to make sure but Gi2/0/4 and Gi3/0/12 are copper links right? These are not fibre optic? If this is so, WHY enable UDLD on copper links? UDLD works very, very well in fibre but I can't say the same for copper. This is why I never enable UDLD on copper links.
07-28-2010 03:21 AM
Hello leolaohoo,
All connectivity is via MM fibre. The 62.5/125 MM fibre runs are at most 50m in length and the 3550 interface uses the WS-G5484 Cisco GBIC ( 850nm wavelength) and the 3750 uses the GLC-SX-MM Cisco SFP ( 850nm wavelength) and all interfaces are genuine Cisco parts.
I have now disabled udld both ends and the ports are not shutting down.
Kind Regards.
07-28-2010 03:27 AM
I didn't fully read your post of the topic. My apologies.
Ok, so you have fibre optic links. This does not make any sense why would UDLD shut itself down unless you have problems with the fibre optic themselves. Change the patch cord, have the 50m link tested with an OTDR ...
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide