3850 udld error on port channel

[Ashwin Jingade]

I have configured 2 switches 3850 with port channel, if I enable UDLD port channel fails, it never established the connection. Even I bounce the port.

Ports will go to "erro-disable" state.

001036: *Sep  7 12:47:01.724 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/1/1, changed state to down
001037: *Sep  7 12:47:01.732 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/1/2, changed state to down
001038: *Sep  7 12:47:01.738 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to down
001039: *Sep  7 12:47:02.731 CET: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/1/1, changed state to down

During my investigation I found :
CSCva24225 - UDLD neighbor Failed to become bi-directional - 1

is there any solution on this ?

I am using IOS:

Switch Ports Model              SW Version        SW Image              Mode
------ ----- -----              ----------        ----------            ----
*    1 16    WS-C3850-12XS      16.3.6            CAT3K_CAA-UNIVERSALK9 BUNDLE

[chrihussey]

Could you post the port-channel and physical switch port interface configs on both sides? It may help in figuring out what is going on.

Also, have you tried to take the interfaces out of the port channel and bring them up individually to see if both links are indeed operational and stable?

Thanks

[Ashwin Jingade]

SW 2:

xx-xx-xx-xx#sh run int t1/1/1
Building configuration...

Current configuration : 167 bytes
!
interface TenGigabitEthernet1/1/1
 description
 switchport trunk native vlan 195
 switchport mode trunk
 channel-group 1 mode desirable non-silent
end

xx-xx-xx-xx#sh run int t1/1/2
Building configuration...

Current configuration : 167 bytes
!
interface TenGigabitEthernet1/1/2
 description
 switchport trunk native vlan 195
 switchport mode trunk
 channel-group 1 mode desirable non-silent
end

xx-xx-xx-xx#sh run int po1
Building configuration...

Current configuration : 128 bytes
!
interface Port-channel1
 description
 switchport trunk native vlan 195
 switchport mode trunk

SW1:

XX-XX-XX-XX#sh run int t1/1/1
Building configuration...

Current configuration : 166 bytes
!
interface TenGigabitEthernet1/1/1
 description
 switchport trunk native vlan 195
 switchport mode trunk
 channel-group 1 mode desirable non-silent
end

XX-XX-XX-XX#sh run int t1/1/2
Building configuration...

Current configuration : 166 bytes
!
interface TenGigabitEthernet1/1/2
 description
 switchport trunk native vlan 195
 switchport mode trunk
 channel-group 1 mode desirable non-silent
end

XX-XX-XX-XX#sh run int po1
Building configuration...

Current configuration : 126 bytes
!
interface Port-channel1
 description
 switchport trunk native vlan 195
 switchport mode trunk
end

[balaji.bandi]

can you post logs shows on other side ? is the same IOS version of other device.

post show udld output

[Ashwin Jingade]

Yes its same IOS recommended by CISCO :

Here is the out put :

SW 2:

sh udld t1/1/1

Interface Te1/1/1
---
Port enable administrative configuration setting: Follows device default
Port enable operational state: Enabled
Current bidirectional state: Unidirectional
Current operational state: Disabled port
Message interval: 7000 ms
Time out interval: 5000 ms

Port fast-hello configuration setting: Disabled
Port fast-hello interval: 0 ms
Port fast-hello operational state: Disabled
Neighbor fast-hello configuration setting: Disabled
Neighbor fast-hello interval: Unknown

No neighbor cache information stored
x-xx-xx-xx#sh udld t1/1/2

Interface Te1/1/2
---
Port enable administrative configuration setting: Follows device default
Port enable operational state: Enabled
Current bidirectional state: Unidirectional
Current operational state: Disabled port
Message interval: 7000 ms
Time out interval: 5000 ms

Port fast-hello configuration setting: Disabled
Port fast-hello interval: 0 ms
Port fast-hello operational state: Disabled
Neighbor fast-hello configuration setting: Disabled
Neighbor fast-hello interval: Unknown

No neighbor cache information stored

error logs:

000086: *Sep  8 12:36:43.559 CET: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/1/2, changed state to up
000087: *Sep  8 12:36:47.004 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/1/2, changed state to up
000088: *Sep  8 12:36:47.164 CET: %UDLD-4-UDLD_PORT_DISABLED: UDLD disabled interface Te1/1/2, aggressive mode failure detected
000089: *Sep  8 12:36:47.164 CET: %PM-4-ERR_DISABLE: udld error detected on Te1/1/2, putting Te1/1/2 in err-disable state

12:46:44.426 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/1/1, changed state to up
000134: *Sep  8 12:46:44.655 CET: %UDLD-4-UDLD_PORT_DISABLED: UDLD disabled interface Te1/1/1, aggressive mode failure detected
000135: *Sep  8 12:46:44.655 CET: %PM-4-ERR_DISABLE: udld error detected on Te1/1/1, putting Te1/1/1 in err-disable state
000136: *Sep  8 12:46:45.657 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/1/1, changed state to down

h int status | i err-disabled
Te1/1/1  x-xx-xx-011        err-disabled 1            full   1000 1000BaseSX SFP
Te1/1/2   x-xx-xx-011        err-disabled 1            full   1000 1000BaseSX SFP

[Leo Laohoo]

I don't see any signs of:
1. The links going down due to error-disable;
2. Error disable was caused by UDLD
Post the complete output to the command "sh interface status err".

[Ashwin Jingade]

SW2 :

sh int status err

Port      Name               Status       Reason               Err-disabled Vlans
Te1/1/1   x-xx-xx-011        err-disabled udld
Te1/1/2   x-xx-xx-011        err-disabled udld

000133: *Sep  8 12:46:44.426 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/1/1, changed state to up
000134: *Sep  8 12:46:44.655 CET: %UDLD-4-UDLD_PORT_DISABLED: UDLD disabled interface Te1/1/1, aggressive mode failure detected
000135: *Sep  8 12:46:44.655 CET: %PM-4-ERR_DISABLE: udld error detected on Te1/1/1, putting Te1/1/1 in err-disable state
000136: *Sep  8 12:46:45.657 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/1/1, changed state to down
000137: *Sep  8 12:46:46.692 CET: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/1/1, changed state to down
000138: *Sep  8 12:46:57.192 CET: %PM-4-ERR_RECOVER: Attempting to recover from udld err-disable state on Te1/1/2
000139: *Sep  8 12:47:02.378 CET: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/1/2, changed state to up
000140: *Sep  8 12:47:05.816 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/1/2, changed state to up
000141: *Sep  8 12:47:06.054 CET: %UDLD-4-UDLD_PORT_DISABLED: UDLD disabled interface Te1/1/2, aggressive mode failure detected
000142: *Sep  8 12:47:06.054 CET: %PM-4-ERR_DISABLE: udld error detected on Te1/1/2, putting Te1/1/2 in err-disable state
000143: *Sep  8 12:47:07.055 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/1/2, changed state to down
000144: *Sep  8 12:47:08.091 CET: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/1/2, changed state to down

SW1:

no err-disable in SW 1 port channels

If I disable "no udld en" in SW2 in  global mode ports will come up

[balaji.bandi]

Do you have VLAN 195 in VLAN Database on both the switches.

show vlan

[Ashwin Jingade]

Yes its available in both switch

[Leo Laohoo]

---

@Ashwin Jingade wrote:

000138: *Sep  8 12:46:57.192 CET: %PM-4-ERR_RECOVER: Attempting to recover from udld err-disable state on Te1/1/2 

---

W-T-F!

Who's the "genius" who configured UDLD auto-recovery????  That person needs to be taken out the back and shot!

UDLD is designed to SAVE & PROTECT the network from network instability (think dynamic routing in a large network).  Enabling auto-recovery means that the link pops back up.  If there is a constant large OSPF or BGP topology changes this can cause the router or switch to crash.  And I've seen this happen before. 

If UDLD protection is enabled, take the auto-recovery OFF.  

If auto-recovery is enabled, then take UDLD aggressive OFF.  

DO NOT HAVE UDLD AGGRESSIVE and AUTO-RECOVERY ENABLED.

Do NOT have both.  

[Ashwin Jingade]

It was configured same way.

As per your solution recommendation do I have to disable no udld aggressive on both switches ?

[Leo Laohoo]

Post the complete output to the command "sh int Ten 1/1/1 controll" and "sh int Ten 1/1/2 controll".

[Ashwin Jingade]

Note: I have disabled UDLD from global mode while taking this output

SW1:

sh int t1/1/1 controller
TenGigabitEthernet1/1/1 is up, line protocol is up (connected)
  Hardware is Ten Gigabit Ethernet, address is 7486.0bbc.380d (bia 7486.0bbc.380d)
  Description: R-UK-CH-011
  MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive not set
  Full-duplex, 1000Mb/s, link type is auto, media type is 1000BaseSX SFP
  input flow-control is off, output flow-control is unsupported
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input never, output 00:00:08, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     518 packets input, 79784 bytes, 0 no buffer
     Received 518 broadcasts (510 multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 510 multicast, 0 pause input
     0 input packets with dribble condition detected
     421 packets output, 60448 bytes, 0 underruns
     0 output errors, 0 collisions, 2 interface resets
     6 unknown protocol drops
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 pause output
     0 output buffer failures, 0 output buffers swapped out
Transmit                  TenGigabitEthernet1/1/1               Receive
        60448 Total bytes                       79784 Total bytes
          146 Unicast frames                        0 Unicast frames
         9344 Unicast bytes                         0 Unicast bytes
          257 Multicast frames                    510 Multicast frames
        49884 Multicast bytes                   79240 Multicast bytes
           18 Broadcast frames                      8 Broadcast frames
         1220 Broadcast bytes                     544 Broadcast bytes
            0 System FCS error frames               0 IpgViolation frames
            0 MacUnderrun frames                    0 MacOverrun frames
            0 Pause frames                          0 Pause frames
            0 Cos 0 Pause frames                    0 Cos 0 Pause frames
            0 Cos 1 Pause frames                    0 Cos 1 Pause frames
            0 Cos 2 Pause frames                    0 Cos 2 Pause frames
            0 Cos 3 Pause frames                    0 Cos 3 Pause frames
            0 Cos 4 Pause frames                    0 Cos 4 Pause frames
            0 Cos 5 Pause frames                    0 Cos 5 Pause frames
            0 Cos 6 Pause frames                    0 Cos 6 Pause frames
            0 Cos 7 Pause frames                    0 Cos 7 Pause frames
            0 Oam frames                            0 OamProcessed frames
            0 Oam frames                            0 OamDropped frames
          201 Minimum size frames                  56 Minimum size frames
          137 65 to 127 byte frames               377 65 to 127 byte frames
            0 128 to 255 byte frames                0 128 to 255 byte frames
           83 256 to 511 byte frames               85 256 to 511 byte frames
            0 512 to 1023 byte frames               0 512 to 1023 byte frames
            0 1024 to 1518 byte frames              0 1024 to 1518 byte frames
            0 1519 to 2047 byte frames              0 1519 to 2047 byte frames
            0 2048 to 4095 byte frames              0 2048 to 4095 byte frames
            0 4096 to 8191 byte frames              0 4096 to 8191 byte frames
            0 8192 to 16383 byte frames             0 8192 to 16383 byte frames
            0 16384 to 32767 byte frame             0 16384 to 32767 byte frame
            0 > 32768 byte frames                   0 > 32768 byte frames
            0 Late collision frames                 0 SymbolErr frames
            0 Excess Defer frames                   0 Collision fragments
            0 Good (1 coll) frames                  0 ValidUnderSize frames
            0 Good (>1 coll) frames                 0 InvalidOverSize frames
            0 Deferred frames                       0 ValidOverSize frames
            0 Gold frames dropped                   0 FcsErr frames
            0 Gold frames truncated
            0 Gold frames successful
            0 1 collision frames
            0 2 collision frames
            0 3 collision frames
            0 4 collision frames
            0 5 collision frames
            0 6 collision frames
            0 7 collision frames
            0 8 collision frames
            0 9 collision frames
            0 10 collision frames
            0 11 collision frames
            0 12 collision frames
            0 13 collision frames
            0 14 collision frames
            0 15 collision frames
            0 Excess collision frames

LAST UPDATE 2000 msecs AGO

sh int t1/1/2 controller
TenGigabitEthernet1/1/2 is up, line protocol is up (connected)
  Hardware is Ten Gigabit Ethernet, address is 7486.0bbc.380e (bia 7486.0bbc.380e)
  Description: R-UK-CH-011
  MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive not set
  Full-duplex, 1000Mb/s, link type is auto, media type is 1000BaseSX SFP
  input flow-control is off, output flow-control is unsupported
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input never, output 00:00:05, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 9000 bits/sec, 16 packets/sec
  5 minute output rate 5000 bits/sec, 9 packets/sec
     25204 packets input, 1840450 bytes, 0 no buffer
     Received 25199 broadcasts (25151 multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 25151 multicast, 0 pause input
     0 input packets with dribble condition detected
     13969 packets output, 1010296 bytes, 0 underruns
     0 output errors, 0 collisions, 2 interface resets
     5 unknown protocol drops
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 pause output
     0 output buffer failures, 0 output buffers swapped out
Transmit                  TenGigabitEthernet1/1/2               Receive
      1010296 Total bytes                     1840450 Total bytes
          155 Unicast frames                        5 Unicast frames
         9920 Unicast bytes                      1151 Unicast bytes
        13813 Multicast frames                  25151 Multicast frames
      1000308 Multicast bytes                 1836043 Multicast bytes
            1 Broadcast frames                     48 Broadcast frames
           68 Broadcast bytes                    3256 Broadcast bytes
            0 System FCS error frames               0 IpgViolation frames
            0 MacUnderrun frames                    0 MacOverrun frames
            0 Pause frames                          0 Pause frames
            0 Cos 0 Pause frames                    0 Cos 0 Pause frames
            0 Cos 1 Pause frames                    0 Cos 1 Pause frames
            0 Cos 2 Pause frames                    0 Cos 2 Pause frames
            0 Cos 3 Pause frames                    0 Cos 3 Pause frames
            0 Cos 4 Pause frames                    0 Cos 4 Pause frames
            0 Cos 5 Pause frames                    0 Cos 5 Pause frames
            0 Cos 6 Pause frames                    0 Cos 6 Pause frames
            0 Cos 7 Pause frames                    0 Cos 7 Pause frames
            0 Oam frames                            0 OamProcessed frames
            0 Oam frames                            0 OamDropped frames
          965 Minimum size frames                  72 Minimum size frames
        12917 65 to 127 byte frames             25039 65 to 127 byte frames
            0 128 to 255 byte frames                0 128 to 255 byte frames
           86 256 to 511 byte frames               91 256 to 511 byte frames
            1 512 to 1023 byte frames               2 512 to 1023 byte frames
            0 1024 to 1518 byte frames              0 1024 to 1518 byte frames
            0 1519 to 2047 byte frames              0 1519 to 2047 byte frames
            0 2048 to 4095 byte frames              0 2048 to 4095 byte frames
            0 4096 to 8191 byte frames              0 4096 to 8191 byte frames
            0 8192 to 16383 byte frames             0 8192 to 16383 byte frames
            0 16384 to 32767 byte frame             0 16384 to 32767 byte frame
            0 > 32768 byte frames                   0 > 32768 byte frames
            0 Late collision frames                 0 SymbolErr frames
            0 Excess Defer frames                   0 Collision fragments
            0 Good (1 coll) frames                  0 ValidUnderSize frames
            0 Good (>1 coll) frames                 0 InvalidOverSize frames
            0 Deferred frames                       0 ValidOverSize frames
            0 Gold frames dropped                   0 FcsErr frames
            0 Gold frames truncated
            0 Gold frames successful
            0 1 collision frames
            0 2 collision frames
            0 3 collision frames
            0 4 collision frames
            0 5 collision frames
            0 6 collision frames
            0 7 collision frames
            0 8 collision frames
            0 9 collision frames
            0 10 collision frames
            0 11 collision frames
            0 12 collision frames
            0 13 collision frames
            0 14 collision frames
            0 15 collision frames
            0 Excess collision frames

LAST UPDATE 212 msecs AGO

[Leo Laohoo]

What is connected to these two ports?

[Ashwin Jingade]

configured as a portchannel and Fiber cables are connected