Solved: EtherChannel problem, 3750X <--> 6509E

fearless.rabbit · ‎03-04-2013

Hello guys,

I desperately need help configuring EtherChannel between 3750-X cross-stack and 6509E switch. I use two ports on 3750s, and two ports on 6509. I just need it as a trunk. For some elusive reason one port on 3750 keeps being err-disabled, and one on 6509 notconnected.

Here is what I did so far.

3750

!

interface GigabitEthernet1/0/22

description ***VSS-RNOC-link***

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 2

speed 1000

duplex full

no vtp

channel-group 15 mode on

end

!

interface GigabitEthernet2/0/22

description ***VSS-RNOC-link***

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 2

speed 1000

duplex full

no vtp

channel-group 15 mode on

end

!

interface Port-channel15

description ***VSS-RNOC-link***

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 2

end

6509

!

interface GigabitEthernet2/1/28

description ***Dom_Vojske_Failover_link***

switchport

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 2

switchport mode trunk

speed 1000

duplex full

channel-group 220 mode on

end

!

interface GigabitEthernet2/1/29

description ***Dom_Vojske_Failover_link***

switchport

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 2

switchport mode trunk

speed 1000

duplex full

channel-group 220 mode on

end

!

interface Port-channel220

description ***Dom_Vojske_Failover_link***

switchport

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 2

switchport mode trunk

no vtp

end

These bastards give me headaches:

GigabitEthernet1/0/22 is down, line protocol is down (err-disabled)

GigabitEthernet2/1/29 is down, line protocol is down (notconnect)

I tried shuting them down, then no shuting them again. Didnt help.

Any ideas?

Peter Paluch · ‎03-04-2013

Hello Milos,

It is a blind shot but I am increasingly getting the feeling that the ports you are configuring for the EtherChannel are not the true ports that interconnect your 3750 and 6509 together.

Note the output you've provided:

show etherchannel summary on 3750:
15     Po15(SD)        LACP      Gi1/0/22(I) Gi2/0/22(D)

The gi1/0/22 is identified as an (I)ndividual port - that means that the opposite device is not speaking LACP to it so it cannot negotiate about even becoming an EtherChannel member link. The gi2/0/22 is (D)own meaning that the opposite port is shutdown or not even connected.

Can you verify using CDP (show cdp neighbor) or LLDP (show lldp neighbor) that the ports you're configuring are indeed the ports that connect your 3750 and 6509 together?

And of course verify that none of the ports you're configuring on either switch aren't put into err-disabled or shutdown state.

Best regards,

Peter

View solution in original post

Peter Paluch · ‎03-04-2013

Hi Milos,

Can you issue the show interfaces status err-disabled and post the output here after the ports on the 3750 get err-disabled? This will reveal the reason why they are being err-disabled.

Very often, ports like these get err-disabled because of EtherChannel Misconfig Guard that gets tripped when the opposite device has not its ports bundled into an EtherChannel yet. However, the configuration of your 6509 seems to be okay for the time being. Are you sure you are bundling the correct ports?

One recommendation - do not use channel-group mode on unless absolutely necessary. Use LACP to negotiate the EtherChannel, i.e. I strongly suggest removing the EtherChannel configuration altogether and instead, reconfigure the EtherChannel on both devices using channel-group mode active. This will both avoid your ports being put into err-disabled state because of problems with EtherChannel, and allow these devices to actually negotiate whether the ports can be bundled.

Best regards,

Peter

fearless.rabbit · ‎03-04-2013

Dear Peter,

Thank you very much for such a quick reply

I have put all interfaces into active mode instead of on, and now Port-channel interfaces are err-disabled as well.

Here is show interfaces status err-disabled

DV-CORE#show interfaces status err-disabled

Port Name Status Reason Err-disabled Vlans

Gi1/0/22 ***VSS-RNOC-link** err-disabled channel-misconfig (STP)

Gi2/0/21 err-disabled channel-misconfig (STP)

Po15 ***VSS-RNOC-link** err-disabled channel-misconfig (STP)

DV-CORE#

Wow, wow wow.. is STP taking them down?

Peter Paluch · ‎03-04-2013

Hello Milos,

Suggestion - these kinds of errors usually occur when you change the mode of the physical interfaces (e.g. from on to active) but do not delete the logical Port-channel interfaces. It appears that the Port-channel interface is internally created with a notion of the signalling protocol being used, and if the protocol is changed on the physical interface, the internal Port-channel setting is mismatched, and as a result, the ports are being put into err-disabled state.

My suggestion:

Remove the channel-group commands from all affected physical interfaces. In other words, tear down the EtherChannel between your 3750 and 6509 completely.
Shut down the physical ports if not shut down already. This will clear the err-disabled state.
Remove the Port-channel interfaces using no interface port-channel X if they still exist. This is a very important step.
Now, start the physical interfaces again using no shutdown and bundle them again using channel-group mode active. Do not manually create the Port-channel interfaces - they will be created automatically.

Please give this a try and let us know how it worked. Thank you!

Best regards,

Peter

fearless.rabbit · ‎03-04-2013

Ok, I did as you said - excluded ports from EtherChannel, shut them down, and deleted the previous Port-channel interfaces on both 3750 and 6509. I have let new Port-channel interfaces be created automatically, unlike me creating them manually before

Now only two interface are up. Other two physical interfaces and both Port-channel interfaces are down, line protocol is down (notconnect).

Maybe I should try having same Port-channel ID on both switches (it is 15 on 3750 and 220 on 6509)? I believe this is just locally significant, but maybe is worth a try?

P.S. I have put them all in active mode.

Ton V Engelen · ‎03-04-2013

Hi

maybe its removed but i dont see "switchport mode trunk" configured on the 3750 port-channel and interfaces.

i have configured the 3750X and the 65009-E w/ channels like this, works like a charm:

3750X

!

interface Port-channel25

switchport trunk encapsulation dot1q

switchport mode trunk

interface GigabitEthernet1/1/1

switchport trunk encapsulation dot1q

switchport mode trunk

channel-protocol lacp

channel-group 25 mode active

interface GigabitEthernet1/1/3

switchport trunk encapsulation dot1q

switchport mode trunk

channel-protocol lacp

channel-group 25 mode active

6509-E

interface Port-channel25

switchport

switchport trunk encapsulation dot1q

switchport mode trunk

mls qos trust dscp

interface GigabitEthernet7/10

switchport

switchport trunk encapsulation dot1q

switchport mode trunk

channel-protocol lacp

channel-group 25 mode active

interface GigabitEthernet8/10

switchport

switchport trunk encapsulation dot1q

switchport mode trunk

channel-protocol lacp

channel-group 25 mode active

fearless.rabbit · ‎03-04-2013

Defaulted them all again, recreated Port-channels.. it still wont come up It`s like it ignores second interface altogether.

show etherchannel summary on 3750:

15 Po15(SD) LACP Gi1/0/22(I) Gi2/0/22(D)

sh etherchannel 15 detail

Group state = L2

Ports: 2 Maxports = 16

Port-channels: 1 Max Port-channels = 16

Protocol: LACP

Minimum Links: 0

Ports in the group:

-------------------

Port: Gi1/0/22

------------

Port state = Up Sngl-port-Bndl Mstr Not-in-Bndl

Channel group = 15 Mode = Active Gcchange = -

Port-channel = null GC = - Pseudo port-channel = Po15

Port index = 0 Load = 0x00 Protocol = LACP

Flags: S - Device is sending Slow LACPDUs F - Device is sending fast LACPDUs.

A - Device is in active mode. P - Device is in passive mode.

Local information:

LACP port Admin Oper Port Port

Port Flags State Priority Key Key Number State

Gi1/0/22 SA indep 32768 0xF 0xF 0x117 0x7D

Age of the port in the current state: 0d:00h:05m:03s

Port: Gi2/0/22

------------

Port state = Down Not-in-Bndl

Channel group = 15 Mode = Active Gcchange = -

Port-channel = null GC = - Pseudo port-channel = Po15

Port index = 0 Load = 0x00 Protocol = LACP

Flags: S - Device is sending Slow LACPDUs F - Device is sending fast LACPDUs.

A - Device is in active mode. P - Device is in passive mode.

Local information:

LACP port Admin Oper Port Port

Port Flags State Priority Key Key Number State

Gi2/0/22 SA down 32768 0xF 0x0 0x217 0x45

Age of the port in the current state: 0d:00h:06m:23s

Port-channels in the group:

---------------------------

Port-channel: Po15 (Primary Aggregator)

------------

Age of the Port-channel = 0d:00h:06m:38s

Logical slot/port = 10/15 Number of ports = 0

HotStandBy port = null

Port state = Port-channel Ag-Not-Inuse

Protocol = LACP

Port security = Disabled

DV-CORE#sh int g1/0/22 status

Port Name Status Vlan Duplex Speed Type

Gi1/0/22 ***VSS-RNOC-link** connected trunk full 1000 10/100/1000BaseTX

DV-CORE#sh int g2/0/22 status

Port Name Status Vlan Duplex Speed Type

Gi2/0/22 ***VSS-RNOC-link** notconnect 1 full 1000 10/100/1000BaseTX

3750 config:

!

interface Port-channel15

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 2

switchport mode trunk

end

!

interface GigabitEthernet1/0/22

description ***VSS-RNOC-link***

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 2

switchport mode trunk

speed 1000

duplex full

no vtp

channel-protocol lacp

channel-group 15 mode active

end

!

interface GigabitEthernet2/0/22

description ***VSS-RNOC-link***

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 2

switchport mode trunk

speed 1000

duplex full

no vtp

channel-protocol lacp

channel-group 15 mode active

end

6509 config:

!

interface Port-channel220

switchport

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 2

switchport mode trunk

end

!

interface GigabitEthernet2/1/28

description ***Dom_Vojske_Failover_link***

switchport

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 2

switchport mode trunk

speed 1000

duplex full

channel-protocol lacp

channel-group 220 mode active

end

!

interface GigabitEthernet2/1/29

description ***Dom_Vojske_Failover_link***

switchport

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 2

switchport mode trunk

speed 1000

duplex full

channel-protocol lacp

channel-group 220 mode active

end

Peter Paluch · ‎03-04-2013

Hello Milos,

The Port-channel number is irrelevant - it can safely be different between different switches.

Would you mind posting the output of the following commands on the 3750?

show cdp neighbor
show ether sum
show ether 25 detail
show int gi1/1/1 status
show int gi1/1/3 status

Thank you!

Best regards,

Peter

Wilson Bonilla · ‎03-04-2013

Hello.

Error disable reason clearly states:

Po15 ***VSS-RNOC-link** err-disabled channel-misconfig (STP)

Spanning-tree inconsistent !!! Check the physical connections, because either your 3750 stack or your VSS chassis

is broken into two. Or you simple have interfaces going to another switch that we don't have in the picture.

Regards.

Wilson B.

fearless.rabbit · ‎03-04-2013

My VSS is in fact two 6509E chassis. I am connecting 3750 to one of them. Could this somehow be the issue?

Peter Paluch · ‎03-04-2013

Hello Milos,

It is a blind shot but I am increasingly getting the feeling that the ports you are configuring for the EtherChannel are not the true ports that interconnect your 3750 and 6509 together.

Note the output you've provided:

show etherchannel summary on 3750:
15     Po15(SD)        LACP      Gi1/0/22(I) Gi2/0/22(D)

The gi1/0/22 is identified as an (I)ndividual port - that means that the opposite device is not speaking LACP to it so it cannot negotiate about even becoming an EtherChannel member link. The gi2/0/22 is (D)own meaning that the opposite port is shutdown or not even connected.

Can you verify using CDP (show cdp neighbor) or LLDP (show lldp neighbor) that the ports you're configuring are indeed the ports that connect your 3750 and 6509 together?

And of course verify that none of the ports you're configuring on either switch aren't put into err-disabled or shutdown state.

Best regards,

Peter

fearless.rabbit · ‎03-04-2013

Well Peter, your blind hit is more accurate than 48 hours of my "troubleshooting". I have been configuring wrong VSS chassis as along! God knows what I f*cked up on the other side

You learn as long as you live I guess. Thank you all immensly nonetheless guys. I would take me at least another 48 hours until I turned my eye on actual cabling