04-13-2022 12:32 PM
I'm looking for some best practices for setting up a trunked Etherchannel between two 9300 switch stacks. We have had issues with Etherchannel setup since moving to the 9300 platform. I've always configured a port channel > then setup my interfaces with the following and they work until we got 9300's. I'm not sure whether the non negotiate commands are a problem or not but this setup below is currently working but we are trying to add another new stack and the interfaces go into err disable state as soon as you do a shut/no shut. Same config as below on the new stack but with a new
Switch 1
interface Port-channel1
switchport mode trunk
switchport nonegotiate
interface GigabitEthernet1/1/1
description Uplink to core
switchport mode trunk
switchport nonegotiate
channel-group 1 mode on
!
interface GigabitEthernet1/1/2
description Uplink to core
switchport mode trunk
switchport nonegotiate
channel-group 1 mode on
Switch 2
interface Port-channel23
description ESX_DEV_SWITCH Trunk
switchport mode trunk
interface GigabitEthernet1/0/5
description ESX Dev Sw1 Uplink
switchport mode trunk
channel-group 23 mode on
!
interface GigabitEthernet1/0/6
description ESX Dev Sw3 Uplink
switchport mode trunk
channel-group 23 mode on
Solved! Go to Solution.
04-18-2022 12:04 PM
We got this working. We did remove the nonnegotiate commands and then brought up the interfaces one at a time until they were joined and working.
04-13-2022 12:34 PM
I wanted to add that I have setup a new port channel for the new stack. port channel 5 on my core. It cut that off my original post.
04-13-2022 12:48 PM - edited 04-13-2022 12:59 PM
What happens if you delete "switchport nonegotiate" from the Portchannel and the physical interfaces?
If removing "switchport nonegotiate" command does not help, can you try the following:
delete the portchannel
delete the channel-group x mode on from the physical interface.
shut down the physical interface
and redo everything this way
Do not use the same Portchannle number that was not working. use a new one.
under the physical interface add channel-group x mode on
un-shut the interfaces
By doing a no sh to the interfaces the new Portchannle should be created and the Portchannel should come up, if not make sure that the Portchannel is not shut and test again.
HTH
04-13-2022 01:00 PM
Same thing, it goes into the err disable state with a shut/no shut.
04-13-2022 12:43 PM - edited 04-13-2022 12:51 PM
Hello,
There could be just some issues with the ether channel configuration to put it into err-disable mode. As I'm sure you know the ports have to be the same config to bundle and pair with the other side. Sometimes speed/duplex are negotiated if not specifically configured and could cause issues. I would hard set these values, as well as the no negotiate command on ALL interfaces participating if thats what you are trying to achieve.
Some ether channel configurations also need a minimum amount of ports to be involved before comin gup.
The err-disable state makes me think that Ether channel Misconfiguration Guard is on (default setting I believe) to detect misconfigurations.
Can you show err-disable command?
and
show spanning-tree summary
-David
04-13-2022 12:59 PM
I currently don't have anything configured for the err-disable so it's using the default.
04-13-2022 01:02 PM - edited 04-13-2022 01:02 PM
Yes I was saying the misconfiguration guard for ether channel is ON by default if I remember correctly.
Tee only difference on your commands I see is the negotiate command. Try removing it from all ports and the ether channel.
04-13-2022 01:23 PM
Can you show the output of the below command:
show spanning-tree summary
-David
04-14-2022 05:02 AM
I checked and I didn't see anything odd.
04-13-2022 01:32 PM
There are many reasons port go in to error disabled, You did not mention which side error is disable ?
Looking at your config : (post-show version from both the switches)
1. Switch 1 you are using the uplink module to connect to the port-channel (how many switches in the stack ) - post-show switch and show modules help here
2. Switch 2 you are using normal gig ports (in the description you mentioned switch3 uplink, are you sure you connecting right ports here ? (I suspect) - show cdp neigh (help here)
are you sure you connecting these 2 switches only, or do any other devices in the network participate here, do you have any small network diagram, I see one side you connecting core and the other side esxi server?
Let's do a simple test.
switch 1 :
(remove the cables)
default interface GigabitEthernet1/1/1
default interface GigabitEthernet1/1/2
interface GigabitEthernet1/1/1
switchport mode trunk
channel-group 1 mode on
!
interface GigabitEthernet1/1/2
switchport mode trunk
channel-group 1 mode on
!
interface Port-channel1
switchport mode trunk
no shutdown
Switch 2
remove the cables
default interface GigabitEthernet1/0/5
default interface GigabitEthernet1/0/6
interface GigabitEthernet1/0/5
description ESX Dev Sw1 Uplink
switchport mode trunk
channel-group 23 mode on
!
interface GigabitEthernet1/0/6
description ESX Dev Sw3 Uplink
switchport mode trunk
channel-group 23 mode on
interface Port-channel23
description ESX_DEV_SWITCH Trunk
switchport mode trunk
no shut
connect only a single cable and what is the outcome.
04-14-2022 04:57 AM
I hadn't considered that the module ports might be a little different that the standard gig ports. This is a stack in an ESX server rack for the many connections an ESX servers needs. The stack it's connecting to is another 9300 stack with other stacks connected to it just fine. I have 3750 stacks and another 9300 stack connected and working fine. On the other 9300 stack that is working I'm also using 4 ports from the module for uplinks to the main 9300 stack. The main 9300 stack is only using the standard gig ports and not the module. It just seems the 9300 stacks don't operate like other switch stacks I've worked with with Etherchannel. Maybe they made changes to STP on the 9300. I may just open up a case with TAC and see what they say. Weird I have one 9300 stack working fine but can't get this 9300 stack up and I've configured the port channel and ports exactly the same for the uplinks.
04-14-2022 05:01 AM
I did end up getting one standard gig port up and was able to pull in my VTP info and assign an IP on our MGMT VLAN so I could get to the device remotely. I did try one port at at time with the module ports and it goes into err disable as soon as I do a shut/no shut. I was only able to get the standard gig port talking so I'm thinking it's something with the module interfaces. Maybe I need to set the speed on those module interfaces to full/1000.
04-14-2022 08:37 AM
Do you have rough diagram how they connected ?
which part is going error disable, can you post the out of the interaces here.
04-18-2022 12:04 PM
We got this working. We did remove the nonnegotiate commands and then brought up the interfaces one at a time until they were joined and working.
04-18-2022 12:39 PM
glad all is good and working, thank you for sharing your input, appreciated.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide