Hello,
I'm in the process of changing my core from two Cat4.5K to a two member stack of cisco SG550XG-24F and a two member stack of cisco SG350XG-24F. Both stacks running 2.4.0.94. The access Switches are a lot of SG300 (all running software version 1.4.9.4) with different port densities. I'm running in several problems regarding Port-Channels.
An example is the following setup. A rack consisting of two SG300s, each one is connected with the two 1G Fiber Ports (gi27 and gi28) with a 2 port LACP port channel (Po2) to two 10G SFP+ ports of the SG550 stack, one port of the portchannel on each stack member (e. g. te1/0/8 and te2/0/8). The network is running rstp and the two rack switches have a two port lacp portchannel crossconnect in between them as a backup with a high spanningtree priority (the plan is to get rid of the crosslinks and spanningtree in the future). Before (with the Cat4.5k) the setup was nearly the same, but each rack switch was beeing connected to one of the Cat4.5K with a port channel.
Now I got the problem, that interfaces are joining and leaving the port channel _very_ often. Here are some log excerpts from the switches (log verbosity is set to debugging (!)):
sg300-A:
26-Oct-2018 10:15:36 :%TRUNK-I-PORTADDED: Port gi28 added to Po2
26-Oct-2018 10:15:36 :%TRUNK-W-PORTREMOVED: Port gi28 removed from Po2
26-Oct-2018 10:10:05 :%TRUNK-I-PORTADDED: Port gi27 added to Po2
26-Oct-2018 10:10:05 :%TRUNK-W-PORTREMOVED: Port gi27 removed from Po2
26-Oct-2018 10:08:35 :%TRUNK-I-PORTADDED: Port gi27 added to Po2
26-Oct-2018 10:08:35 :%TRUNK-W-PORTREMOVED: Port gi27 removed from Po2
26-Oct-2018 10:04:05 :%TRUNK-I-PORTADDED: Port gi27 added to Po2
26-Oct-2018 10:04:05 :%TRUNK-W-PORTREMOVED: Port gi27 removed from Po2
SG300-B:
26-Oct-2018 11:26:17 :%TRUNK-I-PORTADDED: Port gi27 added to Po2
26-Oct-2018 11:26:17 :%TRUNK-W-PORTREMOVED: Port gi27 removed from Po2
26-Oct-2018 11:24:47 :%TRUNK-I-PORTADDED: Port gi27 added to Po2
26-Oct-2018 11:24:47 :%TRUNK-W-PORTREMOVED: Port gi27 removed from Po2
26-Oct-2018 11:10:17 :%STP-W-PORTSTATUS: Po2: STP status Forwarding
26-Oct-2018 11:10:17 :%STP-W-PORTSTATUS: Po1: STP status Blocking
26-Oct-2018 11:10:16 :%TRUNK-I-PORTADDED: Port gi28 added to Po2
26-Oct-2018 11:10:16 :%LINK-I-Up: Po2
26-Oct-2018 11:10:16 :%TRUNK-I-PORTADDED: Port gi27 added to Po2
26-Oct-2018 11:10:16 :%LINK-W-Down: Po2
26-Oct-2018 11:10:16 :%STP-W-PORTSTATUS: Po1: STP status Forwarding
26-Oct-2018 11:10:15 :%TRUNK-W-PORTREMOVED: Port gi28 removed from Po2
26-Oct-2018 11:10:15 :%TRUNK-W-PORTREMOVED: Port gi27 removed from Po2
26-Oct-2018 11:07:16 :%TRUNK-I-PORTADDED: Port gi28 added to Po2
26-Oct-2018 11:07:16 :%TRUNK-W-PORTREMOVED: Port gi28 removed from Po2
26-Oct-2018 11:07:15 :%TRUNK-I-PORTADDED: Port gi27 added to Po2
26-Oct-2018 11:07:15 :%TRUNK-W-PORTREMOVED: Port gi27 removed from Po2
SG550-Stack, same time window as above excerpts, Po24 is to SG300-B, Po14 to SG300-A:
26-Oct-2018 11:13:17 :%TRUNK-I-PORTADDED: Port te1/0/12 added to Po24
26-Oct-2018 11:13:16 :%TRUNK-W-PORTREMOVED: Port te1/0/12 removed from Po24
26-Oct-2018 10:44:15 :%TRUNK-I-PORTADDED: Port te2/0/3 added to Po2
26-Oct-2018 10:44:14 :%TRUNK-W-PORTREMOVED: Port te2/0/3 removed from Po2
26-Oct-2018 10:05:36 :%TRUNK-I-PORTADDED: Port te2/0/11 added to Po14
26-Oct-2018 10:05:35 :%TRUNK-W-PORTREMOVED: Port te2/0/11 removed from Po14
example configurations for the ports on the sg550:
interface TengigabitEthernet1/0/12
storm-control broadcast level 1
storm-control multicast level 5
channel-group 24 mode auto
switchport mode trunk
interface Port-Channel14
negotiation 1000f
description SW-R4-A:Po2
switchport mode trunk
no macro auto smartport
for the sg300:
interface gigabitethernet27
ip arp inspection trust
ip dhcp snooping trust
storm-control broadcast level 10
storm-control include-multicast unknown-unicast
channel-group 2 mode auto
interface gigabitethernet27
description "Uplink SW-BB-B:gi1/25"
ip arp inspection trust
ip dhcp snooping trust
storm-control broadcast level 10
storm-control include-multicast unknown-unicast
channel-group 2 mode auto
Connected to the Cat4.5K using the same cables and the same SFP's (additionally using the "lacp timeout short" option on the sg300 interfaces) this setup was running smoothly for several years. I have this problems with all the racks migrated to the new SG550 core. The ones still beeing connected to the Cat4.5k _all_ are fine. I even have this problem with two SG300 beeing directly connected with 1.5m of CAT6 Cable and two RJ45 SFPs to the SG550.
On one Rack I set the port-channels to static on for testing. This one runs smoothly since then.
Actually it gets even more "interesting", as I also have the same problem now with the _untouched_ portchannels of the crossconnect on the rack switches, interfaces leaving and joining the portchannel over and over again.
The configs is the same as above, with only "spanning-tree cost 200000000" added. Again: this is only happening to the rack switches connected to the sg550 stack!
I couldn't find any information about the debug mode for smb's. I'd really like to debug the lacp traffic/connection.
Anyone knows how to do that on SMBs?
Any suggestions are more than appreciated! Thanks!
Apostille:
This is also happening with a two member 2 x 10G LACP portchannel between the SG550 and the SG350 stacks! This portchannel is also distributed over the stack members.
Maybe I should have a look what happens if I don't use cross-stack portchannels...
