Edison, Jon and everybody,
I think I've got it. Please bear with me while I explain my thoughts.
As we have discussed here, a pure Etherchannel between just two switches, with one switch being configured for Etherchannel and the other not, may result in frames being reflected (bounced) back but not in frames going around in circles. A frame received on a member port of an Etherchannel bundle will not be forwarded back through any member port of the same bundle, thus, the loop cannot be formed that way. I have tested this in our lab - I have connected two switches together with two links, one switch was configured for Etherchannel, the other not. No broadcast storm occured as the result although I have intentionally flooded the topology with broadcast traffic (a videostream sent to a broadcast IP address). As soon as I stopped transmitting the stream, the topology remained silent - no frames got caught in a loop, and no network collapse ensued whatsoever.
What I realized is that the loop must be formed by an additional redundant link in the topology that somehow - when combined with a misconfigured Etherchannel - results in multiple paths between switches, thus forming a loop. The networks we have seen to collapse under Etherchannel misconfiguration were always more redundant than just a single Etherchannel between switches. I began to suspect that there must be additional redundancy present in the network so that the loop can be formed.
I have therefore analyzed the simplest scenario fulfilling this requirement - adding a third standalone link between the two switches. To better visualize the concept, please have a look at the exhibit Example1.png I have attached. There are two switches - the Distribution Switch (DS) and the Access Switch (AS). The standalone link is the Fa0/1. Furthermore, DS has ports Fa0/2-3 bundled in an Etherchannel while AS has no Etherchannel configured. The DS is configured as STP Root. What will now happen?
- Because DS is STP Root, all its ports (Fa0/1 and Po1) are Designated Forwarding
- AS will receive BPDUs via Fa0/1 and via exactly one of the links of the Etherchannel from DS. Let's assume that the link is Fa0/2
- AS will declare Fa0/1 to be its Root port (the lowest sender port ID) and Fa0/2 as Alternate Discarding.
- However, AS receives no BPDUs on Fa0/3. Therefore, it declares the Fa0/3 as Designated Forwarding.
And voila! - we have a loop here - two links completely unblocked and forwarding: Fa0/1 and Fa0/3. A single broadcast now starts the usual broadcast storm that I've have striving for so long!
And then it all suddenly began making sense. The true loop with frames endlessly circulating is not actually created by the presence of the misconfigured Etherchannel itself but rather by the modified operation of STP over an Etherchannel bundle - that the BPDUs for a particular VLAN are sent through a single link in the entire bundle. All other bundled ports that do not carry BPDUs can be mistakenly considered as eligible for Designated Forwarding by the switch with a missing/misconfigured Etherchannel, and that forms the basis of the actual loops.
I have subsequently analyzed a rather common scenario with two distribution layer switches and an access switch connected to both distribution switches. Please see first the Example2.png. The DS1 is configured as STP Root, the DS2 is configured as STP Secondary Root. In this exhibit, the AS has the ports Fa0/1-2 unconfigured by mistake (or not yet). Assuming that the bundle on AS towards DS2 becomes the root port (Etherchannel has a lower cost than individual links so with enough links in an Etherchannel of an appropriate speed, this may happen by default), one of the ports Fa0/1-2 on AS becomes Alternate Discarding (because it receives BPDUs from DS1) and the other becomes Designated Forwarding as it receives no BPDUs. A loop is thus formed.
The Example3.png depicts another common scenario with ports Fa0/5-6 unbundled on AS. DS1 is again STP Root, DS2 is STP Secondary Root. Here, the bundle on AS towards DS1 will be declared Root port, and because the DS2 has a lower BID than AS (it is Secondary Root), its bundle towards AS will be declared as Designated Forwarding. Again, AS will declare one of the ports Fa0/5-6 as Alternate Discarding and the other as Designated Forwarding, and here we have the loop again.
I assume this is actually what brought down the networks with misconfigured Etherchannels.
I have experimentally tested all three scenarios in our lab and I have been able to easily create the broadcast storm in each of these cases. Furthermore, I have not deactivated the STP Etherchannel Misconfig Guard. Even with this guard left active, there was absolutely no problem in creating these loops as described earlier. The reason is that this guard basically reacts to arrival of BPDUs sent from differing MAC addresses on ports bundled in an Etherchannel which is not expected in correct configuration. However, in my particular topology, each bundle consisted of two links. Whenever AS declared one of these links as Alternate Discarding and the second as Designated Forwarding, the Etherchannel received BPDUs only via the Forwarding link and so the EC Misconfig Guard had no reason to kick in. It would be probably different if my bundles consisted of at least 3 links but for two-link bundles, this guard is helpless (which is logical - it performs only local decisions and so has very limited information).
Phew I would like to thank ANYONE that has joined this thread so far and helped me to finally resolve this mystery!
Best regards,
Peter
