topic Re: How does a loop form in a misconfigured Etherchannel? in Switching

How does a loop form in a misconfigured Etherchannel?

Peter Paluch — Wed, 06 Mar 2019 22:21:29 GMT

Dear friends,

It is a commonly seen and practically proven issue that if two switches are interconnected by a number of parallel links which are bundled into an Etherchannel on one switch (obviously using the on mode) while being unbundled on the second switch, a Layer2 loop may well be created. However, I do not understand the exact mechanism of the formation of this loop.

I am well aware of the basic principles behind: I know that STP treats the Port-channel interface as a single interface, and thus all member links bundled in this Etherchannel share the same STP state/role. I also understand that a broadcast/multicast/unknown unicast frame sent by a port in the Etherchannel will reach the opposite switch and get flooded over all remaining links, eventually arriving back at the switch with the configured Etherchannel.

And this precise moment is where my understanding ends: The frame arrived back and its destination is still unknown. However, from the viewpoint of the switch, the frame came in through a particular Port-channel interface. If this switch floods the frame, it will flood it through all remaining ports except the port through which the frame came in, meaning that the frame will never be sent back through the Port-channel. How does the loop get created, then?

Thank you very much for helping me out with this!

Best regards,

Peter

Re: How does a loop form in a misconfigured Etherchannel?

Edison Ortiz — Sun, 05 Dec 2010 12:40:34 GMT

Peter,

Packets will be flooded out of all interfaces for unknown broadcast/unicast packets with the exception of the interface where the flood came from.

In a 2 switch topology, Switch A (which has the bundle) will flood the packet out of the members of the bundle expecting Switch B to receive the packet on its bundle.

However Switch B will only receive the packet on one of its physical interfaces and can potentially flood back to Switch B out of its other connected physical interface causing the STP loop.

Regards,

Edison.

Re: How does a loop form in a misconfigured Etherchannel?

Jon Marshall — Sun, 05 Dec 2010 12:43:21 GMT

Peter

My undestanding is -

sw1 = switch with etherchannel gi0/1 - 4

sw2 = switch without etherchannel gi0/1 - 4 not etherchanneled

sw1 selects a port in the etherchannel and transmits the broadcast eg gi0/2. It arrives at sw2 as you say and because the there is no etherchannel configured sw2 floods the broadcast on ports gi0/1, 3 & 4 but not 2 because it received it on that port.

sw1 receives the packet back on gi0/1, 3 & 4 but not gi0/2. So sw1 then refloods the packet on gi0/2 back to sw2 and the whole process starts over again.

Jon

Re: How does a loop form in a misconfigured Etherchannel?

Peter Paluch — Sun, 05 Dec 2010 12:46:26 GMT

Hi Edison,

Thanks so much for your reply.

In a 2 switch topology, Switch A (which has the bundle) will flood the packet out of the members of the bundle expecting Switch B to receive the packet on its bundle.

However Switch B will only receive the packet on one of its physical interfaces and can potentially flood back to Switch B out of its other connected physical interface causing the STP loop.

Exactly! And this is precisely what I wrote in my first post. But my doubts go further, and I will repost them here for clarity:

The frame arrived back to the switch A that has the bundle configured and the frame's destination is still unknown. However, from the viewpoint of the switch, the frame came in through a particular Port-channel interface. If this switch floods the frame, it will flood it through all remaining ports except the port through which the frame came in, meaning that the frame will never be sent back through the Port-channel.

Is this assumption correct?

Best regards,

Peter

Re: How does a loop form in a misconfigured Etherchannel?

Jon Marshall — Sun, 05 Dec 2010 12:46:37 GMT

ediortiz wrote:

Peter,

Packets will be flooded out of all interfaces for unknown broadcast/unicast packets with the exception of the interface where the flood came from.

In a 2 switch topology, Switch A (which has the bundle) will flood the packet out of the members of the bundle expecting Switch B to receive the packet on its bundle.

However Switch B will only receive the packet on one of its physical interfaces and can potentially flood back to Switch B out of its other connected physical interface causing the STP loop.

Regards,

Edison.

Edison

Okay, i'm confused now.

Does etherchannel send a broadcast across all of it links or just pick one of them ?

You also say when switch A floods the broadcast out of all the members of the bundle, switch B will only receive it one interface. Not understanding this as they are still physical connections so if A sends on all links B will receive on all links ?

Could you explain a bit more ?

Jon

Re: How does a loop form in a misconfigured Etherchannel?

Peter Paluch — Sun, 05 Dec 2010 12:53:01 GMT

Hi Jon,

Thank you very much for joining this thread!

sw1 receives the packet back on gi0/1, 3 & 4 but not gi0/2. So sw1 then refloods the packet on gi0/2 back to sw2 and the whole process starts over again.

Right, the sw1 will receive the frame back on Gi0/1, Gi0/3 and Gi0/4. But for the sw1, all these ports are just members of the Port-channel interface. A frame arriving through any of the member links is treated as arriving through the Port-channel itself. Even the MAC addresses are learned on Port-channel interfaces, not on physical member ports. Therefore, when sw1 forwards the frame, it - by an elementary switch logic - cannot forward it back through the original ingress interface which is Port-channel.

And this is my trouble here.

Best regards,

Peter

Re: How does a loop form in a misconfigured Etherchannel?

Edison Ortiz — Sun, 05 Dec 2010 12:53:21 GMT

Jon,

Your explanation and mine are the same - I don't understand how you can be confused.

The EC will hash out of one link but the logical link will be treated as one for the STP loop so it will eliminate this interface as a candidate for the egress flood.

*** Edited for Clarification

Re: How does a loop form in a misconfigured Etherchannel?

Jon Marshall — Sun, 05 Dec 2010 12:58:09 GMT

I don't understand how you can be confused.

Ohh, that can happen all too easily

No problem, must be the way i was reading it.

Jon

Re: How does a loop form in a misconfigured Etherchannel?

Edison Ortiz — Sun, 05 Dec 2010 13:05:53 GMT

You are correct but the frame that was sent originally came back via the same interface - in essence that's a loop.

As you stated, it won't send the frame back out but take into account a network with tons of unknown of multicast/broadcast traffic.

Re: How does a loop form in a misconfigured Etherchannel?

Edison Ortiz — Sun, 05 Dec 2010 13:06:14 GMT

Re: How does a loop form in a misconfigured Etherchannel?

Peter Paluch — Sun, 05 Dec 2010 13:18:39 GMT

Hello Edison,

You are correct but the frame that was sent originally came back via the same interface - in essence that's a loop.

Certainly, I agree with this. But this kind of loop does not result in the same broadcast frame to loop forever. Yet, we have seen the network being brought to its knees by misconfigured Etherchannels. A broadcast frame hitting a switch twice and being dropped afterwards, pardon me... I don't believe it can cause such a massive outage.

So what happens in such instances?

Best regards,

Peter

Re: How does a loop form in a misconfigured Etherchannel?

Jon Marshall — Sun, 05 Dec 2010 13:28:34 GMT

Peter

Etherchannel can be configured to learn addresses in 2 ways -

1) aggregate port learning ie. the mac-addresses are associated with the port-channel interface

2) physical port learning - speaks for itself really

If the switch was using physical port learning then there could indeed be a loop created by the method Edison and I described as far as i can see.

However, having said that,i noticed your recent post (yesterday) on the etherchannel broadcast storm and after checking the config guides for both 2960 and 3750 switches it states that physical port learning is not supported on those switches so it doesn't answer the previous posts issue.

Jon

Re: How does a loop form in a misconfigured Etherchannel?

Edison Ortiz — Sun, 05 Dec 2010 13:57:12 GMT

Peter,

Packet duplication in a large network can certainly put any switch on its knees. The cases may be different on each EC misconfiguration. It depends on the network state. I've seen networks with EC misconfiguration running for a while until heavy traffic traverses those links.

Say you deploy multicast on that segment after those switches have been misconfigured for a while and hell break loose. You would attribute the problem with the multicast deployment while the culprit was a bad design to begin with.

Re: How does a loop form in a misconfigured Etherchannel?

Peter Paluch — Sun, 05 Dec 2010 14:03:13 GMT

Hi Jon,

Thanks for your insights. Correct, I have almost forgotten about the physical learning method for Etherchannels. But even if taking that into account, I still do not believe that it could cause frames to be caught in a loop. Let me explain.

We all agree that a frame can be returned back to switch A (with the Etherchannel) if the neighboring switch B does not have the Etherchannel configured. No doubt about that. Jon, now, your supposition is that because the unknown frame is forwarded back to switch A through almost all member links of the bundle, the switch A will somehow re-flood the frame back to switch B. Is that correct?

Note that this situation is not specific only for Etherchannel misconfiguration. The switch B can have the Etherchannel configured just fine but it suddenly receives a burst of frames with different destination MACs, all unknown, or its load-balancing mechanism may react to something different that just the source/destination MAC combination. From the switch A viewpoint, the situation is no different, and yet, no looping ensues - even with physical address learning, otherwise the Etherchannel would be unusable per se.

Best regards,

Peter

Re: How does a loop form in a misconfigured Etherchannel?

Jon Marshall — Sun, 05 Dec 2010 14:09:38 GMT

Peter

A viewpoint, the situation is no different, and yet, no looping ensues - even with physical address learning, otherwise the Etherchannel would be unusable per se.

That's a good point, it would be no different than an etherchannel receiving an unknown unicast or broadcast for that matter from a fully functioning etherchanel at the other end.

Jon

Re: How does a loop form in a misconfigured Etherchannel?

Peter Paluch — Sun, 05 Dec 2010 14:09:42 GMT

Edison,

I get the point. I am now moving on a thin ice as I am talking about something I believe to have seen in the past but I am not entirely sure: with a misconfigured Etherchannel, an obvious storm ensued, with the switches indicating heavy traffic on all their LEDs (blinking wildly). That would mean the frames received by a Port-channel interface were reflooded back which is actually the reason behind this entire thread. Have you had a similar experience?

Best regards,

Peter

Re: How does a loop form in a misconfigured Etherchannel?

Peter Paluch — Sun, 05 Dec 2010 14:23:27 GMT

Edison,

Still one more comment, though:

Packet duplication in a large network can certainly put any switch on its knees.

Agree. But the switch having the Etherchannel configured should actually not be bothered so much with the traffic "reflected" back from the opposite unconfigured switch - it would merely drop it which is performed in the hardware as one of the basic switching functions. Even the unconfigured switch should not be loaded in a significantly higher way - it is in the same position as a switch receiving a broadcast/multicast traffic on a port and replicating the flow on remaining ports. Neither of these switch is actually going to process and forward any such frame twice.

Best regards,

Peter

Re: How does a loop form in a misconfigured Etherchannel?

Edison Ortiz — Sun, 05 Dec 2010 14:33:30 GMT

Peter,

I haven't seen the case where packets are being flooded back via the EC - if that was your intended question. However, I wouldn't discard this scenario which can be attributed to a software defect along with poor design implementation.

Re: How does a loop form in a misconfigured Etherchannel?

Edison Ortiz — Sun, 05 Dec 2010 14:34:15 GMT

We need to take into account where the STP Root may be located as well.

Re: How does a loop form in a misconfigured Etherchannel?

Jon Marshall — Sun, 05 Dec 2010 15:01:04 GMT

Do either of you guys know what would happen if a switch received a packet on a port-channel interface and the destination mac was pointing back out the same po1 interface eg.

po1 = gi0/1 - 4

switch receives packet with dst mac aa.bb on gi0/2 and has recorded in it's cam table that aa.bb is reachable via po1 ? Would it simply select one of it'sl links and send the packet back out ?

Jon