Broadcast storm : How to handle it ?

[ITexpert]

Hello @Julio E. Moisa @Francesco Molino @Joseph W. Doherty  

yesterday, i got calls that everything is slow on network.  When I see on switches, there was lots of broadcast packets.

According to my knowledge, Broadcast means, when packet is sent to everybody, so more broadcast means flooding of packets etc.

I am not sure why devices or switches generate broadcast packets, how i can reduce the broadcast traffic in my network ?

what will happen after configuring the broadcast storm, does it will shutdown or error-disable any port ?

I also See the DropsTx. on trunk interfaces, how i can reduce these numbers ?

Thanks 

[Julio E. Moisa]

Hi

You could configure storm control on the ports, 2 actions are available: shutdown the port when the threshold is reached and the other one is generate alert notifications, it can be useful to determine the top offender port and track the source. 

It could be an attack so you could identify any sources and block them.

[Francesco Molino]

Hi

as @Julio E. Moisa said you can enable storm-control on ports.

Broadcast can be the consequence of so many issues, could be spanning-tree loop issue.

You can, for example, follow spanning-tree change topology using the command sh spanning-tree detail | i ieee|occur|from|is exec

In this output, you will see which port (meaning which device behind this port) has initiated a topology change. And if you follow it by cascade, you will maybe see the switch causing issues with a loop.

It could be a device crashing sending lot of broadcast and for that, you'll need to capture the traffic and see what's the source mac address and search it in your lan using command sh mac add add xxxx.xxxx.xxxx. When found disconnect it to validate, it was the real device causing the issue and check why..

these are just few examples

[Joseph W. Doherty]

"I am not sure why devices or switches generate broadcast packets, how i can reduce the broadcast traffic in my network ?"

Depends on the cause of the broadcast traffic. One way to "reduce" it is by reducing the size of your networks. Example moving from a /24 to a /25, etc. Of course, if the broadcasts are due to some kind of DoS attack, you want to eliminate the cause.

"what will happen after configuring the broadcast storm, does it will shutdown or error-disable any port ?"

Broadcast storm, generally, begins to drop broadcast packets above a certain amount. Unfortunately, although this make keep such packets from totally disrupting your network, as often broadcast packets have a purpose, just dropping them can cause other issues.

"I also See the DropsTx. on trunk interfaces, how i can reduce these numbers ?"

That depends on the device, and as drops are also often used to trigger flow rate control, trying to totally eliminate them can be problematic. Ideally drop management is part of your QoS strategy.

[ITexpert]

Hello @Joseph W. Doherty @Francesco Molino @Julio E. Moisa  

 This  is the issue now,   I have configured MSTP in default mode with just one instance because everything is in default vlan at site.

 I am getting MSTP Toplogy change notification on each switch on internal network now after few minutes, 

 yesterday I also got some notifications + notification that all LLDP neighbours delete and then Create  and same time I also got notifications that all ports goes down and then come up.  the time diff bw port going down and up was 2 or 3 seconds.

It was on all switches.  That was the time when people complaint about speed issues to access network media

Thanks

[Joseph W. Doherty]

If you only have one VLAN (or very few VLANs - and just Cisco switches), good chance rapid-PVST would have been fine.

In any case, do you have port-fast configured? If not, edge ports going up/down may be causing your excessive CNs (along with taking longer to bring an edge port fully on-line).

[Leo Laohoo]

---

@ITexpert wrote:

I am not sure why devices or switches generate broadcast packets, how i can reduce the broadcast traffic in my network 

---

Create or use smaller subnets.

[ITexpert]

Hello @Leo Laohoo @Richard Burts @Joseph W. Doherty @Francesco Molino @Julio E. Moisa 

Now i am confused and i believe i am missing some entry level cocepts, 

 I can totally understand that by making smaller subnets, the broadcast traffic will decrease as number of availble IP's decrease.

but i want to know ,

Why Broadcast packets exist, whats the purpose of them  ?  Why devices originate that ?  please explain unicast vs multicast vs Broadcast traffic ? I know the defination but i want to understand it with real scenario examples ?

Does BPDU's are type of broadcast messages ?

@Joseph W. Doherty  How edge devices can generate TCN's if they do not produce BPDU's ?  I am really sorry but want to undestand in depth ?

Thanks in Advance 

[Joseph W. Doherty]

"Why Broadcast packets exist, whats the purpose of them ?"

It's for messages that one host may believe all other hosts, or many (including unknown to it) might want to receive. It's also done with just one frame/packet rather than sending packets to each other hosts (especially problematic if they are "unknown"). Simple example of a broadcast, ARP request, i.e. who's got this IP?

"Why devices originate that ?"

Any can.

"please explain unicast vs multicast vs Broadcast traffic ?"

Unicast - Frame/packet to sent to one other host. Multicast - Frame/packet to sent to a group (0 to N number) of hosts. Broadcast - Frame/packet to sent to all other hosts (usually limited to sender's network).

"How edge devices can generate TCN's if they do not produce BPDU's ? I am really sorry but want to undestand in depth ?"

NB: I'm rusty on L2 STP, as generally don't use it except for accidental loop prevention on L3 networks. I.e. take what I say on this subject with a grain of salt.

However, I recall (?), without port-fast, an edge port going up/or down triggers a TCN from the hosting (switch) device, as it's "unknown" whether the topology might have changed until STP resolves the situation. With port-fast, the hosting (switch) devices "knows" the topology shouldn't be impacted because the port should be a network edge, i.e. a loop should not happen. (BTW, on Cisco switches, STP is still monitoring port-edge ports, and will revert to full STP operation if it sees an BPDU on the port, but since it skips the initial STP steps, it's possible a port-fast port can cause a L2 loop before STP blocks the loop.)

(I recall [?] setting edge-ports to port-fast stops needless TCNs, besides allowing an edge port to fully join the L2 topology much faster.)