Broadcast storm, loop, flood how to find from where the dissasters camming?

[tanner.zaitt]

Hi, how can I find, what is wrong with my network?

My topology is with access Cisco switches, distribution switches and Cisco  core switches but the Cisco core switches they are in stack.
And everything was fine..

Now we migrated old core Cisco switches to new Juniper core QFX and EX access switches in  stack in ring topology, two QFX are In one location like one stack, two QFX are in other location and one QFX in other location.
And all 5 QFX are in stack like one in ring topology.

In Juniper QFX we create Link aggreggation with lacp and we connect all our cisco equipment with channel groups (port channels in lacp mode) to new Juniper QFX  core switches.


And the network is too slow  now...


The problem is all our end user systems in access level work so slowly ...

The all our network is toooooooooo slow..............................

How can we troubleshot this and why it's happened?

[chrihussey]

Think right now there needs to be some clarification to be able to help.

Other than being slow, what are the indicators that there is a broadcast storm, loop or flood? Is there unexplained high utilization on the links? High CPU?

Are spanning tree root bridges defined and do the all the switches see the proper root, especially when looking at the Cisco in tandem with the Junipers switches?

Thanks

[tanner.zaitt]

We are close to solve our problem, we found in one switch in our network unicast and
broadcast activity from Hp thin client witch windows 7 embedded system.

The pings in all our network now is okay and all systems now operate normally.
The monitor messages on Juniper core switch now we receive them in time.

We turned off all our Cisco switches in our network and we started to turn them one by one and we find the switch from where the problems are started.
We played with it, first we turned off it and we saw that the our network is fine, we turned on it and we saw that our network  is not okay.
We find in access ports of this switch Hp thin clients with Windows 7 Embedded.
Soon we will analyze them with Wireshark....

Why and how these unicast and broadcast from n hp thin clients can down all our network....to slow performance..

[chrihussey]

There could be several reasons why and it all depends on the network environment. In the end a broadcast storm or loop causes the network itself to be come a traffic generator. A single broadcast is constantly propagated and looped from switch to switch. So as devices send additional broadcasts, they too are repeated and it just grows and grows.

1- Being that you are using both Cisco and Juniper, is the spanning tree domain stable with root bridge(s) defined and all switches seeing the proper roots? If there are inconsistencies or compatibility issues then that may cause problems.

2- A faulty switch can have the same affect where as it causes a breakdown in the spanning tree domain and causes this looping. Even if the physical topology is "loop free"it can cause problems.

3- In general, excessive host broadcast traffic is more detrimental than unicast. Excessive broadcasting from a host can cause the issues you experienced. Broadcasts have to go to every switch port on that VLAN. So if a device is sending thousands a second that will definitely cause problems. You could look into implementing broadcast suppression or limiting at the port level to try and mitigate it at that level. 

Glad you found the problem.

Regards

[tanner.zaitt]

Junipers are core and root bridge for all Cisco equipment.
The cisco switch is okay where we found the problems because, we just isolate only these ports with unicast and broadcast activity and our network is back on business.
STP is default in cisco and juniper equipment.
Broadcast storm control is not configured I think in cisco equipment.
But in juniper equipment broadcast storm is with default configuration and we change these settings and we try to increase the limit to high value, but it doesn't help us.
We saw in juniper monitor messages DDoS protection messages.
We talk about internal L2 and L3 switching.

I am surprised to see how two end device can down one big L2/L3 network.
We didn't see the loops..
Maybe micro loops?
I am not sure and I don't know what is the problem.

I am too young and without experience.

Can you tell me someone here electronic books for learning networks with free access?

I was study CCNA and CCNA SECURITY, but they are not enough to troubleshooting problems on network or to integrating some network solution to IT infratructures.

What is the best way to learn new big networks?
If documentation is not good or you are there in midle of changes, old to new topology?

I  have heard about Microsoft Vision software for documentation network purposses ?

I think it's a best way to start learn new unknown topology, with starting write documentation and diagramas with optical traces and network equipment locations and connectivity and all information about links, ports, connectivities, vlans, IP addresses, domain names and ....all to description of topology..

What do you think?

[chrihussey]

What is the best way to learn new big networks?

Everybody is different with their on ways that works best for them. There will always be different degrees of documentation, network discovery and learning tools, etc. These all certainly help, but I've never relied on them to upgrade or change a network. For me accessing the network devices, diagramming the network and familiarizing one self with the connectivity, links, traffic patterns, etc., has always worked best. Certainly on larger networks it is more labor intensive, but it often pays off. Especially if one has to embark on changing the network topology. You need a thorough understanding of what exists before making changes.

If you are young and want to learn, try diagramming as much of the network as you can. The degree of detail is up to you. I'm sure once you do that, it will give you a greater understanding and will provide a path to expand.

Regards.