Connection to gateway randomly lost

[Martijn de Loos]

Hello,

I am having an odd problem in a client's network and it is causing big issues. Please see the (simple) star topology below:

5x Cisco Small Business Switch SG220-50

1x Fortinet FortiWifi 60D firewall

A whole bunch of desktops and printers and servers 

The problem we are having is that at very random times, no consistency whatsoever, internal clients lose connectivity to only the gateway which is at x.x.x.1. When this happens the entire office loses their internet connection. All internal resources such as servers and printers are still available and reachable, except the gateway.

When this problem occurs I ran an infinite ping -t to the gateway's IP and what I saw is intermittent replies and timeouts. I thought, because only the gateway is affected, that there would be a machine in the network assuming the gateway's IP address and so causing an IP conflict, but when checking the arp on a computer and checking the MAC address table on the switches, I do not see anything conflicting. Also, when I disconnect the internal interface of the firewall from the network, all pings timeout so there is no other device in the network that is assuming the gateway's IP address.

Now here comes the weird part I cannot explain. While working on this issue I was convinced there was a device in the network causing this. I disconnected cables one by one from the switches and then at some point the connectivity to the gateway is restored. After tracing the cable to the specific workstation I found a computer in sleep mode, so it wasn't even on. I turned it on and did an ipconfig. It had a normal IP address from the DHCP pool. Anyway, the connectivity to the gateway was restored and I called it a night. The next day the office's connection ran perfectly fine until the end of the day. Then the issue started occurring again. To fix it I had to do the exact same thing, but this time the connection got restored after disconnecting different cables on another switch. Again when tracing the cable to a workstation, there is no IP conflict on the computer. Also, after disconnecting the cables and the connection is restored to the gateway, I reconnected the workstations to the switch and everything was still working fine. However, the connection to the gateway keeps going down randomly and the only way to fix it is by disconnecting cables from the switches. I can't figure out what is going on and the times it happens is randomly and also every time I have to disconnect different cables in order to fix the problem.

Also, when this problem occurs I tried connecting my laptop straight into the inside interface of the fortinet firewall and that was working perfectly fine so I do not think the problem is caused by the firewall. 
What can be the issue here?

Any help is greatly appreciated.

[paul driver]

Hello

Did  you apply any L2 port security?

if not I would suggest to do so Also chexk the spanning tree  as to the switch trunks forwarding or any other edge ports types showing other than edge status 

[paul driver]

Would you be able to post the config of the switches 

also 

Sh spanning tree

sh spanning tree summary

sh spanning tree interface detail

sh int trunk

Sh interface counters 

[Martijn de Loos]

Hi Paul,

Thanks again for your replies. I will check in the afternoon as I am not in the office right now. Before I left I did see something interesting in the STP port statusses/port roles on the coreswitch. It said the root bridge was connected to some random port on the switch. When tracing the cable, I found an old ISP cable modem. I confirmed with the client what it's for and if it is still needed and they said it was from their old ISP so not in use anymore. As soon as I disconnected the modem, the STP topology got updated and now elected the coreswitch as the root bridge. I have also immediately enabled bpdu guard on all ports but the ones where the actual switches are connected to.

Do you think this might have caused an issue in any way? There was no loop as the modem was only connected to the coreswitch (with 1 cable)

[paul driver]

Hello

You definitely need to make sure your core switch is the stp root for ALL vlans, do this manually dont let stp negotiate it.

What stp mode are you running and how many vlans do you have?

Secondly be careful setting the coreswitch ports with root guard usually you only need to do this when it connecting to another stp domain so as to not let that other network make your core switch a root port, you should not have to do this in your own internal network, so only apply it ONLY on boundary ports.

[Martijn de Loos]

Right now all switches are running STP, not RSTP. Its a simple flat network, so no VLANs are present (beside the default VLAN 1). Switches are in a Star setup (see picture in my first post).

The reason I set bdpu guard on the switchports is because for some reason the old ISP modem got elected as root bridge, so I wanted to make sure that nothing but the actual switches are exchanging these packets with each other.

[paul driver]

Hello

Bpdugaurd is good I was pertaining to rootguard

Dont forget to make your coreswitch root  - spanning-tree vlan 1 priority 0