Hello Everyone,

Stephen Sisson · ‎10-02-2015

Hello Everyone,

We have a weird problem that we can't figure out on our own and need your expertise.

We have dozens of people reporting a network outage at our branch office, after arriving onsite we see everyone down, they use our Cisco 3750 /48 port switch at 192.168.50.1 /24 as their default gateway, to access resources online or at our primary site. we have our ISP router upstream from that Cisco 3750 at 192.168.50.2, this router seems OK - Tech support for our ISP reports, all our users cant connect to anything right now. We changed the default gateway from the Cisco 3750 192.168.50.1 to use our ISP router 192.168.50.2 and everything works for our users - internet access, resources from our primary site. We no longer can ping the Cisco switch 192.168.50.1 from any location or our ISP router, when they clear the Arp table on their router problem resolved for about twenty minutes, after that time the Cisco 192.168.50.1 address is down and we have lost access to the switch.

I'm not sure what the problem is, what should we look at - the Cisco 3750 switch, the ISP Cisco 2800 router or do we have an IP conflict on our network, another piece of equipment using the same IP address - if this why do we lose the Cisco switch 192.168.50.1 and if another piece of equipment using the same IP address they stay up - maybe they are down, maybe no ip conflict - as you see we need your help to resolve this crazy issue.

Maybe someone has seen this problem before.

Thank you all

Mark Malone · ‎10-02-2015

Hi you need to check both the 3750 and the 2800 that are local on site , are there logs providing any information from when the issue occurred in show logg

when you say they cant connect to anything is that external/ internal or both ? you say there down are the actual ports down or they just cant reach anything have your ran any tracerts from local pcs to see where they stop is the 37 or 28 ?

Stephen Sisson · ‎10-02-2015

Hi Mark,

When I say their down means no access beyond their local subnet, when trying to access the internet or resources at any other location with default gateway the Cisco 3750 switch 192.168.50.1 this fails to work. When we change them all to use the upstream ISP 2800 router as their default gateway 192.168.50.2 they have access to the internet and resources at all remote sites.

The Cisco 3750 only uses two ports right now, with one going to the ISP Cisco 2800 router, the other going to a 3com 48 port switch that our users connect to. Neither port on the Cisco 3750 go down nor the Vlan associated with the 192.168.50.x subnet is up up but the default gateway we can no longer ping until we have the ISP clear arp on the port connecting the equipment.

After we change the default gateway on all equipment to point the ISP router on the same subnet as the Cisco 3750 everything works - like by-passing the switch and using the ISP 2800 router, all traffic must go through the switch before reaching the router

Thank you

Mark Malone · ‎10-02-2015

Is there anything showing up in your logs on the 3750 to indicate an issue , memory problems ,cpu ,duplicate address etc ?

Stephen Sisson · ‎10-02-2015

Nothing in the logs - or anything for the CPU /memory, the ISP is seeing lots of devices not connecting or having connection errors for several devices, when they clear the arp everything resolves just fine.

After clearing the arp or resetting the port between the equipment everything works for twenty minutes up to three hours before we have an issue again.

We have a work around as to change all equipment / DHCP server to use the ISP IP address as their default gateway until we figure out the problem, everything works great - three days since the changes.

The ISP states nothing wrong our side and must be something on the local LAN network causing the problem

Cisco Network Problem