Network Problems and Symptoms

[oighodaro1]

Hello Everyone,

This is my first post and I am hoping I have come to the right place. 

Background: I work for a non-profit and we are hoping to get new switches this summer. Our networking infrastructure consist of LAN and WLAN. We are currently using Cisco Catalyst 6509 for the LAN and Meraki APs for WLAN. 

Problems/Symptoms: 

We having experiencing different issues on different fronts. We are experiencing huge network slow downs when editing over the Network, the connections slow down almost to a halt.  Also whenever users attempt to download a file large than 100mb their download halts between 50 to 70 mb. Additionally the log is showing constant flapping. 

Log: has been attached.

some questions: What should I be looking for in the network connections ? Is the flapping causing the slow down ? 

Any assistance/help would be greatly appreciated.

Thanks

[Reza Sharifi]

Hi,

What is your current design looks like?

How is everything connected? Can you provide a diagram?

[oighodaro1]

Everything is connected using copper 1 gigE cables. The switches are used for layer 2, the Windows AD server is handling DNS and DHCP.

Thanks so much

[Jon Marshall]

The flapping could well be a symptom of what is causing the slowdown.

Firstly you need to identify what devices are connected to the ports mentioned in your log.

Note you don't need to identify all the mac addresses but can you confirm what those ports connect to.

Secondly we need to know a bit more about your network ie.

1) how many vlans do you have or is everything in one vlan  ?

2) what version of STP are you running on the 6500s and the Dell switches because there may be an interoperability problem between them

3) are your wireless APs autonomous because there is no sign of a controller in your diagram?  

Should just say I have very little knowledge of wireless but some of the people in this forum do so they may add to it.

Is this happening all the time or is it at specific times. 

Finally for the moment are most of the clients connected directly to either the 6500 or Dell switches.

If so can you post the configuration of a typical host port from the 6500 and also the configuration of the 6500 port that connects to the Dell switch.

These kind of problems can be difficult to track down so it could be a long process.

Jon

[oighodaro1]

Hey

Sorry I was away for a bit.

1. We have 3 active vlans, however, vlan 1 the default vlan is the only one being used. The other two vlans don't have any ports attached to them.

2.

Switch is in rapid-pvst mode
Root bridge for: VLAN0001, VLAN0010, VLAN0020
EtherChannel misconfig guard is enabled
Extended system ID           is disabled
Portfast Default             is disabled
PortFast BPDU Guard Default  is enabled
Portfast BPDU Filter Default is disabled
Loopguard Default            is disabled
UplinkFast                   is disabled
BackboneFast                 is disabled
Pathcost method used is short

Name                   Blocking Listening Learning Forwarding STP Active
---------------------- -------- --------- -------- ---------- ----------
VLAN0001                     0         0        0         45         45
VLAN0010                     0         0        0          2          2
VLAN0020                     0         0        0          2          2
---------------------- -------- --------- -------- ---------- ----------
3 vlans                      0         0        0         49         49

2. the controller for the wireless AP is host by Meraki, so it is off prem. 

I was able to isolate the problems on to the 7th floor, whenever I am connected to the LAN or APs on the 6th floor I don't experience the problems.

[Jon Marshall]

Did you manage to identify which devices were connected to the flapping links in your log ?

Also the output above shows what version of STP is running on the 6500, what about the Dell switches ?

Jon

[oighodaro1]

So, 

the port doing the flapping is connected to a network HUB that is currently connecting all the Meraki APs.  Also the Dell Switch doesn't support STP.

Thanks

[Jon Marshall]

How is the hub connected to the rest of the network ?

Jon

[oighodaro1]

It is connected using 1 Gig E copper cable to the Cisco 6509. I have updated the network diagram

[oighodaro1]

Hi Jon,

I have done a couple of things: 

1. removed the hub between the APs and Cisco switch

2. read through the logs and located the mac address to it's client

The flapping is happening within on the AP ports at the switch. The devices flopping are happening to devices moving from AP to AP. I found my brand new iPhone flapping between ports. So I can eliminated virus or torrent-ing.

I can't tell what else should I be looking at to find why the switch is performing slowly.

:( 

Thanks

[Jon Marshall]

Removing the hub is a good start.

When you get bad performance on the 7th floor are the source and destination IPs on the same floor ie. the communication never goes down the 6th floor ?

Have you checked all the interfaces for end clients to make sure there no collisions on the ports ?

If the server you are copying to and from when editing is on the other floor also check the ports that connect your switches together.

I am sorry for all the questions but these things take time to trace down and we need to eliminate things one by one.

Jon

[oighodaro1]

Hey Jon

Thanks for the questions and continuous response. It is very helpful.

1. We get bad communication when the source is on the 7th floor and the destination doesn't matter. 

2. How to I check for collisions exactly ? 

3. the port that connects both floors is using portchanneling 1/ trunk. What exactly should I be looking for there ?

Observations:

The flapping is happening whenever a person moves from one AP to another, but it is happening too many times per second. 

Should I put the APs on a different vlan to help reduce the broadcasting ? If yes, would I have to change the IP address for the AP-vlan? could I keep the IP address the same ? 

Sorry for all the questions. I really appreciate the help .

Odaro

[Jon Marshall]

No problem with the questions.

So how many mac changes are you seeing every minute for example ?

This could be causing issues with the 6500 if there are too many of them but I don't know why the other floor is not experiencing those issues.

I am not a wireless person but I believe you can see mac notifications if a person does roam but if you are seeing a lot of them in a short time the obvious question I guess is why are there so many roams ?

What I would like to do is try and rule out the wireless and the mac flapping to see if you then get good performance with copying and editing but I appreciate it is not that simple to do.

Where are the servers that are used for storing the data ie. which floor ?

Jon

[Jon Marshall]

From the 6500 can you post the output of -

"sh spanning-tree vlan 1"

Jon

[oighodaro1]

VLAN0001
  Spanning tree enabled protocol rstp
  Root ID    Priority    8196
             Address     00d0.0405.0c01
             This bridge is the root
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    8196 
             Address     00d0.0405.0c01
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time 300

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Gi1/2            Desg FWD 4         128.2    P2p Peer(STP) 
Gi4/4            Desg FWD 4         128.388  P2p 
Gi4/7            Desg FWD 4         128.391  P2p 
Gi4/9            Desg FWD 4         128.393  P2p 
Gi4/10           Desg FWD 19        128.394  P2p Peer(STP) 
Gi4/12           Desg FWD 100       128.396  P2p 
Gi4/13           Desg FWD 4         128.397  P2p 
Gi4/14           Desg FWD 19        128.398  P2p 
Gi4/21           Desg FWD 4         128.405  P2p 
Gi4/23           Desg FWD 19        128.407  P2p 
Gi4/30           Desg FWD 19        128.414  P2p 
Gi4/35           Desg FWD 4         128.419  P2p 
Gi4/38           Desg FWD 4         128.422  P2p 
Gi4/39           Desg FWD 4         128.423  P2p 
Gi4/40           Desg FWD 4         128.424  P2p 
Gi4/43           Desg FWD 4         128.427  P2p 
Gi4/44           Desg FWD 4         128.428  P2p 
Gi4/47           Desg FWD 4         128.431  P2p 
Gi5/1            Desg FWD 19        128.513  P2p 
Gi5/3            Desg FWD 4         128.515  P2p 
Gi5/10           Desg FWD 19        128.522  P2p 
Gi5/14           Desg FWD 4         128.526  P2p 
Gi5/21           Desg FWD 100       128.533  P2p 
Gi5/25           Desg FWD 100       128.537  P2p 
Gi5/31           Desg FWD 4         128.543  P2p 
Gi5/33           Desg FWD 4         128.545  P2p 
Gi5/34           Desg FWD 4         128.546  P2p 
Gi5/35           Desg FWD 4         128.547  P2p 
Gi6/20           Desg FWD 4         128.660  P2p 
Gi7/5            Desg FWD 4         128.773  P2p 
Gi7/6            Desg FWD 4         128.774  P2p Peer(STP) 
Gi7/16           Desg FWD 4         128.784  P2p 
Gi7/31           Desg FWD 4         128.799  P2p 
Gi7/38           Desg FWD 4         128.806  P2p 
Gi7/39           Desg FWD 4         128.807  P2p 
Gi8/11           Desg FWD 4         128.907  P2p 
Gi8/12           Desg FWD 4         128.908  P2p 
Gi8/22           Desg FWD 4         128.918  P2p 
Gi8/24           Desg FWD 4         128.920  P2p 
Gi8/35           Desg FWD 100       128.931  P2p 
Po3              Desg FWD 3         128.1667 P2p