Re: Intermittent WiFi Connections

NetworkNewbie · ‎09-25-2019

Hello,

I have kind of a weird one here.. Starting around the 1st of the month, (start of the school year, so more wireless clients have joined) I have been getting complaints about intermittent wifi connections. It doesn't matter if its a Windows, Apple or Android device, I have received complaints from them all.

The user will say that they are connected, and all of a sudden, nothing will load. If they look at their wifi, they will still show connected, but the internet doesn't load. I was able to reproduce this error on my iPhone. When I experienced it, I still had an IP address listed, but if I tried to ping it, I would just get "Request Timed Out"

We have multiple models of APs, ranging from 1500, 1815, 2800, and 3700. We are using a 5508 Wireless Controller on code 8.5.140.0 and our wlans are using WPA2, AES Encryption and 802.1X authentication. We have a session timeout of 1800 seconds and the idle timeout is 7200 seconds. To test session timeout, I created a separate wlan without session timeout and still experienced drops.

When I look for logs on the controller, I see the authentication process, but as I use "debug client <mac address>" nothing shows up when I drop internet connection. If I do a "show client detail <mac address>" it shows I am still associated and in the RUN state, so now I am not sure it has anything to do with the controller..

I took a look at our DHCP server and we have not exhausted the pool, so I don't think it's that unless someone else has something that we should look at there.

Has anyone else experienced something like this? What should I look at next?

Thank you,

Flavio Miranda · ‎09-25-2019

Hi

When clients looses connectivity it happens to all clients under the same Access Points or is it randomly ? I have a similar problem but in my case all clients under a specific AP looses connectivity and the workaround is reboot the AP.

-If I helped you somehow, please, rate it as useful.-

NetworkNewbie · ‎09-25-2019

Hi Flavio,

We have seen clients dropping from multiple access points in multiple buildings on campus. We also looked into it being a roaming issue, but the clients are stationary at their desks, so they should be staying on the same AP.

Leo Laohoo · ‎09-25-2019

Check the logs of your switches. See if you see something like "MAC flapping".

NetworkNewbie · ‎09-26-2019

Bingo.

DT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:05:28.318 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:05:47.898 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Gi1/0/48 and port Te1/1/4
Sep 26 09:06:31.864 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:06:46.795 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Gi1/0/48 and port Te1/1/4
Sep 26 09:07:44.125 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Gi1/0/48 and port Te1/1/4
Sep 26 09:08:52.075 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:09:23.477 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:10:02.279 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:10:15.345 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:10:29.785 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:11:47.638 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Gi1/0/48 and port Te1/1/4
Sep 26 09:12:00.342 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:12:17.546 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:12:36.063 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:12:49.470 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:13:21.428 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Gi1/0/48 and port Te1/1/4
Sep 26 09:13:50.954 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:14:02.479 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Gi1/0/48 and port Te1/1/4
Sep 26 09:14:20.775 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Gi1/0/48 and port Te1/1/4
Sep 26 09:14:32.894 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:14:50.155 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:14:57.619 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Gi1/0/48 and port Te1/1/4
Sep 26 09:15:14.487 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:15:49.183 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:16:13.724 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:17:14.538 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:17:46.851 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:18:04.573 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:18:14.679 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 38de.ad03.be3e in vlan 50 is flapping between port Te1/1/4 and port Gi1/0/48
Sep 26 09:24:19.849 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 70ef.004c.a6ff in vlan 56 is flapping between port Po1 and port Te1/1/4

NetworkNewbie · ‎09-30-2019

After doing some research, those flaps are because of the wireless clients roaming to different APs correct? Or is there something else that I should be concerned about for those?

Leo Laohoo · ‎10-01-2019

AP traffic is going from one "link" to another.
Someone has plugged two (or more) uplink without configuring EtherChannel.
Either take off that link or enable EtherChannel.

NetworkNewbie · ‎10-07-2019

It appears our Vlan interface that we use for our wireless clients is not handling all the traffic. We are seeing a ton of input queue drops and flushes. See below:

We are waiting to clear the counters, but from Friday at 11:30am to this morning at 9:20am, the input queue drops has increased 272,000 times and the flushes counter has also gone up 10,000.

None of our other campuses have the interface statistics that this one has.

Could this be due to old hardware?

MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 30/255, rxload 7/255
Encapsulation ARPA, loopback not set
Keepalive not supported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output never, output hang never
Last clearing of "show interface" counters 1y1w
Input queue: 0/75/2335758/683883 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 28226000 bits/sec, 6634 packets/sec
5 minute output rate 120370000 bits/sec, 12466 packets/sec
L2 Switched: ucast: 3620588982 pkt, 1223867696539 bytes - mcast: 0 pkt, 0 bytes
L3 in Switched: ucast: 97271997873 pkt, 30174575235171 bytes - mcast: 0 pkt, 0 bytes
L3 out Switched: ucast: 226661750501 pkt, 279734982558318 bytes - mcast: 0 pkt, 0 bytes
98593010467 packets input, 30027893475861 bytes, 0 no buffer
Received 0 broadcasts (0 IP multicasts)
0 runts, 0 giants, 28062 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
226963406722 packets output, 278877329354103 bytes, 0 underruns
0 output errors, 0 interface resets
0 unknown protocol drops
0 output buffer failures, 0 output buffers swapped out

patoberli · ‎10-07-2019

Could be caused by overloaded uplink ports to your distribution. The port can buffer some, but once it's full, it starts to drop.
Or wrong/bad QoS on the switch could also cause this.

NetworkNewbie · ‎10-09-2019

Thanks for the reply, patoberli

I ran the command "show int | inc output drop" and none of the interfaces have anything recorded.

As for the bad QoS config, I don't believe we have anything actually configured. For our 6807-XL switch, I did a little research and the command "show platform qos" shows that it is enabled, but that's enabled by default correct? Unless I am looking in the wrong spot..

We also ran some debug netdr commands, and saw a lot of traffic for Bonjour. We were advised to create an ACL to permit those IP addresses, but we are still seeing the input queue drops go up. I also should add that we have multiple vlans with input queue drops and flushes.

Thanks again!

Leo Laohoo · ‎10-07-2019

I'd look at the uplink for any "total output drops".