<30mbps throughput on 3702 over 5GHz with Mobility Express controller

[Jeff A.]

I'm been working through troubleshooting this issue for the past two weeks and have checked and changed every setting I can think of within the controller without any luck. I have a single 2802 running the current 8.10.130 Mobility Express image acting as a controller with two additional 3702 AP's joined in CAPWAP mode to the controller. I'll also add that I'm still fairly new to the technical side of the Cisco ecosystem so please be patient with me.

When I am connected over 5GHz to the 2802 I have no throughput issues, able to get up to about 600mbps on my devices (iPhones, MacBooks, iPads) and everyone connected is happy. However, on my 3702 the connection starts off alright at about 400mbps and then starts to deteriorate over hours/days until eventually the same devices are getting less than 30mbps total throughput. Re-associating with the network sometimes kicks it back into gear and other times it doesn't.

Funnily enough, I have observed cases where a MacBook associated with the same 3702 is able to pull 300mbps but an iPhone connected was only able to pull 30mbps. Then the next day it was reversed.

I have tried just about every setting under the sun on the Mobility Express controller. Tried to set everything up based on the Cisco Best Practices. Tried the defaults. Tried suggestions in articles floating online. Nothing seems to make a difference to the throughput measured. I even reset the whole system - every AP, and controller, back to defaults – and this helped for a day, but then back to the old antics.

I have confirmed that the switch is providing PoE+ and all the AP's are confirming they are happy displaying Full Power.

I'm not sure where to go from here – there's a couple of posts in these forums and on Reddit with 3702 devices having lower than 300mbps throughput, but that seems to be a different issue than what I'm having. If anyone could shed some light or point me in the right direction to further troubleshoot this weird issue I'd be very grateful.

[Leo Laohoo]

This issue is only affecting the 3700, right?  

Check the switch ports for any speed/duplex mismatch.  If the switch is a Cisco Catalyst switch, post the complete output to the command 

sh controll e <PORT>

I want to see  the output from the ports of the 3700 all the way up to the 2800.

[Jeff A.]

It’s affecting both of the 3702’s. They’re both on a full-duplex gigabit connection, as is the 2802. I don’t have a Cisco switch at the moment but I have both a HPE and FS switch and getting the same results on both.

I don’t have shell access to the current switch this very moment but I can see from my SNMP monitoring tool that the three interfaces are showing full duplex on gigabit speeds.

All AP’s are connected to the same switch. The iperf server is connected to the same switch as well and is also full duplex gigabit.

If I’m not wrong, the AP’s are in FlexConnect local mode by default in Mobility Express, so they shouldn’t be passing all the data through the controller. If I bring the 2802 offline or reboot it, the 3700’s keep serving clients and data.

Since I’m still in testing I have not yet added any VLAN’s, so all traffic on the whole network for right now is untagged.

[Jeff A.]

I have now tried 8.10.130, 8.8.130, and 8.5.161, each with vanilla settings (i.e. set up controller, attach AP's, leave and observe) and the issue persists.

What I have found is this:

• When connected to the 2802 there are no issues whatsoever
• When connected to an 3702 the issues are intermittent and appear on devices without rhyme or reason
  • Sometimes the issue is that the speed is <30mbps, I have even observed a single device being <1mbps despite being connected with 3 streams
  • Other times I cannot even reach the controllers IP from those AP's
  • The issues attach themselves to a device, as an example right now:
    • MacBook is getting full speed on all AP's and is able to access the controllers IP
    • iPhone X is getting only 20mbps on the 3702's and cannot access the controllers IP
    • iPhone 6 is getting 1mbps on the 3702's and can access the controllers IP

After checking, it looks like the support agreement has run out, so I can't raise a TAC case either. I'm pretty much ready to throw the 3702's out the window at this stage as they seem to be the only consistency in this issue.

[Scott Fella]

Well sorry to hear that... I got rid of mine a while back but had no issues because I was using a controller.

[Jeff A.]

I could try a vWLC trial and see if that helps, but not sure if that would really make a difference.

The only thing I can think of is that the wireless network and the wired network are currently on a native VLAN, which encompasses IPv4 and IPv6 networking. Could this behaviour have anything to do with that and is it worth separating the VLAN's?

[Scott Fella]

I doubt it because all aps need to be untagged for one and it’s easy to disable ipv6 if you really want to.

[Jeff A.]

So currently I have everything on the switch untagged VLAN 1. If I move all the AP’s to untagged 2 and tagged 1 and broadcast the wireless network on tagged 1 you’re saying that won’t make a difference?

[Scott Fella]

Well a few things, you should have a management vlan which is untagged. Then you place wireless clients on another vlan. So for example:

Management: vlan 20 (native vlan)
Wireless Clients: vlan 30
Wired clients: vlan 40

[Jeff A.]

So currently the management network, wired network, and wireless network are all untagged.

For simplicity’s sake, I’d like to bridge the wired network (VLAN 1) with the wireless network.

If I simply add a management network on VLAN 2 and have that as untagged just between the controller and AP’s, do you think that will have any effect on the behaviour I’ve been experiencing?

[Scott Fella · Accepted solution]

You should download and setup the 9800-CL, that supports wave 1 and 2 AP’s. It’s much better than the vWLC.

[Jeff A.]

I tried setting up the Mobility Express controller with all AP’s entirely in VLAN 2 untagged and a wireless network on VLAN 1 as tagged traffic. Worked fine on the 2802 but the 3702’s wouldn’t even join. Factory reset and they joined but then wouldn’t pass traffic at all. Didn’t bother looking further at logs at this point.
I set up an evaluation of the 9800-CL and it’s been running for 24 hours in FlexConnect mode without any issues. Throughput is solid everywhere, devices connected with less problems, and the whole network is running in a much more stable manner.