High RSSI, High SNR, but high Data Retries rate (Multipath effect?)

[Tymofii Dmytrenko]

Hello gents,

Trying to troubleshoot a very strange issue. We are seeing voice quality issues when calls are made using wireless connection (from a corporate laptop) - sound is choppy one way, or both ways. Remote end is wired. We don't see such issues in our other offices, where we use the same type of APs (it's 3702i), or other models, such as 1142s/2600s.

I have noticed that RSSI is reported at -55dBm and SNR is at 40dBm, which is quite an awesome. However, at the same time Data Retries counter is increasing consistently.

I am trying to understand if Data Retries corresponds to Packets Received or Packets Sent. In any case, at the moment it says 875746 packets received and 1021059 packets sent, with 280174 data retries. It is very high re-transmission rate in my humble opinion. It's about 27.5% if data reties correspond to outbound direction, and even high if it's inbound - 32%.

I know it is very hard to troubleshoot multipath issues - there's no way to capture it (correct me if I am wrong). We have metal furniture here, and APs are installed on metal tiles (i.e. it's like a metal screen behind them). I believe this is causing issues to us, but I am not an expert yet to make such conclusions.

How do you identify the root cause for data retries having that RSSI and SNR values are high, there's no co-channel interference and it's 5GHz band only with 40MHz channel. Spectrum analyzer doesn't show/report anything unusual either. Feel a bit lost at the moment. Shouldn't multipath have a constructive effect on MIMO systems (or is it not always the case?).

[Tymofii Dmytrenko]

There are as many opinions on that one as wireless engineers :)

Peter Mackenzie (CWNE#33) told us he is trying to avoid RRM. Basically, because of that.. it doesn't use all the channels, it is very complex algorithm which always makes mistakes... and will never be as optimal as human :)

My concern is... if I have 50 offices, I certainly don't want to manage RF manually. This is an overkill. While he said... well, you have Ch44 where my client can see 5 APs (ours). If you statically assign it, then it will be 1 AP per Ch and if someone turns his radio On using the same channel it will be 2 APs per Ch worst case scenario... I see sense in this, but... when I think about OAM, it looks like an overhead for me.

[Scott Fella]

We manage over 500 sites with both Aruba and Cisco Wireless.  If you plan on setting static, then you still have to watch out for AP's that are not on static that will have the same channel as your static. Unless you disable DCA, then you don't have to worry about it until users start complaining.  It depends on how you want to manage statics in operations. We just can't do that with over 50,000 AP's.  We have very high density in a bunch of our building and we will see overlap, but the channel plan looks fine.  In our offices in shared tenant space, we can't control the RF by our neighbors but you make changes to make your RF better for your users.

-Scott 

*** Please rate helpful posts *** 

[Tymofii Dmytrenko]

Scott, I have just updated the driver to the latest version (19.10.2.1), my NIC is Dual Band Wireless-AC 7265. It's too early to say anything but I can see that they improved the reporting capabilities of PROSet Wireless utility. Previously it wasn't showing "real" TX power levels. It was showing 1dBm even with the highest power level - now it shows 25mW for the highest power level. Also, I have noticed it reports a very high rate of Full Scans (roaming aggressiveness is set to Medium). I have 37K scans in the last 15 minutes. I think it's an overkill! I will try to change this to less aggressive.

I am going to decrease Tx power level to medium or medium-low to match my AP settings, which is using Level 4, or 8dBm according to both, WLC and AP.

GBMAN1APF09-10#show controllers dot11Radio 1 powercfg
Configured TxPower:             8 dBm (Level Index 4)
Allowed Power Levels:           17 14 11  8  5  2  dBm
Allowed Client Power Levels:    17 14 11  8  5  2  dBm

    Tx Power
      Num Of Supported Power Levels ............. 6
      Tx Power Level 1 .......................... 17 dBm
      Tx Power Level 2 .......................... 14 dBm
      Tx Power Level 3 .......................... 11 dBm
      Tx Power Level 4 .......................... 8 dBm
      Tx Power Level 5 .......................... 5 dBm
      Tx Power Level 6 .......................... 2 dBm
      Tx Power Configuration .................... AUTOMATIC
      Current Tx Power Level .................... 4
      Tx Power Assigned By ...................... DTPC

I will also disable Interim updates (although it is not clear why this can be a root cause at the moment). I will keep an eye on it tomorrow. Had a call in an empty environment post driver upgrade and quality was ok. Also noticed that AP and client have used almost symmetric MCS (according to PROSet).

Thanks!

[Scott Fella]

During upgrades, features can be enabled by default. This was one feature that I know was an issue before so just verify that it is not checked. Setting the max and mix txpower is also a good thing so you can keep the txpower similar.  If you have AP's that are 6db difference, you might have more clients on AP's that have higher txpower. 

Driver versions are something that really needs to be tested. We have seen old drivers being an issue along with some of the latest drivers.   Look at enabling band select also if you have many dual band clients on that wlan.  Also what you can do is if you have a dense deployment, you can disable more data rates.  You can make 24mbps or 36mbps for example as the lowest mandatory rate and support the higher rates.  Things to play with to get a better performance from the wireless.  Make sure all you clients works when playing with data rates.

-Scott 

*** Please rate helpful posts *** 

[Tymofii Dmytrenko]

Thanks

Think band select on a corporate WLAN is not required because I set WLAN to 5GHz only - so all clients by default use 5GHz to join it. I have disabled 6/9Mbps already, will continue next week by disabling 12Mbps and potentially 18Mbps. I wouldn't say our environment is high density, but when we planned it, we decided to have 1AP per 10 users, smth like that... because every user carries 1-3 devices.

Also, do you usually tune RRM by setting lower and higher limits for TPC? I've seen some people recommend to change default value for lower limit (which is -10dBm by default - means no limit). In my understanding, if it stays at default value, APs can lower power down to Level 5 or 6, which is very very low power in dBm and mW (according to above it is 2 and 5dBm, literally nothing, a microcell). So, some people recommend to set lower limit to smth like 7-8dBm to make sure AP doesn't go lower than Level 4 for example.

The more I read about RF/RRM the more tricky this subject becomes. For example, RRM is based on RF NDP, where (if I am not wrong) NDP packets are sent using LOWEST DATA RATE (regardless of what is configured - always uses 1Mbps) and HIGHEST POWER (regardless of what is configured - it uses Regulatory Domain's highest power). So... RRM uses neighborship information which is based on rates that I am not even using, and can end up with suboptimal power levels (?).

Plus, in my understanding, when I have to troubleshoot issues like this... Passive Site survey is not an option? It is based on beacons/probes... which are sent at the lowest mandatory rate... so... Do I have to perform an active site survey only (with throughput testing/iperf) and look at real data rates?

Thank you very much and I do apologize for so many questions. I have just passed CCNP Wireless Security... and going to do Design next (will do Ekahau course first). This issue I am troubleshooting will certainly help to get more familiar with the subject.

Will keep an eye on the network tomorrow (with interim updates disabled) and will post the latest update here. Thanks again

[Tymofii Dmytrenko]

Hi Scott

I know it's too early to make any conclusion, but... I see a dramatic improvement today as to how my wireless network perform. First of all, we've noticed the voice quality has improved significantly. Secondly, I have noticed my ping latency to the gateway does not fluctuate anymore... I still have some fluctuation, but... yesterday it was 3, 100, 200, 3, 500, 1, 250ms... not stable at all, while now it is 3-4ms all the time with some rare cases of 11-19ms and even more rare 100+. Average is 3ms. This is drastic

I only disabled interim updates on the corporate network yesterday and forced DCA/TPC to start from scratch. I have frozen both in the morning, to make sure we have no channel changes during the day and I completely disabled EDRRM. That's pretty much all I did.

We have a pilot group of 8 users who haven't complained at all. I had few voice calls today, even with remote VPN party and it was much/much better. I am going to leave it until next week (we return here on Tuesday), just to make sure pilot group is happy. I will then test it in the evening by turning interim updates back on and checking my ping latency.

I wasn't able to find any detailed info about this command, but apparently it sends RADIUS accounting message after every packet and waits for response back until AP transfers anything back? Maybe not after every packet, but certainly after every few - hence we have this fluctuations.

If that was the root cause, I am going to kill Cisco :) And if you're planning to visit Cisco Live in Berlin, I owe you a proper dinner. That's for sure :)

I'll update this thread next week, but I have my fingers crossed.