cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1781
Views
15
Helpful
8
Replies

user-defined wred max threshold higher than default queue-limit

_|brt.drml|_
Level 1
Level 1

Cisco IOS :Cisco IOS Software, C2900 Software (C2900-UNIVERSALK9-M), Version 15.7(3)M6, RELEASE SOFTWARE (fc1)

 

Policy-Map used:

        Class-map: CM_QOS_C_INTERACT (match-any)
          0 packets, 0 bytes
          5 minute offered rate 0000 bps, drop rate 0000 bps
          Match:  dscp af31 (26) af32 (28) af33 (30)
            0 packets, 0 bytes
            5 minute rate 0 bps
          Match: qos-group 31
            0 packets, 0 bytes
            5 minute rate 0 bps
          Match: qos-group 32
            0 packets, 0 bytes
            5 minute rate 0 bps
          Match: qos-group 33
            0 packets, 0 bytes
            5 minute rate 0 bps
          Queueing
          queue limit 128 packets
          (queue depth/total drops/no-buffer drops/flowdrops) 0/0/0/0
          (pkts output/bytes output) 0/0
          bandwidth remaining 20%
          Fair-queue: per-flow queue limit 32 packets

            Exp-weight-constant: 9 (1/512)
            Mean queue depth: 0 packets
            dscp       Transmitted      Random drop      Tail/Flow drop Minimum Maximum Mark
                        pkts/bytes       pkts/bytes      pkts/bytes   thresh  thresh  prob

            af31           0/0               0/0              0/0                100           128  1/10
            af32           0/0               0/0              0/0                 90           128  1/10
            af33           0/0               0/0              0/0                 80           128  1/10
          police:
              cir 90 %
              cir 4455000 bps, bc 139218 bytes
            conformed 0 packets, 0 bytes; actions:
              transmit
            exceeded 0 packets, 0 bytes; actions:
              drop
            conformed 0000 bps, exceeded 0000 bps

Resulted in the logging in the error: %QOS-6-WRED_QLIMIT_OUT_OF_SYNC: On interface Tunnel12 user-defined wred max threshold higher than default queue-limit

 

Upon removing the queue depth and use the system default, no more error.

Anyone has an idea?

8 Replies 8

Joseph W. Doherty
Hall of Fame
Hall of Fame

Idea for what?

From what you describe, it might be the default queue limit doesn't exceed the WRED limit or "knowing" the default queue limit is being used, the "sanity" checked isn't applied.

As the queue-limit, in what you posted, is 128, and WRED tail limits are all also 128, the WRED tail limits don't exceed the global queue limit.  These stats are when the error message doesn't appear, correct?

Interesting, though, since FQ appears active, and is using flow queue limits of 32, WREDs min and max limits should never be reached!  NB: I don't recall bumping into this sanity check, so I'm wondering if yours as a more recent IOS version, 15.7, where it's a "new" feature.  If so, again from what you posted, it doesn't appear to take into account FQ limits.

Also BTW, on the whole I recommend only QoS experts use WRED, and especially recommend it not being used with FQ.

Joseph,



Thank you for the reply.

Indeed, we are not QoS experts. Routing and switching we perform better :).

The config we have is a legacy from previous architect. As you notice, we are in a learning process.

- Fair-queue is enabled with WRED...

o I noticed FQ Queue depth that is much less than the WRED. I guess that this is not good. As I expect that the FQ overrides the WRED config.

- I got the WRED error when configuring the queue depth.

o The queue is 128, and I followed the Cisco Qos book, tail-limit at 100%.



Also, I do not understand why FQ and WRED may not be enabled together.

Why wouldn't you? -> in the cisco book, they are both.[and frankly I do not understand the reason, or I did not read it correct]



Understand your advice, and I shall test this on the lab.

However, learning to become a small expert in this.



The fair queue ?

If I mark traffic with AF11, 12 or 13. Will FQ create three queues? And with a default queue limit? In some cases its half of the default depth (64).



If you have a suggestion for good books/video, I would appreciate it.



Sincerely,



Bart




"I noticed FQ Queue depth that is much less than the WRED. I guess that this is not good. As I expect that the FQ overrides the WRED config."

Laugh, depends how you define "good".  It's not so much FQ overrides WRED, just the FQ and WRED thresholds, in your posted stats don't overlap.  I.e. WRED, in this case, should never see queues large enough to react.

"Also, I do not understand why FQ and WRED may not be enabled together."

They can be used together, although I cannot think of a good reason to do so.  Using FQ and FRED together, might be useful, though, but I don't recall any Cisco platform supporting both of those on the same platform.

"Also, I do not understand why FQ and WRED may not be enabled together.

Why wouldn't you?"

For starters, FQ drops per flow while WRED drops against, logically, a single FIFO queue.  I.e. doesn't make much sense to me, unless, perhaps, you using WRED more as a WTD method or you really believe you need to drop (some?) AFx3 before AFx2 before AFx1, which you may, but with overlap?

"If I mark traffic with AF11, 12 or 13. Will FQ create three queues?"

No, FQ queues are per flow, i.e. same source/dest. addresses and port.  ToS tag not considered.  (Which is again, a possible case, as noted in answer to your prior question, to use WRED to drop some less important flows, in the same FQ class, before other flow, in the same FQ class.)

"And with a default queue limit? In some cases its half of the default depth (64)."

FQ has it's own ways of determining its default number of FQ queues and their depths.  Some platforms will allows you to adjust FQ queue depths, but I don't recall any that support determining number of FQ queues.

"If you have a suggestion for good books/video, I would appreciate it."

Sorry, I have not found any that I really consider "good".  IMO, QoS isn't really too complex, but to understand it, and use it well, you need a broad knowledge base.  Most QoS material doesn't get into any queuing theory, or delve very deeply in something like TCP "goodput".  Or, I don't recall any that touch upon a very similar (at least to me) problem set, which is how a resource like CPU is managed.  Lastly, much material doesn't take into account the impact of other changes, like how recent TCP stacks do much more than monitor drops for flow (rate) control.

As an aside, many, I believe, don't appreciate how useful "good" QoS can be.

Years ago, I contracted with a fairly large European company that had a world wide footprint.  I did a lot of work developing QoS for the Americas (about 100 locations).

The WAN lead believed QoS was just "voodoo" until the day one of the main WAN routers, at the America's HQ, rebooted itself, overnight, and somehow dropped all its QoS settings.

That morning, the WAN lead said his phone was lit like a Christmas tree, with complaints of "what's wrong with the network".

He discovered the one WAN router missing its QoS configuration, reapplied it, and the complaints stopped.  Laugh, a new believer of how effective QoS can be.

Or, a couple of times a year, the company would have all the word wide network leads meet, in one of the regions.  One time when they were meeting in the US, I was invited to attend.  The EMEA and Asia/Pac leads were complaining that no matter how often they kept buying more and more bandwidth (with the corresponding expense), all they continued to get were complaints the network was still slow.  Cough, I mentioned during the last several years the Americas have upgraded very few WAN link's bandwidth, and our user base isn't complaining?  Did you know, we, the Americas, use QoS?  (Alas, our European HQ engineers, also thought QoS was just worthless "voodoo".  [A couple of years later, they finally decided QoS might be worthwhile, but, of course, no point in studying what we've been doing for the last five years.])

In any case, don't get too excited about the approach many QoS books, including Cisco's, suggest.

Joseph



It all falls into place 'LOL'.

I got the difference between FQ and WRED.

In my company I have to deal with a lot of ships with low bandwidth and high delays (512kbps, 1Mbps and 600ms delay).

So they are convinced that even slight changes to the settings do have an positive or negative effect.



In the design I mark packets at switching level, and I do use the drop probability to differentiate in 'services' that need 'prioritizing' within the same 'application-types'.

However, this type of solution is in my opinion highly 'end-user' experience.



I'll test FQ only, and the WRED solution on the different platforms.



I thank you a lot for clarifying the 'finesse' of QoS



Sincerely



Bart


In years past, I've dealt with international WAN links even offering less than 512 Kbps, and WAN link, literally running half way around the world.  So, not totally unfamiliar with low bandwidth and/or high latency and some QoS changes can make an impact, and as you note, both for the good or bad.

On the mention of ships and 600 ms latency, satellite links?

Are you using any WAAS/WAFS?  Such can cut down much on the issues of low bandwidth and/or high latency links, although they do suffer from cold vs. warm data transfers.

Such devices, sometimes, can also provide "better" transmission protocols to deal with links where some transmission corruption might arise.

With low bandwidth, sometimes you need to consider LFI.

Also with links which offer less bandwidth than port bandwidth, shaping is often needed for egress.

Also what can help is some 3rd party traffic shaping appliances.  (Unsure there's still a market for these, e.g. appliances like Packeteer's, of years ago.)

Anyway, again, although, in principle, WRED seems simple and ideal for flow rate management, it's surprisingly difficult to get it "right".

Lastly, for low bandwidth links, you might want to consider using a very small router (e.g. 800/900/1000 series) as often their QoS feature set is much better than a switch's.

Indeed those are satlinks. Many services and clients behind them. Yes, I do use Cisco Waas as a Wan Optimizer, however the tool is outdated and is currently under 'research' to replace. 

Nowadays, I get the feeling that with the test equipment the QoS config is going to disappear, but that is quickly said.

 

As a router we use the ISR3954 or 2911. Depending on the amount of throughput we like to generate. 

 

On the other hand we are convincing manager to use servers on board I think this would greatly help the clients. But that is another discussion.

 

Thanlkyou and I wish you all the best.

 

 

Are you dealing with Cruise Ships?  If so, would certainly explain many clients.  Also if so, I can see some of the problems trying to support so many, with such little bandwidth, and such high latency; plus modern application just love to use bandwidth and many developers won't take into consideration dealing with little bandwidth and/or having lots of latency (hey, application works great using the server under my desk and having a full gig between host and server).

"General" Cisco QoS isn't going to do as much, and ideally, as you need.  Again, shaping for available path bandwidth is important, and, again, FQ is one of the best options to trying to keep some heavy bandwidth flows from being very adverse to low bandwidth flows.

One technique I came up with years ago was using two tiers of FQ, one for low bandwidth flows and one for high bandwidth flows.  The former getting priority over the latter.  The true secret of this, though, was I also dynamically shifted flows back and forth between those two classes.

In your case, things like WAAS, your idea for local servers, and perhaps specially traffic conditioning appliances, could do more than just any QoS technique on a router.

Since you mentioned the possibility, in the future, using local servers, one story I have, the company I was working for, noted in prior post, was doing a trade show on the other side of USA from their servers.  Surprise!  They discovered application performance was horrible compared to running the same application, locally, even though they had similar LAN vs. WAN bandwidth.  What's wrong with the network?  Do something!  Fix it!!!

When they brought me in on the issue, I noted, there's bandwidth and latency.  We can increase the former, but not much we can do about (distance based) latency.  I suggested shipping a server to show.  They did, problem solved.

For your, possibly in the not too distance future, LEO satellites will provide more bandwidth and less latency.

In the meantime, at least when ship is in port, possibly a WiFi relay might be used.

Good luck!

Sounds so familiar

Thank you for the idea's and information. Drafting up my documents and tests. Looks great. 

 

All the best,

 

B

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Review Cisco Networking products for a $25 gift card