07-09-2021 04:47 AM
Cisco IOS :Cisco IOS Software, C2900 Software (C2900-UNIVERSALK9-M), Version 15.7(3)M6, RELEASE SOFTWARE (fc1)
Policy-Map used:
Class-map: CM_QOS_C_INTERACT (match-any) 0 packets, 0 bytes 5 minute offered rate 0000 bps, drop rate 0000 bps Match: dscp af31 (26) af32 (28) af33 (30) 0 packets, 0 bytes 5 minute rate 0 bps Match: qos-group 31 0 packets, 0 bytes 5 minute rate 0 bps Match: qos-group 32 0 packets, 0 bytes 5 minute rate 0 bps Match: qos-group 33 0 packets, 0 bytes 5 minute rate 0 bps Queueing queue limit 128 packets (queue depth/total drops/no-buffer drops/flowdrops) 0/0/0/0 (pkts output/bytes output) 0/0 bandwidth remaining 20% Fair-queue: per-flow queue limit 32 packets Exp-weight-constant: 9 (1/512) Mean queue depth: 0 packets dscp Transmitted Random drop Tail/Flow drop Minimum Maximum Mark pkts/bytes pkts/bytes pkts/bytes thresh thresh prob af31 0/0 0/0 0/0 100 128 1/10 af32 0/0 0/0 0/0 90 128 1/10 af33 0/0 0/0 0/0 80 128 1/10 police: cir 90 % cir 4455000 bps, bc 139218 bytes conformed 0 packets, 0 bytes; actions: transmit exceeded 0 packets, 0 bytes; actions: drop conformed 0000 bps, exceeded 0000 bps
Resulted in the logging in the error: %QOS-6-WRED_QLIMIT_OUT_OF_SYNC: On interface Tunnel12 user-defined wred max threshold higher than default queue-limit
Upon removing the queue depth and use the system default, no more error.
Anyone has an idea?
07-09-2021 09:08 AM
Idea for what?
From what you describe, it might be the default queue limit doesn't exceed the WRED limit or "knowing" the default queue limit is being used, the "sanity" checked isn't applied.
As the queue-limit, in what you posted, is 128, and WRED tail limits are all also 128, the WRED tail limits don't exceed the global queue limit. These stats are when the error message doesn't appear, correct?
Interesting, though, since FQ appears active, and is using flow queue limits of 32, WREDs min and max limits should never be reached! NB: I don't recall bumping into this sanity check, so I'm wondering if yours as a more recent IOS version, 15.7, where it's a "new" feature. If so, again from what you posted, it doesn't appear to take into account FQ limits.
Also BTW, on the whole I recommend only QoS experts use WRED, and especially recommend it not being used with FQ.
07-10-2021 07:52 AM
07-10-2021 10:29 AM
"I noticed FQ Queue depth that is much less than the WRED. I guess that this is not good. As I expect that the FQ overrides the WRED config."
Laugh, depends how you define "good". It's not so much FQ overrides WRED, just the FQ and WRED thresholds, in your posted stats don't overlap. I.e. WRED, in this case, should never see queues large enough to react.
"Also, I do not understand why FQ and WRED may not be enabled together."
They can be used together, although I cannot think of a good reason to do so. Using FQ and FRED together, might be useful, though, but I don't recall any Cisco platform supporting both of those on the same platform.
"Also, I do not understand why FQ and WRED may not be enabled together.
Why wouldn't you?"
For starters, FQ drops per flow while WRED drops against, logically, a single FIFO queue. I.e. doesn't make much sense to me, unless, perhaps, you using WRED more as a WTD method or you really believe you need to drop (some?) AFx3 before AFx2 before AFx1, which you may, but with overlap?
"If I mark traffic with AF11, 12 or 13. Will FQ create three queues?"
No, FQ queues are per flow, i.e. same source/dest. addresses and port. ToS tag not considered. (Which is again, a possible case, as noted in answer to your prior question, to use WRED to drop some less important flows, in the same FQ class, before other flow, in the same FQ class.)
"And with a default queue limit? In some cases its half of the default depth (64)."
FQ has it's own ways of determining its default number of FQ queues and their depths. Some platforms will allows you to adjust FQ queue depths, but I don't recall any that support determining number of FQ queues.
"If you have a suggestion for good books/video, I would appreciate it."
Sorry, I have not found any that I really consider "good". IMO, QoS isn't really too complex, but to understand it, and use it well, you need a broad knowledge base. Most QoS material doesn't get into any queuing theory, or delve very deeply in something like TCP "goodput". Or, I don't recall any that touch upon a very similar (at least to me) problem set, which is how a resource like CPU is managed. Lastly, much material doesn't take into account the impact of other changes, like how recent TCP stacks do much more than monitor drops for flow (rate) control.
As an aside, many, I believe, don't appreciate how useful "good" QoS can be.
Years ago, I contracted with a fairly large European company that had a world wide footprint. I did a lot of work developing QoS for the Americas (about 100 locations).
The WAN lead believed QoS was just "voodoo" until the day one of the main WAN routers, at the America's HQ, rebooted itself, overnight, and somehow dropped all its QoS settings.
That morning, the WAN lead said his phone was lit like a Christmas tree, with complaints of "what's wrong with the network".
He discovered the one WAN router missing its QoS configuration, reapplied it, and the complaints stopped. Laugh, a new believer of how effective QoS can be.
Or, a couple of times a year, the company would have all the word wide network leads meet, in one of the regions. One time when they were meeting in the US, I was invited to attend. The EMEA and Asia/Pac leads were complaining that no matter how often they kept buying more and more bandwidth (with the corresponding expense), all they continued to get were complaints the network was still slow. Cough, I mentioned during the last several years the Americas have upgraded very few WAN link's bandwidth, and our user base isn't complaining? Did you know, we, the Americas, use QoS? (Alas, our European HQ engineers, also thought QoS was just worthless "voodoo". [A couple of years later, they finally decided QoS might be worthwhile, but, of course, no point in studying what we've been doing for the last five years.])
In any case, don't get too excited about the approach many QoS books, including Cisco's, suggest.
07-11-2021 10:45 PM
07-12-2021 09:42 AM - edited 07-12-2021 09:42 AM
In years past, I've dealt with international WAN links even offering less than 512 Kbps, and WAN link, literally running half way around the world. So, not totally unfamiliar with low bandwidth and/or high latency and some QoS changes can make an impact, and as you note, both for the good or bad.
On the mention of ships and 600 ms latency, satellite links?
Are you using any WAAS/WAFS? Such can cut down much on the issues of low bandwidth and/or high latency links, although they do suffer from cold vs. warm data transfers.
Such devices, sometimes, can also provide "better" transmission protocols to deal with links where some transmission corruption might arise.
With low bandwidth, sometimes you need to consider LFI.
Also with links which offer less bandwidth than port bandwidth, shaping is often needed for egress.
Also what can help is some 3rd party traffic shaping appliances. (Unsure there's still a market for these, e.g. appliances like Packeteer's, of years ago.)
Anyway, again, although, in principle, WRED seems simple and ideal for flow rate management, it's surprisingly difficult to get it "right".
Lastly, for low bandwidth links, you might want to consider using a very small router (e.g. 800/900/1000 series) as often their QoS feature set is much better than a switch's.
07-12-2021 10:23 PM
Indeed those are satlinks. Many services and clients behind them. Yes, I do use Cisco Waas as a Wan Optimizer, however the tool is outdated and is currently under 'research' to replace.
Nowadays, I get the feeling that with the test equipment the QoS config is going to disappear, but that is quickly said.
As a router we use the ISR3954 or 2911. Depending on the amount of throughput we like to generate.
On the other hand we are convincing manager to use servers on board
Thanlkyou and I wish you all the best.
07-13-2021 08:21 AM
Are you dealing with Cruise Ships? If so, would certainly explain many clients. Also if so, I can see some of the problems trying to support so many, with such little bandwidth, and such high latency; plus modern application just love to use bandwidth and many developers won't take into consideration dealing with little bandwidth and/or having lots of latency (hey, application works great using the server under my desk and having a full gig between host and server).
"General" Cisco QoS isn't going to do as much, and ideally, as you need. Again, shaping for available path bandwidth is important, and, again, FQ is one of the best options to trying to keep some heavy bandwidth flows from being very adverse to low bandwidth flows.
One technique I came up with years ago was using two tiers of FQ, one for low bandwidth flows and one for high bandwidth flows. The former getting priority over the latter. The true secret of this, though, was I also dynamically shifted flows back and forth between those two classes.
In your case, things like WAAS, your idea for local servers, and perhaps specially traffic conditioning appliances, could do more than just any QoS technique on a router.
Since you mentioned the possibility, in the future, using local servers, one story I have, the company I was working for, noted in prior post, was doing a trade show on the other side of USA from their servers. Surprise! They discovered application performance was horrible compared to running the same application, locally, even though they had similar LAN vs. WAN bandwidth. What's wrong with the network? Do something! Fix it!!!
When they brought me in on the issue, I noted, there's bandwidth and latency. We can increase the former, but not much we can do about (distance based) latency. I suggested shipping a server to show. They did, problem solved.
For your, possibly in the not too distance future, LEO satellites will provide more bandwidth and less latency.
In the meantime, at least when ship is in port, possibly a WiFi relay might be used.
Good luck!
07-13-2021 10:10 PM
Sounds so familiar
Thank you for the idea's and information. Drafting up my documents and tests. Looks great.
All the best,
B
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide