topic Re: Multi-Level Priority Queue QOS in Routing and SD-WAN

Multi-Level Priority Queue QOS

Gabriel Hill — Fri, 18 Oct 2019 18:18:15 GMT

I am doing a QOS config in the following scenario.

I have a 250mbps MPLS circuit, where voice and data will both be passing. I know there are multiple ways to do this, just wanting some peer review on the plan below and see if I may be missing anything. I setup three different classes. One for voice, where I assigned level 1 priority and am restricting it to 15% of BW given congestion (though I do not expect the voice would consume that much, I want to be safe). The data class is for VDI traffic (which will be essentially the remaining usage, and will most likely be the cause of any spikes given multiple agents logging in at one time. Then the third class is for IT mgmt and NMS, so that we may have a small reservation of bandwidth for SSH / SNMP, etc.

ip access-list extended voip
permit ip VOIP-NET any

ip access-list extended data
permit ip DATA-NET any

ip access-list extended IT
permit ip host IT-IP any
permit ip host IT-IP any
permit ip host IT-IP any

class-map match-any voip
match access-group name voip

class-map match-any data
match access-group name data

class-map match-any IT
match access-group name IT

policy-map QOS
class voip
priority level 1
police cir percent 15
class data
priority level 2
class IT
police cir percent 2

Re: Multi-Level Priority Queue QOS

paul driver — Fri, 18 Oct 2019 19:14:51 GMT

Hello

instead of policing and dropping or Re-classifying/marking down out of policy packets suggest you shape your traffic instead using LLq for the priority class.and add another class for you routing process

policy-map wan-child

class voip

priority percent 15

class data

bandwidth percent xx

class IT

bandwidth percent xx

class routing

Bandwidth xxx

class class-default

bandwidth percent remaining 100

fair queue

policy-map wan-parent

class class-default

shape-average 250000000

service-policy wan-child

Re: Multi-Level Priority Queue QOS

Gabriel Hill — Fri, 18 Oct 2019 20:00:09 GMT

Thank you for your reply Paul.

For the routing class, we are doing BGP, I assume I could just mark the class for cs6?

class-map match-any routing
match dscp cs6

Also, in regards to the bandwidth xx, do you know what would be the recommended allocation for a BGP session?

Last one, the bandwidth percent command, that only applies during congestion correct?

Re: Multi-Level Priority Queue QOS

Gabriel Hill — Fri, 18 Oct 2019 20:31:21 GMT

This may be an extremely stupid question but if you were simply doing static routing over such a circuit, it is fair to say no class allocation would be needed for "routing".

Re: Multi-Level Priority Queue QOS

Joseph W. Doherty — Fri, 18 Oct 2019 21:18:43 GMT

If you expect spikes in VDI to fill your link, using LLQ for it is not, I suggest, the best approach.

Try something like (if your device supports):

policy-map QOS
class voip
priority level 1
police cir percent 15
class foreground !for VDI, BGP
bandwidth remaining percent 81
fair-queue
class background !for future low-priority bulk, e.g. data backup traffic
bandwidth remaining percent 1
fair-queue
class class-default !for most traffic
bandwidth remaining percent 9
fair-queue

Assuming your 250 Mbps has a gig hand-off, you'll want a shaper, something like:
policy-map ShapeToCIR
class class-default
shape average 250000000 !you may need to shape about 15% slower if device doesn't count L2 overhead
policy QOS

If your link is logically multi-point you'll want to use MPLS QoS support and bandwidth management becomes complicated.

Re: Multi-Level Priority Queue QOS

Joseph W. Doherty — Fri, 18 Oct 2019 21:20:24 GMT

"For the routing class, we are doing BGP, I assume I could just mark the class for cs6?"

Cisco routers mark CS6, for BGP, by default. Also I recall BGP traffic will obtain PAK priority.

Re: Multi-Level Priority Queue QOS

Joseph W. Doherty — Fri, 18 Oct 2019 21:20:47 GMT

Correct.

Re: Multi-Level Priority Queue QOS

paul driver — Fri, 18 Oct 2019 23:06:39 GMT

@joseph

isn't wfq applicable to only for packets dropped into the class-default queue and not for the specific define classes?

Re: Multi-Level Priority Queue QOS

Joseph W. Doherty — Fri, 18 Oct 2019 23:42:06 GMT

"isn't wfq applicable to only for packets dropped into the class-default queue and not for the specific define classes?"

Technically, not since HQF QoS.

Logically, if the class can congest, and if flows in the class are VBR, and if the two prior criteria are detrimental to the network applications using that class, FQ is often a benefit. (Basically same criteria applies to any class, class-default included.)

As an example, notice I suggested placing BGP into the same class as VDI. If there were a large BGP update, it might impact concurrent VDI flows, but with FQ, it probably won't. The alternative would be to place BGP into its own defined class, but then we need to juggle bandwidth allocation of it relative to other classes, and that allocation would de-prioritize VDI relative to other traffic.

FQ, in a class, often is one queue per flow (NB: not always though), and when than happens, you generally don't need tiered priority, but when you do, you can do it for a whole class in CBWFQ.

Sometimes to show how well FQ works, I'll have a telnet session going and then start a bandwidth test in the same class. W/o FQ, telnet performance becomes almost impossible to use. Activate FQ in the class, telnet again operates much like it does w/o the bandwidth test running concurrently. Again, the alternative would be to place the telnet and bandwidth test traffic into different defined classes, which works, but with FQ, you often don't need too.

Since FQ is dynamic, it also avoids nasty surprises, such as (true story), the day we had our "interactive traffic" class, w/o FQ and which had both telnet and SSH mapped into it, crushed by someone doing SCP.

Re: Multi-Level Priority Queue QOS

Gabriel Hill — Sat, 19 Oct 2019 02:57:39 GMT

Thank you, I appreciate the discussion.

It makes me a bit paranoid to place BGP within the same class as the VDI traffic. I am not worried about BGP affecting VDI (small tables), but worried about VDI affecting BGP. I understand your point with FQ, and it probably would never be an issue, just something that would make me paranoid.

How would I know if my device counts L2 overhead (to adjust the shape appropriately)? I'm using 4331 ISR's.

Would IP SLA's configured on the ISR fall into any QOS class?

Thank you,

Re: Multi-Level Priority Queue QOS

Georg Pauwen — Sat, 19 Oct 2019 06:35:23 GMT

Hello,

on a side note, and I might have missed that somewhere in this discussion, what exactly are you permitting in the access lists ? What are the objects VOIP-NET DATA-NET IT-IP ?

Re: Multi-Level Priority Queue QOS

Joseph W. Doherty — Sat, 19 Oct 2019 13:35:42 GMT

Well, if you're paranoid (laugh), go ahead and place the BGP (NB: remember BGP, unlike [?] VDI, uses TCP, which has it's own implications) traffic into it's own class. Again, though, there shouldn't be a much, if any, risk of VDI impacting it using FQ; also you need to determine this new class's and other classes bandwidth allocation (as I mentioned earlier [I harp on this because although you're defining bandwidth "allocations", what's actually happening is you're defining relative dequeuing priorities.]).

BTW, if you're worried about VDI impacting BGP, your original policy had data with priority level 2, which has priority over all other traffic (except priority level 1). That's much more likely to impact concurrent (i.e. other level 2, because of shared FIFO queue) and subordinate priority traffic, such as your BGP (if it's also not in same class).

"How would I know if my device counts L2 overhead (to adjust the shape appropriately)? I'm using 4331 ISR's. "

Send a "known" flow rate of traffic and "see" what the device reports for xfer rate. (NB: on software based routers, such as your 4331, such features can vary between IOS versions.)

"Would IP SLA's configured on the ISR fall into any QOS class?"

Good question! I'm not sure because sometimes traffic sourced on the same device is excluded from some egress functions. I would expect it to be fully subject to your policy, but best to test your device to insure it is.

Such information isn't always documented, or if it is, you need to find it some supplemental note or white paper. Good example (true story), years ago I had a QoS policy on a 7500 which, to me, just didn't seem to be providing the results I expected. Came across a Cisco white paper explaining how QoS on ATM interfaces (NB: I wasn't using ATM) "overflowed" from the interface TX ring FIFO queue into the CBWFQ queues; and that the TX ring queue might mitigate function of CBWFQ. The "solution" was to decrease the TX ring queue. I did that, and lo and behold, traffic QoS behavior began to meet my expectation. (NB: seems TX ring is an issue with CBWFQ on other interface types too, and I recall coming across later IOS version release notes mentioning that on these later versions the IOS would automatically decrease the default TX ring size when CBWFQ applied [although I found the newer defaults, IMO, still too large. Also, BTW, shouldn't be an issue for you since you should be using a shaper.].)

Re: Multi-Level Priority Queue QOS

paul driver — Sat, 19 Oct 2019 15:16:40 GMT

@joseph

i did have Hqos in mind when I queried it with you but I couldn’t remember ever reading any different behaviour for it regards FQ so I was quite certain in my thinking I was correct.

After your very informative reply I did manage to find a HQF doc from 2008 which obviously supports what you have said -

Concerning part is I have a couple of Cisco press books on qos later than 2008 that either don’t cover hqos or even come close explaining what you have stated here.

I am fully aware as engineers we need to keep abreast of the latest feature sets enhancements etc regards any new software/ hardware Cisco bring out and I am also aware how dynamic networking g is but sometimes this kind of thing not only p@@@ me off it makes me think some of these authors are just in it to make a few quid and plagiarise Cisco documentation to meet their own ends and not provide a full picture of the topic

Anyway rant over and thanks once again for the clarification - very much appreciated

Re: Multi-Level Priority Queue QOS

Joseph W. Doherty — Sat, 19 Oct 2019 16:44:02 GMT

Paul, you're most welcome, and I appreciate your kind remarks.

The link you provide, BTW, would be one I would have referenced for the QoS changes that came out with HQF. FQ changes were actually pretty significant. So much so, in some aspects, I was troubled that the same syntax would/could cause different operational results. I thought, new parameters or keywords should have been provided so you would be "forced" to confront the different operational results. (Ideally, it would have been nice to have both an old way, pre-HQF, and new way, HQF, functionally supported for some period of IOS upgrades; again, ideally, with different syntax.)

Regarding QoS books after 2008, don't forget there's a time lag in book publishing, so books published within two or there years of an IOS release stand a good chance of being out-of-date, even when hot off the press. (NB: if the author has an "in" with the vendor, they might be provided NDA information so that a book might be concurrent with an IOS release at about the same time.)

I'm sure some authors do indeed don't delve into the subject as deeply as they should, possibly for some of the reasons you note, but on the subject of QoS, I believe many just don't really understand the subject too well. Either they don't grasp how important QoS can be (e.g. all you really need is to obtain more bandwidth, and/or prioritize VoIP [heck even Internet2 uses the bandwidth premise]), or they approach it in a manner that, I think, is not as effective as it might be (e.g. let's have 12 [RFC] classes of kinds of traffic; let's have three different drop levels, except when we don't; let's have . . .). (BTW, I believe the best way to grok QoS is much like resource management is done for CPU or disk access.)

Laugh, and speaking of rants, we could get into OER, PfR, etc,, again often not appreciated or well understood too. Of course, in my first couple of decades of IT I was a software developer, so to me, many issues should be solved with a program. So, I do look at things a bit differently from the "typical" network engineer. 😉

Re: Multi-Level Priority Queue QOS

Gabriel Hill — Sat, 19 Oct 2019 22:53:01 GMT

I made a class for icmp, sure enough it would hit that class every time the SLA initiated it's ping.

QOS has always been a subject I've never had to dive into much, but this has been good for me. I appreciate the help from you Joseph and Paul.

I slept on it, and then read up a little more on FQ. I am going to take your suggestion and run with it. I did have around 5 classes yesterday (voice, data, IT (SNMP, SSH, etc), routing, class-default). I took data, IT and routing and bundled them and set the class with FQ.

I think this will be my last question - On a policy-map, if you do not define the class-default at the end, is it still there no matter what? Like, look below where I have it at the end, with the remaining 5 % of BW defined. If I did not put that there, what setting would assume the default?

policy-map QOS
class voip
priority level 1
police cir percent 15
class data
bandwidth remaining percent 80
fair-queue
class class-default
bandwidth remaining percent 5
fair-queue

Re: Multi-Level Priority Queue QOS

Joseph W. Doherty — Sat, 19 Oct 2019 22:33:30 GMT

"On a policy-map, if you do not define the class-default at the end, is it still there no matter what?"

Yes, but it will only be FIFO and bandwidth allocation will be . . .

"Like, look below where I have it at the end, with the remaining 5 % of BW defined. If I did not put that there, what setting would assume the default?"

. . . I really don't know what the bandwidth allocation would be. That's an item I don't recall seeing documented.

BTW, when you use bandwidth remaining, those classes can total up to 100% because its the bandwidth not used by the classes defined without it. In your case, your class data is 80% of 85%.