how to config QoS in Cisco Catalyst 3560 Series Switches

[Majid Jalinousi]

Hi

There is a problem in configuring QoS in switch 3560 that i completely confused with it. I've a Cisco 3560 switch that is running IOS 12.2(25r).

I've configured following commands for QoS: actually this commands have generated by auto qos.

mls qos map cos-dscp 0 8 16 26 32 46 48 56
mls qos srr-queue input bandwidth 90 10
mls qos srr-queue input threshold 1 8 16
mls qos srr-queue input threshold 2 34 66
mls qos srr-queue input buffers 67 33
mls qos srr-queue input priority-queue 2 bandwidth 10
mls qos srr-queue input cos-map queue 1 threshold 2  1
mls qos srr-queue input cos-map queue 1 threshold 3  0
mls qos srr-queue input cos-map queue 2 threshold 1  2
mls qos srr-queue input cos-map queue 2 threshold 2  4 6 7
mls qos srr-queue input cos-map queue 2 threshold 3  3 5
mls qos srr-queue input dscp-map queue 1 threshold 2  9 10 11 12 13 14 15
mls qos srr-queue input dscp-map queue 1 threshold 3  0 1 2 3 4 5 6 7
mls qos srr-queue input dscp-map queue 1 threshold 3  32
mls qos srr-queue input dscp-map queue 2 threshold 1  16 17 18 19 20 21 22 23
mls qos srr-queue input dscp-map queue 2 threshold 2  33 34 35 36 37 38 39 48
mls qos srr-queue input dscp-map queue 2 threshold 2  49 50 51 52 53 54 55 56
mls qos srr-queue input dscp-map queue 2 threshold 2  57 58 59 60 61 62 63
mls qos srr-queue input dscp-map queue 2 threshold 3  24 25 26 27 28 29 30 31
mls qos srr-queue input dscp-map queue 2 threshold 3  40 41 42 43 44 45 46 47
mls qos srr-queue output cos-map queue 1 threshold 3  5
mls qos srr-queue output cos-map queue 2 threshold 3  3 6 7
mls qos srr-queue output cos-map queue 3 threshold 3  2 4
mls qos srr-queue output cos-map queue 4 threshold 2  1
mls qos srr-queue output cos-map queue 4 threshold 3  0
mls qos srr-queue output dscp-map queue 1 threshold 3  40 41 42 43 44 45 46 47
mls qos srr-queue output dscp-map queue 2 threshold 3  24 25 26 27 28 29 30 31
mls qos srr-queue output dscp-map queue 2 threshold 3  48 49 50 51 52 53 54 55
mls qos srr-queue output dscp-map queue 2 threshold 3  56 57 58 59 60 61 62 63
mls qos srr-queue output dscp-map queue 3 threshold 3  16 17 18 19 20 21 22 23
mls qos srr-queue output dscp-map queue 3 threshold 3  32 33 34 35 36 37 38 39
mls qos srr-queue output dscp-map queue 4 threshold 1  8
mls qos srr-queue output dscp-map queue 4 threshold 2  9 10 11 12 13 14 15
mls qos srr-queue output dscp-map queue 4 threshold 3  0 1 2 3 4 5 6 7
mls qos queue-set output 1 threshold 1 138 138 92 138
mls qos queue-set output 1 threshold 2 138 138 92 400
mls qos queue-set output 1 threshold 3 36 77 100 318
mls qos queue-set output 1 threshold 4 20 50 67 400
mls qos queue-set output 2 threshold 1 149 149 100 149
mls qos queue-set output 2 threshold 2 118 118 100 235
mls qos queue-set output 2 threshold 3 41 68 100 272
mls qos queue-set output 2 threshold 4 42 72 100 242
mls qos queue-set output 1 buffers 10 10 26 54
mls qos queue-set output 2 buffers 16 6 17 61
mls qos

I configured above commands on switch, then i generated following prioritized streams by an open source traffic generator (Ostinato) toward switch:

1.an ICMP stream with DSCP 46 (40 megabit/sec)

2.an ICMP stream with DSCP 24 (24 megabit/sec)

3.an ICMP stream with DSCP 16 (18 megabit/sec)

4.an ICMP stream with DSCP 0 (12 megabit/sec)

On switch port connected to my traffic generator i set command "mls qos trust dscp".

The above streams are destined to a trunk port that i configured following commands:

   int fa 0/12

      switchport trunk encapsulation dot1q

      switchport mode trunk

      srr-queue bandwidth shape 10 0 0 0

      srr-queue bandwidth share 10 40 30 20

      priority-queue out

On the other side this switch(switch1) is connected to a trunk port on another switch(switch2), on switch 2 i mirror this trunk port to a another port that is connected to a computer with PRTG(packet analyzer) installed on it. I used packet sniffer sensor of PRTG to capture the traffic, but the amount of traffic that is received like this:

1. ICMP stream with DSCP 46 = 5 megabit/sec

2. ICMP stream with DSCP 24 = 24 megabit/sec

3. ICMP stream with DSCP 16 = 18 megabit/sec

4. ICMP stream with DSCP 0 = 12 megabit/sec

Then i decided to monitor the entrance port on switch1 that is connected to my traffic generator and i saw the exactly traffic pattern was captured on that port. I thought ingress QoS was applied on switch, then i changed tow commands:

mls qos srr-queue input bandwidth 80 20

mls qos srr-queue input priority-queue 2 bandwidth 20

But nothing changed in traffic pattern that i monitored. Any amount that i configured on tow above commands didn't change nothing. I completely confused and i don't know what to do. anyone can help me? it can be out of IOS version that is installed on switch?

Thanks in advance.

[Joseph W. Doherty]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

As I understand it, ingress QoS is for managing bandwidth from ports to fabric (or the 3750 stack ring - n/a for your 3560).

As the original 3560/3750 have a 32G fabric, you might need, I believe, to be able to have more than 16G of ingress traffic before ingress QoS would have an effect.

[Majid Jalinousi]

Hi Joseph,

Thanks for your response.

I understand what you say and i believe you, but i don't understand behavior of switch, total traffic that i send to switch is less than 100 Megabit per second, but the stream with DSCP 46 or COS 5 always is limited to 4 Megabit per second, disregarding the amount of this input stream. Other stream with other COS or DSCP tags is normal. Can it be out of IOS version?

[Joseph W. Doherty]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Sorry, I've misread your original post.  I thought the issue was you were not seeing ingress prioritization.

Yes, your IOS version is rather (very) old.  Those who use 3560/3750 have found (55)SE9 about the most stable (and Cisco too has marked that version as recommended [the only version I believe they have so marked]).  If you have maintenance, it might be worthwhile to upgrade to that version for many reasons.

Otherwise, I see your egress port has shaping configured, but with PQ also configured.  I believe PQ is supposed to override the shaping, but I've haven double checked the documentation for that version.  Try setting shaping to zero for all four queues and see if that makes any difference.

[Majid Jalinousi]

Thanks for your rapid response.

I'll upgrade IOS version and i'll see result of it. About the second recommendation, i thought when you use shaping on a priority queue then you limit that queue doesn't exceed to bandwidth of other queues even when other queues don't use of their allocated bandwidth. this means this queue always limit to specified amount of the bandwidth, but other queues because of sharing can use bandwidth of each other when they don't use of their allocated bandwidth and they never can exceed to bandwidth of priority queue.

Is it correct or i misunderstood?

[Joseph W. Doherty]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Right, when you shape an individual queue, it shouldn't limit bandwidth of the other queues, which is what your flow rates appears to show for the other queues, correct?  (Note, I haven't taken the time, though, to work through all your DSCP to queue mappings - I've been assuming your four flows maps to four different egress queues.)

[Majid Jalinousi]

Sorry for bothering you.

Yes, the four streams should be assigned to four different queues. All of my traffic(include four different types) send to the switch is 94 megabit/sec and i expect QoS wouldn't active, because no congestion occurs, but when i capture the egress port i see the traffic with DSCP 46 is limited to 5 megabit/sec, you know and surprisely when i capture ingress port i see the same behaviour.

Just for my clarity, can you answer these questions please?

I read from CCIE router and switch book version 4, when you shape a queue then that queue wouldn't exceed to allocated bandwidth of other queues even if they dont use their bandwidths, and other queues cant exceed to allocated bandwidth of this queue, is it correct? with my configuration the queue 1 is shaped queue and the other queues are in shared mode. Why do you think my configuration can cause of this problem? Can you explain that? Is not my configuration correct?

Thanks

[Joseph W. Doherty]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Your questions are not a bother.

Correct, shape queues are limited by their shaper.  Bandwidth to other queues doesn't matter, well beyond whether there's sufficient bandwidth allocated to the shaped queue to allow it to hit the shaper's maximum value.

I'm confused by "... and other queues cant exceed to allocated bandwidth of this queue ...", not sure what you're asking.

The reason I wondered about your configuration is because Q1 is in fact shaped, it could limit Q1 traffic.  As I've previously noted, as you've also activated PQ, the Q1 shaper should, I believe, be ignored, but if it's not, it could explain why only your EF traffic isn't hitting full rate.

This is also why I suggested deactivation of your shaper, to see if it makes any difference.  Did you try that?

I've also previously noted, your IOS version is very old.  It's not totally uncommon for some features to not work as exactly described, which is one reason Cisco puts out later releases.  This is why, if possible, I recommended upgrading to (55)SE9 which appears to be very solid.

[Majid Jalinousi]

Hi again

I did what you recommended and upgrading IOS version to 12.55 SE9 hoping everything gonna be OK, but nothing changed.

Again problem was in port the traffic flows came in. I monitored this port to another port that the monitoring server was connected and i saw only traffic related to ICMP with DSCP 46 limited to 4 megabit/sec disregarding to amount of this traffic. When i set command " no mls qos" the amount of traffic was OK. This behavior occurred disregarding to which commands of srr-queue was set on the output port as you said for test.

Actually when i sent flow with DSCP 40 41 42 43 44 45 46 47 or COS 5 this limitation occured and it was a big surprise for me. why did switch behave like this?

If you see my configuration you will see the mentioned DSCPs and COS assigned to queue 2 threshold 3 in ingress port, but my question is why for another COS like 3 and another DSCPs like 24,25,26, etc that assigned to queue 2 threshold 3 too, the limitation doesn't occur? I completely confused with this behavior of switch. Is it cause of my configuration? can you please send me a known and worked configuration? Is there a way for tracing to find problem in switch? I know with "show commands" related to QoS there is no lots of options to see.

Thanks

[Aaron Harrison]

Have you tried removing the shape/priority commands on the egress port to see if it stops reducing the rate of the EF class?

Also have you tried streaming another type of traffic (e.g. UDP) rather than ICMP? 

[Majid Jalinousi]

Aaron,

Thanks for your response,

Yes i removed shape/priority command but nothing changed, but as i said i think QoS happens in ingress port, but i don't know why? I also changed type of traffic to UDP but nothing changed. I also changed my switch to another same switch with the same configuration but nothing changed.

Is my configuration true? Is there any special things I've forgotten to configure?

I generated my flows with all payload of zeros. Can it effect?I think just the tag is important. Isn't it?

Can i have a known and worked configuration from you?

Thanks

[Majid Jalinousi]

Joseph,

Yes my config is same as before, but now without any configs on egress port. you know because traffic is limited in ingress port, i completely forgot egress port now.

If you see my configuration the COS 3 and DSCP 24, 25, 26, etc are also mapped to this queue with this threshold but flows tagged with this values don't limit. i think switch is sensitive just to DSCP 4x and COS 5, is it smart!!!?

In my ingress port, i just trusted to DSCP or COS. is it true?

Thanks again.

[Majid Jalinousi]

Hi again

I've been waiting for your response. Is there a solution for this issue?

Thanks in advance.

[Joseph W. Doherty]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Well, if the problem goes away when you disable QoS, it would seem to be a QoS issue, either in your configuration or with actual correct operation.

Likely the reason you see the issue with any DSCP 4x values and/or CoS 5, they may all map to the same egress queue.

If I get the time, I'll study your OP configuration, which I haven't yet.  Your config is still the same as that?  How are ingress ports configured?