Ask the Expert: QoS on Catalyst Switches.

[ciscomoderator]

With Shashank Singh  and

Welcome to the Cisco Support Community Ask the Expert conversation. This is an opportunity to learn from Cisco experts Shashank Singh and Sweta Morga about implementation and working and troubleshooting QoS on Cisco Catalyst 2960, 3650, 3750, 4500 and 6500 switches.

Shashank Singh  graduated in 2009 with a bachelor's degree in Computer Science and Engineering from VIT University, Vellore India. Prior to joining Cisco he worked at General Electric as a software engineer. Later on he joined the Cisco Technical Assistance Center as an engineer in October of 2009. He has been working on LAN Switching technologies in TAC since then. Shashank also holds a CCNP certificate. QoS on Catalyst switches is one of the areas of his interest.

Sweta Mogra is a Computer Science & Engineering graduate from VIT University, India. She has worked as a consultant with Tata Consultancy Services before joining Cisco's Technical Assistance Center (TAC) in 2011. She is currently working on LAN Switching technologies and QoS as one of her areas of expertise.

Remember to use the rating system to let Shashank and Sweta know if you have received an adequate response. 

Shashank and Sweta might not be able to answer each question due to the volume expected during this event. Remember that you can continue the conversation on the Network Infastructure sub-communityLan Switching forum shortly after the event. This event lasts through June 1, 2012. Visit this forum often to view responses to your questions and the questions of other community members.

[jerome.gomez]

Hi,

i'm trying to setup qos on 3750X (15-0(1) IP BASE) without success

I want to mark each packets comming from downlinks (several switches on LAN)

The downlink i test is g 1/0/1. I only want to mark DSCP value on each IP Packet entering this interface.

i tried to set it up on svi with same result.

when i look at policy map, no packets are shown in result and access lists don't hit.

Any idea to help me ?

thanks

here is the implemented configuration

**************************

mls qos

class-map match-any DATA-1

  match access-group name DATA-1

class-map match-any DATA-2

  match access-group name DATA-2

class-map match-any VISIO

  match access-group name VISIO

class-map match-all VOIX-RTP

  match ip dscp ef

class-map match-any VOIX-SIG

  match ip dscp cs5

!

policy-map MARK

class VOIX-RTP

   set dscp ef

class VOIX-SIG

   set dscp cs5

class VISIO

   set dscp af41

class DATA-1

   set dscp af31

class DATA-2

   set dscp af21

class class-default

   set dscp af11

interface GigabitEthernet1/0/1

service-policy input MARK

ip access-list extended DATA-1

permit tcp any any eq telnet

permit tcp any any eq 2300

permit tcp any any eq 88

permit udp any any eq 88

permit tcp any any eq 464

permit udp any any eq 464

permit tcp any any eq 3268

permit tcp any any eq 389

permit tcp any any range 3200 3210

permit tcp any any range 3300 3310

permit tcp any any range 8000 8010

permit tcp any any eq 449

permit tcp any any eq 8476

permit tcp any any eq 4955

permit tcp any any eq 22

permit tcp any any

permit udp any any

permit tcp any any eq domain

permit udp any any eq domain

permit tcp any any eq 3389

permit tcp any any eq 10001

permit tcp any any eq 1494

permit tcp any any eq 2598

permit tcp any any eq 902

permit udp any any eq 902

permit tcp any any eq 903

permit tcp any any eq 5405

permit tcp any any eq 7788

permit tcp any any eq 1515

permit tcp any any range 27000 27009

permit tcp any eq telnet any

permit tcp any eq 2300 any

permit tcp any eq 88 any

permit udp any eq 88 any

permit tcp any eq 464 any

permit udp any eq 464 any

permit tcp any eq 3268 any

permit tcp any eq 389 any

permit tcp any range 3200 3210 any

permit tcp any range 3300 3310 any

permit tcp any range 8000 8010 any

permit tcp any eq 449 any

permit tcp any eq 8476 any

permit tcp any eq 4955 any

permit tcp any eq 22 any

permit tcp any eq domain any

permit udp any eq domain any

permit tcp any eq 3389 any

permit tcp any eq 10001 any

permit tcp any eq 1494 any

permit tcp any eq 2598 any

permit tcp any eq 902 any

permit udp any eq 902 any

permit tcp any eq 903 any

permit tcp any eq 5405 any

permit tcp any eq 7788 any

permit tcp any eq 1515 any

permit tcp any range 27000 27009 any

ip access-list extended DATA-2

permit ip 0.0.1.110 255.255.0.0 any

permit ip host 10.57.1.1 any

permit tcp any any eq 161

permit udp any any eq snmp

permit icmp any any

permit tcp any any eq www

permit tcp any any eq 443

permit ip any 0.0.1.110 255.255.0.0

permit ip any host 10.57.1.1

permit tcp any eq 161 any

permit udp any eq snmp any

permit tcp any eq www any

permit tcp any eq 443 any

ip access-list extended VISIO

permit udp any any eq 1718

permit udp any any eq 1719

permit tcp any any eq 1720

permit tcp any any eq 1731

permit tcp any any eq 1503

permit tcp any any range 3230 3253

permit udp any any range 3230 3253

permit udp any eq 1718 any

permit udp any eq 1719 any

permit tcp any eq 1720 any

permit tcp any eq 1731 any

permit tcp any eq 1503 any

permit tcp any range 3230 3253 any

permit udp any range 3230 3253 any

here are some traces i took

sh mls qos

QoS is enabled

QoS ip packet dscp rewrite is enabled

sh mls qos interface g 1/0/1

GigabitEthernet1/0/1

Attached policy-map for Ingress: MARK

trust state: not trusted

trust mode: not trusted

trust enabled flag: ena

COS override: dis

default COS: 0

DSCP Mutation Map: Default DSCP Mutation Map

Trust device: none

qos mode: port-based

sh mls qos maps

   Policed-dscp map:

     d1 :  d2 0  1  2  3  4  5  6  7  8  9

     ---------------------------------------

      0 :    00 01 02 03 04 05 06 07 08 09

      1 :    10 11 12 13 14 15 16 17 18 19

      2 :    20 21 22 23 24 25 26 27 28 29

      3 :    30 31 32 33 34 35 36 37 38 39

      4 :    40 41 42 43 44 45 46 47 48 49

      5 :    50 51 52 53 54 55 56 57 58 59

      6 :    60 61 62 63

   Dscp-cos map:

     d1 :  d2 0  1  2  3  4  5  6  7  8  9

     ---------------------------------------

      0 :    00 00 00 00 00 00 00 00 01 01

      1 :    01 01 01 01 01 01 02 02 02 02

      2 :    02 02 02 02 03 03 03 03 03 03

      3 :    03 03 04 04 04 04 04 04 04 04

      4 :    05 05 05 05 05 05 05 05 06 06

      5 :    06 06 06 06 06 06 07 07 07 07

      6 :    07 07 07 07

   Cos-dscp map:

        cos:   0  1  2  3  4  5  6  7

     --------------------------------

       dscp:   0  8 16 24 32 40 48 56

   IpPrecedence-dscp map:

     ipprec:   0  1  2  3  4  5  6  7

     --------------------------------

       dscp:   0  8 16 24 32 40 48 56

   Dscp-outputq-threshold map:

     d1 :d2    0     1     2     3     4     5     6     7     8     9

     ------------------------------------------------------------

      0 :    02-01 02-01 02-01 02-01 02-01 02-01 02-01 02-01 02-01 02-01

      1 :    02-01 02-01 02-01 02-01 02-01 02-01 03-01 03-01 03-01 03-01

      2 :    03-01 03-01 03-01 03-01 03-01 03-01 03-01 03-01 03-01 03-01

      3 :    03-01 03-01 04-01 04-01 04-01 04-01 04-01 04-01 04-01 04-01

      4 :    01-01 01-01 01-01 01-01 01-01 01-01 01-01 01-01 04-01 04-01

      5 :    04-01 04-01 04-01 04-01 04-01 04-01 04-01 04-01 04-01 04-01

      6 :    04-01 04-01 04-01 04-01

   Dscp-inputq-threshold map:

     d1 :d2    0     1     2     3     4     5     6     7     8     9

     ------------------------------------------------------------

      0 :    01-01 01-01 01-01 01-01 01-01 01-01 01-01 01-01 01-01 01-01

      1 :    01-01 01-01 01-01 01-01 01-01 01-01 01-01 01-01 01-01 01-01

      2 :    01-01 01-01 01-01 01-01 01-01 01-01 01-01 01-01 01-01 01-01

      3 :    01-01 01-01 01-01 01-01 01-01 01-01 01-01 01-01 01-01 01-01

      4 :    02-01 02-01 02-01 02-01 02-01 02-01 02-01 02-01 01-01 01-01

      5 :    01-01 01-01 01-01 01-01 01-01 01-01 01-01 01-01 01-01 01-01

      6 :    01-01 01-01 01-01 01-01

   Cos-outputq-threshold map:

              cos:  0   1   2   3   4   5   6   7

              ------------------------------------

  queue-threshold: 2-1 2-1 3-1 3-1 4-1 1-1 4-1 4-1

   Cos-inputq-threshold map:

              cos:  0   1   2   3   4   5   6   7

              ------------------------------------

  queue-threshold: 1-1 1-1 1-1 1-1 1-1 2-1 1-1 1-1

   Dscp-dscp mutation map:

   Default DSCP Mutation Map:

     d1 :  d2 0  1  2  3  4  5  6  7  8  9

     ---------------------------------------

      0 :    00 01 02 03 04 05 06 07 08 09

      1 :    10 11 12 13 14 15 16 17 18 19

      2 :    20 21 22 23 24 25 26 27 28 29

      3 :    30 31 32 33 34 35 36 37 38 39

      4 :    40 41 42 43 44 45 46 47 48 49

      5 :    50 51 52 53 54 55 56 57 58 59

      6 :    60 61 62 63

sh mls qos queue-set

Queueset: 1

Queue     :       1       2       3       4

----------------------------------------------

buffers   :      25      25      25      25

threshold1:     100     200     100     100

threshold2:     100     200     100     100

reserved  :      50      50      50      50

maximum   :     400     400     400     400

Queueset: 2

Queue     :       1       2       3       4

----------------------------------------------

buffers   :      25      25      25      25

threshold1:     100     200     100     100

threshold2:     100     200     100     100

reserved  :      50      50      50      50

maximum   :     400     400     400     400

sh mls qos input-queue

Queue     :       1       2

----------------------------------------------

buffers   :      90      10

bandwidth :       4       4

priority  :       0      10

threshold1:     100     100

threshold2:     100     100

sh class-map

Class Map match-any DATA-1 (id 1)

   Match access-group name DATA-1

Class Map match-any DATA-2 (id 2)

   Match access-group name DATA-2

Class Map match-any class-default (id 0)

   Match any

Class Map match-any VISIO (id 3)

   Match access-group name VISIO

Class Map match-all VOIX-RTP (id 4)

   Match ip  dscp ef (46)

Class Map match-any VOIX-SIG (id 5)

   Match ip  dscp cs5 (40)

sh policy-map

  Policy Map MARK

    Class VOIX-RTP

      set dscp ef

    Class VOIX-SIG

      set dscp cs5

    Class VISIO

      set dscp af41

    Class DATA-1

      set dscp af31

    Class DATA-2

      set dscp af21

    Class class-default

      set dscp af11

sh policy-map int g 1/0/1

GigabitEthernet1/0/1

  Service-policy input: MARK

    Class-map: VOIX-RTP (match-all)

      0 packets, 0 bytes

      5 minute offered rate 0 bps, drop rate 0 bps

      Match: ip dscp ef (46)

    Class-map: VOIX-SIG (match-any)

      0 packets, 0 bytes

      5 minute offered rate 0 bps, drop rate 0 bps

      Match: ip dscp cs5 (40)

        0 packets, 0 bytes

        5 minute rate 0 bps

    Class-map: VISIO (match-any)

      0 packets, 0 bytes

      5 minute offered rate 0 bps, drop rate 0 bps

      Match: access-group name VISIO

        0 packets, 0 bytes

        5 minute rate 0 bps

    Class-map: DATA-1 (match-any)

      0 packets, 0 bytes

      5 minute offered rate 0 bps, drop rate 0 bps

      Match: access-group name DATA-1

        0 packets, 0 bytes

        5 minute rate 0 bps

    Class-map: DATA-2 (match-any)

      0 packets, 0 bytes

      5 minute offered rate 0 bps, drop rate 0 bps

      Match: access-group name DATA-2

        0 packets, 0 bytes

        5 minute rate 0 bps

    Class-map: class-default (match-any)

      0 packets, 0 bytes

      5 minute offered rate 0 bps, drop rate 0 bps

      Match: any

sh ip access

Extended IP access list DATA-1

    10 permit tcp any any eq telnet

    20 permit tcp any any eq 2300

    30 permit tcp any any eq 88

    40 permit udp any any eq 88

    50 permit tcp any any eq 464

    60 permit udp any any eq 464

    70 permit tcp any any eq 3268

    80 permit tcp any any eq 389

    90 permit tcp any any range 3200 3210

    100 permit tcp any any range 3300 3310

    110 permit tcp any any range 8000 8010

    120 permit tcp any any eq 449

    130 permit tcp any any eq 8476

    140 permit tcp any any eq 4955

    150 permit tcp any any eq 22

    160 permit tcp any any eq domain

    170 permit udp any any eq domain

    180 permit tcp any any eq 3389

    190 permit tcp any any eq 10001

    200 permit tcp any any eq 1494

    210 permit tcp any any eq 2598

    220 permit tcp any any eq 902

    230 permit udp any any eq 902

    240 permit tcp any any eq 903

    250 permit tcp any any eq 5405

    260 permit tcp any any eq 7788

    270 permit tcp any any eq 1515

    280 permit tcp any any range 27000 27009

    290 permit tcp any eq telnet any

    300 permit tcp any eq 2300 any

    310 permit tcp any eq 88 any

    320 permit udp any eq 88 any

    330 permit tcp any eq 464 any

    340 permit udp any eq 464 any

    350 permit tcp any eq 3268 any

    360 permit tcp any eq 389 any

    370 permit tcp any range 3200 3210 any

    380 permit tcp any range 3300 3310 any

    390 permit tcp any range 8000 8010 any

    400 permit tcp any eq 449 any

    410 permit tcp any eq 8476 any

    420 permit tcp any eq 4955 any

    430 permit tcp any eq 22 any

    440 permit tcp any eq domain any

    450 permit udp any eq domain any

    460 permit tcp any eq 3389 any

    470 permit tcp any eq 10001 any

    480 permit tcp any eq 1494 any

    490 permit tcp any eq 2598 any

    500 permit tcp any eq 902 any

    510 permit udp any eq 902 any

    520 permit tcp any eq 903 any

    530 permit tcp any eq 5405 any

    540 permit tcp any eq 7788 any

    550 permit tcp any eq 1515 any

    560 permit tcp any range 27000 27009 any

Extended IP access list DATA-2

    10 permit ip 0.0.1.110 255.255.0.0 any

    20 permit ip host 10.57.1.1 any

    30 permit tcp any any eq 161

    40 permit udp any any eq snmp

    50 permit icmp any any

    60 permit tcp any any eq www

    70 permit tcp any any eq 443

    80 permit ip any 0.0.1.110 255.255.0.0

    90 permit ip any host 10.57.1.1

    100 permit tcp any eq 161 any

    110 permit udp any eq snmp any

    120 permit tcp any eq www any

    130 permit tcp any eq 443 any

Extended IP access list VISIO

    10 permit udp any any eq 1718

    20 permit udp any any eq 1719

    30 permit tcp any any eq 1720

    40 permit tcp any any eq 1731

    50 permit tcp any any eq 1503

    60 permit tcp any any range 3230 3253

    70 permit udp any any range 3230 3253

    80 permit udp any eq 1718 any

    90 permit udp any eq 1719 any

    100 permit tcp any eq 1720 any

    110 permit tcp any eq 1731 any

    120 permit tcp any eq 1503 any

    130 permit tcp any range 3230 3253 any

    140 permit udp any range 3230 3253 any

[Matthew Hall]

In order to tell if it is working you are going to have to setup a sniffer downstream or use sh mls qos interface statistic on the downstream switch and look at the incoming dscp tables.

The show policy map interface command doesn't work on ANY 3750/3560 platform.  It will show you only the configured policy but the counters will alsways show 0, even if it is working...I know bummer.

[jerome.gomez]

Hi Matthew,

thanks for your help, in fact i found another post on this point.

So it works perfectly

regards

[smogra]

Hi Jerome,

As stated by Dahua and Matthew, 'show policy-map interface' command is not supported on 3750 and 3560 switches, even though it is allowed to be typed in the CLI. You always have an option to do a sniffer capture to confirm if the traffic is getting marked or not or use "show mls qos interface x/y stat" to watch out for the packets.

Regards,

Sweta

[jerome.gomez]

Hi Sweta,

thanks for your help, in fact i found another post on this point.

So it works perfectly

regards

[lcd_shouldit]

Hi  Shashank ，

For QoS configuration assistance and best practices, I would suggest following the Campus QoS design guide located at 

http://www.cisco.com/en/US/docs/solutions/Enterprise/WAN_and_MAN/QoS_SRND/QoSDesign.html

I find there are Queuing Model for Catalyst 29**, 35**, 37**, 45**, 65**, look like they are some standards, am i right?

If it is,then I will use these model in future, because i think standards is very important, it will make the network consistent.

And the most great thing is the queuing recommendations config part in this document, I think the queuing config part is very importand and very difficult, including threshold for each queue, share &shape config in interface level......

I want to ask, if I want to implement END-TO-END Catalyst QOS in a new Campus network, where should I begin,how should I consider, what is the most important part during implementing? And Are there some real cases, which can let me

know how cool the QOS is 

The last question, this document guide version is 3.3,November 2005, Is there some update, and the latest version  thank you very much！！！

[smogra]

Hi Changdong,

Please find the answers inline:

I find there are Queuing Model for Catalyst 29**, 35**, 37**, 45**, 65**, look like they are some standards, am i right?

The values displayed under 'sh mls qos maps' are the default or rather you can say 'standard' values. These are the Cisco recommended ones . But if they don't suit your network requirements, you can tweak them as per your need.

if I want to implement END-TO-END Catalyst QOS in a new Campus network, where should I begin,how should I consider, what is the most important part during implementing? And Are there some real cases, which can let me know how cool the QOS is ..

First you need to understand if you expect to have congestion in your netwrok. If you have congestion, you will need to  find out the amount of traffic for each type and which traffic is lesser important than others and can be dropped.

Try looking at below link for some sample examples to get a better understanding of qos.:

http://www.cisco.com/en/US/products/hw/switches/ps5023/products_tech_note09186a0080883f9e.shtml#qds

The last question, this document guide   version is 3.3,November 2005, Is there some update, and the latest  version..

I believe this document covers all the required topics and their explanations. You can rely on this for your need.

Regards,

Sweta

[dahua.huang]

If we apply "service-policy output xxx",  is that means from outside traffic into server get market only?  this confused me.if we apply "service-policy output xxx" to all etherent interfaces, does it will cause tcam out of resources issue?

service-policy output xxx will affect traffic that is going out of that interface(egress traffic). Applying the same service policy on all interfaces should not cause your TCAM to run out of space.

Hi,

ouput and input still a little bit confused me.

We have a lab with two 4507 (sup2)

pc1, vlan 100 --> F3/1(SwitchA, 4507, SUP2+)----6509----(SwitchB, 4507, SUP2+)F4/1---->pc2, vlan 200

SwitchA:

INT F3/1

qos vlan-based

int vlan 100

service-policy input QOSMARK

qos trust dscp to 6509

SwitchB:

INT F4/1

qos vlan-based

int vlan 200

service-policy input QOSMARK

qos trust dscp to 6509

Switch6509

int vlan100

ip address x.x.x.x

int vlan200

ip address x.x.x.x

qos trust dscp to switch A & B

We put a wireshark in PC2.

I can see packet with  correct DSCP value from PC1 to PC2 if set as " service-policy input QOSMARK"

I can NOT see correct value if I changed to " service-policy ouput QOSMARK"

so the 4507E sup6 support " service-policy ouput QOSMARK" only kind of confused me.

Please advise.

Thanks and have a great day.

and you guys are so great and should publish a QOS-cookbook.

[Shashank Singh]

Thanks for the compliments To answer your question, when traffic flows from pc1 to pc2, it is considered as ingress traffic on the following interfaces marked with XX:

pc1 ------->XX-F3/1(SwitchA)--------->XX-6509---------->XX-(SwitchB)F4/1------------>pc2

As per your switch A and switch B config, pc1--->pc2 traffic will hit the input policy map ONLY on switch A (F3/1). This traffic is egress traffic on f4/1 on switch B and hence will not hit input policy map on switch B. I hope that explains why pc1--->pc2 traffic is not affected when you change the policy map to output on switch A.

Regards,

Shashank

[dahua.huang]

Hi, Shashank

based on

***************

pc1 ------->XX-F3/1(SwitchA)--------->XX-6509---------->XX-(SwitchB)F4/1------------>pc2

As per your switch A and switch B config, pc1--->pc2 traffic will hit the input policy map ONLY on switch A (F3/1). This traffic is egress traffic on f4/1 on switch B and hence will not hit input policy map on switch B. I hope that explains why pc1--->pc2 traffic is not affected when you change the policy map to output on switch A.

**************

4507 sup6 only support output

pc1 ------->XX1-F3/1(SwitchA) yy1--------->XX2-6509 YY2---------->XX3-(SwitchB)F4/1 YY3------------>pc2

where the traffic will be marked (PC1 TO PC2)? YY1?

and how about L2 traffic in the same switch? will be marked?

Thanks a lot.

There are over 300 4507 needs to be replaced. I need to fully understand this.

Thanks.

[Shashank Singh]

Hi Dahua,

Yes, traffic from PC1 TO PC2 will hit the output service-policy on yy1 (if configured). L2 traffic will also get marked if you are using qos vlan-based on L2 interface.

Regards,

Shashank

[Andrey.Miroshnichenko]

Hello!

I have Catalyst 3560 with "mls qos trust dscp" on some interfaces.

This is only qos option, applyed on a switch. In documentation I readed, what 2 ingress and 4 egress queues on each interface exist, to provide QoS.

Can you explain, where I can see drops in this queues?

"show interfaces f0/0" displays actual statistics on input and output, but in general (without displaying drops on particular queues).

"show platform port-asic stats drop f0/0" displays detailed statistics on Tx, but any statistics on Rx. Also, it displays statistics since last swtch restart, what isn't very actual.

Thanks!

[Shashank Singh]

Hi Andrey,

Please find the answers inline.

Can you explain, where I can see drops in this queues?

If drops are present on any queue, they would be seen in "sh mls qos int gix/y stat" output. Please check out the blog for the sample output indicating drops. (link provided below)

"show interfaces f0/0" displays actual statistics on input and output, but in general (without displaying drops on particular queues).

Yes, show interface output does not give us queue level drops. But as in your case there are no drops at all, it is likely that packets are not getting dropped in first place. Is there a reason like degraded performance that tells you that packets should be getting dropped?

"show platform port-asic stats drop f0/0" displays detailed statistics on Tx, but any statistics on Rx. Also, it displays statistics since last swtch restart, what isn't very actual.

Yes this output shows drops only on TxQueue. Infact most of the times drops happen only on TxQueue on switches. Drops in input queue may not necessarily indicate a QoS issue, as they represent packets going to CPU and are most likely not CEF switched. And as you correctly pointed out, the counters are from the time of last reboot. So the correct thing to do is to run this output multiple times to check if the counters are incrementing at a particular moment or not.

You may find the following blog useful which I wrote sometimg back. This talks about troubleshooting output queue drops due to QoS on this platform.

https://supportforums.cisco.com/community/netpro/network-infrastructure/switching/blog/2011/04/11/output-drops-due-to-qos-on-296035603750-switches

Hope that helps.

Regards,

Shashank

[Kishore Chennupati]

Shashank and Sweta,

Man , I must admit that you guys are doing a bloody good job at this. You have shared so much wealth that it could have taken engineers days or weeks or even months to find. I would really thank you from the bottom of my heart and please keep adding value to the CSC as you always do.

I do have heaps of questions but not that I can ask one now.. 

I would like to request everyone who has posted their question here to rate the experts by generously clicking on the 5 stars if they reply has helped you. These guys deserve it. I have done it to begin with

Regards, Kishore

[Shashank Singh]

Hi Kishore,

Thanks for all your kind words, really appreciate it! It is immensely satisfying that you found this discussion helpful which compensates for all our efforts here!

Regards,

Shashank