Clarity needed please for how QoS works on 3750/3560 Cat Switches - thanks

[richard.thomas]

Hi,

Hopefully someone can help!

I need answers to the specific questions below which concentrate solely on how QoS works on 3750/3560 Cat Switches.

Having studying in depth the following URL: https://supportforums.cisco.com/docs/DOC-8093, I like to think I have a fairly good understanding of how it all works, but I really need some clarity on a few specifics.

I understand how enabling mls qos creates all the ingress and egress queues etc... and I've seen flow diagrams that show how traffic passes through them but what the documentation doesn't seem to say is whether on egress the queues are used only during times of interface congestion. So my first question is:

Q1.)  Does a  packet that needs to be transmitted out an interface ALWAYS get placed into one of the interfaces egress Tx queues, regardless of whether interface congestion is being experienced?

My thoughts are that it wouldn’t as queuing a packet is only necessary when either there is no room on the interfaces Tx ring or when a packet is already in an egress Tx queue awaiting transmission scheduling. So, in other words when the interface is uncongested, packets are not placed in the egress Tx queues.

Q2.) This sort of depends on the answer to Q1.) but I’ll assume for now my thoughts on it are correct. The document above states:-

“ When the expedite queue is enabled, it’s no longer shaped or shared and whenever there’s a packet enqueued to the egress tx queue 1, the egress logic will service it. All weights (shaped or shared) for tx queue 1 are ignored. This needs to be taken into consideration when configuring the shared weights for the other queues. The shared weight configured for tx queue 1 is not included in the ratio computation. If you use the default weights of 25, 25, 25, 25 for tx queues 1-4 respectively, then tx queues 2-4 would each be guaranteed 33% of the egress bandwidth.”

I understand this statement but my question again relates to what happens during interface congestion. From the statement, it infers that it is possible that traffic enqueued to egress tx queue 1 can consume all the interface bandwidth starving bandwidth from the other egress Tx queues.  If this is true, then the scheduling bandwidth assigned to tx-queues 2-4 using the shared weighted ratios cannot be guaranteed.

Therefore, how does bandwidth to tx queues 2-4 get guaranteed when a priority queue could potentially starve them of it?

My thoughts are, as soon as any of the Tx queues begins to fill, this would indicate interface congestion. When this occurs and as soon as the scheduler detects the priority queue is empty, it services the other tx queues that have packets waiting, according to the weighted ratio derived from the configuration and empty Tx queue states. So as long as no traffic enters the priority queue during this period, the shared weighted bandwidth ratios can be guaranteed. Realistically, as it’s more than likely traffic will enter the priority queue, the scheduler will not be able to service the other Tx queues continuously at the rate defined, so as long as it’s not interrupted to much servicing the priority queue, the figures quoted are based on average bandwidth guarantees.

Q3.) We have Catalyst switches that use Gigabit Ethernet 802.1q trunks as their uplink to a PE router. IP packets are tagged with an 802.1q frame as they leave the PE router towards the Catalyst switch. The switches Ethernet 802.1q trunk is configured to trust DSCP. Am I correct in saying that despite receiving the 802.1q tag first as the IP packet is played in, the switch performs packet inspection on the DSCP field in the IP header?

My thoughts are that it does because trust DSCP is configured. I’ve read somewhere this is what happens but could really do with confirmation.

I would really appreciate any feedback regarding these questions.

Many thanks.

Regards

Richard

[Joseph W. Doherty]

Disclaimer

The  Author of this posting offers the information contained within this  posting without consideration and with the reader's understanding that  there's no implied or expressed suitability or fitness for any purpose.  Information provided is for informational purposes only and should not  be construed as rendering professional advice of any kind. Usage of this  posting's information is solely at reader's own risk.

Liability Disclaimer

In  no event shall Author be liable for any damages whatsoever (including,  without limitation, damages for loss of use, data or profit) arising out  of the use or inability to use the posting's information even if Author  has been advised of the possibility of such damage.

Posting

Q1: I don't recall seeing documentation whether the 3560/3750 maintains a physically separate tx queue from the QoS enabled egress queues.  I would suspect not, but even if there is a tx queue I also suspect it might use hardware pointers to the same physical buffer space (this to avoid physically moving the packet's bytes, within the buffers, from logical egress queue to tx queue).  It might also simplify the logic as you don't have to deal with overflowing a tx queue's packet into the correct logical egress queue.

Q2: My understanding of egress PQ is it can indeed starve the other queues.  The statement that's confusing would be better worded as ". . . then tx queues 2-4 would each be guaranteed 33% of the remaining egress bandwidth."

Q3: I don't think it really matters what inspected or the sequence of inspection.  What matters is what you've told the port to "trust" for QoS purposes.  If you told it to "trust" DSCP, then that's what's used.

[richard.thomas]

Hi Joseph,

Thanks’ for responding.

I take your point about there may not be a physically separate Tx output queue, but having just looked again at a typical switch Fast Ethernet interface (with mls qos enabled), when you do a 'show interface fax/y' it displays the interface queuing as FIFO. Furthermore, it displays the number of output queue drops different to the number of drops shown when you do a 'show mls qos interface fax/y stat'. That sort of indicated to me that packets leave the Fast Ethernet interface from the FIFO output queue, having been placed there by the scheduler servicing the 4 egress Tx queues. So again, this leaves me a bit confused!

Perhaps someone could enlighten me please. Would really appreciate it!

Many thanks

Richard

emnewol-3560-1#sh int fa0/4

FastEthernet0/4 is up, line protocol is up (connected)

Hardware is Fast Ethernet, address is fcfb.fb3a.7286 (bia fcfb.fb3a.7286)

MTU 1500 bytes, BW 10000 Kbit, DLY 1000 usec,

     reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full-duplex, 10Mb/s, media type is 10/100BaseTX

input flow-control is off, output flow-control is unsupported

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:30, output 00:00:00, output hang never

Last clearing of "show interface" counters never

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 7421965

Queueing strategy: fifo

Output queue: 0/0 (size/max)

5 minute input rate 60000 bits/sec, 48 packets/sec

5 minute output rate 60000 bits/sec, 46 packets/sec

     1103175252 packets input, 375560922647 bytes, 0 no buffer

     Received 1379913 broadcasts (1379648 multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 1379648 multicast, 0 pause input

     0 input packets with dribble condition detected

     1563641206 packets output, 917109616747 bytes, 0 underruns

     0 output errors, 0 collisions, 3 interface resets

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 PAUSE output

     0 output buffer failures, 0 output buffers swapped out

show mls qos int fa0/4 stat

FastEthernet0/4 (All statistics are in packets)

dscp: incoming

-------------------------------

0 - 4 :   496626375           0           16           0         122

5 - 9 :           0           0           0     5362970           0

10 - 14 :           0           0           0           0           0

15 - 19 :           0           0           0     27741158           0

20 - 24 :           0            0           0           0           0

25 - 29 :           0           0           0           0           0

30 - 34 :           0           0           0           0     27741598

35 - 39 :           0           0           0           0            0

40 - 44 :           0           0           0           0           0

45 - 49 :           0   758413731           0     14125468           0

50 - 54 :           0           0           0           0           0

55 - 59 :          0           0           0           0           0

60 - 64 :           0           0           0           0

dscp: outgoing

-------------------------------

0 - 4 :   625784587           0           0           0           0

5 - 9 :          0           0           0       13192           0

10 - 14 :           0           0           0           0           0

15 - 19 :           0           0           0     27541096           0

20 - 24 :           0           0          0           0           0

25 - 29 :           0           0           0           0           0

30 - 34 :           0           0           0           0     27552706

35 - 39 :           0           0           0           0           0

40 - 44 :           0           0           0           0           0

45 - 49 :           0   721854689           0           53           0

50 - 54 :           0           0           0           0           0

55 - 59 :           0          0           0           0           0

60 - 64 :           0           0           0           0

cos: incoming

-------------------------------

0 - 4 : 1337018185           0           0           0           0

5 - 7 :           0           0           0

cos: outgoing

-------------------------------

0 - 4 :   625788524       13192     27541096           0     27552706

5 - 7 :   721854689           53           0

output queues enqueued:

queue: threshold1 threshold2 threshold3

-----------------------------------------

queue 0:   721854691           0           0

queue 1:   625788641           0   37871565

queue 2:   27554289           0           0

queue 3:   27552760           0           0

output queues dropped:

queue: threshold1 threshold2 threshold3

-----------------------------------------

queue 0:           0           0           0

queue 1:     7318631           0           0

queue 2:           0           0           0

queue 3:           0           0           0

Policer: Inprofile:           0 OutofProfile:           0

[Joseph W. Doherty]

Disclaimer

The   Author of this posting offers the information contained within this   posting without consideration and with the reader's understanding that   there's no implied or expressed suitability or fitness for any purpose.   Information provided is for informational purposes only and should not   be construed as rendering professional advice of any kind. Usage of  this  posting's information is solely at reader's own risk.

Liability Disclaimer

In   no event shall Author be liable for any damages whatsoever (including,   without limitation, damages for loss of use, data or profit) arising  out  of the use or inability to use the posting's information even if  Author  has been advised of the possibility of such damage.

Posting

I believe I recall, hardware stats don't always sync up to interface stats, as you might expect.  For example, try clearing your interface stats and then compare those with the hardware stats.

[richard.thomas]

Hi Joseph,

Thanks for response!

OK, on the basis packets are placed in the 4 egress queues regardless of congestion and there is no Tx queue, given the following scenario, would I be right in thinking that packets could potentially be dropped before the interface becomes congested.

In this example all 4 egress queues are configured to share their bandwidth (Not shape and no priority queue enabled). The weights configured for each queue proportionally allocate 40%: 30%:20%:10% interface bandwidth to Q1,Q2,Q3 and Q4 respectively.

Lets say in this example Q4 is the only egress queue that is recieving packets at a rate that consumes 30% of the interfaces bandwidth. (30% load). As all other egress queues are empty, the bandwidth assigned to them is given to Q4. Hence, Q4 could happily handle frames entering it at a rate that consumes 100% of the interface bandwidth without packet droppage.

Now lets say Q3 also starts receiving packets at a rate that would consume 20% of the interfaces bandwidth (20% load). As both Q1 and Q2 are empty, their bandwidth is shared equally between Q3 and Q4. So far the total loading from Q3 and Q4 is 50% interface bandwidth. Note: The loading Q3 and Q4 experience is well within their scheduling bandwidths at this point.

Now finally Q2 starts receiving packets at a rate that consumes 10% of the interfaces bandwidth (10% load). As Q1 is still empty, its bandwidth is shared equally between Q2, Q3 and Q4. Now the total loading from Q2, Q3 and Q4 is 60% interface bandwidth.

This is where I find it gets interesting. Q4's loading (still 30%) now exceeds its 23% scheduling bandwidth. If Q4's loading continues for some time, more packets will enter Q4 than can leave and the number of Tx buffers it needs to borrow will grow untill at some point a threshold will be reached and tail drop occurs.

In effect, packet discard occurs before the all the interfaces bandwidth is utilised. Or put another way, packet discard occurs before the interface experiences congestion.(Remember there was only 60% bandwidth loading).

Although the interface doesn't experience congestion, egress Q4 does!

What I want to know is, am I correct in my understand from my example I've provided?

If anyone could advise, it would be much appreciated!

Many thanks

Richard

jjjjjjjjjjjjjjjjjjjjjjjjj

jjjjjjjjjjjjjj

lllllllllllllllllllllllllllllllllllllllllllll

[rakdutt]

Hi RIchard,

As 3560/3750 as 4 tx queue & every queue has some buffers. please have a look at example below:

3560-A#show mls qos interface g0/2 buffers

GigabitEthernet0/2

QoS is disabled. When QoS is enabled, following settings will be applied

The port is mapped to qset : 1

The allocations between the queues are : 25 25 25 25 

For above interface when mls qos enabled each queue will get 25% buffers.

In case the limit of buffers is excedded the packets will be dropped. At the same time you will see the output drops counter incremeting in show interface fax/x output.

HTH

Rakesh

[Joseph W. Doherty]

Disclaimer

The    Author of this posting offers the information contained within this    posting without consideration and with the reader's understanding that    there's no implied or expressed suitability or fitness for any  purpose.   Information provided is for informational purposes only and  should not   be construed as rendering professional advice of any kind.  Usage of  this  posting's information is solely at reader's own risk.

Liability Disclaimer

In    no event shall Author be liable for any damages whatsoever  (including,   without limitation, damages for loss of use, data or  profit) arising  out  of the use or inability to use the posting's  information even if  Author  has been advised of the possibility of such  damage.

Posting

No, in your example, Q4 shouldn't drop packets.  This because the overall offered load is only 60%.  SRR deals with bandwidth proportions when there's congestion, but as the overall offered rate is lower then the egress bandwidth, in theory each queue could receive a packet and transmit that packet while the other two queues are empty.  SRR shouldn't hold a packet just because it's above its allocated bandwidth allocation when there's no competing packets for that bandwidth.