3 Tier QoS Policy for MPLS

[hypnotoad]

I'm working on a three tier QOS policy for use on an MPLS network.  We are not running MPLS, the provider is.  We are peering via BGP as 22 locations.  The circuit speeds vary by site.  My routers are ISR4351 in the offices and ASR1001-X in the two data centers.

Look at this configuration and see if I'm on the right track.  I'm using LiveAction to monitor QOS.  I'm seeing drops on an interface because of shaping.  I have the shaping rate set to 100Mbps on a 100Mbps circuit so can't see why I would see class drops.

class-map DC1-SHAPING
 match access-group name DC1-SHAPING-ACL

class-map DC2-SHAPING
 match access-group name DC2-SHAPING-ACL  

class-map DEN-SHAPING
 match access-group name DEN-SHAPING-ACL  

class-map LAX-SHAPING
 match access-group name LAX-SHAPING-ACL  

class-map VOICE-QUEUE
  match dscp 40
  match dscp 46
 
class-map VIDEO-INTERACTIVE-QUEUE
  match dscp 32
  match dscp 34
  match dscp 36
  match dscp 38

class-map VIDEO-STREAMING-QUEUE
  match dscp 56

class-map NETWORK-CONTROL-QUEUE
  match dscp 48

class-map CALL-SIGNALING-QUEUE
  match dscp 24
  match dscp 26
  match dscp 28
  match dscp 30

class-map CRITICAL-DATA-QUEUE
  match dscp 16
  match dscp 18
  match dscp 20
  match dscp 22

class-map BULK-DATA-QUEUE
  match dscp 8
  match dscp 10
  match dscp 12
  match dscp 14
 
 
policy-map WAN-EGRESS-POLICY
  class class-default
    shape average 100000000
      service-policy SHAPING-POLICY


policy-map SHAPING-POLICY
  class DC1-SHAPING
    shape average 100000000
    service-policy QUEUING-POLICY                            

  class DC2-SHAPING
    shape average 100000000
    service-policy QUEUING-POLICY                

  class DEN-SHAPING
    shape average 100000000
    service-policy QUEUING-POLICY            

  class LAX-SHAPING
    shape average 100000000
    service-policy QUEUING-POLICY                


policy-map QUEUING-POLICY
  class VOICE-QUEUE
    priority level 1
    police rate percent 10
!
  class VIDEO-INTERACTIVE-QUEUE
    priority level 2
    police rate percent 15
!
  class VIDEO-STREAMING-QUEUE
    bandwidth percent 5
!
  class NETWORK-CONTROL-QUEUE
    bandwidth percent 5
!
  class CALL-SIGNALING-QUEUE
    bandwidth percent 15
!
  class CRITICAL-DATA-QUEUE
    bandwidth percent 20
    fair-queue
    random-detect dscp-based
!
  class BULK-DATA-QUEUE
    bandwidth percent 20
!
  class class-default
    bandwidth percent 10
    fair-queue
    random-detect dscp-based
!
interface GigabitEthernet0/0/0
  service-policy output WAN-EGRESS-POLICY

[Georg Pauwen]

Hello,

my first thought is: are the drops on the classes configured with the 'bandwidth' percentage ? If that is the case, you might simply be sending more than the 100Mbps, because the 'bandwidth' is only active when there is actual congestion.

That said, does it make a difference if you configure the bc on your shape averages ?

shape average 100000000 10000000

[hypnotoad]

I just made this change on the 1st tier shaper as shown below.

policy-map QOS-WAN-EGRESS
 class class-default
  shape average 100000000 10000000  
   service-policy SHAPING-POLICY

[hypnotoad]

I also made the the same change to the two data center shapers that are at tier two. 

policy-map SHAPING-POLICY
 class DC1-SHAPING
  shape average 100000000 10000000  
   service-policy QUEUING-POLICY
 class DC2-SHAPING
  shape average 100000000 10000000  
   service-policy QUEUING-POLICY

This circuit has no congestion.  It is running at 9% peak input and 5% peak output.

--Patrick

[Georg Pauwen]

Hello Patrick,

I don't think congestion is causing the drops, it looks like a problem with the queue size. There are a few classes with drops, e.g.:

DC1-SHAPING

DC2-SHAPING

Try and double the size of the queue limit in these classes:

'queue-limit 832'

Does that reduce the drops ?

[hypnotoad]

I called TAC and they said the same thing.  I doubled the queue limit for class default to 128 packets.  I cleared the counters and am watching for drops.

TAC also said that I could disable fair-queue as a plan B.

 class class-default
  bandwidth percent 10
  fair-queue
  random-detect dscp-based
  queue-limit 128 packets

[Georg Pauwen]

Hello,

disabling the fair queue would be another option. Curious to know if either of those config changes reduces the drops...

[hypnotoad]

Doubling the queue helped but I still see drops.  I'm wondering if I should double it again or just get rid of fair-queue. 

[Georg Pauwen]

Hello,

as far as I can tell, the classes where the drops occur have a queue limit of 416, that is why I suggested 832 (double the current size). Can you try these values ?

[hypnotoad]

Here's a snip of the 'show policy-map' output.

            Class-map: class-default (match-any)  
              6378 packets, 3946026 bytes
              5 minute offered rate 87000 bps, drop rate 0000 bps
              Match: any
              Queueing
              queue limit 256 packets
              (queue depth/total drops/no-buffer drops/flowdrops) 0/0/0/0
              (pkts output/bytes output) 6378/3946026
              bandwidth 10% (10000 kbps)
              Fair-queue: per-flow queue limit 64 packets
                Exp-weight-constant: 4 (1/16)
                Mean queue depth: 0 packets

I just changed it to 256 from 128.

policy-map QUEUING-POLICY
 class class-default
  bandwidth percent 10
  fair-queue
  random-detect dscp-based
  queue-limit 256 packets

[hypnotoad]

I was still seeing drops so I removed fair-queue.  The office I'm testing with is my smallest by head count.  The config looks like this.

policy-map QUEUING-POLICY

 class class-default
  bandwidth percent 10
  random-detect dscp-based
  queue-limit 256 packets

--Patrick

[Yahia CHARIF]

Hi,

Can you please post the output of "show policy-map interface GigabitEthernet0/0/0 output" command when you see the drops.

[hypnotoad]

See the attachment.  Thanks for looking.

--Patrick

[Yahia CHARIF]

Hi,

I think there some problems when "bandwidth" and "fair-queue" are applied together.

Can you please try to remove the "bandwidth percent 10" from the "class-default", keep the "fair-queue" command and try again.

class class-default
    fair-queue
    random-detect dscp-based

!

[hypnotoad]

I removed the bandwidth statement from class-default.  This is what the queuing policy looks like now.

policy-map QUEUING-POLICY
 class VOICE-QUEUE
  police rate percent 10
  priority level 1
 class VIDEO-INTERACTIVE-QUEUE
  police rate percent 15
  priority level 2
 class VIDEO-STREAMING-QUEUE
  bandwidth percent 5
 class NETWORK-CONTROL-QUEUE
  bandwidth percent 5
 class CALL-SIGNALING-QUEUE
  bandwidth percent 15
 class CRITICAL-DATA-QUEUE
  bandwidth percent 20
  fair-queue
  random-detect dscp-based
 class BULK-DATA-QUEUE
  bandwidth percent 20
 class class-default
  random-detect dscp-based
  fair-queue

--Patrick