cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Announcements
22599
Views
15
Helpful
21
Replies
Highlighted
Community Manager

Ask the Expert: Troubleshooting WAN Links Using QoS

Read the bioTroubleshooting WAN Links Using QoS
with Sarala Akella

Welcome to the Cisco Support Community Ask the Expert conversation. This is an opportunity to learn how to troubleshoot congestion on WAN links by using QoS and buffering best practices to optimize traffic flow. This includes queueing techniques (like WFQ, CBWFQ, or LLQ,) congestion avoidance (like WRED and CAR) as well as policing and traffic shaping mechanisms. Sarala is a customer support engineer at the Cisco Technical Assistance Center. She currently works in the WAN team where she focuses on various WAN related issues along with QoS issues on various interfaces. Sarala has been with Cisco for 11 years and has worked as a software engineer in the Network Software and Systems Technology Group. She holds a master's degree in computer engineering from Santa Clara University and a master's degree in mathematics from Osmania University, India. She also holds CCIE certification (#29921) in Routing and Switching.

Remember to use the rating system to let Sarala know if you have received an adequate response. 

Sarala might not be able to answer each question due to the volume expected during this event. Remember that you can continue the conversation on the Network Infrastructure WAN, Routing and Switching discussion forum shortly after the event.   This event lasts through January 13, 2012. Visit this forum often to view responses to your questions and the questions of other community members.

21 REPLIES 21
Beginner

Ask the Expert: Troubleshooting WAN Links Using QoS

1.     How does software queue and hardware queue works on WAN links.

2.     What is the reason of output drops on WAN links? How can we minimize them?

3.     What is the function of bandwidth configured on WAN link?

Beginner

Re: Ask the Expert: Troubleshooting WAN Links Using QoS

1: How does software queue and hardware queue works on WAN links

Cisco routers have two types of queues: a hardware queue and a software queue. The hardware

queue, which is sometimes referred to as the transmit queue (TxQ), always uses FIFO queuing, and only

when the hardware queue is full does the software queue handle packets.

Therefore, your queuing configuration only takes effect during periods of interface congestion, when the

hardware queue has overflowed.

Congestion must occur on the interface first, which causes packets to be held in the TX Ring/TX Queue.

When the TX Ring/TX Queue fills, IOS enables the queuing function on the interface

Software queue is configured using qos. When there is congestion on the link the software queue will kick it and prioritize the packets and put it in the hardware queue.

Depending on the type of interface you have you can configure different type of Qos to implement software queue.

2: What is the reason of output drops on WAN links? How can we minimize them?

Output drops are caused by a congested interface. For example, the traffic rate on the outgoing interface cannot accept all packets that should be sent out. The ultimate solution to resolve the problem is to increase the line speed. However, there are ways to prevent, decrease, or control output drops when you do not want to increase the line speed. You can prevent output drops only if output drops are a consequence of short bursts of data. If output drops are caused by a constant high-rate flow, you cannot prevent the drops. However, you can control them;

If you see short bursts that fill up the output queue buffer causing output drops, you can increase the hold queue to help with the burst.

Router(conf-if)# hold-queue length out

This command is only for burst traffic.

If you are over congesting the interface you cannot prevent output drops, but you can make sure that priority traffic is not dropped by configuring qos on the interface.

This will drop low priority traffic at time of congestion and make sure you priority traffic is not affected.

Sometimes it could be a bug where the queues are not getting freed after use and that would need more troubleshooting to isolate and fix in code.

3: What is the function of bandwidth configured on WAN link?

Appling a bandwidth statement on a interface does not change the performance of the interface in any way.

The main function of using the bandwidth statement is for routing metrics.EIGRP uses the bandwidth of the link set by the bandwidth command when calculating the metrics.

Beginner

Re: Ask the Expert: Troubleshooting WAN Links Using QoS

He I have Question about WAN. How can We send all traffic true central router from localization router? We have 2901 router in central or 891 router in localization. Problem is when We have that connection WWW works terible in localization.

Beginner

Re: Ask the Expert: Troubleshooting WAN Links Using QoS

Hi

I have question

We are using Cisco 3845 router in this router we created many Sub interfaces Ok

at present i have configured rate-limit for bandwidth limit as per our required , now i need to limit the bandwidth via QoS (both input/output) on sub interface.

e g one of our Customer having 5mb on interface gi0/1.5 , they need 3mb for voice and 2mb for data , what is the exact command i can configure on subinterface as per my require.

How can i achive on subinterface in cisco 3845 router via Qos.

Thanks in ADV,

Beginner

Re: Ask the Expert: Troubleshooting WAN Links Using QoS

An Ethernet subinterface is a logical interface in Cisco IOS. You can use the modular QoS command-line interface (CLI) (MQC) to create and apply a service policy to an Ethernet subinterface

Cisco IOS logical interfaces do not inherently support a state of congestion and do not support the direct application of a service policy that applies a queueing method. Instead, you first need to apply shaping to the subinterface using either generic traffic shaping (GTS) or class-based shaping

you must configure a hierarchical policy with the shape command at the parent level. Use the bandwidth command for CBWFQ, or the priority command for Low Latency Queueing (LLQ) at lower levels. Class-based shaping limits the output rate and (we can assume) leads to a congested state on the logical subinterface. The subinterface than applies "backpressure," and Cisco IOS begins queueing the excess packets that are held by the shaper.

Applying a Hierarchical Policy

Follow these steps to apply a hierarchical policy:

1.       Create a child or lower-level policy that configures a queueing mechanism. In the example below, we configure LLQ using the priority

command and CBWFQ using the bandwidth command.

            policy-map child 
            class voice 
                    priority 512

2. Create a parent or top-level policy that applies class-based shaping. Apply the child policy as a command under the parent policy since the admission control for the child class is done based on the shaping rate for the parent class.

            policy-map parent
            class class-default 
              shape average 2000000 
       service-policy child



3 Apply the parent policy to the subinterface.

            interface ethernet0/0.1
    service-policy parent 

    Beginner

    Re: Ask the Expert: Troubleshooting WAN Links Using QoS

    The question is not clear to me, I  will  try to answer the best from my interpretation

    We will be needing to ensure that the traffic leaving the 891 router QoS is configured on the outgoing interface to the central router.

    class-map match-any voice

    match access-group name sip-voip

    !

    policy-map QOS

    class voice

    priority 2000

    class class-default

    !

    policy-map parentQOS

    class class-default

    shape average 5000000

    service-policy QOS

      For central router please let me what kind of connection is it? If it is it Hub and spoke kind of connection.

    Then on central router we will have config in similar lines....

    Step 1: Create class-maps     

    class-map match-all VIDEO_SIGNAL

      match access-group 65

      match access-group 165

    class-map match-all VOICE_SIGNAL

      match access-group 165

    class-map match-all CITRIX

      match protocol citrix

    class-map match-all FTP

      match protocol ftp

    class-map match-all VIDEO

      match access-group 65

    class-map match-any VOICE

      match  dscp ef

    class-map match-all location1

      match access-group 161           (create access list to match destination subnets)

    class-map match-all location2

      match access-group 162           (create access list to match destination subnets)

    class-map match-all location3

      match access-group 163           (create access list to match destination subnets)

    class-map match-all location4

      match access-group 164           (create access list to match destination subnets)

    class-map

    ==================================

    Step 2: Create your child policy maps.

    policy-map LLQ2

      class VOICE_SIGNAL

       bandwidth percent 2

      class VIDEO_SIGNAL

       bandwidth percent 3

      class VOICE

       priority percent 10

      class VIDEO

       bandwidth percent 20

       police 9000000 conform-action transmit  exceed-action drop

      class CITRIX

       bandwidth percent 25

      class FTP

       police 5000000 conform-action transmit  exceed-action drop

    ==================================

    Step 3: Create the parent policy map.

    policy-map Parent1

    class location1

      shape average 3000000              (exmaple shaping value)

      service-policy LLQ2

    class location2

      shape average 1500000                (exmaple shaping value)

      service-policy LLQ2

    class location3

      shape average 1000000                (exmaple shaping value)

      service-policy LLQ2

    class location4

      shape average 750000               (exmaple shaping value)

      service-policy LLQ2

    ==================================

    Step 4: Apply map class interface

    ON WAN interface

    service-policy output Parent1

    If  I have not answered  you question., please provide me the config for 891 and 2911 and the  problem. I will be able to help better.

    Participant

    Re: Ask the Expert: Troubleshooting WAN Links Using QoS

    It's our understanding that it's best to perform QoS marking as close to the source as possible, so that's what we're doing:

    We mark appropriate DSCP values on our datacenter aggregation switches.

    However, our Nexus 7K's & Cat 6K's don't seem to support classification of Citrix ICA Traffic by ICA Tag Number using NBAR.

    We're running 5.1(3) on the 7K's, and 12.2(33)SXI5 on the 6K's.

    Could you please tell us what versions of NX-OS & IOS for Catalyst support Citrix ICA Tag Number through NBAR?

    IOS 12.4(24)T5 on an ISR 2800 supports it.

    http://www.cisco.com/en/US/docs/ios/ios_xe/qos/configuration/guide/clsfy_traffic_nbar_xe.html#wp1167057

    =======

    N7K(config)# class-map ICA
    N7K(config-cmap-qos)# match protocol ?
      arp       IP ARP
      bridging  Bridging
      cdp       Cisco Discovery Protocol
      clns      ISO CLNS
      clns_es   ISO CLNS End System
      clns_is   ISO CLNS Intermediate System
      dhcp      Dynamic Host Configuration
      isis      Intermediate System Intermediate System Protocol
      ldp       Label Distribution Protocol
      netbios   NetBIOS

    N7K(config-cmap-qos)#

    ===============

    C6K(config)#class-map ICA

    C6K(config-cmap)#match protocol citrix ?
      app  Match Application Name String
     

    C6K(config-cmap)#

    ===============

    ISR(config)#class-map ICA

    ISR(config-cmap)#match protocol citrix ?
      app      Published App in Server Browser Mode
      ica-tag  Citrix ICA tag 0-high 1-medium 2-low 3-background
     

    ISR(config-cmap)#

    Classification of Citrix ICA Traffic by ICA Tag Number

    Beginner

    Re: Ask the Expert: Troubleshooting WAN Links Using QoS

    Hi

    Unfortunatley There is no NBAR support on n7k for now and foreseeable future, The 5.2 feature set is documented in the release notes.

    http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/release/notes/52_nx-os_release_note.html#wp388984

    N7K(config-cmap-qos)# match protocol ?

    arp       IP ARP

    bridging  Bridging

    cdp       Cisco Discovery Protocol

    clns      ISO CLNS

    clns_es   ISO CLNS End System

    clns_is   ISO CLNS Intermediate System

    dhcp      Dynamic Host Configuration

    isis      Intermediate System Intermediate System Protocol

    ldp       Label Distribution Protocol

    netbios   NetBIOS

    All the above option

    if nbar is supported, I would expect to see things like "telnet" "http" "ftp" and the like....something that would require looking at L4 info or using signatures .

    The work around would be to configure QoS using MQC with ACL etc.

    Here is the document to configure QoS on Nexus

    http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/configuration/guide/cli/QoS.html

    But please note that this is OUt of scope for the disscussion as it falls in datacenter swicthing domain. Our topic of disscussion is QoS on WAN.

    I would request you to open a TAC case for more questions on nexus.

    Beginner

    Re: Ask the Expert: Troubleshooting WAN Links Using QoS

    Hi

    I already submitted this question on the LAN switching board before I noticed your thread. I have re-submitted it here as I may hit the QOS "jackpot" for knowledge

    I have a query about the following command

    srr-queue bandwidth shape 30 0 0 0

    I understand what the command does and how the weights work.

    With the example above weight1 is 1/30 of the interface bandwidth but.....

    is this 1/30 of the physical port (say 1Gb) or can it be 1/30 of the configured bandwidth command?

    The reason behind this is I have a throttled link from my ISP.

    I connect to the CPE locally via a 1Gb sfp (fibre), the link between my two sites in question is also a 1Gb link but for now has been software throttled by the ISP to 100Mb (trying to get this changed as the link is on its way to some congestion in the near future).

    Can i simply put the bandwidth statement bandwidth 100000 on my interface and the original command above use this in its calculation?

    I dont want to use the speed command on my interface.

    any help is alway appreciated

    regards

    Stuart

    Beginner

    Re: Ask the Expert: Troubleshooting WAN Links Using QoS

    Here is the response to the question:

    srr-queue bandwidth shape 0 0 0

    The above command will calculate percentage of bandwidth for queue 1 based on the link speed (But not the interface bandwidth you configure).

    Eg:

    config t

    int gi1/0/1

    srr-queue bandwidth shape 10 0 0 0

    The bandwidth weight for queue 1 in here is 1/10 == 10% of link speed (i.e. 10% of 1gig = 100Mb)

    Here is the link which gives more information:

    http://www.cisco.com/en/US/partner/docs/switches/lan/catalyst3750/software/release/12.2_55_se/commmand/reference/cli3.html#wp1947494

    As I understand correctly, You want to rate limit traffic to 100 Mb on gig interface right?

    Then you can either change the link speed (which you dont want to do) or shape the traffic to 10% of gig interface i.e. "srr-queue bandwidth shape 10 0 0 0" under the interface.

    Configuring bandwidth 10000 will not help here, interface bandwidth configuration is used for metric calculations in routing protocols.

    Beginner

    Ask the Expert: Troubleshooting WAN Links Using QoS

    Sarala

    thankyou for the confirmation about the bandwidth statement being ignored as I couldn't really test this out on the live link.

    Beginner

    Ask the Expert: Troubleshooting WAN Links Using QoS

    Hello Sarala,

    My question is in regards to QoS on an MFR interface/subinterfaces. We have a remote site with two bundled T1's terminating on a 2951 router for a total bandwidth of 3072. The circuit is provided by Paetec and the subinterfaces are designated for internet and MPLS traffic respectively. The issue we are facing is with outbound voice quality. It seems that no matter how we apply QoS, either to the main MFR interface or the MFR subinterfaces, voice packets do not seem to be prioritized. We tried FRTS, which slowed the entire link down to a crawl, we tried applying a class map to the main interface as well as a service policy, none of which seemed to affect anything. Please see below for current partial configuration. Any input will be greatly appreciated.

    class-map match-all VOICE

    match ip dscp ef

    class-map match-any SIGNALING

    match ip dscp af31

    match ip dscp cs3

    class-map match-all AZ-SERVERS

    match access-group 10

    !

    !

    policy-map VOICE-POLICY

    class VOICE

        priority 640

    class SIGNALING

        bandwidth 64

      set dscp af41

    class AZ-SERVERS

        police 1000000

    class class-default

        fair-queue

    !

    !

    !

    !

    !        

    interface Loopback1

    no ip address

    !

    interface Tunnel1

    no ip address

    !

    interface MFR1

    no ip address

    ip flow ingress

    ip flow egress

    load-interval 30

    frame-relay lmi-type ansi

    service-policy output VOICE-POLICY

    !

    interface MFR1.501 point-to-point

    description => Internet via PAETEC

    ip vrf forwarding internet

    ip address 63.255.X.X 255.255.255.252
    ip flow ingress
    ip flow egress
    no cdp enable

    frame-relay interface-dlci 501 IETF  

    !

    interface MFR1.502 point-to-point

    description => MPLS VPN via PAETEC

    ip address 63.253.X.X 255.255.255.252
    ip flow ingress
    ip flow egress
    no cdp enable
    frame-relay interface-dlci 502 IETF  

    Beginner

    Re: Ask the Expert: Troubleshooting WAN Links Using QoS

    Hello Sarala,

    My question is in regards to QoS on an MFR interface/subinterfaces. We have a remote site with two bundled T1's terminating on a 2951 router for a total bandwidth of 3072. The circuit is provided by Paetec and the subinterfaces are designated for internet and MPLS traffic respectively. The issue we are facing is with outbound voice quality. It seems that no matter how we apply QoS, either to the main MFR interface or the MFR subinterfaces, voice packets do not seem to be prioritized. We tried FRTS, which slowed the entire link down to a crawl, we tried applying a class map to the main interface as well as a service policy, none of which seemed to affect anything. Please see below for current partial configuration. Any input will be greatly appreciated.

    class-map match-all VOICE

    match ip dscp ef

    class-map match-any SIGNALING

    match ip dscp af31

    match ip dscp cs3

    class-map match-all AZ-SERVERS

    match access-group 10

    !

    !

    policy-map VOICE-POLICY

    class VOICE

        priority 640

    class SIGNALING

        bandwidth 64

      set dscp af41

    class AZ-SERVERS

        police 1000000

    class class-default

        fair-queue

    !

    !

    !

    !

    !        

    interface Loopback1

    no ip address

    !

    interface Tunnel1

    no ip address

    !

    interface MFR1

    no ip address

    ip flow ingress

    ip flow egress

    load-interval 30

    frame-relay lmi-type ansi

    service-policy output VOICE-POLICY

    !

    interface MFR1.501 point-to-point

    description => Internet via PAETEC

    ip vrf forwarding internet

    ip address 63.255.X.X 255.255.255.252
    ip flow ingress
    ip flow egress
    no cdp enable

    frame-relay interface-dlci 501 IETF  

    !

    interface MFR1.502 point-to-point

    description => MPLS VPN via PAETEC

    ip address 63.253.X.X 255.255.255.252
    ip flow ingress
    ip flow egress
    no cdp enable
    frame-relay interface-dlci 502 IETF

    Beginner

    Re: Ask the Expert: Troubleshooting WAN Links Using QoS

    You have mentioned that you tried FRTS. Can you please confirm what configuration you had. It should have worked with FRTS. if you are not seeing any match  we may need to troubleshoot if the packets coming into the router are marked correctly.

    Here is the sample config for FRTS

    !

    class-map match-all MEDIA

    match  dscp ef

    class-map match-all SIGNALING

    match  dscp af41

    !

    !

    policy-map VOIP

    class MEDIA

    priority percent 50

    class SIGNALING

    priority percent 20

    !

    !

    !

    !

    !

    interface MFR1

    no ip address

    ip route-cache flow

    load-interval 30

    no arp frame-relay

    frame-relay traffic-shaping   <<<< do this command last

    !

    interface MFR1.501 point-to-point

    description CONNECTED TO INTERNET

    ip address 74.10.15.246 255.255.255.252

    no arp frame-relay

    frame-relay interface-dlci 501 IETF  

    class VOIPovFR        <<<<<<<<<<<<<<

    !

    interface MFR1.502 point-to-point

    description CONNECTED TO MPLS

    ip address 74.10.22.26 255.255.255.252

    no arp frame-relay

    frame-relay interface-dlci 502 IETF  

    class VOIPovFR        <<<<<<<<<<<<<<

    !

    !

    map-class frame-relay VOIPovFR

    no frame-relay adaptive-shaping

    frame-relay cir 64000    <<<< CIR = MINCIR

    frame-relay bc 640    <<< bc = CIR / 100

    frame-relay be 0

    frame-relay mincir 64000  <<<< Change as needed

    service-policy output VOIP

    !

    please provide show policy-map output. We may have to open TAC case as would be needing interactive/troubleshootingsession.

    CreatePlease to create content
    Content for Community-Ad
    July's Community Spotlight Awards