01-03-2012 10:55 AM - edited 03-04-2019 02:48 PM
Troubleshooting WAN Links Using QoS
with Sarala Akella
Welcome to the Cisco Support Community Ask the Expert conversation. This is an opportunity to learn how to troubleshoot congestion on WAN links by using QoS and buffering best practices to optimize traffic flow. This includes queueing techniques (like WFQ, CBWFQ, or LLQ,) congestion avoidance (like WRED and CAR) as well as policing and traffic shaping mechanisms. Sarala is a customer support engineer at the Cisco Technical Assistance Center. She currently works in the WAN team where she focuses on various WAN related issues along with QoS issues on various interfaces. Sarala has been with Cisco for 11 years and has worked as a software engineer in the Network Software and Systems Technology Group. She holds a master's degree in computer engineering from Santa Clara University and a master's degree in mathematics from Osmania University, India. She also holds CCIE certification (#29921) in Routing and Switching.
Remember to use the rating system to let Sarala know if you have received an adequate response.
Sarala might not be able to answer each question due to the volume expected during this event. Remember that you can continue the conversation on the Network Infrastructure WAN, Routing and Switching discussion forum shortly after the event. This event lasts through January 13, 2012. Visit this forum often to view responses to your questions and the questions of other community members.
01-04-2012 09:28 PM
1. How does software queue and hardware queue works on WAN links.
2. What is the reason of output drops on WAN links? How can we minimize them?
3. What is the function of bandwidth configured on WAN link?
01-06-2012 03:05 PM
1: How does software queue and hardware queue works on WAN links
Cisco routers have two types of queues: a hardware queue and a software queue. The hardware
queue, which is sometimes referred to as the transmit queue (TxQ), always uses FIFO queuing, and only
when the hardware queue is full does the software queue handle packets.
Therefore, your queuing configuration only takes effect during periods of interface congestion, when the
hardware queue has overflowed.
Congestion must occur on the interface first, which causes packets to be held in the TX Ring/TX Queue.
When the TX Ring/TX Queue fills, IOS enables the queuing function on the interface
Software queue is configured using qos. When there is congestion on the link the software queue will kick it and prioritize the packets and put it in the hardware queue.
Depending on the type of interface you have you can configure different type of Qos to implement software queue.
2: What is the reason of output drops on WAN links? How can we minimize them?
Output drops are caused by a congested interface. For example, the traffic rate on the outgoing interface cannot accept all packets that should be sent out. The ultimate solution to resolve the problem is to increase the line speed. However, there are ways to prevent, decrease, or control output drops when you do not want to increase the line speed. You can prevent output drops only if output drops are a consequence of short bursts of data. If output drops are caused by a constant high-rate flow, you cannot prevent the drops. However, you can control them;
If you see short bursts that fill up the output queue buffer causing output drops, you can increase the hold queue to help with the burst.
Router(conf-if)# hold-queue length out
This command is only for burst traffic.
If you are over congesting the interface you cannot prevent output drops, but you can make sure that priority traffic is not dropped by configuring qos on the interface.
This will drop low priority traffic at time of congestion and make sure you priority traffic is not affected.
Sometimes it could be a bug where the queues are not getting freed after use and that would need more troubleshooting to isolate and fix in code.
3: What is the function of bandwidth configured on WAN link?
Appling a bandwidth statement on a interface does not change the performance of the interface in any way.
The main function of using the bandwidth statement is for routing metrics.EIGRP uses the bandwidth of the link set by the bandwidth command when calculating the metrics.
01-05-2012 05:18 AM
He I have Question about WAN. How can We send all traffic true central router from localization router? We have 2901 router in central or 891 router in localization. Problem is when We have that connection WWW works terible in localization.
01-05-2012 06:15 AM
Hi
I have question
We are using Cisco 3845 router in this router we created many Sub interfaces Ok
at present i have configured rate-limit for bandwidth limit as per our required , now i need to limit the bandwidth via QoS (both input/output) on sub interface.
e g one of our Customer having 5mb on interface gi0/1.5 , they need 3mb for voice and 2mb for data , what is the exact command i can configure on subinterface as per my require.
How can i achive on subinterface in cisco 3845 router via Qos.
Thanks in ADV,
01-06-2012 02:58 PM
An Ethernet subinterface is a logical interface in Cisco IOS. You can use the modular QoS command-line interface (CLI) (MQC) to create and apply a service policy to an Ethernet subinterface
Cisco IOS logical interfaces do not inherently support a state of congestion and do not support the direct application of a service policy that applies a queueing method. Instead, you first need to apply shaping to the subinterface using either generic traffic shaping (GTS) or class-based shaping
you must configure a hierarchical policy with the shape command at the parent level. Use the bandwidth command for CBWFQ, or the priority command for Low Latency Queueing (LLQ) at lower levels. Class-based shaping limits the output rate and (we can assume) leads to a congested state on the logical subinterface. The subinterface than applies "backpressure," and Cisco IOS begins queueing the excess packets that are held by the shaper.
Applying a Hierarchical Policy
Follow these steps to apply a hierarchical policy:
1. Create a child or lower-level policy that configures a queueing mechanism. In the example below, we configure LLQ using the priority
command and CBWFQ using the bandwidth command.
policy-map child
class voice
priority 512
2. Create a parent or top-level policy that applies class-based shaping. Apply the child policy as a command under the parent policy since the admission control for the child class is done based on the shaping rate for the parent class.
policy-map parent
class class-default
shape average 2000000
service-policy child
3 Apply the parent policy to the subinterface.
interface ethernet0/0.1
service-policy parent
01-06-2012 03:49 PM
The question is not clear to me, I will try to answer the best from my interpretation
We will be needing to ensure that the traffic leaving the 891 router QoS is configured on the outgoing interface to the central router.
class-map match-any voice
match access-group name sip-voip
!
policy-map QOS
class voice
priority 2000
class class-default
!
policy-map parentQOS
class class-default
shape average 5000000
service-policy QOS
For central router please let me what kind of connection is it? If it is it Hub and spoke kind of connection.
Then on central router we will have config in similar lines....
Step 1: Create class-maps
class-map match-all VIDEO_SIGNAL
match access-group 65
match access-group 165
class-map match-all VOICE_SIGNAL
match access-group 165
class-map match-all CITRIX
match protocol citrix
class-map match-all FTP
match protocol ftp
class-map match-all VIDEO
match access-group 65
class-map match-any VOICE
match dscp ef
class-map match-all location1
match access-group 161 (create access list to match destination subnets)
class-map match-all location2
match access-group 162 (create access list to match destination subnets)
class-map match-all location3
match access-group 163 (create access list to match destination subnets)
class-map match-all location4
match access-group 164 (create access list to match destination subnets)
class-map
==================================
Step 2: Create your child policy maps.
policy-map LLQ2
class VOICE_SIGNAL
bandwidth percent 2
class VIDEO_SIGNAL
bandwidth percent 3
class VOICE
priority percent 10
class VIDEO
bandwidth percent 20
police 9000000 conform-action transmit exceed-action drop
class CITRIX
bandwidth percent 25
class FTP
police 5000000 conform-action transmit exceed-action drop
==================================
Step 3: Create the parent policy map.
policy-map Parent1
class location1
shape average 3000000 (exmaple shaping value)
service-policy LLQ2
class location2
shape average 1500000 (exmaple shaping value)
service-policy LLQ2
class location3
shape average 1000000 (exmaple shaping value)
service-policy LLQ2
class location4
shape average 750000 (exmaple shaping value)
service-policy LLQ2
==================================
Step 4: Apply map class interface
ON WAN interface
service-policy output Parent1
If I have not answered you question., please provide me the config for 891 and 2911 and the problem. I will be able to help better.
01-05-2012 06:27 AM
It's our understanding that it's best to perform QoS marking as close to the source as possible, so that's what we're doing:
We mark appropriate DSCP values on our datacenter aggregation switches.
However, our Nexus 7K's & Cat 6K's don't seem to support classification of Citrix ICA Traffic by ICA Tag Number using NBAR.
We're running 5.1(3) on the 7K's, and 12.2(33)SXI5 on the 6K's.
Could you please tell us what versions of NX-OS & IOS for Catalyst support Citrix ICA Tag Number through NBAR?
IOS 12.4(24)T5 on an ISR 2800 supports it.
=======
N7K(config)# class-map ICA
N7K(config-cmap-qos)# match protocol ?
arp IP ARP
bridging Bridging
cdp Cisco Discovery Protocol
clns ISO CLNS
clns_es ISO CLNS End System
clns_is ISO CLNS Intermediate System
dhcp Dynamic Host Configuration
isis Intermediate System Intermediate System Protocol
ldp Label Distribution Protocol
netbios NetBIOS
N7K(config-cmap-qos)#
===============
C6K(config)#class-map ICA
C6K(config-cmap)#match protocol citrix ?
app Match Application Name String
C6K(config-cmap)#
===============
ISR(config)#class-map ICA
ISR(config-cmap)#match protocol citrix ?
app Published App in Server Browser Mode
ica-tag Citrix ICA tag 0-high 1-medium 2-low 3-background
ISR(config-cmap)#
01-06-2012 02:07 PM
Hi
Unfortunatley There is no NBAR support on n7k for now and foreseeable future, The 5.2 feature set is documented in the release notes.
N7K(config-cmap-qos)# match protocol ?
arp IP ARP
bridging Bridging
cdp Cisco Discovery Protocol
clns ISO CLNS
clns_es ISO CLNS End System
clns_is ISO CLNS Intermediate System
dhcp Dynamic Host Configuration
isis Intermediate System Intermediate System Protocol
ldp Label Distribution Protocol
netbios NetBIOS
All the above option
if nbar is supported, I would expect to see things like "telnet" "http" "ftp" and the like....something that would require looking at L4 info or using signatures .
The work around would be to configure QoS using MQC with ACL etc.
Here is the document to configure QoS on Nexus
http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/configuration/guide/cli/QoS.html
But please note that this is OUt of scope for the disscussion as it falls in datacenter swicthing domain. Our topic of disscussion is QoS on WAN.
I would request you to open a TAC case for more questions on nexus.
01-10-2012 05:29 AM
Hi
I already submitted this question on the LAN switching board before I noticed your thread. I have re-submitted it here as I may hit the QOS "jackpot" for knowledge
I have a query about the following command
srr-queue bandwidth shape 30 0 0 0
I understand what the command does and how the weights work.
With the example above weight1 is 1/30 of the interface bandwidth but.....
is this 1/30 of the physical port (say 1Gb) or can it be 1/30 of the configured bandwidth command?
The reason behind this is I have a throttled link from my ISP.
I connect to the CPE locally via a 1Gb sfp (fibre), the link between my two sites in question is also a 1Gb link but for now has been software throttled by the ISP to 100Mb (trying to get this changed as the link is on its way to some congestion in the near future).
Can i simply put the bandwidth statement bandwidth 100000 on my interface and the original command above use this in its calculation?
I dont want to use the speed command on my interface.
any help is alway appreciated
regards
Stuart
01-11-2012 07:10 PM
Here is the response to the question:
srr-queue bandwidth shape
The above command will calculate percentage of bandwidth for queue 1 based on the link speed (But not the interface bandwidth you configure).
Eg:
config t
int gi1/0/1
srr-queue bandwidth shape 10 0 0 0
The bandwidth weight for queue 1 in here is 1/10 == 10% of link speed (i.e. 10% of 1gig = 100Mb)
Here is the link which gives more information:
As I understand correctly, You want to rate limit traffic to 100 Mb on gig interface right?
Then you can either change the link speed (which you dont want to do) or shape the traffic to 10% of gig interface i.e. "srr-queue bandwidth shape 10 0 0 0" under the interface.
Configuring bandwidth 10000 will not help here, interface bandwidth configuration is used for metric calculations in routing protocols.
01-12-2012 01:06 AM
Sarala
thankyou for the confirmation about the bandwidth statement being ignored as I couldn't really test this out on the live link.
01-11-2012 06:19 AM
Hello Sarala,
My question is in regards to QoS on an MFR interface/subinterfaces. We have a remote site with two bundled T1's terminating on a 2951 router for a total bandwidth of 3072. The circuit is provided by Paetec and the subinterfaces are designated for internet and MPLS traffic respectively. The issue we are facing is with outbound voice quality. It seems that no matter how we apply QoS, either to the main MFR interface or the MFR subinterfaces, voice packets do not seem to be prioritized. We tried FRTS, which slowed the entire link down to a crawl, we tried applying a class map to the main interface as well as a service policy, none of which seemed to affect anything. Please see below for current partial configuration. Any input will be greatly appreciated.
class-map match-all VOICE
match ip dscp ef
class-map match-any SIGNALING
match ip dscp af31
match ip dscp cs3
class-map match-all AZ-SERVERS
match access-group 10
!
!
policy-map VOICE-POLICY
class VOICE
priority 640
class SIGNALING
bandwidth 64
set dscp af41
class AZ-SERVERS
police 1000000
class class-default
fair-queue
!
!
!
!
!
interface Loopback1
no ip address
!
interface Tunnel1
no ip address
!
interface MFR1
no ip address
ip flow ingress
ip flow egress
load-interval 30
frame-relay lmi-type ansi
service-policy output VOICE-POLICY
!
interface MFR1.501 point-to-point
description => Internet via PAETEC
ip vrf forwarding internet
ip address 63.255.X.X 255.255.255.252
ip flow ingress
ip flow egress
no cdp enable
frame-relay interface-dlci 501 IETF
!
interface MFR1.502 point-to-point
description => MPLS VPN via PAETEC
ip address 63.253.X.X 255.255.255.252
ip flow ingress
ip flow egress
no cdp enable
frame-relay interface-dlci 502 IETF
01-11-2012 06:20 AM
Hello Sarala,
My question is in regards to QoS on an MFR interface/subinterfaces. We have a remote site with two bundled T1's terminating on a 2951 router for a total bandwidth of 3072. The circuit is provided by Paetec and the subinterfaces are designated for internet and MPLS traffic respectively. The issue we are facing is with outbound voice quality. It seems that no matter how we apply QoS, either to the main MFR interface or the MFR subinterfaces, voice packets do not seem to be prioritized. We tried FRTS, which slowed the entire link down to a crawl, we tried applying a class map to the main interface as well as a service policy, none of which seemed to affect anything. Please see below for current partial configuration. Any input will be greatly appreciated.
class-map match-all VOICE
match ip dscp ef
class-map match-any SIGNALING
match ip dscp af31
match ip dscp cs3
class-map match-all AZ-SERVERS
match access-group 10
!
!
policy-map VOICE-POLICY
class VOICE
priority 640
class SIGNALING
bandwidth 64
set dscp af41
class AZ-SERVERS
police 1000000
class class-default
fair-queue
!
!
!
!
!
interface Loopback1
no ip address
!
interface Tunnel1
no ip address
!
interface MFR1
no ip address
ip flow ingress
ip flow egress
load-interval 30
frame-relay lmi-type ansi
service-policy output VOICE-POLICY
!
interface MFR1.501 point-to-point
description => Internet via PAETEC
ip vrf forwarding internet
ip address 63.255.X.X 255.255.255.252
ip flow ingress
ip flow egress
no cdp enable
frame-relay interface-dlci 501 IETF
!
interface MFR1.502 point-to-point
description => MPLS VPN via PAETEC
ip address 63.253.X.X 255.255.255.252
ip flow ingress
ip flow egress
no cdp enable
frame-relay interface-dlci 502 IETF
01-11-2012 07:13 PM
You have mentioned that you tried FRTS. Can you please confirm what configuration you had. It should have worked with FRTS. if you are not seeing any match we may need to troubleshoot if the packets coming into the router are marked correctly.
Here is the sample config for FRTS
!
class-map match-all MEDIA
match dscp ef
class-map match-all SIGNALING
match dscp af41
!
!
policy-map VOIP
class MEDIA
priority percent 50
class SIGNALING
priority percent 20
!
!
!
!
!
interface MFR1
no ip address
ip route-cache flow
load-interval 30
no arp frame-relay
frame-relay traffic-shaping <<<< do this command last
!
interface MFR1.501 point-to-point
description CONNECTED TO INTERNET
ip address 74.10.15.246 255.255.255.252
no arp frame-relay
frame-relay interface-dlci 501 IETF
class VOIPovFR <<<<<<<<<<<<<<
!
interface MFR1.502 point-to-point
description CONNECTED TO MPLS
ip address 74.10.22.26 255.255.255.252
no arp frame-relay
frame-relay interface-dlci 502 IETF
class VOIPovFR <<<<<<<<<<<<<<
!
!
map-class frame-relay VOIPovFR
no frame-relay adaptive-shaping
frame-relay cir 64000 <<<< CIR = MINCIR
frame-relay bc 640 <<< bc = CIR / 100
frame-relay be 0
frame-relay mincir 64000 <<<< Change as needed
service-policy output VOIP
!
please provide show policy-map output. We may have to open TAC case as would be needing interactive/troubleshootingsession.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide