QoS Policy on physical interface vs Tunnel Interface

[Chad Westog]

Physical vs Virtual design issue that I'm looking for feedback on, I have a WAN architecture that links outlying user sites back to a core site via Ethernet Private line service.  Three sites connected via three "ethernet" links.  I then layer a VTI across each link to encrypt all the traffic.  I'm doing the standard nested output QoS policy on the tunnel like so:

service-policy wan-shaper

  class class-default

    shape average 200000000

       service-policy wan-child

service-policy wan-child

  class mission-crit1

    bandwidth percent 15

  class mission-crit2

    bandwidth percent 10

  class network-mgmt

    bandwidth percent 10

    random-detect dscp-based

  class enterprise-ad

    bandwidth percent 10

  class patching

    bandwidth percent 25

  class voice

    priority level 1 percent 2

  class voice-c2

    bandwidth percent 1

  class class-default

    random-detect dscp-based

    fair-queue

My question is do I also need a policy on the physical interface as well?  All traffic moves through the tunnel, and I see traffic hitting the appropriate classes, but I'm seeing a lot of output drops on the physical interface.

BTW:  WAN routers are all Cisco 4451's

Any feedback would be appreciated.

Thanks

[Joseph W. Doherty]

Insufficient information to say what you need, but in general, yes, you may need QoS on both tunnel interface and the physical interface.

For example, consider a hub with a 100 Mbps physical interface, but a contracted CIR of 50 Mbps, and 3 branches, also with 100 Mbps physical interfaces, but each with a contracted CIR of 20 Mbps.

On the hub, we want to shape each tunnel so to not over run what the branch can accepts, i.e. in this case, shape each tunnel to 20 Mbps.

However, if all 3 tunnels try to push 20 Mbps, they will overrun our hub's CIR of 50 Mbps, so we need to also shape it for that 50 Mbps.

Conversely, though, the 3 branch tunnels (at the branches), even if shaped at 20, could overrun the hub's 50, so you should shape them that their aggregate cannot exceed the 50 Mbps.

BTW, some providers will only police CIR as it enters their network.  I.e. possible, you hub could get, from the branches, all 60 Mbps.

[Chad Westog]

Ok...I'll check what each of the "ethernet" CIR's are set for...I thought I had matched my tunnel shaping to match the CIR

[Joseph W. Doherty]

I'm sorry, I might misunderstand you topology.

At the core, each branch link has a physical port just for each branch Ethernet private link?

If so, if there are tunnels whose aggregate shaped bandwidth cannot exceed the physical port bandwidth, one can wonder how the physical port might have drops.

Well, what might be happening, one need to remember, shapers still transmit at maximum possible speed.  For example a gig ingress shaped to 50 Mbps sent to a FE egress, packets can arrive at 10x the capacity of the egress.  What a shaper does, is limit overall transmission volume for some time period, not actual frame/packet transmission rate.  I.e. for the shaper's Tc, packets might be sent to the egress interface so fast, the egress queues are exceeded.  Normally, I wouldn't expect this to happen much, but with tunnels, unless you're also working to avoid fragmentation, you could be sending more packets then the source generated, and traditionally egress queues count packets.  Possible, a variation of a micro burst (and micro burst).

What you might try is increasing the egress interface queues (tx-ring and/or software queue).

[MHM Cisco World]

For QoS you need only under vti not WAN interface (which sure use as tunnel source)

Other important things 

Is do show ip interface tunnel

Ypu will see two important value 

Receive and send 

Usually it set to 8000 kbps this can also adjust.

I dont have more info about these two value and it relate to BW but sure it work like policer of traffic more than it the traffic will start drop

MHM

[Joseph W. Doherty]

---

@MHM Cisco World wrote:

Other important things 

Is do show ip interface tunnel

Ypu will see two important value 

Receive and send 

Usually it set to 8000 kbps this can also adjust.

I dont have more info about these two value and it relate to BW but sure it work like policer of traffic more than it the traffic will start drop

---

Hmm, don't recall those settings acting as VTI policers.  You say you're sure they are.  You've seen this policing actually happen based on those settings?

---

@MHM Cisco World wrote:

For QoS you need only under vti not WAN interface (which sure use as tunnel source)

Since there's much information not provided in the OP, you know this is the fact how, exactly?

[Chad Westog]

To all,

So I have three physical 1GB interfaces connected to cmcl provider equipment.  Each is a separate L2 MPLS service provided to me...across the MPLS I have a CIR of 200MB for each "ethernet line"

I did check the tunnels and all are set with a "Tunnel transmit bandwidth 8000 (kbps)" and "Tunnel  receive bandwidth 8000 (kbps)

Never really tracked what those lines actually did as I would set a bandwidth 2000000 statement

[Chad Westog]

Additionally, each ethernet is a private 172.16.1.0/24, 172.16.2.0/24, 172.16.3.0/24 subnet and the VTI's are tun source/destination to the addresses on each side (.1 and .2) of each subnet.  Basically force all traffic through an encrypted tunnel.

[Joseph W. Doherty]

Now knowing that, unless somehow your shaper is allowing a multi gig micro burst within a Tc, cannot see how physical interface has egress drops.

Are these egress ports built-in or modular?  What other physical ports are active, like LAN port?

Also what throughput license is being used?

[Chad Westog]

Two are built on the chassis ports and one is on an add-on L3 port card.  Going to go back and re-validate all my shaping numbers and look at any fragmentation

[Joseph W. Doherty]

Just those 3 ports?  Just WAN ports?  No other active ports, for example, for LAN?  Output drops, about the same loss percentage(which is?) on all 3 WAN ports?

[Chad Westog]

I think I need to do a better job classifying traffic.  Most of my drops are in the default class, followed by my patching/repo class.  I have a lot of traffic thats hitting the default dscp plus CS1 and CS4 that I need to run down what it is.  

My LAN port only has a policing policy that limits traffic when they are pulling updates from the web...though I'm looking to drop that as we now have a dedicated Internet router with it own port/path for pulling updates.  So no real qos policy on the LAN port...all of my marking happens in the LAN 

[Chad Westog]

I'm thinking about enabling NBAR on the LAN port to get a better handle on the unclassified traffic