Best Practices Metro Ethernet QoS?

[pnicolette]

Our sites connect via routers into a single Metro Ethernet (TLS) VLAN. The main site with most servers and the Internet link uses a 100Mb TLS circuit, same for another big site; the rest are 10Mb. Majority of traffic is outbound from main site to others. There's no VoIP, limited streaming video, but occasional congestion when a large file transfer hogs a 10Mb pipe with default FIFO queue.

Where to start for QoS?

Can the main site router enforce separate 10Mb output limits for each destination, even though they share one interface?

Or should I apply input service policies to the TLS interfaces of the other routers?

Or a better way....?

Thanks much for any ideas!

Paul

[lamav]

Paul:

I had a similar situation a while back.

Can you post a diagram?

Can you tell us more about the traffic flows?

You may have no VoIP, but what other delay-sensitive applications do you have?

I implemented a nice little QoS scheme because we were having a problem between our DR site -- which housed our servers -- and the main site whenever a data backup would be running.

[pnicolette]

Hi lamav,

Diagram (not Visio):

__________________________

|.......|.......|.......|B|.......|.......|A|.....|

every site is in same VLAN, sites A & B have bigger pipes, and A originates most traffic.

Is there a config to let the outbound interface at site A do a distinct CBWFQueue for each small-pipe destination, each w/its own 10Mb ceiling?

I'm less worried about the specific apps than the queueing strategy.

[lamav]

Paul:

I'm not sure what that drawing is, but it looks like something from a cave in Mesopotamia. :-)

But all kidding aside, given what you have told us, I imagine that this is a multipoint-to-multipoint (any-to-any) topology, in which all sites are in the same TELCO VLAN.

My client was running that same set up and TELCO was basically performing some dot1q tunneling.

The problem we had was that the database replication and storage/backup jobs were killing the connection between the DR site and the main site. So, what I did was classify all the mission critical traffic: VoIP, server heartbeat, and a few delay-intolerant applications, and then configured QoS to prioritize the traffic accordingly.

For example, at one site we had the following at one site:

Class-map voip

match ip dscp ef

class-map data

match access-group 100

!

policy-map TLS

class voip

set ip dscp ef

priority percent 20

class data

bandwidth percent 30

set ip dscp af31

class class-default

fair-queue

!

interface g0/0

service-policy output TLS

!

!

access-list 100 permit ip 172.16.0.0 0.0.0.255 any

..and at site 2:

class-map hb

match access-group 100

class-map data

match access-group 101

!

policy-map TLS

class hb

set ip dscp af41

class data

set ip dscp af31

!

interface g0/1

!

priority-queue out

!

service-policy input TLS

!

Access-list 100 permit tcp any any eq 580

!

Access-list 100 permit udp any any eq 580

!

Access-list 101 permit ip 172.16.0.0 0.0.1.255 any

!

[end]

So, at site 1, I prioritized the VoIP traffic by placing it in an expedited queue (ef) and alloted a certain amount of bandwidth, too. I also took the general LAN traffic data and classified it as af31 and assigned it 30 percent bandwidth.

At site 2, I identified the mission critical server heartbeat traffic and assigned it an af of 41 and the rest of the general LAN traffic was assigned as af31.

So, the methodology I used was to identify the mission critical traffic, mark and queue it accordingly, and the rest of the general user traffic would fall into a lower priority queue.

I don't know if I have helped you, but this was my experience with a TLS topology and QoS. I am not a QoS guru, but there are others on here who are, so you can stay tuned for them.

HTH

If so, kindly rate this post.

Thanks

Victor

[pnicolette]

it looks like something from a cave in Mesopotamia. :-)

It should. We've got 3.5G here now!

Victor, thanks for taking the time to share your config & experience. I'm starting to learn enough QoS so it looks fine to me - though I try to use named access lists to remember what I did (gets hazy after a few millennia).

Paul

[Joseph W. Doherty]

Two concerns:

Are we missing shaping?

Don't know what platform you're going to attempt this on. L3 switches may not accept a CBWFQ output policy.

[pnicolette]

mea culpa...we missed shaping entirely!

[Joseph W. Doherty]

Assuming you have a hub and spoke, if not physically, at least for almost all traffic flows, the QoS model you might want to start with is a 10 Mbps shaper using FQ, one dedicated for each remote site on the main site, outbound.

Whether the prior can be easily accomplished is dependent on the hardware/software platform's features and perhaps how many remotes you need to support.

[pnicolette]

Joseph,

Exactly! So... 3745, 12.4T, how to configure multiple dedicated shapers?

I can ACL by destination, and mark by traffic type, but am losing my few remaining hairs trying to shape the marked traffic to 10Mb per destination.

Possible?

[Joseph W. Doherty · Accepted solution]

NB: syntax likely off a bit!

ip access-list extended site1

permit ip any x.x.x.x x.x.x.x

ip access-list extended siteN

permit ip any x.x.x.x x.x.x.x

class-map site1

match access-list site1

class-map siteN

match access-list siteN

policy map yourname

class site1

shape 10000000

(might need a bandwidth statement) bandwidth percent 1

class siteN

shape 10000000

(might need a bandwidth statement) bandwidth percent 1

interface fastethernet0

service-policy output yourname

Shapers appear to implement FQ within themselves, so you both keep from overrunning far side and keep one flow from grabbing all the shape bandwidth.

There is a limitation to the number of classes, hopefully you don't have that many sites.

[pnicolette]

NB = NP (No Problem. My IOS wants "match access-group name site1" - ugh!)

Just the FQ within shapers will be big improvement over existing. Thanks.

Now for sun AND stars. In one router, can I cascade classes or policy maps or otherwise somehow

1) mark traffic by type, AND

2) FQ it by destination?

Resulting in per-destination CBWFQ (w/WRED cherry on top).

Hmmmm....create a virtual interface that redirects (how?) to the physical interface and

do 1 on virtual and 2 on physical?

[pnicolette]

or simpler...mark on input (other interfaces) and shape on output? Too obvious?? :-)

[Joseph W. Doherty]

The answer is yes to both your questions, although you would restrict your inbound marking to just traffic types not traffic destinations.

The key to the outbound is usage of a heirarchical policy.

e.g. (NB: syntax likely incorrect)

class-map important

match protocol rtp audio

class map notimportant

match protocol ftp

class-map realtime

match dscp ef (might need to do this with an ACL)

class-map bulk

match dscp af11 (ditto)

policy-map classifymyway

class important

set dscp ef

class notimportant

set dscp af11

class class-default

set dscp be

policy-map queuemyway

class realtime

priority percent 30

class bulk

bandwidth percentage remaining 1

random-detect

class class-default

fair-queue

need stuff from prior post, amended:

policy map yourname

class site1

service-policy queuemyway

shape 10000000

(might need a bandwidth statement) bandwidth percent 1

class siteN

service-policy queuemyway

shape 10000000

(might need a bandwidth statement) bandwidth percent 1

(inbound interface)

interface fastethernet 0

service-policy input classifymyway

(outbound interface)

(as prior post)

[pnicolette]

Wonderful, thanks.

How do I tell if need a bandwidth statement in service-policy queuemyway? And it is percent 1 not percent remaining 1?

[Joseph W. Doherty]

You'll want a bandwidth statement in queuemyway so that you can set the ratios between traffic.

Percent # is fine too. (Oh, just looked it up, the variant is "remaining percent".)