Re: QoS on Nexus 7000...

jpazahanick · ‎05-21-2010

Does anyone have a tried and true QoS configuration for a Nexus 7k?

I'm trying to replicate the standard voice/priority, signalling/bandwidth, default WRED type policy.

Here is what I hacked together...

class-map type qos match-any 1p7q4t-out-q2
match dscp 24,26
class-map type qos match-all 1p7q4t-out-pq1
match dscp 46
policy-map type queuing 10G_Metro_OUT
class type queuing 1p7q4t-out-pq1
    priority level 1
class type queuing 1p7q4t-out-q2
    bandwidth remaining percent 3
class type queuing 1p7q4t-out-q-default
    random-detect cos-based

Am I missing anything? Is there a way to police the priority queue?

Ganesh Hariharan · ‎05-21-2010

Does anyone have a tried and true QoS configuration for a Nexus 7k?

I'm trying to replicate the standard voice/priority, signalling/bandwidth, default WRED type policy.

Here is what I hacked together...

class-map type qos match-any 1p7q4t-out-q2
match dscp 24,26
class-map type qos match-all 1p7q4t-out-pq1
match dscp 46
policy-map type queuing 10G_Metro_OUT
class type queuing 1p7q4t-out-pq1
    priority level 1
class type queuing 1p7q4t-out-q2
    bandwidth remaining percent 3
class type queuing 1p7q4t-out-q-default
    random-detect cos-based

Am I missing anything? Is there a way to police the priority queue?

Hi,

Check out the below link for Modular QoS CLI (MQC) objects that can be used for configuring QoS features using the Cisco Nexus 7000 Series NX-OS software:

http://www.cisco.com/en/US/docs/switches/datacenter/sw/4_1/nx-os/qos/configuration/guide/statistics.html

Hope to Help !!

Ganesh.H

gotney · ‎07-14-2010

I needed to implement a QoS architect to support a Call Manager/Cisco IPT environment. We worked with a Cisco SE and tried to use the 6500 Voice Deployment Guide white paper as a base, (http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/prod_white_paper0900aecd804dc7ae.pdf) and converted it into Nexus 7K configs.

I can’t promise it’s correct, and would like some feedback from others. The frustration for us field engineers is that there is plenty of reference material available for QoS on a 6500, 3750, or other switch, but not much exists yet for the Nexus line. There is some decent theory stuff, but nothing that provides actual config script like you see in the white paper mentioned above.

Again, please review and post back with any changes/suggestions.

One gig card:

class-map type queuing match-any 1p3q4t-out-pq1
match cos 5
class-map type queuing match-any 1p3q4t-out-q2
match cos 2-4,6-7
class-map type queuing match-any 1p3q4t-out-q3
match cos 0
class-map type queuing match-any 1p3q4t-out-q-default
match cos 1
!
policy-map type queuing ONE_GIG-QUEUE-OUT
class type queuing 1p3q4t-out-pq1
    priority level 1
    queue-limit percent 30
class type queuing 1p3q4t-out-q2
    random-detect cos-based
    random-detect cos 7 minimum-threshold percent 90 maximum-threshold percent 100
    random-detect cos 6 minimum-threshold percent 80 maximum-threshold percent 90
    random-detect cos 3 minimum-threshold percent 70 maximum-threshold percent 80
    random-detect cos 2,4 minimum-threshold percent 60 maximum-threshold percent 70
    queue-limit percent 40
    bandwidth remaining percent 55
class type queuing 1p3q4t-out-q3
    random-detect cos-based
    random-detect cos 0 minimum-threshold percent 80 maximum-threshold percent 100
    queue-limit percent 25
    bandwidth remaining percent 40
class type queuing 1p3q4t-out-q-default
    random-detect cos-based
    random-detect cos 1 minimum-threshold percent 80 maximum-threshold percent 100
    queue-limit percent 5
    bandwidth remaining percent 5
!
interface eth 1/1
service-policy type queuing output ONE_GIG-QUEUE-OUT

Ten gig card:

class-map type queuing match-any 1p7q4t-out-pq1
match cos 5
class-map type queuing match-any 1p7q4t-out-q2
match cos 7
class-map type queuing match-any 1p7q4t-out-q3
match cos 6
class-map type queuing match-any 1p7q4t-out-q4
match cos 3
class-map type queuing match-any 1p7q4t-out-q5
match cos 2
class-map type queuing match-any 1p7q4t-out-q6
match cos 4
class-map type queuing match-any 1p7q4t-out-q7
match cos 0
class-map type queuing match-any 1p7q4t-out-q-default
match cos 1
!
policy-map type queuing TEN_GIG-QUEUE-OUT
class type queuing 1p7q4t-out-pq1
    priority level 1
    queue-limit percent 30
class type queuing 1p7q4t-out-q2
    random-detect cos-based
    random-detect cos 7 minimum-threshold percent 80 maximum-threshold percent 100
    bandwidth remaining percent 5
    queue-limit percent 5
class type queuing 1p7q4t-out-q3
    random-detect cos-based
    random-detect cos 6 minimum-threshold percent 80 maximum-threshold percent 100
    bandwidth remaining percent 5
    queue-limit percent 5
class type queuing 1p7q4t-out-q4
    random-detect cos-based
    random-detect cos 3 minimum-threshold percent 80 maximum-threshold percent 100
    bandwidth remaining percent 20
    queue-limit percent 10
class type queuing 1p7q4t-out-q5
    random-detect cos-based
    random-detect cos 2 minimum-threshold percent 80 maximum-threshold percent 100
    bandwidth remaining percent 20
    queue-limit percent 10
class type queuing 1p7q4t-out-q6
    random-detect cos-based
    random-detect cos 4 minimum-threshold percent 80 maximum-threshold percent 100
    bandwidth remaining percent 20
    queue-limit percent 10
class type queuing 1p7q4t-out-q7
    random-detect cos-based
    random-detect cos 0 minimum-threshold percent 80 maximum-threshold percent 100
    bandwidth remaining percent 25
    queue-limit percent 25
class type queuing 1p7q4t-out-q-default
    random-detect cos-based
    random-detect cos 1 minimum-threshold percent 80 maximum-threshold percent 100
    bandwidth remaining percent 5
    queue-limit percent 5
!
interface eth 10/1
service-policy type queuing output TEN_GIG-QUEUE-OUT

bradburyh · ‎07-21-2010

Did you ever figure out whether this solution worked? I am assuming you put this queueing policy on all trunks to other switches?

I am looking at a one way audio issue ever since my customer moved their gateway from their old CATOS 6500 to the Nexus switch (I

think they also upgraded the gateway at the same time). And I am wondering if there is a possibility that QOS issues are occuring.

Thanks,

gotney · ‎07-21-2010

Unfortunately, the "voice" portion of the project was delayed for budgeting reasons, so the specific QoS settings below were never tested on the production network. The voice portion was just re-approved and we should pick back up on it within a month or two, however, we'll be adding a couple TelePresense CTS-1000 and CTS-500 systems also which will mean updating the configs to allow for larger Priority Queues. I'll post the resulting configs once we get things going.

One way audio? I always start that troubleshooting by taking some sniffer traces at different points in the network to see if one voice stream is dropped or if the DSCP is stripped. The Nexus trusts DSCP by default, so it *shouldn't* be stripping those QoS values there. In my experience, as a "network guy” asymmetric routes through a WAN or Firewall issues (or both!) are good candidates to cause one way audio..

bradburyh · ‎07-21-2010

I would normally have all the same ideas you suggested, However the gateway sits in the same vlan as the phones and as the CallManager. There is no firewall, etc in between. And it is not all calls, so trying to think out of the box. I am trying to get sniffer traces from both the port connected to the gateway and the phone. The hardest part is that I am not local to the site, so getting all the neccesary information has been difficult and replicating the issue has also been difficult. I am also looking into DSP or other issues on the gateway.

aamercado · ‎10-13-2010

10 Gig QoS seems to work fine...still testing:

1 Gig QoS has bug which I seem to have hit but is unreproducible in Cisco labs testing:

CSCtf76095 N7k QoS: ERRMSG: Total bandwidth remaining is greater than 100%

The below changes worked for me. My topology was a VPC to a 65XX and I got an error when I applied it to my Port-Channel. It didn't populate to my interface so I had to manually apply it to the each member of the port-channel.

interface port-channel27
description ****S-EDGE-7-IDF-WEST-C6509****
switchport
switchport mode trunk
vpc 27
switchport trunk allowed vlan 71-72,171-172
spanning-tree port type normal
logging event port link-status
logging event port trunk-status
service-policy type queuing output ONE_GIG-QUEUE-OUT

class-map type queuing match-any 1p3q4t-out-pq1

match cos 5

class-map type queuing match-any 1p3q4t-out-q2

match cos 2-4,6-7

class-map type queuing match-any 1p3q4t-out-q3

match cos 0

class-map type queuing match-any 1p3q4t-out-q-default

match cos 1

!

policy-map type queuing ONE_GIG-QUEUE-OUT

class type queuing 1p3q4t-out-pq1

priority level 1

queue-limit percent 30

class type queuing 1p3q4t-out-q2

random-detect cos-based

random-detect cos 7 minimum-threshold percent 90 maximum-threshold percent 100

random-detect cos 6 minimum-threshold percent 80 maximum-threshold percent 90

random-detect cos 3 minimum-threshold percent 70 maximum-threshold percent 80

random-detect cos 2,4 minimum-threshold percent 60 maximum-threshold percent 70

queue-limit percent 40

bandwidth remaining percent 50 (=½)

class type queuing 1p3q4t-out-q3

random-detect cos-based

random-detect cos 0 minimum-threshold percent 80 maximum-threshold percent 100

queue-limit percent 25

bandwidth remaining percent 25 (=¼)

class type queuing 1p3q4t-out-q-default

random-detect cos-based

random-detect cos 1 minimum-threshold percent 80 maximum-threshold percent 100

queue-limit percent 5

bandwidth remaining percent 25 (=¼)

Bandwidth Ratio Values:

1/2 = 50

1/3 = 33

1/4 = 25

1/5 = 20

aamercado · ‎10-26-2010

10-Gig Qos doesn't work due to:

http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/qos/configuration/guide/cpl.html#wp1089345

moving cos0 and 1 on 1p7q4t-out-q-default which by default had 82% to another queue which defaults to 1% messed up my traffic...so now re-evaluating the Qos

Krzysztof Zaleski · ‎08-26-2011

Hello,

Has anyone of you managed to configure QoS on N7K already? It's been a while since you posted this message, and I can see that there is still not many good documents/recommendations for QoS on N7K. My concern is how to properly use "queue-limit", "random-detect", and "bandwidth" commands in certain queues and scenarios. I have seen many examples where all commands are used, but why? From what I understand, "bandwidth" guarantees some minimum part of available bandwidtht during congestion, just like in plain CBWFQ. We can add random-detect for different COS values to proactively manage congestion. But, what is the real need to limit the queue with "queue-limit", except priority class?

In the above example there is a "queue-limit percent 40" which means packets are tail-dropped at that level of total queue depth (6,2MB per port for 1G port), but there is still "random-detect remaining percent 50", which means a) 50 percent of total available buffer size (excluding priority class, as it is remaining percent, but still it's more than queue-limit!!!), or b) 50 percent of that "queue-limit percent 40"?

Cheers,

Krzysztof

Chris Ingram · ‎03-15-2012

I have this basic QoS policy I'm using for all of our 7k switches (2 at each data center that are basically connected by a UVN ring).

It has served the purpose until lately we need to classify some datadomain traffic and utilize another queue.

policy-map type queuing BANDWIDTH

class type queuing 1p3q4t-out-pq1

priority level 1

queue-limit percent 30

class type queuing 1p3q4t-out-q2

class type queuing 1p3q4t-out-q3

class type queuing 1p3q4t-out-q-default

random-detect cos-based

random-detect cos 0-1 minimum-threshold percent 60 maximum-threshold percent 70

random-detect cos 2 minimum-threshold percent 80 maximum-threshold percent 90

random-detect cos 3-4 minimum-threshold percent 90 maximum-threshold percent 100

bandwidth remaining percent 70

VoIP goes to the priority queue (1p3q4t-out-pq1).

Everything else goes to the default queue (1p3q4t-out-q-default) and is subject to random dropping by WRED.

Now we need to move our CoS 2 traffic to queue 2 and I have only one problem. I cannot edit exisiting class-maps of type queuing and I cannot create new ones (only type qos, or network-qos).

Here's what the class-map confiuration looks like:

    class-map type queuing match-any 1p3q4t-out-pq1
      Description: Classifier for egress priority queue of type 1p3q4t
      match cos 5-7

class-map type queuing match-any 1p3q4t-out-q2
Description: Classifier for egress queue 2 of type 1p3q4t

class-map type queuing match-any 1p3q4t-out-q3
Description: Classifier for egress queue 3 of type 1p3q4t

    class-map type queuing match-any 1p3q4t-out-q-default
      Description: Classifier for egress default queue of type 1p3q4t
      match cos 0-4

I just need to figure out how to match CoS 2 under class-map type queuing match-any 1p3q4t-out-q2.

Does anyone know how?

Thanks,

Chris

Chris Ingram · ‎03-15-2012

Now I have just re-written this policy to reflect my intentions of matching CoS 2 under the 2nd queue.

policy-map type queuing BANDWIDTH

class type queuing 1p3q4t-out-pq1

priority level 1

queue-limit percent 30

class type queuing 1p3q4t-out-q2

bandwidth remaining percent 10

random-detect cos-based

random-detect cos 2 minimum-threshold percent 90 maximum-threshold percent 100

class type queuing 1p3q4t-out-q3

class type queuing 1p3q4t-out-q-default

bandwidth remaining percent 60

random-detect cos-based

random-detect cos 0-1 minimum-threshold percent 60 maximum-threshold percent 70

random-detect cos 3-4 minimum-threshold percent 90 maximum-threshold percent 100

ametcalf · ‎06-21-2012

Chris

I guess you have answered your own questions by now, but I am not sure that issuing

policy-map type queuing BANDWIDTH

class type queuing 1p3q4t-out-q2

random-detect cos-based

random-detect cos 2 minimum-threshold percent 90 maximum-threshold percent 100

commands when no traffic of that cos value is mapped to the queue will help.

I assume you also from the default/admin VDC configured this

class-map type queuing match-any 1p3q4t-out-q2

match cos 2

or did it actually work ?

For me I find a bit of a pain/restrcitive because this is then applied globally. The documentation suggests that you can copy the defaults and play with them but so far I have not managed, but I have not dedicated that muich time to it (just one night when I had to get it in and working very quickly)