Re: Trust state in Nexus 7000

e-chuah · ‎03-03-2011

Hi According to the doc, by default, N7K trust dscp and CoS and preserve the value at ingress. A packet traverse N5K which is trunked to N7K. The packet contains both DSCP and COS value. In such scenario, (1) Will the N7K just use the COS value in the packet to perform egress queueing? Or it will use the DSCP value, then check against the default dscp-cos mapping table, and use that COS value to perform egress queueing? The reason i asked is because if i have a scenario the packet consists of COS=4, DSCP=24, egress queuing (based on CoS) will be different. If N7K simply uses COS=4, then it will be mapped to Queue4 for example. If N7K uses DSCP=24, dscp-cos mapping maps to COS=3, then egress queuing will be maped to Queue3. Any idea? Eng Wee

Andrew Gossett · ‎04-26-2011

Packets are queued on egress based on CoS. The CoS value is derived based on the type of traffic (bridged or routed). For bridged traffic, the CoS value is copied from the received CoS value. For routed traffic, the CoS value is derived from the 3 most-significant bits of the DSCP value. If you do a "show table-map" you will see:

Table-map dscp-cos-map

default copy

The mapping values will be as follows

CoS

DSCP 0 - 7 = 000xxx = 0

DSCP 8 - 15 = 001xxx = 1

DSCP 16 - 23 = 010xxx = 2

DSCP 24 - 31 = 011xxx = 3

DSCP 32 - 39 = 100xxx = 4

DSCP 40 - 47 = 101xxx = 5

DSCP 48 - 55 = 110xxx = 6

DSCP 56 - 63 = 111xxx = 7

You cannot currently change the dscp-cos mapping. On L2 links, the CoS value is derived from the CoS value received in the 802.1q header. Note that on L2 access links there is no trunk header so, by default, if traffic is received on an access port and bridged then it will egress the switch with CoS 0. The DSCP value will not be changed. You can manually set the CoS value in a policy-map via three different methods:

1) Table map on input policy (to derive CoS from DSCP value in the same manner as routed traffic):

!

table-map copy_dscp

default copy

!

policy-map type qos copy_dscp

class class-default

set dscp dscp table copy_dscp

!

interface

service-policy type qos input copy_dscp

2) Egress QoS policy with "set cos"

!

policy-map type qos set_cos

class custom_class

set cos x

3) Ingress or Egress QoS policy with "set dscp" command. The CoS value will, again, be derived from 3 MSB of the dscp value.

!

policy-map type qos set_dsxp

class custom_class

set dscp x

-Andy

finn.poulsen · ‎05-10-2012

Hi Andy,

I'm struggeling with QoS on a N7K (mixed F1 and M1 cards) and stumbled across your reply.

Assume that you've meant COS 0,1,2,3,4,5,6,7 in the rigth coloumn.

I need to classify, mark and queue IP-storage traffic (NFS and iSCSI) from a NetApp FAS, which cannot mark trafic (I've been told be NetApp).

Therefore I need to mark ("remark") the traffic to AF21/COS2 on the 802.1Q trunk (actually running as a VPC), from/towards the NetAPP, which is connected to an F132XP-15 port.

According to the manual I cannot do COS marking in ingress, but only DSCP.

My config :

ip access-list NetApp-storage

10 permit ip 10.xxx.yyy.0/24 any <-- used for NFS

20 permit ip 10.xxx.zzz.0/24 any <-- used for iSCSI

class-map type qos match-all IPstorage-IN

match access-group name NetApp-storage

policy-map type qos NetApp-IN

class IPstorage-IN

set dscp 18

class class-default

set dscp 0

int po xxx <--- VPC towards NetApp FAS

service-policy type qos input NetApp-IN

So based on your comments the "storage" traffic will exit the switch with COS=0 - correct ?

In order to set the COS properly upon egress, I would need to additional configure :

policy-map type qos NetApp-OUT

class IPstorage-IN

set cos 2

class class-default

set Cos 0

int eth xx/yy or port-channel zzz <--- ports towards "storage users"

service-policy type qos output NetApp-OUT

on all ports ???

So based on your comments the "storage" traffic will exit the switch with COS=0 - correct ?

In order to set the COS properly upon egress, I would need to additional configure :

policy-map type qos NetApp-OUT

class IPstorage-IN

set cos 2

class class-default

set Cos 0

int eth xx/yy or port-channel zzz <--- ports towards "storage users"

service-policy type qos output NetApp-OUT

on all ports ???

Additional queueing questions :

Will the egress queing be done correctly, by only setting the DSCP upon ingress ?

My config :

qos copy policy type queuing default-4q-8e-out-policy prefix QQ_

policy-map type queuing QQ_4q-8e-out

class type queuing 1p3q1t-8e-out-pq1

priority level 1

class type queuing 1p3q1t-8e-out-q2

bandwidth remaining percent 1

class type queuing 1p3q1t-8e-out-q3 <--- COS 2 should go here

bandwidth remaining percent 49

class type queuing 1p3q1t-8e-out-q-default

bandwidth remaining percent 50

int eth xx/yy or port-channel zzz <--- ports towards "storage users"

service-policy type queuing output QQ_4q-8e-out

service-policy type qos output NetApp-OUT

Is this correct ?

Best Regards

Finn Poulsen

Andrew Gossett · ‎05-21-2012

Hi Finn,

You are absolutely correct on the dscp-to-cos mapping. I have updated the table in the previous post.

In regards to your question, if you create an input policy that remarks DSCP value, then the CoS value will automatically be rewritten based on the 3 most significant bits of the dscp value. So the simplest solution would be to apply the input QoS policy that matches on the NFS/iSCSI traffic and use a 'set dscp af21'. This will mark the dscp value in the IP header to af21 and the packet will also egress with a CoS value of 2. There's no need for the extra egress qos policy to set cos:

policy-map type qos NetApp-IN

class IPstorage-IN

set dscp 18

class class-default

set dscp 0

int po xxx <--- VPC towards NetApp FAS

service-policy type qos input NetApp-IN

The above is all you need to ensure both DSCP and CoS are set correctly.

-Andy

finn.poulsen · ‎05-22-2012

Thanks a lot.

Maybee you should try to get your table incorporated into the CCO documentation, haven't been able to find anything describing this so precisely as you did ;-)

From the Queuing perspective is that the way to do it on a F1 linecard (there is no FCoE) traffic and no priority traffic as well ?

Best Regards

Finn

Andrew Gossett · ‎05-22-2012

Hi Finn,

Yes, assuming that you're using the default 4q-8e queuing type on the F1, then all the same concepts discussed above are still applicable.

-Andy

finn.poulsen · ‎05-29-2012

Hi Andy,

Have now tried implementing this - Wireshark reveals that :

1) Apparently the NetAPP sends traffic with DSCP=default and COS=4 (despite what they told us prevoiusly).

2) this traffic exits the Nexus 7000 with COS=4 and DSCP=AF21

So from above it seems that the ingress COS is being preserved across the switch.

This would also indicated the the "wrong" queing will be used.

Best regards

Finn