QOS marking on trunk links

[Clem58]

Hello,

We have some issues with network congestion and VOIP QOS marking.

Here is our network topology :

We have cisco phones, and WAPs (for wireless phones) that are connected Access switches (C3650) ports level.

These Access switches are connected via trunk links on a Core (C3850) switch that is doing InterVLAN routing, and finally this Core switch has a default route to an ISP router for WAN access.

Currently the deskphones and Wireless phones are marking cos 5 on switches ports level, the ports are configured on trusted cos (autoqos trust cos).

The questions are :

• Do we also need to set QOS trust on trunk links, on source and destination ports ? And on cos or DSCP?
• Do we also need to set QOS trust on the access port connected to the ISP router, on cos or DSCP ?

Many thanks by advance for your help.

[balaji.bandi]

If you are looking end to end you need to mark to work as expected.

[Joseph W. Doherty]

"If you are looking end to end you need to mark to work as expected."

BTW, actually on many QoS capable switches and routers, they don't need CoS or ToS tags to support QoS; even end-to-end.

The real purpose of such tags is to reduce the processing load of every transit device to (deeply) examine a packet for classification purposes.  The "ideal" was the edge (ideally the sending host) would mark the frame/packet appropriately.  Unfortunately, because of the potential for abuse, network devices generally should be configured, at least at some "trust boundary", to verify a frame's/packet's marking, and remark if needed.

[balaji.bandi]

I may be misled people with that statement - what I meant was device end (connected) to mean Voice device.

I understand some of the edge to the internet we do not have control or some not support and not possible, we do need to consider reserve some bandwidth for better quality for voice.

[Joseph W. Doherty]

Unsure "we're on the same page", yet. 

An end device, generally regardless of whether its a VoIP device or not, doesn't need to tag frames/packets to support QoS treatment.  Again, CoS/ToS tags just make examination of a frame/packet, for QoS treatment, more efficient.  (As an extreme example, we might [if supported on the device] do a "deep" packet analysis, using NBAR, for classification purposes.  Rather than doing this on every transit device, it's much more efficient to do it once, at first network device, tag the packet to indicate how it should be treated, and then all subsequent transit devices only need to examine the tag.)

Regarding VoIP and Internet, it's often "easy" to "guarantee" bandwidth as we send the traffic to the Internet.  Within the Internet, itself, we have no control.  (Fortunately, Internet providers often try to insure their circuits rarely congest.)

The problem is with Internet is bandwidth management from the Internet to our site.  This too we have no control over except in cases where we don't use the Internet for general access.  If we only use the Internet for traffic between our sites, we can manage bandwidth from the Internet by managing the bandwidth as we send it into the Internet.  E.g. site A <> Internet <> Site B - again, not using either Site A or B for general Internet access, Site A can manage traffic to Site B and the converse, too.  (Where it gets "messy" is with multiple sites, conversing with each other, i.e. a multi-point topology.)

[balaji.bandi]

Sure Agreed - yes based on CoS /ToS action, But in some cases, if you have enough bandwidth you do not need clarification  - it's unnecessary overhead in my personal point of view - again this depends on requirements and uses case..  but good to have where link congestion takes place due to other overloaded links - this can be any link - not limited.

yes some of them beyond our control and we rely on what promised as delivered. (this may be internet or MPLS or VPLS so on) - as long as it meets business goals there is no issue - how internally service provider-managed is a different discussion. (since we do expect  most of the SP provided links to be optimal)

thank you for your bit more clarity.

[Joseph W. Doherty]

Earlier Cisco switches, with QoS disabled (the default), don't touch QoS tags, so no need to "trust".  Those same switches, when QoS enabled, by default, will reset the QoS tag unless you either "trust" or have a policy assigned for ingress.

Later Cisco switches, like your 3650 (?), implicitly "trust" ingress markings unless configured otherwise.  I don't know whether they have an active default egress QoS policy.

Just "trusting" your QoS markings, alone, does not guarantee the egress treatment your VoIP traffic may require.  On older switches you'll need to activate QoS to enable egress QoS treatment.  On any switch, with active QoS, you either need to confirm the default QoS policy meets your QoS needs or configure one that does.  (BTW, QoS features often vary much between different Cisco switch series.)

Also BTW, if you're sending your VoIP traffic across the Internet, the Internet doesn't honor or provide QoS treatment.  Depending on your Internet topology, however, you can sometimes still provide effective QoS treatment.

Lastly, when possible, avoid using L2 CoS, rather use L3 ToS.  (Why?  L2 CoS only lasts until the first L3 hop; it also requires a VLAN tagged frame.  Further, many Cisco L2 switches can use L3 ToS for QoS support.)

[Clem58]

Thanks for you answer, but for another site, in our Access switches WS-C2960X- model,  in configuration we have this statements :

• For ports connected to PC and Cisco Phones:

interface GigabitEthernet1/0/5
description Desktop with Phone Configuration
switchport access vlan 8
switchport mode access
switchport voice vlan 13
srr-queue bandwidth share 1 30 35 5
priority-queue out
mls qos trust device cisco-phone
mls qos trust cos
macro description cisco-phone
auto qos voip cisco-phone
spanning-tree portfast edge
spanning-tree bpduguard enable
service-policy input AUTOQOS-SRND4-CISCOPHONE-POLICY
end

• If I do a "show mls qos int" on this interface:

#show mls qos int Gi1/0/5
GigabitEthernet1/0/5
Attached policy-map for Ingress: AUTOQOS-SRND4-CISCOPHONE-POLICY
trust state: not trusted
trust mode: trust cos
trust enabled flag: dis
COS override: dis
default COS: 0
DSCP Mutation Map: Default DSCP Mutation Map
Trust device: cisco-phone
qos mode: port-based

• For trunk links to the Core switch :

#show run int Gi1/0/25
Building configuration...

Current configuration : 188 bytes
!
interface GigabitEthernet1/0/25
description +UPLINK : GI4/0/1 : TRUNK
switchport trunk native vlan 1000
switchport mode trunk
load-interval 30
channel-group 1 mode on
end

#show mls qos int Gi1/0/25
GigabitEthernet1/0/25
trust state: not trusted
trust mode: not trusted
trust enabled flag: ena
COS override: dis
default COS: 0
DSCP Mutation Map: Default DSCP Mutation Map
Trust device: none
qos mode: port-based

We can see the trunk links are in 'not trusted" state.

• For the Core switch that is a 3750:

• On a trunk link connected to Access switch:

interface GigabitEthernet3/0/3
description +UPLINK :  GI1/0/25 : TRUNK
switchport trunk encapsulation dot1q
switchport trunk native vlan 1000
switchport mode trunk
load-interval 30
channel-group 3 mode on
end

#show mls qos int Gi3/0/3
GigabitEthernet3/0/3
trust state: not trusted
trust mode: not trusted
trust enabled flag: ena
COS override: dis
default COS: 0
DSCP Mutation Map: Default DSCP Mutation Map
Trust device: none
qos mode: port-based

• On the access port connected to ISP router :

interface GigabitEthernet2/0/24
description ATT Router FE0/0
switchport access vlan 991
switchport mode access
load-interval 30
speed 100
duplex full
spanning-tree portfast edge
end

#show mls qos int Gi2/0/24
GigabitEthernet2/0/24
trust state: not trusted
trust mode: not trusted
trust enabled flag: ena
COS override: dis
default COS: 0
DSCP Mutation Map: Default DSCP Mutation Map
Trust device: none
qos mode: port-based

So I'm a bit confused, my understanding is we need to trust cos on switch ports L2 for end devices, but DSCP on trunk links + access port for ISP router.

For our configuration it seems this is not the case ?

[Joseph W. Doherty]

I recall (?) the 2960 and 3560/3750 switches, unless changed by a later IOS, all function as I described for "earlier switches".  I.e. you don't need a trust statement if you have an ingress policy attached to the interface.  (Why?  Because the policy determines what's trusted or not, and by default, trusts all.)

For those switch ports not using an ingress policy, then you'll need to use a "trust" statement (on access or trunk [or routed - if 3K switch] port) if QoS is enabled (which, again, for those families of switches, should be off, by default).

Also BTW, regarding the need to trust CoS for end devices, that's not needed unless they don't also support L3 ToS tagging (which almost all should).  If the end devices do not set L3 ToS, have the edge switch set the L3 ToS (which could be determined by examining just the L2 CoS, if desired).  (NB: again if end devices don't use VLAN tagged frames, there will be no CoS.  However, edge switch, if "smart/enhanced", should be able to examine L3 attributes and set L3 ToS.)

[Clem58]

Thank you Joseph for this comprehensive answer.

On our 2960/3560/3750 switches, qos is enabled :

#show mls qos
QoS is enabled
QoS ip packet dscp rewrite is enabled

On our 3850 Core switches I don't really know how to see if QOS is enabled. We also have Core C4500 switches, and I don't know to verifiy if QOS is enabled or not.

Anyway according to what you explained, we should disable QOS on those switches in order to place every ports in trusted state, correct ?

But as QOS is enabled on our switches, we need to trust "manually" each ports end to end.

Anyway I admit QOS is not easy !

[Joseph W. Doherty]

On 2960/3560/3750 switches, if QoS is disabled, CoS/ToS is passed through untouched (egress is a single FIFO queue).  If QoS is enabled, you either need to explicitly trust or have an ingress policy on an interface.  (Policies implicitly trust, unless you configure them not to.)  On egress, CoS and ToS, by default, are split between four queues, although PQ isn't enabled by default.

On the 3650/3850 switches, I believe (?) they trust by default.  QoS might also be enabled by default.  I recall (?) they support 8 egress queues.

On the 4500, QoS features vary much by supervisor.  On the later sups, sup 6 (?) and later, I recall (?) their QoS is somewhat like a router's, i.e. they trust by default, but only use a single FIFO egress queue unless you configure an egress policy (limited compared to a Cisco router's).  (NB: one of the earlier 4500 sups, the 4 [?], had/has an "interesting" QoS.)

BTW, when you use Auto-QoS, it generally creates a new set of ingress and egress QoS treatment.  Auto-QoS settings tends to vary based on IOS "age" and also features provided by platform.  When Auto-QoS is enabled, it make lots of changes, which can be individually changed.  (Generally, to fully remove it, you need to "no" every command it inserted.)

[Clem58]

Thanks Joseph.

The problem we have, we have different switches models and with different software version.

I have verified another 3650,  WS-C3650-48PD software version: 16.6.6

On a PC+deskphone port we have this config :

description Office Desk Port
switchport access vlan 10
switchport mode access
switchport voice vlan 11
trust device cisco-phone
auto qos voip cisco-phone
spanning-tree portfast
service-policy input AutoQos-4.0-CiscoPhone-Input-Policy
service-policy output AutoQos-4.0-Output-Policy

On the trunk links, we don't have any policy applied :

TenGigabitEthernet1/1/3
description +UPLINK :  TE1/0/1 : TRUNK
switchport trunk native vlan 1000
switchport mode trunk
load-interval 30
channel-group 1 mode active

So we don't have any policy applied on the trunk links, as this switch is "MQC QOS", does that mean the ports are implicitely trusted even if there is not policy assigned on the port ?

[Joseph W. Doherty]

I believe (?) the 3650 (and 3850) implicitly trust (like a router).  If so, you wouldn't need to "trust" on any port.

In your case of your access port, I believe (?) the "trust device cisco-phone" does more than just trust ToS, it may also help control what auto-QoS ingress policy is applied.  (For example, it might only accept DSCP EF from a recognized Cisco VoIP phone.)

Again, for your trunk port, I believe (?) you don't need to explicitly "trust", however if you're doing QoS to insure needed bandwidth for VoIP traffic, even though you have 10g Etherchannel, you should probably also have an egress QoS policy, to guarantee PQ for VoIP traffic.

(BTW, in you're wondering why all the "I believe [?]"s, my direct experience with Catalyst 3K switches stopped with the 3560/3750 series.  At the time Cisco released the 3650/3850 series, the company I was working for adopted another vendor's [cough - brand "J"] L3 switches.)

[Clem58]

Thanks Joseph for all these comprehensive answers, I think the main thing I've understood is when the QoS is active on MLS ones, you need to explicitely trust QOS on the concerned interfaces