MPLS QoS marking problems

[smailmilak]

Hi,

I have a problem with MPLS EXP marking between two devices. One is Cisco ME3600X and the other a 7609 RSP720 with ES+ card,

I mark the traffic on the ME3600X on the ingress interfaces, and on the egress interface I successfully match the marked packets (MPLS EXP bit 5).

Note: The ME3600X has a MPLS FRR tunnel configured and all traffic is passing through this tunnel (in OSPF routing table the next hop is the FRR tunnel).

The problem is that I don't see the marked EXP bit 5 on the neighbor router, but I see only EXP bit 0.

Trusting is configured on the 7609, well looks like it is enabled by default.

Do you have an idea or a document where I see how MPLS QoS should be configured when using mpls tunnels?

I already have the configuration guide for the ME3600X but the is no information how to solve this issue.

Is there a debug command which could help me.

[Vaibhava Varma]

Hi smailmilak

Is this an MPLS VPN Customer which is injecting traffic at the ingress and being set to MPLS EXP 5 and when checked at Egress the MPLS EXP is 0.

From my personal understanding it seems that PHP is not disabled here and Short Pipe mode is being used for the MPLS VPN QoS.

Is this a normal ICMP Traffic coz ICMP is best-effort service with IPP 0 and even though we are setting up MPLS EXP to 5 at Egress on Top Most Label Becuase of PHP the topmost label with EXP 5 is being removedat the PHP Hop and on the Ultimate Hop we receive the Bottom Label with original EXP 0 set for the ICMP Traffic.

We need to disable PHP in this case and implement Uniform Mode QoS.

Hope this helps to understand the issue.

Regards

Varma

[smailmilak]

Hi,

I read about uinform qos too, and tried to enable it but I don't see a way how to enable uniform mode QoS on ME3600X, only on 7609 with mls mpls qos input uniform-mode.

Yes it is ICMP, but I still mark and match the traffic on ME3600X, but on the neighor router is no EXP 5 bit.

I am using a class-deault class map to mark all kind of traffic with EXP 5.

I even tried to mark the traffic on the output interface where I am policing it with

"police 150000000 conform-action set-mpls-exp-imposition-transmit 5 exceed-action set-mpls-exp-imposition-transmit 5"

but I get a configuration error. The problem is that the ME3600X can not mark on the output direction, only on input.

Could the mpls tunnel be the problem?

I attached a picture of the topology.

Here is the QoS config on ME3600X:

ingress interface

interface GigabitEthernet0/5

switchport trunk allowed vlan none

switchport mode trunk

mtu 1526

service instance 799 ethernet

  encapsulation dot1q 799

  rewrite ingress tag pop 1 symmetric

  service-policy input SET_EXP_5_PW

  bridge-domain 799

policy-map SET_EXP_5_PW

class class-default

  set mpls experimental topmost 5

Egress interface:

class-map match-any MPLS_VOICE

match mpls experimental topmost 3  5

class-map match-any MPLS_CONTROL

match mpls experimental topmost 6

match ip dscp cs6

class-map match-any MPLS_VIDEO

match mpls experimental topmost 4

match ip dscp cs4

class-map match-any MPLS_MANAGEMENT

match mpls experimental topmost 1

match ip dscp cs1

class-map match-any MPLS_CUSTOMER

match mpls experimental topmost 2

match ip dscp cs2

policy-map MPLS_LLQ_WAN_ES20

class MPLS_VOICE

  police 150000000

  priority

class MPLS_CONTROL

  bandwidth percent 1

class MPLS_VIDEO

  bandwidth percent 30

class MPLS_CUSTOMER

  bandwidth percent 20

class MPLS_MANAGEMENT

  bandwidth percent 4

class class-default

  bandwidth percent 30

interface GigabitEthernet0/24

description To_7609

no switchport

mtu 9216

ip address 10.100.97.73 255.255.255.252

ip pim sparse-mode

ip ospf network point-to-point

ip ospf hello-interval 1

ip ospf 8000 area 0

load-interval 30

mpls ip

mpls traffic-eng tunnels

service-policy output MPLS_LLQ_WAN_ES20

ip rsvp bandwidth

ip rsvp signalling hello

interface Tunnel244

description primarni PE_ME3600->PE_7609

ip unnumbered Loopback1

mpls ip

tunnel mode mpls traffic-eng

tunnel destination 10.100.96.244

tunnel mpls traffic-eng autoroute announce

tunnel mpls traffic-eng path-option 1 dynamic

tunnel mpls traffic-eng fast-reroute

So any idea?

A debug command would help me a lot but I don't can't find the right one.

[Vaibhava Varma]

Hi smailmilak

I was earlier thinking in another direction of it being an L3-VPN customer with issues of seeing EXP-0 on egress instead of EXP-5 which was being set at the ingress.

So here we have a requirement of implementing QoS for MPLS L2VPN and we are not seeing the desired marking of EXP5 on the neighbouring router attached to the Ingress PE.

As you are matching the default-class IPP-0 and marking it for MPLS-EXP5 that will not happen for a Layer 2 VPN as in Layer 2 we have the L2 Frame and L3 Packet is encapsulated inside the Payload of the L2 Header. We just read the L2 Frame Header at the ingress and the only way to mark the L2-Frame with a desired MPLS -EXP is to either do a match-any on the Class-Map r do a Match-Cos on the class-map and set the desired EXP.

So it will be as below

class-map match-any PW

match any

or

class-map match-all PW_Cos

match cos  5

policy-map SET_EXP_5_PW

class PW ( or PW_Cos)

  set mpls experimental topmost 5

I would sugegst you to you use any of the above ways of match in the class-map to set the desired MPLS-EXP 5 which would be transported transparently across the MPLS Cloud and now at the egress depending upon whether we have PHP enabled or disabled and the mode of QoS under use there would be 4 cases:

a) PHP-Short-Pipe Mode--The PHP hop pops the topmost label and the MPLS Packet arrives with EXP0 at egress router and the Bottom Label is Popped and the L2-Frame is sent out with the same Cos with which it was sent at ingress.

b)No-PHP-Short-Pipe Mode--The PHP hop swaps the topmost label with Explicit Null lable 0 and the MPLS Packet arrives with two labels at egress router and the Both-Lables are Popped and the L2-Frame is sent out with the same Cos with which it was sent at ingress.

c)PHP-Uniform Mode--The PHP hop pops the topmost label and the MPLS Packet arrives with EXP0 at egress router and the Bottom Label is Popped and MQC is used to change the Cos of the L2-Frame

d)No-PHP-Uniform Mode--The PHP hop swaps the topmost label with Explicit Null lable 0 and the MPLS Packet arrives with two labels at egress router and the Both-Lables are Popped and the

MQC is used to change the Cos of the L2-Frame

So here the issue is not lying with the TE-Tunnel but with the Ingress Marking and Classification and Setting the MPLS-EXP But itself. Were you able to check on the egress interface of ME-3600 when you issued a ping that MPLS-EXP markings were incrementing. I believe it would not be happening because the MPLS-EXP was never being set if my understanding is correct.

Uniform Mode of QoS operation is achieved via MQC and setting the the egress PHB manually depending upon the MPLS-Lable EXP which arrives at the egress router from the Core.

Disabling PHP is desirable in Uniform Mode when we want to set the egress PHB depending on the changes made to MPLS-EXP Topmost lable within the Core.

This is from my best understanding.

Hope this helps to clarify your question. Do let me know for any doubts.

Regards

Varma

[smailmilak]

Hi,

first thank you very much. You are definitely helping.

The part with the L2 frame is clear to me.

But still I don't understand why the EXP 5 bit is not in the MPLS packet header arriving on the neighbor router.

probably beacuse the label is poped

Take a look. The VC destination is 10.100.96.233

ME3600_TEST_SA#sh mpls l2transport vc 799

Local intf     Local circuit              Dest address    VC ID      Status   

-------------  -------------------------- --------------- ---------- ----------

Vl799          Eth VLAN 799               10.100.96.233   799        UP       

ME3600_TEST_SA#sh mpls forwarding-table 10.100.96.233

Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop   

Label      Label      or Tunnel Id     Switched      interface             

35    [T]  Pop Label  10.100.96.233/32 0             Tu233      point2point

As I already said I tried to configure uniform mode. I used this document

http://www.cisco.com/en/US/docs/ios/12_2t/12_2t13/feature/guide/ftdtmode.html#wp1155788

but the problem is that this platform (ME3600X) does not support marking in output direction.

So HOW to disable PHP on this platfrom .

I tried with mpls ldp explicit-label on both routers but it still says POP Label for 10.100.96.233.

EDIT:

I managed to disable PHP. The VC destination was using another MPLS FRR tunnel and after I disabled it and again enabled mpls ldp explicit-null

the POP label was gone and a label 44 appeard. BUT still I can not match EXP 5 on the neighbor router.

[Vaibhava Varma]

Hi smailmilak

The PW under question is being created between ME3600 and DM2 and so we need to disable PHP at ME3600 and DM2 so the loopbacks of them are learned as Explicit Null EXP0...

The reason why we are not seeing the EXP5 at the neighbouring router DM3 is becuase the Policy which we are applying on the attachment circuit on ME3600 is not taking effect as its referencing a class-map which is not able to do the correct match as an L2 Frame is arriving on the AC and it can't read the IPP 0 in that.

Did you try using the match-any or match-cos and checked at the neighbouring router DM3.

If the neighbouring router is not seeing EXP5 even after applying the above class-map then I think you are correct the PoP-Label action on the Tunnel headend is removing the top EXP label 5. Did you trying using setting exp5 using imposition

On a side note Label 44 is not explicit null and we should be seeing something as "explicit-n"

Have you enables MPLS on the TE Tunnel. Is your TE Tunnel between the two PEs which are PW endpoints ?

Regards

Varma

[smailmilak]

Hi,

i forgot to say that I tried this two class-maps. The match any is not present and match cos 5 gives an error

QoS: Configuration failed.  "Cannot add class PW_Cos to port based policy SET_EXP_5_PW"

I added then insted of "set mpls experimental topmost 5" i used set cos. The egress interface matched then this COS 5 but still the neighboring router did not recieved COS 5 nor EXP 5, well COS 5 could not received anyway if I am not wrong.

And yes label 44 is not explicit null because the PW endpoint is a few hops away. There was a POP label because there was an another TE tunnel to the VC endpoint and that is why poped it. After I disabled it and left only the other TE tunnel the label 44 appeared.

MPLS IP is on the TE tunnels. There was a TE tunnel between the PEs which are PW endpoints but this tunnel is shutdown and label 44 is now used for forwarding. I said this above, too.

Now I really don't understand why there is not and matching on the neighbor ingress interface, only EXP 0. On top of the label should be EXP 5 bit. What am I missing? Every traffic that is entering the interface should me marked, no matter what type, ICMP, TCP...??

I desperately try to debug this but I don't know how.

[Vaibhava Varma]

Hismailmilak

Untill and Unless we do the right match on the ingres we would not be able to set the MPLS EXP which is not happening here and causing the issue.

Do you mean to say that you have no options to match any in ME3600 as below

CGN64(config)#class-map match-any L2VPN

CGN64(config-cmap)#match ?

  access-group         Access group

  any                  Any packets

  class-map            Class map

  cos                  IEEE 802.1Q/ISL class of service/user priority values

  destination-address  Destination address

  discard-class        Discard behavior identifier

  dscp                 Match DSCP in IP(v4) and IPv6 packets

  flow                 Flow based QoS parameters

  fr-de                Match on Frame-relay DE bit

  fr-dlci              Match on fr-dlci

  input-interface      Select an input interface to match

  ip                   IP specific values

  mpls                 Multi Protocol Label Switching specific values

  not                  Negate this match result

  packet               Layer 3 Packet length

  precedence           Match Precedence in IP(v4) and IPv6 packets

  protocol             Protocol

  qos-group            Qos-group

  source-address       Source address

  vlan                 VLANs to match

CGN64(config-cmap)#match any

CGN64(config-cmap)#

What match options do you have for class-maps on ME3600 ?

So you are setting the COS to 5 at Ingress and its not being reflecting in EXP5 on neighbouring router. Can you apply a policy map on the Egress Interface of ME3600 and do a match for EXP5 and see if the EXP5 is leaving out from ME3600 Egress interafce or not ?

From my best understanding this is an issue of being unable to match the L2 Frame and set the MPLS EXP.

Regards

Varma

[smailmilak]

Hi,

here are the options on the ME switch:

access-group   Access group

cos            IEEE 802.1Q/ISL class of service/user priority values

discard-class  Discard behavior identifier

dscp           Match DSCP in IPv4 and IPv6 packets

ip             IP specific values

mpls           Multi Protocol Label Switching specific values

precedence     Match Precedence in IPv4 and IPv6 packets

qos-group      Qos-group

vlan           VLANs to match

I am not quite sure what you mean by matching on the ingress interface on ME3600X. A normal ICMP packet arrives on this interface and only with class-default class map I can match "any" and set the COS or EXP 5 bit. If it would be a Cisco IP phone that can mark traffic than I could match COS 5, but it is just a normal PC with is connected to a ME3400 L2 switch and the switch to the ME3600X Switch which is a PE router.

I even tried to set COS 5 instead of EXP 5 on the ingress interface. On the egress interface the same router matched EXP 5 bits. So the LSR is probably rewriting the COS 5 to EXP 5 on the router. But still only EXP 0 arrives on the neighbor router.

I don't have any ideas, I tried a lot.

Maybe it is just a small syntax to solve this problem but which one

By my understanding every possible traffic that is arriving on the ingress interface should be marked, which is succefully marked, and every traffic on the physical egress interface (irrevelant if TE tunnel or PW are used) should be

forwareded as a label or more with the EXP 5 bit on top.

Here is a nice text of Jeff Doyle about explicit null and COS

http://www.networkworld.com/community/node/24143

"The question arises, then: If implicit null is more efficient, why do we need explicit null at all?

The answer is Class of Service.

When a packet or Ethernet frame is encapsulated in  MPLS, you have the option of copying the IP precedence or 802.1p bits to  the three CoS bits of the MPLS header (unfortunately called the  Experimental or EXP bits), so that the MPLS LSP provides the same CoS  behavior, or you can set the EXP bits independently, so that the LSP CoS  behavior has a designated CoS behavior that is independent of any  encapsulated payload.

In this second case, you would want to use the  explicit null label between the penultimate and egress LSRs. If a POP is  performed at the penultimate LSR, as in Figure 2, the EXP bits in the  MPLS header are no longer available as a reference for queuing and the  packet is queued on the outgoing interface according to the CoS behavior  of the underlying payload. An explicit null, on the other hand, leaves  the MPLS header in place until it reaches the egress, preserving the LSP  CoS behavior across the entire LSP."

Well I tried this but it did not help.

[Vaibhava Varma]

Hismailmilak

So the marking is happening at the Ingress interface of ME3600 and the same is verified on the Egress Interface of ME3600 also. But at DM3 the EXP5 marking is lost and only EXP0 marking is being seen ?

Can you do an extended ping from the CE with TOS as 0xB8 and see if the neighbouring router sees the EXP5 ?

Regards

Varma

[smailmilak]

Hi,

I will do this tomorrow morning.

I tested it for now with an extended ping on the 3600X to a router in the network.

In this case the the neighbor router matched all  300 packets with TOS 0xb8 as EXP 5.

Should I on the the L2 switch where the PC is connected enable qos trust with mls qos trust or mls qos trust cos? I think I should use mls qos trust cos

With mls qos trust I get this

FastEthernet0/24

trust state: trust dscp

trust mode: trust dscp

trust enabled flag: ena

COS override: dis

default COS: 0

[smailmilak]

Take a look what I have found in the 3600X documentation.

"In the case of L2 to L3 propagation, exp value will be set to 0 by default"

This is my case!!!

Packet marking

http://www.cisco.com/en/US/docs/switches/metro/me3600x_3800x/software/release/15.1_2_ey/configuration/guide/swqos.html#wp1044145

It is late now, I gonna continue tomorrow.

[Vaibhava Varma]

Hismailmilak

I also think to trust cos on access port to PC as its L2 Port.

Did you try setting the mpls exp imposition on the ME3600 ingress yet or not and see any difference ?

I have been thinking over this whole day for the strange behaviuor on different aspects and hopefully would be reaching a conclusion and solution.

Late for me too..Going to hit the bed..too late for morning :-)

Regards

Varma

[smailmilak]

Hi,

I ping now with ef bit and the ingress interface on ME3600X is matching the ef bit, and the egress policy-map, too.

But still there is no EXP5 bit on the neighbor router, only EXP 0.

With  L3 interface on the 3600X which is the GW for the host, which means  there is no PW, the packets are matched on all routers in the network.

Don't know what to do anymore.

Here is what the documentation says about MPLS EXP on 3600X

Default MPLS and EoMPLS QoS Configuration

QoS is disabled. Packets are not modified, and the CoS, DSCP,  and IP  precedence values in the packet are not changed. Traffic is  switched in  pass-through mode (packets are switched without any  rewrites and  classified as best effort without any policing).

The default behavior for VLAN and port-based EoMPLS packets is  to use a  value of 0 in the EXP bits of the virtual-connection and  tunnel labels.  The default behavior for L3VPN MPLS packets is to relay  the IP  Precedence bits into the EXP bits of the virtual-connection and  tunnel  labels. You can change the default behavior for VLAN- or  port-based  EoMPLS by applying a hierarchical QoS policy.

MPLS QoS Configuration Guidelines

•The switch supports these MPLS QoS features:

–MPLS   can tunnel the QoS values of a packet (that is, QoS is transparent  from  edge to edge). With QoS transparency, the IP marking in the IP  packet  is preserved across the MPLS network.  

Why do I have a problem then

–You   can mark the MPLS EXP field differently and separately from the  per-hop  behavior (PHB) marked in the IP precedence, DSCP, or CoS field.

•One   label-switched path (LSP) can support up to eight classes of traffic   (that is, eight PHBs) because the MPLS EXP field is a 3-bit field.

•MPLS   DiffServ tunneling modes support E-LSPs. An E-LSP is an LSP on which   nodes determine the QoS treatment for MPLS packet exclusively from the   EXP bits in the MPLS header.

•The switch does not support egress QoS marking.  Hmm I am marking on ingress, and the marked packet should just be forwarded without       modifiyng it

•MPLS QoS classification does not work for bridged MPLS packets. What are "bridged" MPLS packets

•For port-based EoMPLS, you cannot match the payload VLAN.

Setting the Priority of Packets with Experimental Bits

MPLS and EoMPLS provide QoS on the ingress router by using 3   experimental bits in a label to determine the priority of packets. To   support QoS between LERs, set the experimental bits in both the   virtual-connection and tunnel labels.

The process includes these steps on the ingress router:

•Configure a class map to classify IP packets according to their DSCP or IP precedence classification.

•Configure a policy map to mark MPLS packets (write their classification into the MPLS experimental field).

•Attach the service policy to the input interface or service instance

[Vaibhava Varma]

Hismailmilak

Did you try setting up mpls experimental impostion already before doing this extended ping ?

Regards

Varma