Jumbo frame support under L2vpn sub-interface on ASR9K

[Eric Guo]

Hi all,

I run  jumbo frame support test  for our l2vpn  xconnect service on Cisco ASR 9K platfrom, XR OS is kind of old as 3.9.2

I setup AC interface under both end router sub-interface. 200, shown as below:

testing IOS router 1 int gi0.200---- int gi0/0/0/1.200 ASR9K router 1--MPLS enable --ASR9K router 2 int gi 0/0/0/0.200-----inter gi0/1.200 testing IOS router 2

I already enable jumbo frame on two ASR9K mpls link and also enable MTU size over 9000 on  ASR 9K router1  main interace gi 0/0/0/1 and 

ASR 9K router2  main interace gi 0/0/0/0, and also  on both end sub-interface as well.

the large  size PING test from testing  IOS router 1 or 2 are show good, as ping through over 9000 packet size without fragment is OK.

the issue is: when I changed  MTU size back to 1518 on  ASR9K router 1 int gi0/0/0/1.200 and ASR9K router 2 int gi 0/0/0/0.200, I still can ping through  large packet size over 9000 without fragment from both IOS testing router end. it is showing  my sub-interface MTU size setting couldn't limit l2vpn packet size,  even the L2vpn xconnect output showing MTU size is been changed back to 1500.

sh l2vpn xconnect detail                 

Fri Nov 29 20:35:21.203 UTC

Group test1, XC test1, state is up; Interworking none

  AC: GigabitEthernet0/0/0/1.200, state is up

    Type VLAN; Num Ranges: 1

    VLAN ranges: [200, 200]

    MTU 1500; XC ID 0x40001; interworking none

    Statistics:

      packets: received 84148, sent 507960

      bytes: received 9043313, sent 37742100

      drops: illegal VLAN 0, illegal length 0

  PW: neighbor 10.10.100.2, PW ID 200, state is up ( established )

    PW class not set, XC ID 0x40001

    Encapsulation MPLS, protocol LDP

    PW type Ethernet, control word disabled, interworking none

    PW backup disable delay 0 sec

    Sequencing not set

      MPLS         Local                          Remote                       

      ------------ ------------------------------ -----------------------------

      Label        16002                          16001                        

      Group ID     0x240                          0x4000080                    

      Interface    GigabitEthernet0/0/0/1.200     GigabitEthernet0/0/0/0.200   

      MTU          1500                           1500                         

      Control word disabled                       disabled                     

      PW type      Ethernet                       Ethernet                     

      VCCV CV type 0x2                            0x2                          

                   (LSP ping verification)        (LSP ping verification)      

      VCCV CC type 0x6                            0x6                          

                   (router alert label)           (router alert label)         

                   (TTL expiry)                   (TTL expiry)                 

      ------------ ------------------------------ -----------------------------

    MIB cpwVcIndex: 2

    Create time: 27/11/2013 19:15:21 (2d01h ago)

    Last time status changed: 29/11/2013 18:25:18 (02:10:02 ago)

    Statistics:

      packets: received 507960, sent 84148

      bytes: received 37742100, sent 9043313

so my question: what's MTU size setup use for under layer 2 sub-interface ? is this same function as bridge MTU setting for control plane signaling ? or is this software bug of OS version 3.9.2 ?

by the way, I did same test for l2vpn bridge group test2, I got same result that sub-interface MTU size couldn't limit jumbe frame packet if main-interface MTU size already enable large MTU size.

could someone have same experience to share with me about your idea ? many thanks !

Eric

[xthuijs]

Hi Eric,

yup I can see how that is a bit misleading, but this is expected, let me explain.

In L2(vpn) mode, the sub-interface MTU is not enforced and merely used for signaling only.

So eventhough a packet is larger then the MTU of the l2transport subinterface, it will still go through as MTU checking

is not performed on the L2 interface side. This was done for good reasons (performance), and conformance too sort-a

speak as in L2 mode we ought to be transparent (but ok to what level right).

Either case the performance gain we get out of this is a massive benefit.

If however the MTU of the main interface is smaller then the packet that wants to get out, then it will get dropped.

Remember that XR will never fragment in hardware.

In L3 mode (that is a (sub)interface with an ip address on it pretty much), MTU is enforced, but when an MTU is mismatched against the CEF forwarding entry the packet is punted to the egress LC cpu for fragmentation.

This is limited to 1000pps (or whatever that lpts policer value is these days), but either case features are NOT applied!

So if you have egress ACL or QOS, this will not get applied to the injected fragments and that is a big pain.

Few more examples that hopefully explain a couple of additional scenarios:

If you have a phy MTU defined, then the packet will get dropped when it is over 
that value.

If you have a phy MTU defined of say 8000 and a subif MTU of 2000 and that 
subif is an L2transport interface then you WILL be able to receive a 3000 byte packet 
on your AC. 

If the AC is an xconnect with a PW going out a core interface, the core interface 
MTU on the egress LC will be enforced and may result in fragmentation.

If you receive a 3000 byte packet on your PW (assuming core can handel that) 
and needs to go out the AC with the subif MTU of 2000, and phy MTU of 8000 this 
WILL go through. (basically your case you described)

Xander Thuijs CCIE #6775
Principal Engineer 
ASR9000, CRS, NCS6000 & IOS-XR

[Eric Guo]

many thanks Alexander. this is clear to me now.

much appreciate !

[shivraj.bhosle]

Hi Alexander,

I have a HVPLS setup and do you say that I can leave the physical mtu (UPE-NPE) on the path to 1500 bytes and this will not affect the VC ?

The