IOS XR Strange MTU on Bundle-Ethernet members

[alain_richard]

Hi,

In order to get a usable MTU of 9198, I declare the mtu 9212 on a bundle-ether port (9198+14).

The configuration is following :

RP/0/RSP0/CPU0:rtr1#sh run int be1

interface Bundle-Ether1

 mtu 9212

RP/0/RSP0/CPU0:rtr1#sh run int te0/0/0/8

interface TenGigE0/0/0/8

 bundle id 1 mode active

RP/0/RSP0/CPU0:rtr1#sh run int te0/0/0/9

interface TenGigE0/0/0/9

 bundle id 1 mode active

The strange thing is that the MTU on the members interfaces are set to the MTU of the bundle-ether plus 4 :

RP/0/RSP0/CPU0:rtr1#sh int be1 | i MTU

  MTU 9212 bytes, BW 0 Kbit

RP/0/RSP0/CPU0:rtr1#sh int te0/0/0/8 | i MTU

  MTU 9216 bytes, BW 10000000 Kbit (Max: 10000000 Kbit)

RP/0/RSP0/CPU0:rtr1#sh int te0/0/0/9 | i MTU

  MTU 9216 bytes, BW 10000000 Kbit (Max: 10000000 Kbit)

I have tested this on an ASR9901 IOS XR 6.5.3 and on an NCS540 IOS XR 6.6.25.

Normally the MTU on the bundle and on interfaces aren't supposed to be the same ? At least this is the case on all other IOS devices I tested.

Regards

Alain RICHARD

[tkarnani]

Hi,

MTU calculation is different on ios-xr than ios.

we have a document here, configured mtu -ethernet header (14) vlan header(4)

https://www.cisco.com/c/en/us/support/docs/ios-nx-os-software/ios-xr-software/116350-trouble-ios-xr-mtu-00.html

Thanks!

[alain_richard]

Yes, I well know this document and if I add a vlan tag, the header is 18 instead of 14.

The problem here is not this one :

- I declare a LACP aggregate BE1 with no encapsulation and an MTU of 9014

- I put 2 interfaces into this aggregate BE1 without declaring an MTU

In that cas, if I look at the MTU of the member interfaces, their MTU is the BE1 MTU plus 4 (in this case 9018).

This is not normal because I have not declared any more encapsulation (there is no 802.1q tag) and the MTU of members are supposed to be the same as the MTU of BE.

Regards,

[tkarnani]

I tried in our lab, asr9k 653

what are you doing that is different to me?

RP/0/RP1/CPU0:ASR9922-A#show run int bundle-ether 3000
Fri Jan 17 11:48:22.335 EST
interface Bundle-Ether3000
 ipv4 address x.x.1.2 255.255.255.252
!

RP/0/RP1/CPU0:ASR9922-A#show run int tenGigE 0/7/0/0
Fri Jan 17 11:48:28.764 EST
interface TenGigE0/7/0/0
 bundle id 3000 mode passive
 cdp
!

RP/0/RP1/CPU0:ASR9922-A#show interfaces bundle-ether 3000 | inc "MTU|MRU"
Fri Jan 17 11:48:42.296 EST
  MTU 1514 bytes, BW 20000000 Kbit (Max: 20000000 Kbit)
RP/0/RP1/CPU0:ASR9922-A#show interfaces tenGigE 0/7/0/0 | inc "MTU|MRU"
Fri Jan 17 11:48:55.142 EST
  MTU 1514 bytes, BW 10000000 Kbit (Max: 10000000 Kbit)
RP/0/RP1/CPU0:ASR9922-A#conf t
Fri Jan 17 11:48:58.388 EST
RP/0/RP1/CPU0:ASR9922-A(config)#interface Bundle-Ether3000
RP/0/RP1/CPU0:ASR9922-A(config-if)#mtu 9014
RP/0/RP1/CPU0:ASR9922-A(config-if)#commit
Fri Jan 17 11:49:16.518 EST
RP/0/RP1/CPU0:ASR9922-A(config-if)#exit
RP/0/RP1/CPU0:ASR9922-A(config)#interface Bundle-Ether3000
RP/0/RP1/CPU0:ASR9922-A#show interfaces bundle-ether 3000 | inc "MTU|MRU"
Fri Jan 17 11:49:25.710 EST
  MTU 9014 bytes, BW 20000000 Kbit (Max: 20000000 Kbit)
RP/0/RP1/CPU0:ASR9922-A#show interfaces tenGigE 0/7/0/0 | inc "MTU|MRU"
Fri Jan 17 11:49:28.652 EST
  MTU 9014 bytes, BW 10000000 Kbit (Max: 10000000 Kbit)
RP/0/RP1/CPU0:ASR9922-A#

[philclemens1835]

I'd be interested to know more about the config.  Are there subinterfaces with dot1q tags?  Is it an l2transport interface?  Routed?  What bundle protocol is used?

If we assume that there are tagged subinterfaces, based on this link, the physical interfaces will handle the tags, and the bundle really only assigns traffic to physical interfaces, so it doesn't need to have the extra 4 bytes to deal with tags.  Is that the idea?

Maybe a "show controller te0/0/0/8 all | i "MTU|MRU" would provide some more info.

[philclemens1835 · Accepted solution]

I also configured this in our lab.  With just a simple bundle of two interfaces, all MTU's are equal.  When I add a subinterface with a dot1q tag to the bundle, then I get the additional 4 bytes on the physical interfaces.

RP/0/RSP0/CPU0:XR2(config)#int gi0/0/0/0
RP/0/RSP0/CPU0:XR2(config-if)#bundle id 1 mode active
RP/0/RSP0/CPU0:XR2(config-if)#int gi0/0/0/1
RP/0/RSP0/CPU0:XR2(config-if)#bundle id 1 mode active
RP/0/RSP0/CPU0:XR2(config-if)#int bundle-ether 1
RP/0/RSP0/CPU0:XR2(config-if)#mtu 9216

RP/0/RSP0/CPU0:XR2#show int gi0/0/0/0 | i MTU
Fri Jan 17 15:53:42.168 UTC
MTU 9216 bytes, BW 1000000 Kbit (Max: 1000000 Kbit)
RP/0/RSP0/CPU0:XR2#show int bundle-ether 1 | i MTU
Fri Jan 17 15:53:53.866 UTC
MTU 9216 bytes, BW 0 Kbit

RP/0/RSP0/CPU0:XR2(config)#int bundle-ether 1.1
RP/0/RSP0/CPU0:XR2(config-subif)#encap dot1q 101

RP/0/RSP0/CPU0:XR2#show int gi0/0/0/0 | i MTU
Fri Jan 17 15:55:03.193 UTC
MTU 9220 bytes, BW 1000000 Kbit (Max: 1000000 Kbit)
RP/0/RSP0/CPU0:XR2#show int bundle-ether 1 | i MTU
Fri Jan 17 15:55:17.050 UTC
MTU 9216 bytes, BW 0 Kbit

[alain_richard]

So yes, I reproduced the problem on a new router :

int be3000
mtu 9014
int te0/0/0/1
bundle id 3000 mode active
int te0/0/0/2
bundle id 3000 mode active

result :
be3000 MTU = 9014
te0/0/0/1 MTU = 9014
te0/0/0/2 MTU = 9014

if then I add

int be3000.10
encapsulation dot1q 10

i get the problem :

be3000 MTU = 9014, OK
be3000 MTU = 9018, OK
te0/0/0/1 MTU = 9018, different of be3000 MTU
te0/0/0/2 MTU = 9018, different of be3000 MTU

This is probably just a cosmetic problem, but it is a strange behaviour.

Regards,

Alain

[philclemens1835]

I believe it is likely what I indicated earlier.  In a bundle, the actual frames traverse the physical interfaces as frames, and it is there where the dot1 tagging actually occurs.  So, by telling the bundle you want an MTU size, it adjusts accordingly for the associated physical interfaces that will actually be handling the tagging.

The end result is that two ASR9K's would have identical MTU's regardless.  The catch is definitely if you were attempting to have an XE OSPF neighbor, where even if you were aware of the 14-byte deal between XR and XE, you still might get caught out with this additional four bytes.  But, maybe XE adds the same 4 bytes to physical interfaces in a port channel.

It would be good to have a BU person chime in on this and maybe provide even more clarification on this specific situation.

BTW, my lab is running 6.4.2sp3 Classic.