slow file transfer between ASR 9000 XR Version 4.3.4

[Eric Guo]

Hi All,

We setup a standard L2VPN bridge group (customerA) and 2 domains (cust-vlan100 and cust-vlan200) underneath on our two PE sites, one l2vpn bridge-domain is for our customer file replication transmit  , other one is for regular service, such as internet access.

In  the between, there are 2 other P/PE routers made multi path connection ( shown below). We have ISIS running as IGP and MPLS layer is on the top, all the devices are ASR 9010/9006 XR version is 4.3.4. Currently, our cutomer file replication speed is only below  150Mbps under vlan 100 tunnel. It is very slow that is against with lab testing result ( up to 300-400 Mbps by using iperf)  

My question is:

how many factors could be impacting this service performance?

such as interface packet loss,  IGP routing issue for MPLS LSP path?

or MPLS issue? L2vpn configuration issue?  hardware issue ? software issue etc ?

Could someone give me some clues for this troubleshooting? I attached L2vpn configuration as reference. 

many thanks,

Eric.

  l2vpn
 bridge group customerA
  bridge-domain cust-vlan100
   interface GigabitEthernet0/0/0/5.100
   !
   vfi 100
    neighbor PE2-lookback ip pw-id 100
     pw-class encap-mpls
    !
    
  bridge-domain cust-vlan200
   interface GigabitEthernet0/0/0/5.200
   !
   vfi 200
    neighbor PE2-lookback pw-id 200
     pw-class encap-mpls
    !
   !
  !
 !
!

[Peter L]

Hi

The configuration looks fine. Shouldn't be anything there that affect the throughput. Could be a lot of different reason for the customer measurement result. But here is some things you could begin and check.

Start troubleshooting on a low level. Check interface counters/duplex and so on. Then move upwards in the layers. Check that there isn't any packet drops between the boxes and that there isn't any policers/shapers that could drop traffic applied on any interfaces.

Do you know how the customer has done there testing, UDP or TCP test? Always test UDP first if possible to rule out TCP problems.

Have there been any test done on the other vlan? Same problem or is that working fine?

/Regards Peter

[Eric Guo]

many thanks Guys for your information ! I will check these document.

First of all, the UDP test result is similar with TCP test one.
Secondary, I forced this service path on only one leg, and our customer reported back to me, the transmit speed just  is right above to 154Mbps, a little bit improvement. 

I will continue to check...

thanks,

Eric

[DIEGO ZORRILLA]

I have use IPERF in the pass, so a couple of questions.

When you test with UDP, whats the max bandwith you are able to reach?

When using TCP you are telling is 150M on you production network, but on lab able to go to 400 or 500M, so what about delay?

If you change from a l2vpn to an xconnect, do you have the same results?

[xthuijs]

hey guys,

take a look at bryan's note, I am almost sure that you are suffering from out of ordering caused by that mac addr 4 or 6.

insert a control word or change the mac addr of the destination to eliminate that 4 or 6 and see if that results in a better performance.

if not, some other tricks may exist, but wanna exclude the most obvious here first.

cheers!

xander

[Bryan Garland]

Eric,

If you have ECMP paths in the core, do you know the dmacs used for your traffic? If starting with 4 or 6 could be issue with arriving out of order and such.  Take a look at here:  

https://supportforums.cisco.com/document/111291/asr9000xr-loadbalancing-architecture-and-characteristics

Try using control word for your PWs.  If you still want to make sure you are getting enough load balancing then you can also use Flow labels as well.

Bryan

[Evgeniy Prichinin]

What mtu do you use on core links?