MX 105 Stack

[SSAS1]

Just within the last month our internet has slowed by 75%. It worked fine up until a month ago. Using a MX 105 that was suggest to us by Meraki to fit our needs. Using a 2 gig SFP only getting 30 to 50 Mbps download. 900 + uploads. It looks like our Trunk ports will only handle MTU of 1500. it needs to be 9000. Does anyone know if we need to MX 250?

[CMR]

It shouldn't matter too much, but @Ryan_Miles said you have 1Gb transceiver in WAN 2, yet you have set it to 3Gb of bandwidth, I would lower it.

Also the download chart clearly shows more that 30-50Mb/s - are you only having the issue with WAN2, is WAN1 working as expected?

[kj4qwt]

Sorry. I forgot to mention that the two different fiber modules were used as a demonstration for Meraki support to rule out that optics overload is not to blame for the close proximity between our providers NID and the MX105.

[SSAS1]

WAN 2 should be ruled out as we only use it as a failover backup circuit.

[CMR]

@SSAS1 so if WAN1 is working at up to at least 220Mb/s download - per the graph above, what is at 30-50Mb/s?

[SSAS1]

Internet is very slow 30 to 50 mbps throughout network. All devices. User complaints.

[CMR]

If you go from a client to fast.com and test there, what do you see?

[SSAS1]

170 Wired, 100 on Wireless

[CMR]

So not 1-2Gbps, but much better than 30-50Mbps!

[CMR]

With the wired connection are you plugged directly into one of the MX105 LAN ports, or via switches etc.? If not direct, have you tried that?

[kj4qwt]

The one thing that we cannot see are the counters that represent the Giant Frames on interfaces 1 and 9 SFP+ 10G Fiber interfaces on the MX105.

I think the issue here is too many resends of packets because of MTU size mismatches between our LAN and the Providers NID networking equimpment and the MX105.

Our providers customer fiber interface facing our MX105 has it's MTU set to 9000 and our main switch is set to 9578.

My understanding is that the MX105 only supports an MTU of 1500 MAX, even if the LAN interface is configured as a trunk port.

[CMR]

Does your provider host servers for you, the internet itself generally never has an MTU above 1500. If the device you are using has an MTU of 1500 and the switch has a max MTU of 9578 the biggest packet will still be 1500 bytes. Do your devices that are accessing the internet have large frames enabled and if so why?

[kj4qwt]

Hello cmr,

The reason for the large MTU size is because we are using trunk interfaces between our switches to more efficiently handle large volumes of traffic, especially when our MX105 is being used for inter-vlan routing.

Normally it's standard to have trunk interfaces have an MTU in the 9000's range for this reason.

Are you suggesting that we need to drop our MTU on our switches and ISP NID down to 1500?

[CMR]

Unless your endpoint devices have large frames enabled, having larger frames on the switches will make no difference. The usual reasons for having large frames is for iSCSI storage networks, though unless you have lots of sequential data it doesn't make a lot of difference in most cases. We don't enable large frames on any networks including the iSCSI ones. However the network switch max frame size remains at 9578.

As requested above, do any of your endpoints have large frames enabled, servers, storage systems or client devices? If not then that is a red herring.

What speed did you get when plugging a client directly into the LAN ports of the MX105?

[kj4qwt]

Sorry cmr, I'm confused.

Are you saying that it matters to have the MTU size match between endpoints, but not between networking equipment?

When we plug in directly into an available copper Ethernet port on our MX105, we do get faster speeds than compared to going through our 10G fiber LAN interface. Like in the 200MB/s range for download and the 900MB/s range for the upload.

We get the same results when completely disconnecting our LAN network from the MX105 and just using one computer on an available copper ethernet interface on our MX105.

[CMR]

@kj4qwt what I'm saying is that the endpoint MTU size must be the same or smaller than anything in the path.

So if you had endpoint (1500) -> switch (9578) -> server (1500) then the data transfers would be in packets of up to 1500 bytes and it should work well.

On the other hand if you had endpoint (6000) -> switch (9578) -> router (1500) -> carrier (9000) -> Internet (1470) -> server (1500) then any packet over 1470 bytes in size would be fragmented and if the client was sending 6000 byte packets then they would be broken into at least four parts when they got to the router and eight parts by time they got to the server.

If at the above the endpoint had a max MTU of 1470 there would be no fragmentation and if it had a max MTU of 1500 then it would only be in 2 fragments.