topic 10% throughput when routing in Routing and SD-WAN

10% throughput when routing

Brad Boudreau — Tue, 28 Nov 2023 15:05:33 GMT

Hello,

I am having an odd issue where if my traffic has to be routed I get approximately 10% of line rate. I have attached a drawing that shows a subset of my network which will hopefully make clear what I am doing.

I have multiple subnets on VLAN 200 (subnet consolidation is in the works so it won't be this way forever). If I transfer files between systems on the same subnet (192.168.1.0/24) I get the 1Gbps line rate however if I try to transfer files between different subnets I get around 80-90Mbps. I am not sure if it is important to mention that when I was using a Cisco c8000v at the router I was getting 700Kbps-3Mbps on routed traffic. When I replaced the Cisco c8000v with a physical router that improved to the 80-90Mbps mark, but is obviously a far cry from 1Gbps.

In my drawing I only show a single host on 172.16.1.0/24, but there are several other devices on that subnet/VLAN and if they talk directly they get 1Gbps so I am confident that all hardware is working properly. I have also confirmed that all devices are 1Gbps/full duplex and there are only a couple people using the network at the time so it "shouldn't" be a congestion issue. I have to be overlooking something in my configuration and I would appreciate any insight as to what may be causing this problem.

Thank you.

Re: 10% throughput when routing

Joseph W. Doherty — Tue, 28 Nov 2023 15:23:13 GMT

Many small Cisco routers do NOT support wire rate for all their or sometimes even for one port.

If would be helpful if you would identify the specific router.

Re: 10% throughput when routing

Brad Boudreau — Tue, 28 Nov 2023 16:33:37 GMT

Thank you for your reply. The router I am using is an ISR4331/K9 on Cisco IOS XE v16.06.04.

Re: 10% throughput when routing

Joseph W. Doherty — Tue, 28 Nov 2023 16:52:47 GMT

The 4k ISR have licensed throughput limits. The 4331 basic license limit is 100 Mbps, which from your OP info, might be what your bumping into.

The performance license limit is 300 Mbps while the no cap boost license provides whatever the hardware is capable of (Cisco says up to 2 Gbps).

Re: 10% throughput when routing

Brad Boudreau — Tue, 28 Nov 2023 16:58:08 GMT

Well then, that is probably the issue. I will look into licensing to see if I can definitively identify that as my bottleneck. I was pulling my hair out trying to figure out if my VLANs were magically crippling the network..but not looping to eventually take everything down.

Thank you very much. If licensing does end up being the problem I will come back in here and mark this as the solution.

Re: 10% throughput when routing

Brad Boudreau — Tue, 28 Nov 2023 17:32:57 GMT

I have confirmed that the router was limited to 100Mbps...didn't even know licensing now covered performance.

Re: 10% throughput when routing

Richard Burts — Tue, 28 Nov 2023 17:53:01 GMT

One aspect to consider is that when it is from one device to another device in the same subnet then it just host to host layer 2 forwarding for the router/switch. And near wire rate is feasible. When it is from one device in a subnet to another device in a different subnet then it is layer 3 forwarding and performance is different.

Re: 10% throughput when routing

Brad Boudreau — Tue, 28 Nov 2023 21:12:49 GMT

Definitely, and I do expect a bit of a drop in throughput due to latency and overhead, just not a 90% drop. I thought there had to be something misconfigured or just a mess since there were several subnets all on the same VLAN being routed over the same wire at times. Colour me surprised when I was told in this thread that I was being held back by licensing. Lots of red herring trouble shooting was going on lol.