Slow speed with ASR 1000

[Secnet DRVBundGQ]

We have 2 ASR1000 Router in our company. Redundancy is provided through HSRP.
After several users complained about a slow line speed, we did some tests with iperf and saw that the result shows always a very bad iperf result.
The test setup was

Notebook1--Switch-ASR1000-Switch-Notebook2
A typical test result is shown in the attachment
The Notebooks aren't the bootleneck, when we made the test with a cross cable (Notebook to Notebook) we got ~950 MBit

The strange thing is, the iperf test with udp gets over 900Mbit, also when we make the iperf test (tcp) with several silutaneous connections, we also get over 900Mbit

The interface on the router shows no errors, also the cpu load is very low (2%).
Everything seems to be perfect, but the line speed is low.
We tried some debug commands, everything is ok. Also, there is no limitation in the config..no special config command..nothing.

I also created a tcpdump file, even the tcpdump shows no error. Some Retransmission, but not enough to be a problem.

any ideas?

[Georg Pauwen]

Hello,

post the configs of all 4 devices involved (2 ASRs, 2 switches)...

[paul driver]

Hello

Could you provide more details regards your topology, I assume these hosts are not directly connected to the asrs?
Do you have access switches interconnected?

Do you get the same results with hosts either in the same or on different broadcast domains?

Have you checked for stp re-transmissions , mtu mismatch etc..

What software/feature set are you running on the ASR's (switches also if applicable)

[Secnet DRVBundGQ]

many thx for your help.

the hosts (Iperf) are not directly connected
we have access switches between the hosts
we always have the same reuslts

we have stfp re-transmissions, but only a few (30x in 30 seconds) and no mtu mismatches
the router version is "Cisco IOS Software, ASR1000 Software (PPC_LINUX_IOSD-ADVENTERPRISEK9-M), Version 15.3(3)S2, RELEASE SOFTWARE (fc3)"

The test was done with no VPN Connection via GRE Tunnel, it was only a routed connection from notebook 1 to Notebook2

I attached 4 Config files...

asr1000_1.txt and asr1000_2.txt   >ASR1000 Config

switch1.txt and switch2.txt > Switch config

Switch1 = cisco WS-C3850-24XU (MIPS) processor (revision G0) with 865070K/6147K bytes of memory.
Switch2 = cisco WS-C3850-24T (MIPS) processor (revision AB0) with 865070K/6147K bytes of memory.

[paul driver]

Hello
I think there is more to your network than you have mentioned.
The two ISR's are your wan rtrs running in a DMVPN environment and their lan facing interlaces running HRSP between them.

What i don't see are the is L3 device(s) responsible for you inter-vlan routing (vlans 200,203, 250) , those two switches you have posted seem to be both access layer switches (sw1 show only L2 vlan for 200) (sw2 shows l2 vlan 203, 250) and for the hosts in these vlans to be able to communicate their traffic needs to be routed.

So do you have another a L3 switch/router or FW performing the  inter-vlan routing -  it will be the device(s) your hosts ip default gateway address refers to.

---

@Secnet DRVBundGQ wrote:

we have stfp re-transmissions, but only a few (30x in 30 seconds) and no mtu mismatches

---

And the above amount of stp transitions shouldn't be happening especially in that short amount of time, so i would say a more detailed review of your current network topology would be applicable and a closer look are those stp transitions and what making them, possible flapping link, stp loop?.

Do you have topology diagram you could share?.

[Secnet DRVBundGQ]

Hello,

i uploaded the config again..i only removed the password and the certificate.

Our network topology was saved as a png file. I hope everything is visible.

Our switches don't do inter-vlan routing, the routing is done by the ASR Router.

[Georg Pauwen]

Hello,

I cannot figure out the traffic flow between your two notebooks. Where is the second HSRP router (VPN24099-44-21) ? Also, you seem to be sending traffic from a NAT inside to a NAT outside interface ? Is your traffic going through the Internet ?

It might be better to make a new, simplified drawing involving only the devices that are in between both notebooks, and post the configs of these devices...

[Secnet DRVBundGQ]

Hello,

it is much easier as you might think.

To find the failure we only made the test between the 2 notebooks (red line.)

Only the 2 notebooks, the 2 switches (marked with a green line) and the ASR1002 (orange line) were involved.
Really this simple test.

- no VPN
- no Gre Tunnel

The traffic went just through the switch, to the ASR1002 Device then to the other switch and go in the the second switch.

The network 203.0.0.0/8 isn't routed to the inside, so we must used a nat to make the test possible.
The only reason for the nat was, to make the connection to the ASR1002 possible.

The network is local, we use internet IP addresses inside..this might confuse you.

Imagine having 2 notebooks, 2 switches and the router on your desk., without conenction to the internet, just these 5 devices and 4 cables.
This was our test scenario.

(small correction..we have the ASR1002 only the ASR1000 Software)
Cisco IOS Software, ASR1000 Software (PPC_LINUX_IOSD-ADVENTERPRISEK9-M), Version 15.3(3)S2, RELEASE SOFTWARE (fc3)
cisco ASR1002 (2RU) processor with 1666625K/6147K bytes of memory.