01-17-2020 02:55 PM
Hi, I've been working on this for 3 days and no resolution. Hopefully some smarter folks here will throw me a bone!
I have a hub and spoke phase 3 topology that works great except for speed- internet speeds (DIA) are as expected, spoke is 100M and hub is 1 gig link.
The hub is an old Cisco 2911 running 15.7(3)M5, the spoke is a small ISR (881W, wireless disabled) running 15.4(3)M8.
Actual bandwidth through the tunnel (iperf3) is only 2.2-3Mbits average- I'm pulling what little hair I have left out over that.
Configs are below, I've played w/ the tcp-mss settings and mtu, it didn't really make much difference. I also removed the tunnel protection as a test and the BW on both sets was actually slightly slower, but a negligible difference.
Hub Config:
interface Tunnel1
bandwidth 1000000
ip vrf forwarding labnetwork
ip address 192.168.185.2 255.255.255.0
no ip redirects
ip mtu 1400
no ip next-hop-self eigrp XXXX
no ip split-horizon eigrp XXXX
ip nhrp authentication XXXX
ip nhrp network-id 69
ip nhrp holdtime 450
ip nhrp redirect
ip virtual-reassembly in max-reassemblies 1000
no ip split-horizon
ip tcp adjust-mss 1365
delay 1000
tunnel source GigabitEthernet0/0.1
tunnel mode gre multipoint
tunnel key XXX
tunnel path-mtu-discovery
tunnel bandwidth transmit 1000000
tunnel bandwidth receive 1000000
tunnel protection ipsec profile ROME
end
Spoke config:
interface Tunnel1
bandwidth 1000000
bandwidth receive 100000
ip address 192.168.185.1 255.255.255.0
no ip redirects
ip mtu 1400
ip nat inside
ip nhrp authentication XXXX
ip nhrp map 192.168.185.2 158.93.X.X
ip nhrp map multicast 158.93.X.X
ip nhrp network-id 69
ip nhrp holdtime 450
ip nhrp nhs 192.168.185.2
ip nhrp shortcut
ip nhrp redirect
ip virtual-reassembly in
ip tcp adjust-mss 1365
delay 1000
tunnel source FastEthernet4
tunnel mode gre multipoint
tunnel key XXX
tunnel path-mtu-discovery
tunnel bandwidth transmit 100000
tunnel bandwidth receive 100000
tunnel protection ipsec profile ROME
Anyone has a solution the beer is on me the next time you're in Augusta, Ga.
Thanks!
Jerry
01-17-2020 03:05 PM - edited 01-17-2020 03:09 PM
Look at the thread and some Limitation :
https://community.cisco.com/t5/routing/50mb-cable-connection-slows-to-3mb-after-dmvpn/m-p/2317120
also attached router performance and support speeds
01-17-2020 03:27 PM
Thanks for the quick response. I fully expected a performance hit, but from 100 down to sub 3 Mb seems way out of line. In any case the speeds were virtually identical with the crypto removed. It does appear to have hardware acceleration:
crypto engine name: Virtual Private Network (VPN) Module
crypto engine type: hardware
State: Enabled
Location: onboard 0
Product Name: Onboard-VPN
FW Version: 1
Time running: 114174 seconds
Compression: Yes
DES: Yes
3 DES: Yes
AES CBC: Yes (128,192,256)
AES CNTR: No
Maximum buffer length: 4096
Maximum DH index: 0000
Maximum SA index: 0000
Maximum Flow index: 1000
Maximum RSA key size: 0000
I'd expect at least 10-15M with crypto.
Thanks!
01-18-2020 05:47 AM
@Georg Pauwen is much ahead of me to reply to you, start with the basic config of DMVPN, not additional fancy stuff start with and see if the performance increases, he suggested more no commands i would agree on wit that too.
let us know the outcome.
01-18-2020 12:11 AM
Hello,
a few things in your configuration don't look right, make the changes/additions marked in bold. Also, can you post the full configs of both the hub and the spoke ? We might be able to spot something else...
HUB
interface Tunnel1
--> no bandwidth 1000000
--> no ip vrf forwarding labnetwork
ip address 192.168.185.2 255.255.255.0
no ip redirects
ip mtu 1400
no ip next-hop-self eigrp XXXX
no ip split-horizon eigrp XXXX
ip nhrp authentication XXXX
--> ip nhrp map multicast dynamic
ip nhrp network-id 69
--> ip nhrp holdtime 300
ip nhrp redirect
ip virtual-reassembly in max-reassemblies 1000
no ip split-horizon
--> ip tcp adjust-mss 1360
--> no delay 1000
tunnel source GigabitEthernet0/0.1
tunnel mode gre multipoint
tunnel key XXX
--> no tunnel path-mtu-discovery
--> no tunnel bandwidth transmit 1000000
--> no tunnel bandwidth receive 1000000
tunnel protection ipsec profile ROME
end
Spoke
interface Tunnel1
--> no bandwidth 1000000
--> no bandwidth receive 100000
ip address 192.168.185.1 255.255.255.0
no ip redirects
ip mtu 1400
--> no ip nat inside
ip nhrp authentication XXXX
ip nhrp map 192.168.185.2 158.93.X.X
ip nhrp map multicast 158.93.X.X
ip nhrp network-id 69
--> ip nhrp holdtime 300
ip nhrp nhs 192.168.185.2
ip nhrp shortcut
--> no ip nhrp redirect
ip virtual-reassembly in
--> ip tcp adjust-mss 1360
--> no delay 1000
tunnel source FastEthernet4
tunnel mode gre multipoint
tunnel key XXX
--> no tunnel path-mtu-discovery
--> no tunnel bandwidth transmit 100000
--> no tunnel bandwidth receive 100000
tunnel protection ipsec profile ROME
01-18-2020 06:47 AM
Thank you- changes made, the only thing that I couldn't remove or lose connectivity from here was the vrf. Basically the same results, average was 2.54. Configs attached, minus passwords, etc. There's probably a few extraneous commands from old configs but there doesn't look like anything that would affect performance. I appreciate your looking at this.
BTW, the nhrp map multicast dynamic appears to be a default on this license, minus static mappings. When it's entered it doesn't show in the running config, but checking the mappings I see it's functioning:
AB-Lab-2911-Gateway#sh ip nhrp multicast
I/F NBMA address
Tunnel1 73.190.199.62 Flags: dynamic (Enabled)
Tunnel1 75.76.206.181 Flags: dynamic (Enabled)
01-18-2020 07:41 AM
Hello,
it is kind of hard to understand your topology. Your interface:
interface GigabitEthernet0/1
bandwidth 10000
ip vrf forwarding labnetwork
ip address 192.168.170.0 255.255.255.254
ip nat inside
ip virtual-reassembly in
duplex auto
speed auto
has a /31 mask, is that right ? Also, you have NAT enabled (both versions), what exactly are you trying to NAT ? Can you provide a schematic drawing of your topology, showing what is connected to what, and what is being routed where ?
01-18-2020 09:19 AM
Hello,
also, from which IP address to which IP address are you testing the connection ?
01-18-2020 12:43 PM
Hi Georg,
Attached is a quick drawing (.jpg, it wouldn't let me attach the Visio file) I put together. The lab changes quite a bit so I never have maintained one.
I think you're looking at the same thing that I've been thinking about, the VRF may be causing issues although it shouldn't and traffic/cpu/crypto/CEF all looks very clean.
Tentative plan is to move the hub from the main lab to the other public IP (here) and swap the 881 for a 2921 I pulled out of mothballs this morning. That way I can easily test the tunnel speed from the other spoke without impacting other lab testing.
It'll probably be Monday or Tues but I'll update on the results. I'd feel better if I could see some dropped packets or debug errors, but nothing- it's just slow. Maybe the VRF is causing issues. Thanks!
01-18-2020 10:48 AM
Thank for the making changes and reported back
you were mentioned, before you deploy a VPN, you able to get real bandwidth as per your expectation.
after deploying DMVPN you have a poor performance like 3mb max ? is this correct?
01-18-2020 01:00 PM
Hi,
You're correct. I know that with a lower end ISR as a spoke I can expect a performance hit, but 90+ % is a little extreme. I should , with this router and low end IPSEC, be getting at least 10-15M.
Thanks!
01-18-2020 10:00 AM
01-18-2020 12:58 PM
Hi Joseph,
Thanks for the reply.
Spoke bandwidth is less, it's commercial internet so runs around 75-80% of 100M but the traffic is pretty light, I've never seen it approach even 50 % of capacity. The hub is true bandwidth, it's connected to a 3gig connection provided by the state. No internet traffic traverses the tunnels, all are split tunneled w/ internet traffic going out locally- only 192.168 traffic goes across the tunnels. UDP traffic is minimal now, only when we do backups of equipment or something like that, although we do plan on testing some voice and video multicast later this spring.
I can probably get rid of the crypto ipsec df-bit clear, at one time I had a fragmentation problem but fixed that. I'll dump that when I start testing again. I probably don't need the tunnel keys w/ the ipsec but it's an organizational standard.
Thanks,
Jerry
01-19-2020 07:03 AM
Hello,
for the sake of testing, you might also want to try a different encryption (on both ends obviously):
crypto isakmp policy 10
encr aes
hash sha256
authentication pre-share
group 5
!
crypto ipsec transform-set LAB esp-aes 256 esp-sha256-hmac
mode tunnel
01-19-2020 09:10 AM
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide