2811 DMPVN-GRE-IPSEC resolving mtu problems

Kevin B · ‎09-18-2014

We are municipality IT which at this point do most Cisco work in house. However our VPN experience with Cisco is limited. This project involves replacing Linux boxes running OPENVPN with Cisco 2811 routers to a few temporary/small locations were site fiber or PTP wireless is not available or cost effective. Our need is a core vpn (presently 2811) unit at the data center (comcast 200 mb internet) to receive multiple site (3-5) routers (2811) with cable business comcast. We would like the capability for dynamic site IPs if possible. All traffic from sites to core is 100% bridged L3, so internet traffic passes through content filtering.

We began with a DMVPN-GRE-IPSEC tutorial(s) and our present sandbox is static routed with default routes to the internal network, ISP route is static. Eventually plan to move to policy based routing so dynamic IP ISP services can be used.

Our sandbox tunnel is up and passing traffic. However an initial indication of problem(s) is we find certain (complex) websites like cnn.com load terribly or time out, simple low content flat websites or video streaming are not a problem. This we've seen is a common indication of the need to adjust mtu settings. After research we have tried various mtu settings/options, on various interfaces without success.

CORE VPN ROUTER

crypto isakmp policy 1
encr aes
authentication pre-share
group 2
crypto isakmp key <psk-text> address 0.0.0.0 0.0.0.0

crypto ipsec transform-set TS esp-aes esp-sha-hmac
crypto ipsec df-bit clear

crypto ipsec profile PROTECT-GRE
set security-association lifetime seconds 86400
set transform-set TS

interface Tunnel0
description mGRE-DMVPN Tunnel
ip address 10.8.8.1 255.255.255.0
no ip redirects
ip nhrp authentication apremont
ip nhrp map multicast dynamic
ip nhrp network-id 1
tunnel source <core external IP>
tunnel mode gre multipoint
tunnel protection ipsec profile PROTECT-GRE

interface FastEthernet0/0
description ** ISP Uplink **
ip address <core external IP> <core external mask>
duplex auto
speed auto

interface FastEthernet0/1
description ** Uplink to LAN **
ip address <ip> 255.255.255.0
duplex auto
speed auto

ip forward-protocol nd
ip route 0.0.0.0 0.0.0.0 <LAN IP>
ip route 10.190.0.0 255.255.0.0 10.8.8.2

SITE VPN ROUTER

crypto isakmp policy 1
encr aes
authentication pre-share
group 2
crypto isakmp key <psk-text> address 0.0.0.0 0.0.0.0

crypto ipsec transform-set TS esp-aes esp-sha-hmac
crypto ipsec df-bit clear

crypto ipsec profile PROTECT-GRE
set security-association lifetime seconds 86400
set transform-set TS

interface Tunnel0
description mGRE-DMVPN Tunnel
ip address 10.8.8.2 255.255.255.0
no ip redirects
ip nhrp authentication apremont
ip nhrp map multicast dynamic
ip nhrp map 10.8.8.1 <core external IP>
ip nhrp map multicast <core external IP>
ip nhrp network-id 1
ip nhrp nhs 10.8.8.1
tunnel source FastEthernet0/0
tunnel mode gre multipoint
tunnel protection ipsec profile PROTECT-GRE

interface FastEthernet0/0
description ** External Interface ***
ip address <ISP externalIP> <ISP external mask>
duplex auto
speed auto

interface FastEthernet0/0/0
description ** workstation subnet**
switchport access vlan 513
spanning-tree portfast

interface Vlan513
description admin-wired
ip address 10.190.13.1 255.255.255.0

ip forward-protocol nd
ip route 0.0.0.0 0.0.0.0 10.8.8.1
ip route <core vpn external ip> 255.255.255.255 <site isp gateway>

Any pointer or assistance would be greatly appreciated.

Kevin B · ‎09-22-2014

If I may, bump.

Marcin Latosiewicz · ‎09-25-2014

Kevin,

Clearing DF bit is a way to kill performance - essentially allowing fragmentation.

I would rather use tunnel path mtu discovery - which indeed might cause problem with some ISPs who would not send ICMP unreachables - with MTU and MSS settings.

We typically start with MTU of 1400 and MSS of 1360. But you might need to check/test what the actual MTU on the DMVPN path is.

M.

Kevin B · ‎09-25-2014

Thanks for the reply Marcin,

I'll try removing crypto ipsec df-bit clear and see what we get. This command was part of the tutorial we started with.

I did some testing of the MTU path, it was unclear which portions should we test against. Should the ping be to the far VPN interface IP, or say from a client to our gateway. Any when we applied the mtu settings we got no appreciable results on different interfaces. What interface do you usually apply the mtu adjustments?

Marcin Latosiewicz · ‎09-25-2014

Kevin,

MTU and MSS (together) on tunnel interface.

Ping between the tunnel IP addresses to begin with - make sure DF-bit is set and that you do not clear it :-)

For most providers we do not need to lower much below 1400 bytes ... :-)

M.

Kevin B · ‎09-25-2014

So I ping'ed (ping -f -l value) from a client off the site router to the IP of the far tunnel interface and found I was fine starting at 1400 to 1402, then no response from 1403 1420, then responses and not until about 1450 would I actually get the fragmentation warnings.

I then removed the crypto ipsec df-bit clear from each router, and from each tunnel 0 added ip mtu 1400 and ip tcp adjust-mss 1360 as a starting point and testing browsing on my client I now get excellent results for browsing.

I know I had worked with the mtu and mss settings before but perhaps I had them on the wrong interfaces, and certainly the df-bit setting still on.

I am getting different throughput however depending on traffic "source" when copying files in Windows environment. I can transfer a file very well from a core client to a remote client (5 mb sec), but the reverse (remote client to core client) is terribly slow (200 kb sec). Between these two (with mtu 1400 set on routers) I get fragmented above 1370.

Marcin Latosiewicz · ‎09-26-2014

Kevin,

Try removing DF-bit clearing and use tunnel path MTU discovery feature (both tunnel interfaces).

And MSS/MTU across the board.

Also when checking MTU, make sure whether length (-l flag) is length of data or entire packet, if it's data, remember that you need to add 20 bytes of IP header and 8 bytes for ICMP header.

M.

Kevin B · ‎09-26-2014

Marcin, thanks again for your help and attention, please bare with me as we've definitely hit my knowledge limit here.

> Try removing DF-bit clearing and use tunnel path MTU discovery feature (both tunnel interfaces).

I had removed the crypto ipsec df-bit clear command from both routers, so I'm not sure what you mean by this.

> And MSS/MTU across the board.

Do you mean apply the 1400 mtu and mss 1360 settings to each interface? To the ISP interface, tunnel int (already in place), the L3 LAN interfaces?

>Also when checking MTU, make sure whether length (-l flag) is length of data or entire packet, if it's data, remember that you need to add 20 bytes of IP header and 8 bytes for ICMP header.

I'm not sure how I would know the length of the packet. I had thought when testing you found your value, then adjusted for overhead and apply that final number as your MTU and MSS values.

Marcin Latosiewicz · ‎09-26-2014

Kevin,

This is a good read for you:

http://www.cisco.com/c/en/us/support/docs/ip/generic-routing-encapsulation-gre/25885-pmtud-ipfrag.html#intro

A bit outdated, but that's probably the best there is a at the moment.

But it does contain some information about tunnel path MTU discovery.

When adjusting MTU and MSS I only meant tunnel interfaces.

Regarding calculating.

If you sent a packet with 1400 bytes of data, the end packet ends up being 1400 + 20 (IP header) + 8 (ICMP header) bytes long.

Depending which tools you're using you would either send 1400 bytes of data or 1400 byte packet. That's what I was saying to be sensitive about.

So start by setting tunnel path MTU discovery and re-do the test :-)

M.