Re: MTU Size issue?

KGrev · ‎04-19-2022

Hi, I'm trying to resolve an issue im having with a vpn connection to a mobile LTE Router (IR 809G)

Normally the connection and throughput is fine, however i'm being required at add an encryption device to one router to form a link to another encryption device inside out network.

Throughput of lte routers normally = many megs
throughput through encryption device through lte router = 56k hard lock
throughput of encryption devices connected to switch = 100meg

I'm wondering if im having an mtu issue pushing a tunnel through a vpn on the lte router.

When I send a ping with df-bit from LTE router (not through encryptor device) it begins to fragment at 1439.

Router vpn terminates to an ASA also.

I've dropped the mtu on the encryption devices well below this but have not noticed a change.

Any ideas?

jamesduv9 · ‎04-19-2022

Can you provide a rough diagram or running configuration of the devices? I don't think we have enough information here to give any good advice.

I'm assuming your topology looks like - VPN SPOKE > INLINE ENCRYPTOR > INLINE ENCRYPTOR > VPN HUB

KGrev · ‎04-19-2022

It looks more like this. Sorry, i'm unable to get any config files as of yet due to the nature of the system.

I would gladly look up anything you have questions about.

jamesduv9 · ‎04-19-2022

Thanks for the diagram. You should also be able to set the MTU size on the encryptor itself. I would refer to the vendors documentation on how much overhead is required, and subtract that from your LTE router's tunnel MTU. I believe a popular INE vendor suggests 50 bytes.

KGrev · ‎04-19-2022

Config for LTE Router

Building configuration...

Current configuration : 19109 bytes
!
! Last configuration change at 14:24:33 UTC Tue Apr 19 2022 by local_login
!
version 15.8
no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
service internal
service sequence-numbers
no service dhcp
!
hostname CS016-PRB1
!
boot-start-marker
boot-end-marker
!
!
security passwords min-length 10
enable secret 5 XXXXXXX
!
aaa new-model
!
!
aaa group server radius ABC-RADIUS
server name I-NPS-01
server name I-NPS-02
ip radius source-interface Loopback0
!
aaa authentication login ABC-AUTH group ABC-RADIUS local
aaa authentication enable default enable
aaa authorization exec ABC-AUTHO group ABC-RADIUS local
!
!
!
!
!
aaa session-id common
!
!
!
!
!
!
no ip source-route
no ip gratuitous-arps
!
!
!
!
!
!
!
!
!
!
!
!

!
!
!
!
no ip bootp server
ip domain lookup source-interface Loopback0
ip domain name ABCis.local
ip name-server X.Y..219.10
ip name-server X.Y..219.11
ip inspect WAAS flush-timeout 10
ip cef
login block-for 60 attempts 5 within 60
login on-failure log
login on-success log
no ipv6 cef
!
multilink bundle-name authenticated
!
!
!
chat-script lte "" "AT!CALL" TIMEOUT 20 "OK"
password encryption aes
!
crypto pki trustpoint ABC_NET_Trust
enrollment terminal
serial-number
ip-address loopback0
revocation-check crl
!
!
*************CRYPTO******************
!
archive
log config
record rc
logging enable
logging size 500
notify syslog contenttype plaintext
hidekeys
object-group network IA-ADMIN-ADDRESS
description IP addresses of IA admin boxes
host X.X2.231
host X.X2.232
host X.X2.235
host X.X2.96
!
object-group service IPSLA-SERVICES
description Ports used for IPSLA testing
udp eq 1967
udp eq 17000
!
object-group service IPv6_TRAFFIC_FILTER_NET_TUNL_001
description Ports used in outdated tunneling schemes
42
93
nos
97
98
udp eq 1723
tcp eq 1723
60
!
object-group network MANAGEMENT-ADDRESSES
description IP ranges of management devices
X.X242.0 255.255.255.0
X.X243.0 255.255.255.0
X.X102.0 255.255.255.0
host X.X0.114
host 10.200.252.101
X.Y..219.0 255.255.255.0
!
object-group service MANAGEMENT-SERVICES
description Ports used for network management
udp eq snmp
tcp eq 22
icmp
udp eq syslog
!
object-group network NTP-SERVERS
description IP Addresses of NTP servers
host X.X102.5
host X.X102.6
!
object-group network RADIUS-SERVERS
description IP Address of radius servers
host X.Y..219.10
host X.Y..219.11
!
object-group service RADIUS-SERVICES
description Ports used for radius servers
udp eq 1645
udp eq 1646
!
object-group service VPN-SERVICES
description VPN traffic
udp eq isakmp
esp
!
vtp mode transparent
************USERNAMES*******************
!
redundancy
notification-timer 120000

!
!
!
!
!
controller Cellular 0
lte sim fast-switchover enable
lte failovertimer 5
no cdp run
!
ip tcp synwait-time 10
!
class-map match-all CoPP_UNDESIRABLE
match access-group name CoPP_UNDESIRABLE
class-map match-any CoPP_IMPORTANT
match access-group name CoPP_IMPORTANT
match protocol arp
class-map match-all CoPP_DEFAULT
match access-group name CoPP_DEFAULT
class-map match-all CoPP_NORMAL
match access-group name CoPP_NORMAL
class-map match-all CoPP_CRITICAL
match access-group name CoPP_CRITICAL
!
policy-map CONTROL_PLANE_POLICY
class CoPP_CRITICAL
police 512000 8000 conform-action transmit exceed-action transmit
class CoPP_IMPORTANT
police 512000 4000 conform-action transmit exceed-action drop
class CoPP_NORMAL
police 128000 2000 conform-action transmit exceed-action drop
class CoPP_UNDESIRABLE
police 8000 1000 conform-action drop exceed-action drop
class CoPP_DEFAULT
police 64000 1000 conform-action transmit exceed-action drop
!
!
!
crypto isakmp policy 1
encr aes 256
authentication pre-share
group 2
!
!
!
!
!
!
crypto ipsec client ezvpn RMCSPROBE
connect auto
group RMCS_BitProbe key 6 QM[D[\gLHEfSPVTgQaYgICbAZOEAAB
mode network-extension
peer X.X0.114
virtual-interface 2
username rmcsprobe-sec password 6 _dcZM^A^YFLQ`HKfALOAgPRP\faV[O^XAK_ZI]iIgYAAB
xauth userid mode local
!
!
!
!
!
!
interface Loopback0
ip address X.X9.32 255.255.255.254
no ip redirects
no ip unreachables
no ip proxy-arp
ip flow ingress
ip flow egress
crypto ipsec client ezvpn RMCSPROBE inside
!
interface GigabitEthernet0
ip address X.X244.1 255.255.255.252
duplex auto
speed auto
no cdp enable
no keepalive
crypto ipsec client ezvpn RMCSPROBE inside
!
interface GigabitEthernet1
no ip address
duplex auto
speed auto
no cdp enable
!
interface GigabitEthernet2
no ip address
duplex auto
speed auto
!
interface Cellular0
ip address negotiated
no ip redirects
no ip unreachables
no ip proxy-arp
encapsulation slip
dialer in-band
dialer idle-timeout 0
dialer string lte
dialer watch-group 1
async mode interactive
crypto ipsec client ezvpn RMCSPROBE
!
interface Cellular1
no ip address
encapsulation slip
shutdown
!
interface Virtual-Template2 type tunnel
ip unnumbered Cellular0
ip access-group ACL-INFRASTRUCTURE-IN in
ip access-group ACL-INFRASTRUCTURE-OUT out
tunnel mode ipsec ipv4
!
interface Async0
no ip address
encapsulation scada
!
interface Async1
no ip address
encapsulation scada
!
!
ip forward-protocol nd
!
no ip http server
no ip http secure-server
ip http client source-interface Loopback0
ip flow-export source Loopback0
ip flow-export version 5
ip flow-export destination X.Y..219.41 2055
!
ip ftp source-interface Loopback0
ip tftp source-interface Loopback0
ip route X.X0.114 255.255.255.255 Cellular0
ip ssh time-out 60
ip ssh version 2
ip ssh server algorithm mac hmac-sha1 hmac-sha1-96
ip ssh server algorithm encryption aes128-cbc aes192-cbc aes256-cbc
ip scp server enable
!
******************ACLs********************
!
ip radius source-interface Loopback0
ip sla responder
ip sla 10
icmp-echo X.Y..219.41 source-interface Loopback0
threshold 2000
frequency 30
ip sla enable reaction-alerts
logging facility local2
logging source-interface Loopback0
logging host X.Y..219.31
dialer watch-list 1 ip 5.6.7.8 0.0.0.0
dialer watch-list 1 delay route-check initial 60
dialer watch-list 1 delay connect 1
ipv6 ioam timestamp
!
!
************SNMP**************************

radius server I-NPS-01
address ipv4 X.Y..219.10 auth-port 1645 acct-port 1646
key 6 NYUDHbXaIJWb`NaTHVei^RBi^HhSHIbTGVMRdeOfcaaJDFcXEMMAAB
!
radius server I-NPS-02
address ipv4 X.Y..219.11 auth-port 1645 acct-port 1646
key 6 ^Id_MA`IAWIc]BWBMhgfNSHWKZ`gL^ODNBYaLB[]G\^JOBiZK]VAAB
!
!
!
control-plane
service-policy input CONTROL_PLANE_POLICY
!
!
banner login ^CCCC
********************BANNER**************
^C
!
line con 0
exec-timeout 5 0
logging synchronous
login authentication ABC-AUTH
transport preferred ssh
transport output ssh
stopbits 1
line 1
stopbits 1
line 2
no activation-character
no exec
transport preferred ssh
transport input ssh
stopbits 1
line 3
exec-timeout 0 0
script dialer lte
modem InOut
no exec
transport preferred lat pad telnet rlogin lapb-ta mop udptn v120 ssh
transport output lat pad telnet rlogin lapb-ta mop udptn v120 ssh
rxspeed 100000000
txspeed 50000000
line 8
no exec
transport preferred lat pad telnet rlogin lapb-ta mop udptn v120 ssh
transport output lat pad telnet rlogin lapb-ta mop udptn v120 ssh
rxspeed 100000000
txspeed 50000000
line 1/3 1/6
transport preferred none
transport output none
stopbits 1
line vty 0 4
access-class Mgmt_Access in
exec-timeout 5 0
authorization exec ABC-AUTHO
logging synchronous
login authentication ABC-AUTH
transport preferred ssh
transport input ssh
transport output ssh
!
no scheduler max-task-time
ntp authentication-key 1 md5 011012075218494E117E6B5B 7
ntp authenticate
ntp trusted-key 1
ntp source Loopback0
ntp access-group peer NTP-SERVERS
ntp update-calendar
ntp server X.X102.5 key 1
ntp server X.X102.6 key 1 prefer
no iox hdm-enable
iox client enable interface GigabitEthernet2
no iox recovery-enable
!
!
!
!
!
!
!
end

KGrev · ‎04-19-2022

James, Thank you for your response.

Ive dropped the MTU down on the encryption devices as low as 1300 with no change.

MHM Cisco World · ‎04-19-2022

Nice draw

MHM Cisco World · ‎04-19-2022

Please can You more elaborate about
LTE-ASA VPN is OK
Encrypt/Decrypt -LTE-ASA-Encrypt/Decrypt <- here is issue ??

KGrev · ‎04-19-2022

MHM, thank you for your response.

IF I am understanding you correctly, Yes, the LTE router vpn is working ok and normal on its own. I am able to plug a laptop into is and access the inner network and transfer files at expected speeds.

The only time I'm having the 56K speeds is when I use the encryption devices at each end of the link.

On their own with just a switch between them, they opperate at full speeds. But when I try to use them over the LTE router link, I get the slow speeds.

My suspicioun has been that possibly I'm having an mtu problem when I push the tunnel of the encryption devices inside of the vpn the LTE devices use. But I dont have much evidence to support that as of yet. Without the encryption devices, I can send pings without fragmentation at 1438 mtu size. When I lower the mtu of the encryption devices down well below this, there is no change so far. I could be looking in the wrong place.

KGrev · ‎04-19-2022

More information,

Here is a picture from the laptop sending pings till the MTU is too high. It drops of at 1397 while connected to the encryption devices.

MHM Cisco World · ‎04-19-2022

So You get the solution, reduce the MTU to be 1397

1439-1397 = 42-44 bytes and that OK if you use GRE/IPSec or L2TP/IPSec.

note that new header for encrypt vpn add to original data.

KGrev · ‎04-19-2022

At which interface should I put this change, I have a few options listed above.

And should I just change the mtu or use tcp-mss adjust instead?

David Ruess · ‎04-19-2022

Hello,

Can you post the interface configs on both sides along with any logs you get as it pertains to the fragmentation? If you're going through a tunnel and encryption you may need to reduce it more along with the tcp-mss. See below

int <int#>

mtu 1400

ip tcp adjust-mss 1360 <- should be 40 less than mtu as good practice

-David

KGrev · ‎04-19-2022

So, nothing so far has reported that there is a fragmentation issue.

The encryptor devices may be having the issue but their logging is pretty empty.

For adjusting the tcp-mss, ive been experimenting in a few different places on where to add that but unsure.

The LTE router has:

G0 - encryptor device port

Virtual-access2 - the template for the vpn

Cellular 0 - the interface it sends to cellular

I can add tcp-mss adjust to each of those or set mtu to a certain size. Unsure which one would need it.

By default the Virtual-access2 mtu says its around 17000, which is interesting. I assume thats something like a virtual link inside the router?

I can adjust the group policy for the vpn at the ASA and change mtu size.

KGrev · ‎04-19-2022

LTE Router Config

Building configuration...

Current configuration : 19109 bytes
!
! Last configuration change at 14:24:33 UTC Tue Apr 19 2022 by local_login
!
version 15.8
no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
service internal
service sequence-numbers
no service dhcp
!
hostname CS016-PRB1
!
boot-start-marker
boot-end-marker
!
!
security passwords min-length 10
enable secret 5 XXXXXXX
!
aaa new-model
!
!
aaa group server radius ABC-RADIUS
server name I-NPS-01
server name I-NPS-02
ip radius source-interface Loopback0
!
aaa authentication login ABC-AUTH group ABC-RADIUS local
aaa authentication enable default enable
aaa authorization exec ABC-AUTHO group ABC-RADIUS local
!
!
!
!
!
aaa session-id common
!
!
!
!
!
!
no ip source-route
no ip gratuitous-arps
!
!
!
!
!
!
!
!
!
!
!
!

!
!
!
!
no ip bootp server
ip domain lookup source-interface Loopback0
ip domain name ABCis.local
ip name-server X.Y..219.10
ip name-server X.Y..219.11
ip inspect WAAS flush-timeout 10
ip cef
login block-for 60 attempts 5 within 60
login on-failure log
login on-success log
no ipv6 cef
!
multilink bundle-name authenticated
!
!
!
chat-script lte "" "AT!CALL" TIMEOUT 20 "OK"
password encryption aes
!
crypto pki trustpoint ABC_NET_Trust
enrollment terminal
serial-number
ip-address loopback0
revocation-check crl
!
!
*************CRYPTO******************
!
archive
log config
record rc
logging enable
logging size 500
notify syslog contenttype plaintext
hidekeys
object-group network IA-ADMIN-ADDRESS
description IP addresses of IA admin boxes
host X.X2.231
host X.X2.232
host X.X2.235
host X.X2.96
!
object-group service IPSLA-SERVICES
description Ports used for IPSLA testing
udp eq 1967
udp eq 17000
!
object-group service IPv6_TRAFFIC_FILTER_NET_TUNL_001
description Ports used in outdated tunneling schemes
42
93
nos
97
98
udp eq 1723
tcp eq 1723
60
!
object-group network MANAGEMENT-ADDRESSES
description IP ranges of management devices
X.X242.0 255.255.255.0
X.X243.0 255.255.255.0
X.X102.0 255.255.255.0
host X.X0.114
host 10.200.252.101
X.Y..219.0 255.255.255.0
!
object-group service MANAGEMENT-SERVICES
description Ports used for network management
udp eq snmp
tcp eq 22
icmp
udp eq syslog
!
object-group network NTP-SERVERS
description IP Addresses of NTP servers
host X.X102.5
host X.X102.6
!
object-group network RADIUS-SERVERS
description IP Address of radius servers
host X.Y..219.10
host X.Y..219.11
!
object-group service RADIUS-SERVICES
description Ports used for radius servers
udp eq 1645
udp eq 1646
!
object-group service VPN-SERVICES
description VPN traffic
udp eq isakmp
esp
!
vtp mode transparent
************USERNAMES*******************
!
redundancy
notification-timer 120000

!
!
!
!
!
controller Cellular 0
lte sim fast-switchover enable
lte failovertimer 5
no cdp run
!
ip tcp synwait-time 10
!
class-map match-all CoPP_UNDESIRABLE
match access-group name CoPP_UNDESIRABLE
class-map match-any CoPP_IMPORTANT
match access-group name CoPP_IMPORTANT
match protocol arp
class-map match-all CoPP_DEFAULT
match access-group name CoPP_DEFAULT
class-map match-all CoPP_NORMAL
match access-group name CoPP_NORMAL
class-map match-all CoPP_CRITICAL
match access-group name CoPP_CRITICAL
!
policy-map CONTROL_PLANE_POLICY
class CoPP_CRITICAL
police 512000 8000 conform-action transmit exceed-action transmit
class CoPP_IMPORTANT
police 512000 4000 conform-action transmit exceed-action drop
class CoPP_NORMAL
police 128000 2000 conform-action transmit exceed-action drop
class CoPP_UNDESIRABLE
police 8000 1000 conform-action drop exceed-action drop
class CoPP_DEFAULT
police 64000 1000 conform-action transmit exceed-action drop
!
!
!
crypto isakmp policy 1
encr aes 256
authentication pre-share
group 2
!
!
!
!
!
!
crypto ipsec client ezvpn RMCSPROBE
connect auto
group RMCS_BitProbe key 6 QM[D[\gLHEfSPVTgQaYgICbAZOEAAB
mode network-extension
peer X.X0.114
virtual-interface 2
username rmcsprobe-sec password 6 _dcZM^A^YFLQ`HKfALOAgPRP\faV[O^XAK_ZI]iIgYAAB
xauth userid mode local
!
!
!
!
!
!
interface Loopback0
ip address X.X9.32 255.255.255.254
no ip redirects
no ip unreachables
no ip proxy-arp
ip flow ingress
ip flow egress
crypto ipsec client ezvpn RMCSPROBE inside
!
interface GigabitEthernet0
ip address X.X244.1 255.255.255.252
duplex auto
speed auto
no cdp enable
no keepalive
crypto ipsec client ezvpn RMCSPROBE inside
!
interface GigabitEthernet1
no ip address
duplex auto
speed auto
no cdp enable
!
interface GigabitEthernet2
no ip address
duplex auto
speed auto
!
interface Cellular0
ip address negotiated
no ip redirects
no ip unreachables
no ip proxy-arp
encapsulation slip
dialer in-band
dialer idle-timeout 0
dialer string lte
dialer watch-group 1
async mode interactive
crypto ipsec client ezvpn RMCSPROBE
!
interface Cellular1
no ip address
encapsulation slip
shutdown
!
interface Virtual-Template2 type tunnel
ip unnumbered Cellular0
ip access-group ACL-INFRASTRUCTURE-IN in
ip access-group ACL-INFRASTRUCTURE-OUT out
tunnel mode ipsec ipv4
!
interface Async0
no ip address
encapsulation scada
!
interface Async1
no ip address
encapsulation scada
!
!
ip forward-protocol nd
!
no ip http server
no ip http secure-server
ip http client source-interface Loopback0
ip flow-export source Loopback0
ip flow-export version 5
ip flow-export destination X.Y..219.41 2055
!
ip ftp source-interface Loopback0
ip tftp source-interface Loopback0
ip route X.X0.114 255.255.255.255 Cellular0
ip ssh time-out 60
ip ssh version 2
ip ssh server algorithm mac hmac-sha1 hmac-sha1-96
ip ssh server algorithm encryption aes128-cbc aes192-cbc aes256-cbc
ip scp server enable
!
******************ACLs********************
!
ip radius source-interface Loopback0
ip sla responder
ip sla 10
icmp-echo X.Y..219.41 source-interface Loopback0
threshold 2000
frequency 30
ip sla enable reaction-alerts
logging facility local2
logging source-interface Loopback0
logging host X.Y..219.31
dialer watch-list 1 ip 5.6.7.8 0.0.0.0
dialer watch-list 1 delay route-check initial 60
dialer watch-list 1 delay connect 1
ipv6 ioam timestamp
!
!
************SNMP**************************

radius server I-NPS-01
address ipv4 X.Y..219.10 auth-port 1645 acct-port 1646
key 6 NYUDHbXaIJWb`NaTHVei^RBi^HhSHIbTGVMRdeOfcaaJDFcXEMMAAB
!
radius server I-NPS-02
address ipv4 X.Y..219.11 auth-port 1645 acct-port 1646
key 6 ^Id_MA`IAWIc]BWBMhgfNSHWKZ`gL^ODNBYaLB[]G\^JOBiZK]VAAB
!
!
!
control-plane
service-policy input CONTROL_PLANE_POLICY
!
!
banner login ^CCCC
********************BANNER**************
^C
!
line con 0
exec-timeout 5 0
logging synchronous
login authentication ABC-AUTH
transport preferred ssh
transport output ssh
stopbits 1
line 1
stopbits 1
line 2
no activation-character
no exec
transport preferred ssh
transport input ssh
stopbits 1
line 3
exec-timeout 0 0
script dialer lte
modem InOut
no exec
transport preferred lat pad telnet rlogin lapb-ta mop udptn v120 ssh
transport output lat pad telnet rlogin lapb-ta mop udptn v120 ssh
rxspeed 100000000
txspeed 50000000
line 8
no exec
transport preferred lat pad telnet rlogin lapb-ta mop udptn v120 ssh
transport output lat pad telnet rlogin lapb-ta mop udptn v120 ssh
rxspeed 100000000
txspeed 50000000
line 1/3 1/6
transport preferred none
transport output none
stopbits 1
line vty 0 4
access-class Mgmt_Access in
exec-timeout 5 0
authorization exec ABC-AUTHO
logging synchronous
login authentication ABC-AUTH
transport preferred ssh
transport input ssh
transport output ssh
!
no scheduler max-task-time
ntp authentication-key 1 md5 011012075218494E117E6B5B 7
ntp authenticate
ntp trusted-key 1
ntp source Loopback0
ntp access-group peer NTP-SERVERS
ntp update-calendar
ntp server X.X102.5 key 1
ntp server X.X102.6 key 1 prefer
no iox hdm-enable
iox client enable interface GigabitEthernet2
no iox recovery-enable
!
!
!
!
!
!
!
end