I am experiencing packet loss when pinging a device from the hub router (10.0.0.1 - Network IP, 10.255.255.254) to a device that is behind a remote spoke router (10.0.4.21 - basically any device on that network excluding the remote gateway IP), however, I can ping the remote gateway (10.0.4.1) and spoke tunnel IP (10.255.255.4) without any issue. I am using DMVPN and OSPF. Pinging from the remote spoke router to the hub or devices directly connected (10.0.0.25 etc) to the hub router experience no packet loss. Spoke routers are on cellular cards. Also, yes I can ping without issue from the local router (10.0.4.1) to the device (10.0.4.21)
It does not appear that the VPN or OSPF is flapping, as both have decent uptimes. If you need any further information or command outputs, just let me know.
Text Network Diagram (device IP's are exemplar):
[Device 10.0.0.25]<->[Hub Router 10.0.0.1/24 LAN, 10.255.255.254/24 Hub Tunnel IP, x.x.x.x/24 WAN IP]<->INTERNET<->[Spoke Router 10.0.4.1/24 LAN, 10.255.255.4/24 Spoke Tunnel IP, CELL IP]<->[Device 10.0.4.21]
Running configs:
Hub 1921 with c1900-universalk9-mz.SPA.156-2.T1.bin:
Management_Hub#sh run
Building configuration...
Current configuration : 4277 bytes
!
! Last configuration change at 23:52:50 UTC Tue Jun 13 2017 by *****
! NVRAM config last updated at 23:52:51 UTC Tue Jun 13 2017 by *****
!
version 15.6
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
!
hostname Management_Hub
!
boot-start-marker
boot-end-marker
!
!
enable secret 5 **********
!
no aaa new-model
ethernet lmi ce
!
!
!
!
!
!
!
!
!
!
!
!
ip domain name **********
ip name-server 8.8.8.8
ip name-server 8.8.4.4
ip cef
login quiet-mode access-class RTR_MGMT
login on-failure log
login on-success log
no ipv6 cef
!
multilink bundle-name authenticated
!
!
!
license udi pid CISCO1921/K9 sn **********
!
!
vtp mode transparent
username ****************
!
redundancy
!
!
!
!
no cdp run
!
!
crypto keyring newkeyring
pre-shared-key address 0.0.0.0 0.0.0.0 key ********
!
!
crypto ipsec transform-set ********* esp-3des esp-sha-hmac
mode transport
!
crypto ipsec profile *********
set transform-set ********
!
!
!
!
!
!
!
interface Tunnel0
bandwidth 10000
ip address 10.255.255.254 255.255.255.0
no ip redirects
ip mtu 1408
ip nhrp authentication ****
ip nhrp map multicast dynamic
ip nhrp network-id 1772778
ip nhrp holdtime 300
ip virtual-reassembly in
ip tcp adjust-mss 574
ip ospf network broadcast
ip ospf priority 255
delay 1000
qos pre-classify
tunnel source GigabitEthernet0/0
tunnel mode gre multipoint
tunnel key 1772778
tunnel path-mtu-discovery
tunnel protection ipsec profile ****** shared
!
interface Embedded-Service-Engine0/0
no ip address
shutdown
!
interface GigabitEthernet0/0
ip address ****** 255.255.255.0
ip nat outside
ip virtual-reassembly in
duplex auto
speed auto
!
interface GigabitEthernet0/1
ip address 10.0.0.1 255.255.255.0
ip nat inside
ip virtual-reassembly in
duplex auto
speed auto
no mop enabled
!
interface GigabitEthernet0/0/0
no ip address
!
interface GigabitEthernet0/0/1
no ip address
!
interface GigabitEthernet0/0/2
no ip address
!
interface GigabitEthernet0/0/3
no ip address
!
interface Vlan1
no ip address
shutdown
!
router ospf 10
router-id 10.255.255.254
redistribute static subnets
network 10.0.0.0 0.0.0.255 area 0
network 10.255.255.0 0.0.0.255 area 0
!
ip forward-protocol nd
!
no ip http server
no ip http secure-server
!
ip nat inside source list NATout-Acl interface GigabitEthernet0/0 overload
ip nat inside source static tcp 10.0.0.25 80 ***** 80 extendable
ip nat inside source static udp 10.0.0.25 80 ***** 80 extendable
ip nat inside source static tcp 10.0.0.25 3389 ***** 3389 extendable
ip nat inside source static udp 10.0.0.25 3389 ***** 3389 extendable
ip nat inside source static tcp 10.0.0.30 80 ***** 10000 extendable
ip nat inside source static udp 10.0.0.30 80 ***** 10000 extendable
ip nat inside source static tcp 10.0.0.30 443 ***** 10001 extendable
ip nat inside source static udp 10.0.0.30 443 ***** 10001 extendable
ip route 0.0.0.0 0.0.0.0 82.145.42.1
ip ssh time-out 60
ip ssh authentication-retries 2
ip ssh source-interface GigabitEthernet0/0
ip ssh version 2
!
ip access-list standard RTR_MGMT
permit *****
permit *****
permit 10.0.0.0 0.255.255.255
!
ip access-list extended NATout-Acl
permit ip 10.0.0.0 0.0.0.255 any
!
!
!
snmp-server community ***** RO
!
control-plane
!
!
!
line con 0
line aux 0
line 2
no activation-character
no exec
transport preferred none
transport output pad telnet rlogin lapb-ta mop udptn v120 ssh
stopbits 1
line vty 0 4
password 7 *****
logging synchronous
login local
transport input telnet ssh
line vty 5 15
login local
transport input telnet ssh
!
scheduler allocate 20000 1000
ntp peer 204.2.134.163
ntp peer 208.79.16.124
ntp peer 50.116.38.157
ntp peer 69.65.40.29
!
end
Management_Hub#
Spoke 809 with ir800-universalk9-mz.SPA.156-3.M2:
Cedars#sh run
Building configuration...
Current configuration : 5980 bytes
!
! No configuration change since last restart
!
version 15.6
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
!
hostname Cedars
!
boot-start-marker
boot system flash:ir800-universalk9-mz.SPA.156-3.M2
boot-end-marker
!
!
enable secret 5 *****
!
no aaa new-model
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
ip domain name *****
ip cef
login block-for 120 attempts 3 within 120
login quiet-mode access-class RTR_MGMT
login on-failure log
login on-success log
no ipv6 cef
!
multilink bundle-name authenticated
!
!
!
chat-script lte "" "AT!CALL1" TIMEOUT 30 "OK"
!
!
license udi pid IR809G-LTE-GA-K9 sn *****
!
!
username *****
!
redundancy
!
!
!
!
!
controller Cellular 0
lte modem link-recovery rssi onset-threshold -110
lte modem link-recovery monitor-timer 20
lte modem link-recovery wait-timer 10
lte modem link-recovery debounce-count 6
!
!
crypto keyring newkeyring
pre-shared-key address 0.0.0.0 0.0.0.0 key *****
!
crypto isakmp policy 1
encr 3des
authentication pre-share
group 2
lifetime 28800
crypto isakmp keepalive 10 3
!
!
crypto ipsec transform-set ***** esp-3des esp-sha-hmac
mode transport
!
crypto ipsec profile *****
set transform-set *****
!
!
!
!
!
!
!
interface Tunnel1
bandwidth 1000
ip address 10.255.255.4 255.255.255.0
no ip redirects
ip mtu 1408
ip nhrp authentication *****
ip nhrp map multicast dynamic
ip nhrp map 10.255.255.254 *****
ip nhrp map multicast *****
ip nhrp network-id 1772778
ip nhrp holdtime 300
ip nhrp nhs 10.255.255.254
ip tcp adjust-mss 574
ip ospf network broadcast
ip ospf priority 0
delay 1000
qos pre-classify
tunnel source Dialer1
tunnel mode gre multipoint
tunnel key 1772778
tunnel path-mtu-discovery
tunnel protection ipsec profile ***** shared
!
interface GigabitEthernet0
ip address 10.0.4.1 255.255.255.0
ip nat inside
ip virtual-reassembly in
duplex auto
speed auto
no mop enabled
!
interface GigabitEthernet1
ip address 192.168.0.38 255.255.255.0
duplex auto
speed auto
!
interface Wpan2
no ip address
shutdown
ieee154 txpower 25
no ieee154 fec-off
!
interface GigabitEthernet2
no ip address
shutdown
duplex auto
speed auto
!
interface Cellular0
ip address negotiated
no ip unreachables
ip nat outside
ip virtual-reassembly in
encapsulation slip
dialer in-band
dialer pool-member 1
dialer-group 1
async mode interactive
routing dynamic
!
interface Cellular1
no ip address
encapsulation slip
shutdown
!
interface Async0
no ip address
encapsulation scada
shutdown
!
interface Async1
no ip address
encapsulation scada
shutdown
!
interface Dialer1
ip address negotiated
no ip unreachables
ip nat outside
ip virtual-reassembly in
encapsulation ppp
dialer pool 1
dialer idle-timeout 0
dialer string lte
dialer persistent
dialer watch-group 1
dialer-group 1
ppp authentication pap callin
ppp pap sent-username ***** password 7 *****
ppp ipcp dns request
!
!
router ospf 10
router-id 10.255.255.4
passive-interface Cellular0
passive-interface Dialer1
network 10.0.4.0 0.0.0.255 area 0
network 10.255.255.0 0.0.0.255 area 0
!
ip forward-protocol nd
!
no ip http server
no ip http secure-server
!
ip nat inside source static tcp 10.0.4.20 80 interface Dialer1 9000
ip nat inside source static udp 10.0.4.20 80 interface Dialer1 9000
ip nat inside source static tcp 10.0.4.21 80 interface Dialer1 9001
ip nat inside source static udp 10.0.4.21 80 interface Dialer1 9001
ip nat inside source static tcp 10.0.4.5 80 interface Dialer1 9010
ip nat inside source static udp 10.0.4.5 80 interface Dialer1 9010
ip nat inside source static tcp 10.0.4.22 80 interface Dialer1 9002
ip nat inside source static udp 10.0.4.22 80 interface Dialer1 9002
ip nat inside source static tcp 10.0.4.23 80 interface Dialer1 9003
ip nat inside source static udp 10.0.4.23 80 interface Dialer1 9003
ip nat inside source static tcp 10.0.4.20 443 interface Dialer1 9011
ip nat inside source static udp 10.0.4.20 443 interface Dialer1 9011
ip nat inside source list NATout-Acl interface Dialer1 overload
ip route 0.0.0.0 0.0.0.0 Dialer1
ip ssh time-out 60
ip ssh authentication-retries 2
ip ssh source-interface Dialer1
ip ssh version 2
ip ssh server algorithm encryption aes128-ctr aes192-ctr aes256-ctr
ip ssh client algorithm encryption aes128-ctr aes192-ctr aes256-ctr
!
ip access-list standard RTR_MGMT
permit *****
permit *****
permit 10.0.0.0 0.255.255.255
!
ip access-list extended NATout-Acl
permit ip 10.0.4.0 0.0.0.255 any
!
dialer watch-list 1 ip 5.6.7.8 0.0.0.0
dialer watch-list 1 delay route-check initial 60
dialer watch-list 1 delay connect 1
dialer-list 1 protocol ip permit
ipv6 ioam timestamp
!
route-map clear-df permit 10
set ip df 0
!
!
snmp-server community ***** RO
snmp-server enable traps wpan
!
control-plane
!
!
!
!
line con 0
stopbits 1
line 1 2
stopbits 1
line 3
script dialer lte
modem InOut
no exec
transport preferred none
transport input all
transport output all
rxspeed 236800
txspeed 118000
line 8
no exec
transport preferred lat pad telnet rlogin lapb-ta mop udptn v120 ssh
transport output lat pad telnet rlogin lapb-ta mop udptn v120 ssh
rxspeed 236800
txspeed 118000
line 1/3 1/6
transport preferred none
transport output none
stopbits 1
line vty 0 4
password 7 *****
login local
transport input telnet ssh
line vty 5 15
password 7 *****
login local
transport input telnet ssh
!
no scheduler max-task-time
ntp peer 204.2.134.163
ntp peer 208.79.16.124
ntp peer 50.116.38.157
ntp peer 69.65.40.29
iox client enable interface GigabitEthernet2
iox hypervisor password 7 *****
!
!
!
!
!
!
end
Cedars#
