Hi,
First I would like to note that I sanitized the IP addresses in these logs. I am by far no expert on VPNs, but I am trying to pinpoint a solution for a far reaching problem we are having. We have a DMVPN setup that has two destinations from the client end. The client is using a Cisco 871 with
c870-advipservicesk9-mz.124-15.T9 Ios image. I should note that we only have access to the client end, so we are unable to make any changes to the other side. At some sites, everything works perfect and there are never any drops. At other sites we get neighbor and tunnel drops that can go on for hours in a cycle, a few minutes apart. Below are logs from one of the sites, showing the type of events that we are seeing.
510505: Dec 24 11:30:50.269 EST: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 175: Neighbor 10.109.147.1 (Tunnel2) is up: new adjacency
510506: Dec 24 11:31:42.284 EST: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=15.28.146.234, prot=50, spi=0x608EF276(1619980918), srcaddr=112.72.37.119
510507: Dec 24 11:32:05.446 EST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel2, changed state to down
510508: Dec 24 11:32:05.446 EST: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 175: Neighbor 10.109.147.1
(Tunnel2) is down: interface down
510509: Dec 24 11:33:20.449 EST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel2, changed state to up
510510: Dec 24 11:33:20.461 EST: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 175: Neighbor 10.109.147.1
(Tunnel2) is up: new adjacency
Sometimes this will be one tunnel with this issue, and sometimes it is both tunnels. The tunnel is mostly for redundancy, but there are some functions unique to each tunnel. We also get the
510506: Dec 24 11:31:42.284 EST: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=15.28.146.234, prot=50, spi=0x608EF276(1619980918), srcaddr=112.72.37.119
message which I would assume is a packet from the previous tunnel coming through. and being rejected because it does not match the new keys. I have looked into the keep alive times and adjusted them both down and up, but the trouble continued. What can cause this kind of flapping? Is there anything that can be done from just the client end to correct this issue? Any help would be greatly appreciated. Below you can see the configuration we have on the tunnel. If you have any questions, please let me know.
Router#sh int tun2
Tunnel2 is up, line protocol is up
Hardware is Tunnel
Description: Tunnel to Destination2
Internet address is 10.129.167.4/17
MTU 1514 bytes, BW 192 Kbit/sec, DLY 7500000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation TUNNEL, loopback not set
Keepalive set (15 sec), retries 3
Tunnel source 10.124.8.6 (Loopback1), destination 219.224.19.22
Tunnel protocol/transport GRE/IP
Key 0x68A92, sequencing disabled
Checksumming of packets disabled
Tunnel TTL 255
Fast tunneling enabled
Tunnel transmit bandwidth 8000 (kbps)
Tunnel receive bandwidth 8000 (kbps)
Last input 00:00:01, output 00:00:07, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 1588
Queueing strategy: fifo
Output queue: 0/0 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
7388052 packets input, 1813050207 bytes, 0 no buffer
Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
7655854 packets output, 3329592353 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 output buffer failures, 0 output buffers swapped out
*Note that we do not always see output drops
