I have a hub and spoke VPN.
Recently the hub was replaced with ISR 3925 with IOS 15.4(3)M1
The spokes are old 851 routers running 12.4 IOS
Now I am facing strange issue.
From time to time it happens that the IPSec SAs (phase 2) to a certain (each time a different) spoke by unknown to me reason disappear from the HUB.
Whenever that happens, the HUB brings down the line protocol of the GRE tunnel (linestate mode reg down) (this behavior wasn't present in the older IOS..).
When that happens, there is no way to bring the tunnel and the phase 2 SAs back up, other than manually:
1) shut/no shut the tunnel
2) clear the isakmp (phase 1) session
I have tried clearing the ipsec sa's (2nd phase), I have tried doing that on the spoke side - nothing happens.
I sniffed the traffic from the hub, and it seems that when that happens (no 2nd phase sa and tunnel goes down), the hub isn't actually trying to create phase 2 ipsec SAs, unless you clear the phase 1 SA..
I think that because the tunnel is down, it wouldn't try to use that tunnel, so that's why it doesn't try to create new SA's - which leads to a paradox..
Is there any way to fix that issue - either make the HUB try to negotiate new phase 2 SAs when it lose them, or make it not bring down the tunnel when that happens?
Any advice would be much appreciated.
Thanks and best regards.
Here is some more information.
HUB#show int tu28
Tunnel28 is up, line protocol is down
Hardware is Tunnel
Description: ** SPOKE **
Internet address is 10.0.0.117/30
MTU 17916 bytes, BW 5120 Kbit/sec, DLY 50000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation TUNNEL, loopback not set
Keepalive not set
Tunnel linestate evaluation down - linestate mode reg down
Tunnel source 18.104.22.168, destination 22.214.171.124
Tunnel protocol/transport GRE/IP
Key disabled, sequencing disabled
Checksumming of packets disabled
Tunnel TTL 255, Fast tunneling enabled
Tunnel transport MTU 1476 bytes
Tunnel transmit bandwidth 8000 (kbps)
Tunnel receive bandwidth 8000 (kbps)
Tunnel protection via IPSec (profile "ipsec")
Last input 01:24:09, output never, output hang never
Last clearing of "show interface" counters 3w4d
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 4
Queueing strategy: fifo
Output queue: 0/0 (size/max)
30 second input rate 0 bits/sec, 0 packets/sec
30 second output rate 0 bits/sec, 0 packets/sec
3905222 packets input, 1150452111 bytes, 0 no buffer
Received 0 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
3827545 packets output, 2347906608 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 output buffer failures, 0 output buffers swapped out
HUB#show tunnel interface tu28
Mode:GRE/IP, Destination 126.96.36.199, Source 188.8.131.52
IP transport: output interface Port-channel1.30 next hop 184.108.40.206
Application ID 1: unspecified
OCE: IP tunnel decap
Provider: interface Tu28, prot 47
Performs protocol check 
Protocol Handler: GRE: opt 0x0
ptype: ipv4 [ipv4 dispatcher: drop]
ptype: ipv6 [ipv6 dispatcher: drop]
ptype: mpls [mpls dispatcher: drop]
ptype: otv [mpls dispatcher: drop]
ptype: generic [mpls dispatcher: drop]
Linestate - current down
Internal linestate - current down, evaluated down - linestate mode reg down
Tunnel Source Flags: Local
Transport IPv4 Header DF bit cleared
HUB#show crypto ipsec sa interface Tunnel28
Crypto map tag: Tunnel28-head-0, local addr 220.127.116.11
protected vrf: vpn
local ident (addr/mask/prot/port): (18.104.22.168/255.255.255.255/47/0)
remote ident (addr/mask/prot/port): (22.214.171.124/255.255.255.255/47/0)
current_peer 126.96.36.199 port 500
#pkts encaps: 2441779, #pkts encrypt: 2441779, #pkts digest: 2441779
#pkts decaps: 2347707, #pkts decrypt: 2347707, #pkts verify: 2347707
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 1, #recv errors 0
local crypto endpt.: 188.8.131.52, remote crypto endpt.: 184.108.40.206
plaintext mtu 1446, path mtu 1500, ip mtu 1500, ip mtu idb Port-channel1.30
current outbound spi: 0x0(0)
PFS (Y/N): N, DH group: none
inbound esp sas:
inbound ah sas:
inbound pcp sas:
outbound esp sas:
outbound ah sas:
outbound pcp sas:
HUB#show crypto isakmp sa | i 220.127.116.11
18.104.22.168 22.214.171.124 QM_IDLE 16169 ACTIVE
HUB#clear crypto isakmp 16169
and after few secs everything came back up..
Here is the config of the Tunnel interface
HUB#show run int tu28
Current configuration : 349 bytes
description ** SPOKE **
ip vrf forwarding vpn
ip address 10.0.0.117 255.255.255.252
ip mtu 1442
ip tcp adjust-mss 1300
tunnel source 126.96.36.199
tunnel destination 188.8.131.52
tunnel protection ipsec profile ipsec
crypto ipsec df-bit clear
service-policy output vpn-out
I have tried putting keepalives, but it didn't bring up the tunnel..