Hi Mohammed, appreciate your reply. I am completely lost in this one, I have been troubleshooting it for weeks now and been posting in this community couple of times. The scenario is, if the clear crypto isakmp sa is issued the whole HUB and SPOKE vpn traffic acts as expected not a single problem at all. But, if the tunnel is left idle for sometime, communication on SPOKE3 becomes one way, no one can communicate to it until SPOKE3 initiate the traffic, this is the only way PHASE 2 becomes completed again. I have red 50 or more articles now about this and tried every solution stated to no avail. I would prefer the vpn not working at all than working and while having this issue, please help. Attached is our ASAs versions and topology and here is the logs of debug crypto ikev1 127 in the HUB when the tunnel is left idle and ping from internal network of SPOKE1 is issued going to internal network of SPOKE3, its always like that after all adjustments I tried:
Apr 13 07:25:42 [IKEv1]Group = (SPK3 public IP), IP = (SPK3 public IP), IKE Initiator: New Phase 2, Intf OUTSIDE_1, IKE Peer (SPK3 public IP) local Proxy Address 10.10.7.0, remote Proxy Address 10.30.10.0, Crypto map (CMAP_1)
Apr 13 07:25:42 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), Oakley begin quick mode
Apr 13 07:25:42 [IKEv1 DECODE]Group = (SPK3 public IP), IP = (SPK3 public IP), IKE Initiator starting QM: msg id = a44f8b3e
Apr 13 07:25:42 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), IKE got SPI from key engine: SPI = 0xdf1156ab
Apr 13 07:25:42 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), oakley constructing quick mode
Apr 13 07:25:42 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), constructing blank hash payload
Apr 13 07:25:42 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), constructing IPSec SA payload
Apr 13 07:25:42 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), constructing IPSec nonce payload
Apr 13 07:25:42 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), constructing proxy ID
Apr 13 07:25:42 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), Transmitting Proxy Id:
Local subnet: 10.10.7.0 mask 255.255.255.0 Protocol 0 Port 0
Remote subnet: 10.30.10.0 Mask 255.255.255.0 Protocol 0 Port 0
Apr 13 07:25:42 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), constructing qm hash payload
Apr 13 07:25:42 [IKEv1 DECODE]Group = (SPK3 public IP), IP = (SPK3 public IP), IKE Initiator sending 1st QM pkt: msg id = a44f8b3e
Apr 13 07:25:42 [IKEv1]IP = (SPK3 public IP), IKE_DECODE SENDING Message (msgid=a44f8b3e) with payloads : HDR + HASH (8) + SA (1) + NONCE (10) + ID (5) + ID (5) + NONE (0) total length : 172
Apr 13 07:25:55 [IKEv1 DEBUG]Group = (SPK1 public IP), IP = (SPK1 public IP), Pitcher: received key delete msg, spi 0x5dfb282c
Apr 13 07:25:55 [IKEv1]Group = (SPK1 public IP), IP = (SPK1 public IP), Connection terminated for peer (SPK1 public IP). Reason: IPSec SA Idle Timeout Remote Proxy 10.10.0.0, Local Proxy 10.10.38.0
Apr 13 07:25:55 [IKEv1 DEBUG]Group = (SPK1 public IP), IP = (SPK1 public IP), sending delete/delete with reason message
Apr 13 07:25:55 [IKEv1 DEBUG]Group = (SPK1 public IP), IP = (SPK1 public IP), constructing blank hash payload
Apr 13 07:25:55 [IKEv1 DEBUG]Group = (SPK1 public IP), IP = (SPK1 public IP), constructing IPSec delete payload
Apr 13 07:25:55 [IKEv1 DEBUG]Group = (SPK1 public IP), IP = (SPK1 public IP), constructing qm hash payload
Apr 13 07:25:55 [IKEv1]IP = (SPK1 public IP), IKE_DECODE SENDING Message (msgid=308b10dc) with payloads : HDR + HASH (8) + DELETE (12) + NONE (0) total length : 68
Apr 13 07:25:55 [IKEv1 DEBUG]Group = (SPK1 public IP), IP = (SPK1 public IP), Active unit receives a delete event for remote peer (SPK1 public IP).
Apr 13 07:25:55 [IKEv1 DEBUG]Group = (SPK1 public IP), IP = (SPK1 public IP), IKE Deleting SA: Remote Proxy 10.10.38.0, Local Proxy 10.10.0.0
Apr 13 07:25:55 [IKEv1 DEBUG]Pitcher: received key delete msg, spi 0x5dfb282c
Apr 13 07:25:59 [IKEv1]Group = (SPK3 public IP), IP = (SPK3 public IP), QM FSM error (P2 struct &0x00007f7e0b52cf80, mess id 0x9e83612)!
Apr 13 07:25:59 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), IKE QM Initiator FSM error history (struct &0x00007f7e0b52cf80) <state>, <event>: QM_DONE, EV_ERROR-->QM_WAIT_MSG2, EV_TIMEOUT-->QM_WAIT_MSG2, NullEvent-->QM_SND_MSG1, EV_SND_MSG-->QM_SND_MSG1, EV_START_TMR-->QM_SND_MSG1, EV_RESEND_MSG-->QM_WAIT_MSG2, EV_TIMEOUT-->QM_WAIT_MSG2, NullEvent
Apr 13 07:25:59 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), sending delete/delete with reason message
Apr 13 07:25:59 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), constructing blank hash payload
Apr 13 07:25:59 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), constructing IPSec delete payload
Apr 13 07:25:59 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), constructing qm hash payload
Apr 13 07:25:59 [IKEv1]IP = (SPK3 public IP), IKE_DECODE SENDING Message (msgid=dbb6f619) with payloads : HDR + HASH (8) + DELETE (12) + NONE (0) total length : 68
Apr 13 07:25:59 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), IKE Deleting SA: Remote Proxy 192.168.185.0, Local Proxy 10.10.32.0
Apr 13 07:25:59 [IKEv1]Group = (SPK3 public IP), IP = (SPK3 public IP), Removing peer from correlator table failed, no match!
Apr 13 07:25:59 [IKEv1 DEBUG]Pitcher: received key delete msg, spi 0xa467bdff
Apr 13 07:26:14 [IKEv1]Group = (SPK3 public IP), IP = (SPK3 public IP), QM FSM error (P2 struct &0x00007f7e0c5baa10, mess id 0xa44f8b3e)!
Apr 13 07:26:14 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), IKE QM Initiator FSM error history (struct &0x00007f7e0c5baa10) <state>, <event>: QM_DONE, EV_ERROR-->QM_WAIT_MSG2, EV_TIMEOUT-->QM_WAIT_MSG2, NullEvent-->QM_SND_MSG1, EV_SND_MSG-->QM_SND_MSG1, EV_START_TMR-->QM_SND_MSG1, EV_RESEND_MSG-->QM_WAIT_MSG2, EV_TIMEOUT-->QM_WAIT_MSG2, NullEvent
Apr 13 07:26:14 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), sending delete/delete with reason message
Apr 13 07:26:14 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), constructing blank hash payload
Apr 13 07:26:14 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), constructing IPSec delete payload
Apr 13 07:26:14 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), constructing qm hash payload
Apr 13 07:26:14 [IKEv1]IP = (SPK3 public IP), IKE_DECODE SENDING Message (msgid=d7eb7703) with payloads : HDR + HASH (8) + DELETE (12) + NONE (0) total length : 68
Apr 13 07:26:14 [IKEv1 DEBUG]Group = (SPK3 public IP), IP = (SPK3 public IP), IKE Deleting SA: Remote Proxy 10.30.10.0, Local Proxy 10.10.7.0
Apr 13 07:26:14 [IKEv1]Group = (SPK3 public IP), IP = (SPK3 public IP), Removing peer from correlator table failed, no match!
Apr 13 07:26:14 [IKEv1 DEBUG]Pitcher: received key delete msg, spi 0xdf1156ab
Apr 13 07:26:20 [IKEv1 DEBUG]Pitcher: received a key acquire message, spi 0x0
Thanks!
