12-13-2012 06:02 PM
We have approx 40 branch offices - all of which are connected to a single core site over VPN Tunnels using various gear. At one particular site, we are having issues with the tunnel dropping sporadically throughout the day - some days it happens 10 times, some days it happens none. This just randomly started happening two weeks ago, without any changes taking place. Since it started happening, I have upgraded the code to latest versions, but still the issue persists.
This particular site has a 2901 and connects back to a 2951.
Below is the output from:
debug crypto ipsec
debug crypto isakmp
Any help would be grealty appreciated!
Dec 13 22:17:22.756 AST: ISAKMP:(4001):DPD/R_U_THERE received from peer 222.222.255.106, sequence 0x2BD738CD
Dec 13 22:17:22.756 AST: ISAKMP: set new node 440073522 to QM_IDLE
Dec 13 22:17:22.760 AST: ISAKMP:(4001):Sending NOTIFY DPD/R_U_THERE_ACK protocol 1
spi 1770341040, message ID = 440073522
Dec 13 22:17:22.760 AST: ISAKMP:(4001): seq. no 0x2BD738CD
Dec 13 22:17:22.760 AST: ISAKMP:(4001): sending packet to 222.222.255.106 my_port 500 peer_port 500 (R) QM_IDLE
Dec 13 22:17:22.760 AST: ISAKMP:(4001):Sending an IKE IPv4 Packet.
Dec 13 22:17:22.760 AST: ISAKMP:(4001):purging node 440073522
Dec 13 22:17:22.760 AST: ISAKMP:(4001):Input = IKE_MESG_FROM_PEER, IKE_MESG_KEEP_ALIVE
Dec 13 22:17:22.760 AST: ISAKMP:(4001):Old State = IKE_P1_COMPLETE New State = IKE_P1_COMPLETE
Dec 13 22:17:29.784 AST: ISAKMP:(4001):purging node -1544238918
Dec 13 22:17:31.776 AST: ISAKMP:(4001):purging node 2072748287
Dec 13 22:17:42.156 AST: ISAKMP (4001): received packet from 222.222.255.106 dport 500 sport 500 Global (R) QM_IDLE
Dec 13 22:17:42.156 AST: ISAKMP: set new node -470636592 to QM_IDLE
Dec 13 22:17:42.160 AST: ISAKMP:(4001): processing HASH payload. message ID = 3824330704
Dec 13 22:17:42.160 AST: ISAKMP:(4001): processing NOTIFY DPD/R_U_THERE protocol 1
spi 0, message ID = 3824330704, sa = 0x684F3BF8
Dec 13 22:17:42.160 AST: ISAKMP:(4001):deleting node -470636592 error FALSE reason "Informational (in) state 1"
Dec 13 22:17:42.160 AST: ISAKMP:(4001):Input = IKE_MESG_FROM_PEER, IKE_INFO_NOTIFY
Dec 13 22:17:42.160 AST: ISAKMP:(4001):Old State = IKE_P1_COMPLETE New State = IKE_P1_COMPLETE
Dec 13 22:17:42.160 AST: ISAKMP:(4001):DPD/R_U_THERE received from peer 222.222.255.106, sequence 0x2BD738CE
Dec 13 22:17:42.164 AST: ISAKMP: set new node 1628771996 to QM_IDLE
Dec 13 22:17:42.164 AST: ISAKMP:(4001):Sending NOTIFY DPD/R_U_THERE_ACK protocol 1
spi 1770341040, message ID = 1628771996
Dec 13 22:17:42.164 AST: ISAKMP:(4001): seq. no 0x2BD738CE
Dec 13 22:17:42.164 AST: ISAKMP:(4001): sending packet to 222.222.255.106 my_port 500 peer_port 500 (R) QM_IDLE
Dec 13 22:17:42.164 AST: ISAKMP:(4001):Sending an IKE IPv4 Packet.
Dec 13 22:17:42.164 AST: ISAKMP:(4001):purging node 1628771996
Dec 13 22:17:42.168 AST: ISAKMP:(4001):Input = IKE_MESG_FROM_PEER, IKE_MESG_KEEP_ALIVE
Dec 13 22:17:42.168 AST: ISAKMP:(4001):Old State = IKE_P1_COMPLETE New State = IKE_P1_COMPLETE
Dec 13 22:17:44.168 AST: ISAKMP:(4001):purging node 590451866
Dec 13 22:18:00.756 AST: ISAKMP:(4001):purging node 1174599381
Dec 13 22:18:05.760 AST: ISAKMP (4001): received packet from 222.222.255.106 dport 500 sport 500 Global (R) QM_IDLE
Dec 13 22:18:05.760 AST: ISAKMP: set new node 184975621 to QM_IDLE
Dec 13 22:18:05.764 AST: ISAKMP:(4001): processing HASH payload. message ID = 184975621
Dec 13 22:18:05.764 AST: ISAKMP:(4001): processing NOTIFY DPD/R_U_THERE protocol 1
spi 0, message ID = 184975621, sa = 0x684F3BF8
Dec 13 22:18:05.764 AST: ISAKMP:(4001):deleting node 184975621 error FALSE reason "Informational (in) state 1"
Dec 13 22:18:05.764 AST: ISAKMP:(4001):Input = IKE_MESG_FROM_PEER, IKE_INFO_NOTIFY
Dec 13 22:18:05.764 AST: ISAKMP:(4001):Old State = IKE_P1_COMPLETE New State = IKE_P1_COMPLETE
Dec 13 22:18:05.764 AST: ISAKMP:(4001):DPD/R_U_THERE received from peer 222.222.255.106, sequence 0x2BD738CF
Dec 13 22:18:05.764 AST: ISAKMP: set new node -1074391907 to QM_IDLE
Dec 13 22:18:05.768 AST: ISAKMP:(4001):Sending NOTIFY DPD/R_U_THERE_ACK protocol 1
spi 1770341040, message ID = 3220575389
Dec 13 22:18:05.768 AST: ISAKMP:(4001): seq. no 0x2BD738CF
Dec 13 22:18:05.768 AST: ISAKMP:(4001): sending packet to 222.222.255.106 my_port 500 peer_port 500 (R) QM_IDLE
Dec 13 22:18:05.768 AST: ISAKMP:(4001):Sending an IKE IPv4 Packet.
Dec 13 22:18:05.768 AST: ISAKMP:(4001):purging node -1074391907
Dec 13 22:18:05.768 AST: ISAKMP:(4001):Input = IKE_MESG_FROM_PEER, IKE_MESG_KEEP_ALIVE
Dec 13 22:18:05.768 AST: ISAKMP:(4001):Old State = IKE_P1_COMPLETE New State = IKE_P1_COMPLETE
Dec 13 22:18:12.756 AST: ISAKMP:(4001):purging node 1858894050
Dec 13 22:18:18.620 AST: %SEC-6-IPACCESSLOGP: list INBOUND denied tcp 216.193.212.102(53) (FastEthernet0/0 0017.cb6e.b040) -> 222.222.3.106(53), 1 packet
Dec 13 22:18:18.880 AST: ISAKMP (4001): received packet from 222.222.255.106 dport 500 sport 500 Global (R) QM_IDLE
Dec 13 22:18:18.880 AST: ISAKMP: set new node 1403645477 to QM_IDLE
Dec 13 22:18:18.884 AST: ISAKMP:(4001): processing HASH payload. message ID = 1403645477
Dec 13 22:18:18.884 AST: ISAKMP:(4001): processing NOTIFY DPD/R_U_THERE protocol 1
spi 0, message ID = 1403645477, sa = 0x684F3BF8
Dec 13 22:18:18.884 AST: ISAKMP:(4001):deleting node 1403645477 error FALSE reason "Informational (in) state 1"
Dec 13 22:18:18.884 AST: ISAKMP:(4001):Input = IKE_MESG_FROM_PEER, IKE_INFO_NOTIFY
Dec 13 22:18:18.884 AST: ISAKMP:(4001):Old State = IKE_P1_COMPLETE New State = IKE_P1_COMPLETE
Dec 13 22:18:18.884 AST: ISAKMP:(4001):DPD/R_U_THERE received from peer 222.222.255.106, sequence 0x2BD738D0
Dec 13 22:18:18.884 AST: ISAKMP: set new node -726317316 to QM_IDLE
Dec 13 22:18:18.884 AST: ISAKMP:(4001):Sending NOTIFY DPD/R_U_THERE_ACK protocol 1
spi 1770341040, message ID = 3568649980
Dec 13 22:18:18.888 AST: ISAKMP:(4001): seq. no 0x2BD738D0
Dec 13 22:18:18.888 AST: ISAKMP:(4001): sending packet to 222.222.255.106 my_port 500 peer_port 500 (R) QM_IDLE
Dec 13 22:18:18.888 AST: ISAKMP:(4001):Sending an IKE IPv4 Packet.
Dec 13 22:18:18.888 AST: ISAKMP:(4001):purging node -726317316
Dec 13 22:18:18.888 AST: ISAKMP:(4001):Input = IKE_MESG_FROM_PEER, IKE_MESG_KEEP_ALIVE
Dec 13 22:18:18.888 AST: ISAKMP:(4001):Old State = IKE_P1_COMPLETE New State = IKE_P1_COMPLETE
Dec 13 22:18:20.616 AST: %SEC-6-IPACCESSLOGP: list INBOUND denied tcp 216.193.212.102(53) (FastEthernet0/0 0017.cb6e.b040) -> 222.222.3.106(53), 1 packet
Dec 13 22:18:21.436 AST: %FW-6-DROP_PKT: Dropping tcp session 208.92.53.140:80 10.10.20.123:61541 due to SYN inside current window with ip ident 0 tcpflags 0x6012 seq.no 395777174 ack 2189112413
Dec 13 22:18:24.612 AST: %SEC-6-IPACCESSLOGP: list INBOUND denied tcp 208.80.55.25(53) (FastEthernet0/0 0017.cb6e.b040) -> 222.222.3.106(53), 1 packet
Dec 13 22:18:31.632 AST: ISAKMP (4001): received packet from 222.222.255.106 dport 500 sport 500 Global (R) QM_IDLE
Dec 13 22:18:31.632 AST: ISAKMP: set new node 794995208 to QM_IDLE
Dec 13 22:18:31.632 AST: ISAKMP:(4001): processing HASH payload. message ID = 794995208
Dec 13 22:18:31.632 AST: ISAKMP:(4001): processing NOTIFY DPD/R_U_THERE protocol 1
spi 0, message ID = 794995208, sa = 0x684F3BF8
Dec 13 22:18:31.632 AST: ISAKMP:(4001):deleting node 794995208 error FALSE reason "Informational (in) state 1"
Dec 13 22:18:31.636 AST: ISAKMP:(4001):Input = IKE_MESG_FROM_PEER, IKE_INFO_NOTIFY
Dec 13 22:18:31.636 AST: ISAKMP:(4001):Old State = IKE_P1_COMPLETE New State = IKE_P1_COMPLETE
Dec 13 22:18:31.636 AST: ISAKMP:(4001):DPD/R_U_THERE received from peer 222.222.255.106, sequence 0x2BD738D1
Dec 13 22:18:31.636 AST: ISAKMP: set new node -1240798570 to QM_IDLE
Dec 13 22:18:31.636 AST: ISAKMP:(4001):Sending NOTIFY DPD/R_U_THERE_ACK protocol 1
spi 1770341040, message ID = 3054168726
Dec 13 22:18:31.636 AST: ISAKMP:(4001): seq. no 0x2BD738D1
Dec 13 22:18:31.640 AST: ISAKMP:(4001): sending packet to 222.222.255.106 my_port 500 peer_port 500 (R) QM_IDLE
Dec 13 22:18:31.640 AST: ISAKMP:(4001):Sending an IKE IPv4 Packet.
Dec 13 22:18:31.640 AST: ISAKMP:(4001):purging node -1240798570
Dec 13 22:18:31.640 AST: ISAKMP:(4001):Input = IKE_MESG_FROM_PEER, IKE_MESG_KEEP_ALIVE
Dec 13 22:18:31.640 AST: ISAKMP:(4001):Old State = IKE_P1_COMPLETE New State = IKE_P1_COMPLETE
Dec 13 22:18:32.160 AST: ISAKMP:(4001):purging node -470636592
Dec 13 22:18:43.624 AST: ISAKMP (4001): received packet from 222.222.255.106 dport 500 sport 500 Global (R) QM_IDLE
Dec 13 22:18:43.624 AST: ISAKMP: set new node 2128351743 to QM_IDLE
Dec 13 22:18:43.628 AST: ISAKMP:(4001): processing HASH payload. message ID = 2128351743
Dec 13 22:18:43.628 AST: ISAKMP:(4001): processing NOTIFY DPD/R_U_THERE protocol 1
spi 0, message ID = 2128351743, sa = 0x684F3BF8
Dec 13 22:18:43.628 AST: ISAKMP:(4001):deleting node 2128351743 error FALSE reason "Informational (in) state 1"
Dec 13 22:18:43.628 AST: ISAKMP:(4001):Input = IKE_MESG_FROM_PEER, IKE_INFO_NOTIFY
Dec 13 22:18:43.628 AST: ISAKMP:(4001):Old State = IKE_P1_COMPLETE New State = IKE_P1_COMPLETE
Dec 13 22:18:43.628 AST: ISAKMP:(4001):DPD/R_U_THERE received from peer 222.222.255.106, sequence 0x2BD738D2
Dec 13 22:18:43.628 AST: ISAKMP: set new node -77912199 to QM_IDLE
Dec 13 22:18:43.628 AST: ISAKMP:(4001):Sending NOTIFY DPD/R_U_THERE_ACK protocol 1
spi 1770341040, message ID = 4217055097
Dec 13 22:18:43.632 AST: ISAKMP:(4001): seq. no 0x2BD738D2
Dec 13 22:18:43.632 AST: ISAKMP:(4001): sending packet to 222.222.255.106 my_port 500 peer_port 500 (R) QM_IDLE
Dec 13 22:18:43.632 AST: ISAKMP:(4001):Sending an IKE IPv4 Packet.
Dec 13 22:18:43.632 AST: ISAKMP:(4001):purging node -77912199
Dec 13 22:18:43.632 AST: ISAKMP:(4001):Input = IKE_MESG_FROM_PEER, IKE_MESG_KEEP_ALIVE
Dec 13 22:18:43.632 AST: ISAKMP:(4001):Old State = IKE_P1_COMPLETE New State = IKE_P1_COMPLETE
Dec 13 22:18:55.764 AST: ISAKMP:(4001):purging node 184975621
Dec 13 22:19:03.436 AST: %FW-6-DROP_PKT: Dropping tcp session 208.92.53.140:80 10.10.20.123:61541 due to SYN inside current window with ip ident 0 tcpflags 0x6012 seq.no 395777174 ack 2189112413
Dec 13 22:19:07.532 AST: ISAKMP (4001): received packet from 222.222.255.106 dport 500 sport 500 Global (R) QM_IDLE
Dec 13 22:19:07.532 AST: ISAKMP: set new node -1277930728 to QM_IDLE
Dec 13 22:19:07.536 AST: ISAKMP:(4001): processing HASH payload. message ID = 3017036568
Dec 13 22:19:07.536 AST: ISAKMP:(4001): processing NOTIFY DPD/R_U_THERE protocol 1
spi 0, message ID = 3017036568, sa = 0x684F3BF8
Dec 13 22:19:07.536 AST: ISAKMP:(4001):deleting node -1277930728 error FALSE reason "Informational (in) state 1"
Dec 13 22:19:07.536 AST: ISAKMP:(4001):Input = IKE_MESG_FROM_PEER, IKE_INFO_NOTIFY
Dec 13 22:19:07.536 AST: ISAKMP:(4001):Old State = IKE_P1_COMPLETE New State = IKE_P1_COMPLETE
Dec 13 22:19:07.536 AST: ISAKMP:(4001):DPD/R_U_THERE received from peer 222.222.255.106, sequence 0x2BD738D3
Dec 13 22:19:07.536 AST: ISAKMP: set new node -1048088339 to QM_IDLE
Dec 13 22:19:07.540 AST: ISAKMP:(4001):Sending NOTIFY DPD/R_U_THERE_ACK protocol 1
spi 1770341040, message ID = 3246878957
Dec 13 22:19:07.540 AST: ISAKMP:(4001): seq. no 0x2BD738D3
Dec 13 22:19:07.540 AST: ISAKMP:(4001): sending packet to 222.222.255.106 my_port 500 peer_port 500 (R) QM_IDLE
Dec 13 22:19:07.540 AST: ISAKMP:(4001):Sending an IKE IPv4 Packet.
Dec 13 22:19:07.540 AST: ISAKMP:(4001):purging node -1048088339
Dec 13 22:19:07.540 AST: ISAKMP:(4001):Input = IKE_MESG_FROM_PEER, IKE_MESG_KEEP_ALIVE
Dec 13 22:19:07.540 AST: ISAKMP:(4001):Old State = IKE_P1_COMPLETE New State = IKE_P1_COMPLETE
My new home on the web: www.closetgeek.ca
12-13-2012 08:54 PM
Do you happen to have the debugs when the issue happens?
From the above logs, it seems to be OK, DPD is getting acknowledge.
If nothing has changed, I would then suspect it is an ISP issue where they could have been dropping some packets hence the DPD either doesn't get there, or the ACK doesn't get to the other side, hence the tunnel is randomly dropping.
10-16-2016 07:46 PM
Hello Adam,
Can you please let me know how where you able to solve this issue? I had the same issue for the past 2 weeks.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide