Hi Vahid,

dnsroot13 · ‎11-17-2015

Hi Guys

I am having serious issue on one of the spoke.

there is an adsl connection on spoke and tunnel keep going down .same config on other spokes having adsl connection have no issue .(there are few drops when ping 8.8.8.8 from spoke router sometime)-not sure if due that having an issue or something else .can you pleae help me to address the issue

below are the logs and spoke config .

sh cry isa sa

IPv4 Crypto ISAKMP SA

dst             src             state          conn-id status

94.56.66.60     192.168.15.96   MM_NO_STATE       2060 ACTIVE (deleted)

sh cry isa sa
dst             src             state          conn-id status

94.56.66.60     192.168.15.96   MM_KEY_EXCH       2061 ACTIVE

94.56.66.60     192.168.15.96   MM_NO_STATE       2060 ACTIVE (deleted)

000768: *Nov 5 12:45:56.522: ISAKMP (2076): received packet from 94.56.66.60 dport 500 sport 500 Global (I) MM_KEY_EXCH

000769: *Nov 5 12:45:56.522: ISAKMP:(2076): phase 1 packet is a duplicate of a previous packet.

000770: *Nov 5 12:45:56.522: ISAKMP:(2076): retransmitting due to retransmit phase 1

000771: *Nov 5 12:45:57.022: ISAKMP:(2076): retransmitting phase 1 MM_KEY_EXCH...

000772: *Nov 5 12:45:57.022: ISAKMP (2076): incrementing error counter on sa, attempt 5 of 5: retransmit phase 1

000773: *Nov 5 12:45:57.022: ISAKMP:(2076): retransmitting phase 1 MM_KEY_EXCH

000774: *Nov 5 12:45:57.022: ISAKMP:(2076): sending packet to 94.56.66.60 my_port 4500 peer_port 4500 (I) MM_KEY_EXCH

000775: *Nov 5 12:45:57.022: ISAKMP:(2076):Sending an IKE IPv4 Packet.cle

KSA_HO#clear cry isa

KSA_HO#

000776: *Nov 5 12:46:01.350: del_node src 192.168.15.96:4500 dst 94.56.66.60:4500 fvrf 0x0, ivrf 0x0

000777: *Nov 5 12:46:01.350: ISAKMP:(2076):peer does not do paranoid keepalives.

000778: *Nov 5 12:46:01.350: ISAKMP:(2076):deleting SA reason "Death by tree-walk" state (I) MM_KEY_EXCH (peer 94.56.66.60)

000779: *Nov 5 12:46:01.350: ISAKMP:(2076):deleting SA reason "Death by tree-walk" state (I) MM_KEY_EXCH (peer 94.56.66.60)

000780: *Nov 5 12:46:01.350: ISAKMP: Unlocking peer struct 0x88F90CC4 for isadb_mark_sa_deleted(), count 0

000781: *Nov 5 12:46:01.350: ISAKMP: Deleting peer node by peer_reap for 94.56.66.60: 88F90CC4

000782: *Nov 5 12:46:01.350: ISAKMP:(2076):deleting node 1501559703 error FALSE reason "IKE deleted"

000783: *Nov 5 12:46:01.350: ISAKMP:(2076):deleting node -304454932 error FALSE reason "IKE deleted"

000784: *Nov 5 12:46:01.350: ISAKMP:(2076):Input = IKE_MESG_INTERNAL, IKE_PHASE1_DEL

000785: *Nov 5 12:46:01.350: ISAKMP:(2076):Old State = IKE_I_MM5 New State = IKE_DEST_SA

000786: *Nov 5 12:46:06.286: ISAKMP:(0): SA request profile is (NULL)

000787: *Nov 5 12:46:06.286: ISAKMP: Created a peer struct for 94.56.66.60, peer port 500

000788: *Nov 5 12:46:06.286: ISAKMP: New peer created peer = 0x88F90CC4 peer_handle = 0x800001B7

000789: *Nov 5 12:46:06.286: ISAKMP: Locking peer struct 0x88F90CC4, refcount 1 for isakmp_initiator

000790: *Nov 5 12:46:06.286: ISAKMP: local port 500, remote port 500

000791: *Nov 5 12:46:06.286: ISAKMP: set new node 0 to QM_IDLE

000792: *Nov 5 12:46:06.286: ISAKMP: Find a dup sa in the avl tree during calling isadb_insert sa = 87F98DE0

000793: *Nov 5 12:46:06.286: ISAKMP:(0):Can not start Aggressive mode, trying Main mode.

000794: *Nov 5 12:46:06.286: ISAKMP:(0):found peer pre-shared key matching 94.56.66.60

000795: *Nov 5 12:46:06.286: ISAKMP:(0): constructed NAT-T vendor-rfc3947 ID

000796: *Nov 5 12:46:06.286: ISAKMP:(0): constructed NAT-T vendor-07 ID

000797: *Nov 5 12:46:06.286: ISAKMP:(0): constructed NAT-T vendor-03 ID

000798: *Nov 5 12:46:06.286: ISAKMP:(0): constructed NAT-T vendor-02 ID

000799: *Nov 5 12:46:06.286: ISAKMP:(0):Input = IKE_MESG_FROM_IPSEC, IKE_SA_REQ_MM

000800: *Nov 5 12:46:06.286: ISAKMP:(0):Old State = IKE_READY New State = IKE_I_MM1

000801: *Nov 5 12:46:06.286: ISAKMP:(0): beginning Main Mode exchange

000802: *Nov 5 12:46:06.286: ISAKMP:(0): sending packet to 94.56.66.60 my_port 500 peer_port 500 (I) MM_NO_STATE

000803: *Nov 5 12:46:06.286: ISAKMP:(0):Sending an IKE IPv4 Packet.

000804: *Nov 5 12:46:06.394: ISAKMP (0): received packet from 94.56.66.60 dport 500 sport 500 Global (I) MM_NO_STATE

000805: *Nov 5 12:46:06.394: ISAKMP:(0):Input = IKE_MESG_FROM_PEER, IKE_MM_EXCH

000806: *Nov 5 12:46:06.394: ISAKMP:(0):Old State = IKE_I_MM1 New State = IKE_I_MM2

000807: *Nov 5 12:46:06.398: ISAKMP:(0): processing SA payload. message ID = 0

000808: *Nov 5 12:46:06.398: ISAKMP:(0): processing vendor id payload

000809: *Nov 5 12:46:06.398: ISAKMP:(0): vendor ID seems Unity/DPD but major 69 mismatch

000810: *Nov 5 12:46:06.398: ISAKMP (0): vendor ID is NAT-T RFC 3947

000811: *Nov 5 12:46:06.398: ISAKMP:(0):found peer pre-shared key matching 94.56.66.60

000812: *Nov 5 12:46:06.398: ISAKMP:(0): local preshared key found

000813: *Nov 5 12:46:06.398: ISAKMP : Scanning profiles for xauth ...

000814: *Nov 5 12:46:06.398: ISAKMP:(0):Checking ISAKMP transform 1 against priority 10 policy

000815: *Nov 5 12:46:06.398: ISAKMP:      encryption 3DES-CBC

000816: *Nov 5 12:46:06.398: ISAKMP:      hash SHA

000817: *Nov 5 12:46:06.398: ISAKMP:      default group 2

000818: *Nov 5 12:46:06.398: ISAKMP:      auth pre-share

000819: *Nov 5 12:46:06.398: ISAKMP:      life type in seconds

000820: *Nov 5 12:46:06.398: ISAKMP:      life duration (VPI) of 0x0 0x1 0x51 0x80

000821: *Nov 5 12:46:06.398: ISAKMP:(0):atts are acceptable. Next payload is 0

000822: *Nov 5 12:46:06.398: ISAKMP:(0):Acceptable atts:actual life: 0

000823: *Nov 5 12:46:06.398: ISAKMP:(0):Acceptable atts:life: 0

000824: *Nov 5 12:46:06.398: ISAKMP:(0):Fill atts in sa vpi_length:4

000825: *Nov 5 12:46:06.398: ISAKMP:(0):Fill atts in sa life_in_seconds:86400

000826: *Nov 5 12:46:06.398: ISAKMP:(0):Returning Actual lifetime: 86400

000827: *Nov 5 12:46:06.398: ISAKMP:(0)::Started lifetime timer: 86400.

000828: *Nov 5 12:46:06.398: ISAKMP:(0): processing vendor id payload

000829: *Nov 5 12:46:06.398: ISAKMP:(0): vendor ID seems Unity/DPD but major 69 mismatch

000830: *Nov 5 12:46:06.398: ISAKMP (0): vendor ID is NAT-T RFC 3947

000831: *Nov 5 12:46:06.398: ISAKMP:(0):Input = IKE_MESG_INTERNAL, IKE_PROCESS_MAIN_MODE

000832: *Nov 5 12:46:06.398: ISAKMP:(0):Old State = IKE_I_MM2 New State = IKE_I_MM2

000833: *Nov 5 12:46:06.398: ISAKMP:(0): sending packet to 94.56.66.60 my_port 500 peer_port 500 (I) MM_SA_SETUP

000834: *Nov 5 12:46:06.398: ISAKMP:(0):Sending an IKE IPv4 Packet.

000835: *Nov 5 12:46:06.398: ISAKMP:(0):Input = IKE_MESG_INTERNAL, IKE_PROCESS_COMPLETE

000836: *Nov 5 12:46:06.398: ISAKMP:(0):Old State = IKE_I_MM2 New State = IKE_I_MM3

000837: *Nov 5 12:46:06.522: ISAKMP (0): received packet from 94.56.66.60 dport 500 sport 500 Global (I) MM_SA_SETUP

000838: *Nov 5 12:46:06.526: ISAKMP:(0):Input = IKE_MESG_FROM_PEER, IKE_MM_EXCH

000839: *Nov 5 12:46:06.526: ISAKMP:(0):Old State = IKE_I_MM3 New State = IKE_I_MM4

000840: *Nov 5 12:46:06.526: ISAKMP:(0): processing KE payload. message ID = 0

000841: *Nov 5 12:46:06.558: ISAKMP:(0): processing NONCE payload. message ID = 0

000842: *Nov 5 12:46:06.558: ISAKMP:(0):found peer pre-shared key matching 94.56.66.60

000843: *Nov 5 12:46:06.558: ISAKMP:(2077): processing vendor id payload

000844: *Nov 5 12:46:06.558: ISAKMP:(2077): vendor ID is Unity

000845: *Nov 5 12:46:06.558: ISAKMP:(2077): processing vendor id payload

000846: *Nov 5 12:46:06.558: ISAKMP:(2077): vendor ID is DPD

000847: *Nov 5 12:46:06.558: ISAKMP:(2077): processing vendor id payload

000848: *Nov 5 12:46:06.558: ISAKMP:(2077): speaking to another IOS box!

000849: *Nov 5 12:46:06.558: ISAKMP:received payload type 20

000850: *Nov 5 12:46:06.558: ISAKMP (2077): NAT found, both nodes inside NAT

000851: *Nov 5 12:46:06.558: ISAKMP:received payload type 20

000852: *Nov 5 12:46:06.558: ISAKMP (2077): My hash no match - this node inside NAT

000853: *Nov 5 12:46:06.558: ISAKMP:(2077):Input = IKE_MESG_INTERNAL, IKE_PROCESS_MAIN_MODE

000854: *Nov 5 12:46:06.558: ISAKMP:(2077):Old State = IKE_I_MM4 New State = IKE_I_MM4

000855: *Nov 5 12:46:06.558: ISAKMP:(2077):Send initial contact

000856: *Nov 5 12:46:06.562: ISAKMP:(2077):SA is doing pre-shared key authentication using id type ID_IPV4_ADDR

000857: *Nov 5 12:46:06.562: ISAKMP (2077): ID payload

        next-payload : 8

        type         : 1

        address      : 192.168.15.96

        protocol     : 17

        port         : 0

        length       : 12

000858: *Nov 5 12:46:06.562: ISAKMP:(2077):Total payload length: 12

000859: *Nov 5 12:46:06.562: ISAKMP:(2077): sending packet to 94.56.66.60 my_port 4500 peer_port 4500 (I) MM_KEY_EXCH

000860: *Nov 5 12:46:06.562: ISAKMP:(2077):Sending an IKE IPv4 Packet.

000861: *Nov 5 12:46:06.562: ISAKMP:(2077):Input = IKE_MESG_INTERNAL, IKE_PROCESS_COMPLETE

000862: *Nov 5 12:46:06.562: ISAKMP:(2077):Old State = IKE_I_MM4 New State = IKE_I_MM5

000863: *Nov 5 12:46:16.522: ISAKMP (2077): received packet from 94.56.66.60 dport 500 sport 500 Global (I) MM_KEY_EXCH

000864: *Nov 5 12:46:16.522: ISAKMP:(2077): phase 1 packet is a duplicate of a previous packet.

000865: *Nov 5 12:46:16.522: ISAKMP:(2077): retransmitting due to retransmit phase 1

000866: *Nov 5 12:46:17.022: ISAKMP:(2077): retransmitting phase 1 MM_KEY_EXCH...

000867: *Nov 5 12:46:17.022: ISAKMP (2077): incrementing error counter on sa, attempt 1 of 5: retransmit phase 1

000868: *Nov 5 12:46:17.022: ISAKMP:(2077): retransmitting phase 1 MM_KEY_EXCH

000869: *Nov 5 12:46:17.022: ISAKMP:(2077): sending packet to 94.56.66.60 my_port 4500 peer_port 4500 (I) MM_KEY_EXCH

000870: *Nov 5 12:46:17.022: ISAKMP:(2077):Sending an IKE IPv4 Packet.

000871: *Nov 5 12:46:26.522: ISAKMP (2077): received packet from 94.56.66.60 dport 500 sport 500 Global (I) MM_KEY_EXCH

000872: *Nov 5 12:46:26.522: ISAKMP:(2077): phase 1 packet is a duplicate of a previous packet.

000873: *Nov 5 12:46:26.522: ISAKMP:(2077): retransmitting due to retransmit phase 1

000874: *Nov 5 12:46:27.022: ISAKMP:(2077): retransmitting phase 1 MM_KEY_EXCH...

000875: *Nov 5 12:46:27.022: ISAKMP (2077): incrementing error counter on sa, attempt 2 of 5: retransmit phase 1

000876: *Nov 5 12:46:27.022: ISAKMP:(2077): retransmitting phase 1 MM_KEY_EXCH

000877: *Nov 5 12:46:27.022: ISAKMP:(2077): sending packet to 94.56.66.60 my_port 4500 peer_port 4500 (I) MM_KEY_EXCH

000878: *Nov 5 12:46:27.022: ISAKMP:(2077):Sending an IKE IPv4 Packet.

000879: *Nov 5 12:46:36.286: ISAKMP: set new node 0 to QM_IDLE

000880: *Nov 5 12:46:36.286: ISAKMP:(2077):SA is still budding. Attached new ipsec request to it. (local 192.168.15.96, remote 94.56.66.60)

000881: *Nov 5 12:46:36.286: ISAKMP: Error while processing SA request: Failed to initialize SA

000882: *Nov 5 12:46:36.286: ISAKMP: Error while processing KMI message 0, error 2.

000883: *Nov 5 12:46:36.522: ISAKMP (2077): received packet from 94.56.66.60 dport 500 sport 500 Global (I) MM_KEY_EXCH

000884: *Nov 5 12:46:36.522: ISAKMP:(2077): phase 1 packet is a duplicate of a previous packet.

000885: *Nov 5 12:46:36.522: ISAKMP:(2077): retransmitting due to retransmit phase 1

000886: *Nov 5 12:46:37.022: ISAKMP:(2077): retransmitting phase 1 MM_KEY_EXCH...

000887: *Nov 5 12:46:37.022: ISAKMP (2077): incrementing error counter on sa, attempt 3 of 5: retransmit phase 1

000888: *Nov 5 12:46:37.022: ISAKMP:(2077): retransmitting phase 1 MM_KEY_EXCH

000889: *Nov 5 12:46:37.022: ISAKMP:(2077): sending packet to 94.56.66.60 my_port 4500 peer_port 4500 (I) MM_KEY_EXCH

000890: *Nov 5 12:46:37.022: ISAKMP:(2077):Sending an IKE IPv4 Packet.

000891: *Nov 5 12:46:46.522: ISAKMP (2077): received packet from 94.56.66.60 dport 500 sport 500 Global (I) MM_KEY_EXCH

000892: *Nov 5 12:46:46.522: ISAKMP:(2077): phase 1 packet is a duplicate of a previous packet.

000893: *Nov 5 12:46:46.522: ISAKMP:(2077): retransmitting due to retransmit phase 1

000894: *Nov 5 12:46:47.022: ISAKMP:(2077): retransmitting phase 1 MM_KEY_EXCH...

000895: *Nov 5 12:46:47.022: ISAKMP (2077): incrementing error counter on sa, attempt 4 of 5: retransmit phase 1

000896: *Nov 5 12:46:47.022: ISAKMP:(2077): retransmitting phase 1 MM_KEY_EXCH

000897: *Nov 5 12:46:47.022: ISAKMP:(2077): sending packet to 94.56.66.60 my_port 4500 peer_port 4500 (I) MM_KEY_EXCH

000898: *Nov 5 12:46:47.022: ISAKMP:(2077):Sending an IKE IPv4 Packet.

000899: *Nov 5 12:46:51.350: ISAKMP:(2076):purging node 1501559703

000900: *Nov 5 12:46:51.350: ISAKMP:(2076):purging node -304454932

000901: *Nov 5 12:46:56.522: ISAKMP (2077): received packet from 94.56.66.60 dport 500 sport 500 Global (I) MM_KEY_EXCH

000902: *Nov 5 12:46:56.522: ISAKMP:(2077): phase 1 packet is a duplicate of a previous packet.

000903: *Nov 5 12:46:56.522: ISAKMP:(2077): retransmitting due to retransmit phase 1

000904: *Nov 5 12:46:57.022: ISAKMP:(2077): retransmitting phase 1 MM_KEY_EXCH...

000905: *Nov 5 12:46:57.022: ISAKMP (2077): incrementing error counter on sa, attempt 5 of 5: retransmit phase 1

000906: *Nov 5 12:46:57.022: ISAKMP:(2077): retransmitting phase 1 MM_KEY_EXCH

000907: *Nov 5 12:46:57.022: ISAKMP:(2077): sending packet to 94.56.66.60 my_port 4500 peer_port 4500 (I) MM_KEY_EXCH

000908: *Nov 5 12:46:57.022: ISAKMP:(2077):Sending an IKE IPv4 Packet.

000909: *Nov 5 12:47:01.350: ISAKMP:(2076):purging SA., sa=86CA5534, delme=86CA5534

000910: *Nov 5 12:47:06.286: ISAKMP: set new node 0 to QM_IDLE

000911: *Nov 5 12:47:06.286: ISAKMP:(2077):SA is still budding. Attached new ipsec request to it. (local 192.168.15.96, remote 94.56.66.60)

000912: *Nov 5 12:47:06.286: ISAKMP: Error while processing SA request: Failed to initialize SA

000913: *Nov 5 12:47:06.286: ISAKMP: Error while processing KMI message 0, error 2.

000914: *Nov 5 12:47:07.022: ISAKMP:(2077): retransmitting phase 1 MM_KEY_EXCH...

000915: *Nov 5 12:47:07.022: ISAKMP:(2077):peer does not do paranoid keepalives.

000916: *Nov 5 12:47:07.022: ISAKMP:(2077):deleting SA reason "Death by retransmission P1" state (I) MM_KEY_EXCH (peer 94.56.66.60)

000917: *Nov 5 12:47:07.022: ISAKMP:(2077):deleting SA reason "Death by retransmission P1" state (I) MM_KEY_EXCH (peer 94.56.66.60)

000918: *Nov 5 12:47:07.022: ISAKMP: Unlocking peer struct 0x88F90CC4 for isadb_mark_sa_deleted(), count 0

000919: *Nov 5 12:47:07.022: ISAKMP: Deleting peer node by peer_reap for 94.56.66.60: 88F90CC4

000920: *Nov 5 12:47:07.022: ISAKMP:(2077):deleting node 1354135933 error FALSE reason "IKE deleted"

000921: *Nov 5 12:47:07.022: ISAKMP:(2077):deleting node -517948856 error FALSE reason "IKE deleted"

000922: *Nov 5 12:47:07.022: ISAKMP:(2077):deleting node -1659841532 error FALSE reason "IKE deleted"

000923: *Nov 5 12:47:07.022: ISAKMP:(2077):Input = IKE_MESG_INTERNAL, IKE_PHASE1_DEL

000924: *Nov 5 12:47:07.022: ISAKMP:(2077):Old State = IKE_I_MM5 New State = IKE_DEST_SA

000925: *Nov 5 12:47:36.286: ISAKMP:(0): SA request profile is (NULL)

000926: *Nov 5 12:47:36.286: ISAKMP: Created a peer struct for 94.56.66.60, peer port 500

000927: *Nov 5 12:47:36.286: ISAKMP: New peer created peer = 0x88F90CC4 peer_handle = 0x80000124

000928: *Nov 5 12:47:36.286: ISAKMP: Locking peer struct 0x88F90CC4, refcount 1 for isakmp_initiator

000929: *Nov 5 12:47:36.286: ISAKMP: local port 500, remote port 500

000930: *Nov 5 12:47:36.286: ISAKMP: set new node 0 to QM_IDLE

000931: *Nov 5 12:47:36.286: ISAKMP: Find a dup sa in the avl tree during calling isadb_insert sa = 86CA5534

000932: *Nov 5 12:47:36.286: ISAKMP:(0):Can not start Aggressive mode, trying Main mode.

000933: *Nov 5 12:47:36.286: ISAKMP:(0):found peer pre-shared key matching 94.56.66.60

000934: *Nov 5 12:47:36.286: ISAKMP:(0): constructed NAT-T vendor-rfc3947 ID

000935: *Nov 5 12:47:36.286: ISAKMP:(0): constructed NAT-T vendor-07 ID

000936: *Nov 5 12:47:36.286: ISAKMP:(0): constructed NAT-T vendor-03 ID

000937: *Nov 5 12:47:36.286: ISAKMP:(0): constructed NAT-T vendor-02 ID

000938: *Nov 5 12:47:36.286: ISAKMP:(0):Input = IKE_MESG_FROM_IPSEC, IKE_SA_REQ_MM

000939: *Nov 5 12:47:36.286: ISAKMP:(0):Old State = IKE_READY New State = IKE_I_MM1

000940: *Nov 5 12:47:36.286: ISAKMP:(0): beginning Main Mode exchange

000941: *Nov 5 12:47:36.286: ISAKMP:(0): sending packet to 94.56.66.60 my_port 500 peer_port 500 (I) MM_NO_STATE

000942: *Nov 5 12:47:36.286: ISAKMP:(0):Sending an IKE IPv4 Packet.

000943: *Nov 5 12:47:36.398: ISAKMP (0): received packet from 94.56.66.60 dport 500 sport 500 Global (I) MM_NO_STATE

000944: *Nov 5 12:47:36.398: ISAKMP:(0):Input = IKE_MESG_FROM_PEER, IKE_MM_EXCH

000945: *Nov 5 12:47:36.398: ISAKMP:(0):Old State = IKE_I_MM1 New State = IKE_I_MM2

000946: *Nov 5 12:47:36.398: ISAKMP:(0): processing SA payload. message ID = 0

000947: *Nov 5 12:47:36.398: ISAKMP:(0): processing vendor id payload

000948: *Nov 5 12:47:36.398: ISAKMP:(0): vendor ID seems Unity/DPD but major 69 mismatch

000949: *Nov 5 12:47:36.398: ISAKMP (0): vendor ID is NAT-T RFC 3947

000950: *Nov 5 12:47:36.398: ISAKMP:(0):found peer pre-shared key matching 94.56.66.60

000951: *Nov 5 12:47:36.398: ISAKMP:(0): local preshared key found

000952: *Nov 5 12:47:36.398: ISAKMP : Scanning profiles for xauth ...

000953: *Nov 5 12:47:36.398: ISAKMP:(0):Checking ISAKMP transform 1 against priority 10 policy

000954: *Nov 5 12:47:36.398: ISAKMP:      encryption 3DES-CBC

000955: *Nov 5 12:47:36.398: ISAKMP:      hash SHA

000956: *Nov 5 12:47:36.398: ISAKMP:      default group 2

000957: *Nov 5 12:47:36.398: ISAKMP:      auth pre-share

000958: *Nov 5 12:47:36.398: ISAKMP:      life type in seconds

000959: *Nov 5 12:47:36.398: ISAKMP:      life duration (VPI) of 0x0 0x1 0x51 0x80

000960: *Nov 5 12:47:36.398: ISAKMP:(0):atts are acceptable. Next payload is 0

000961: *Nov 5 12:47:36.398: ISAKMP:(0):Acceptable atts:actual life: 0

000962: *Nov 5 12:47:36.398: ISAKMP:(0):Acceptable atts:life: 0

000963: *Nov 5 12:47:36.398: ISAKMP:(0):Fill atts in sa vpi_length:4

000964: *Nov 5 12:47:36.398: ISAKMP:(0):Fill atts in sa life_in_seconds:86400

000965: *Nov 5 12:47:36.398: ISAKMP:(0):Returning Actual lifetime: 86400

000966: *Nov 5 12:47:36.398: ISAKMP:(0)::Started lifetime timer: 86400.

000967: *Nov 5 12:47:36.398: ISAKMP:(0): processing vendor id payload

000968: *Nov 5 12:47:36.398: ISAKMP:(0): vendor ID seems Unity/DPD but major 69 mismatch

000969: *Nov 5 12:47:36.398: ISAKMP (0): vendor ID is NAT-T RFC 3947

000970: *Nov 5 12:47:36.398: ISAKMP:(0):Input = IKE_MESG_INTERNAL, IKE_PROCESS_MAIN_MODE

000971: *Nov 5 12:47:36.398: ISAKMP:(0):Old State = IKE_I_MM2 New State = IKE_I_MM2

000972: *Nov 5 12:47:36.402: ISAKMP:(0): sending packet to 94.56.66.60 my_port 500 peer_port 500 (I) MM_SA_SETUP

000973: *Nov 5 12:47:36.402: ISAKMP:(0):Sending an IKE IPv4 Packet.

000974: *Nov 5 12:47:36.402: ISAKMP:(0):Input = IKE_MESG_INTERNAL, IKE_PROCESS_COMPLETE

000975: *Nov 5 12:47:36.402: ISAKMP:(0):Old State = IKE_I_MM2 New State = IKE_I_MM3

000976: *Nov 5 12:47:36.526: ISAKMP (0): received packet from 94.56.66.60 dport 500 sport 500 Global (I) MM_SA_SETUP

000977: *Nov 5 12:47:36.526: ISAKMP:(0):Input = IKE_MESG_FROM_PEER, IKE_MM_EXCH

000978: *Nov 5 12:47:36.526: ISAKMP:(0):Old State = IKE_I_MM3 New State = IKE_I_MM4

000979: *Nov 5 12:47:36.530: ISAKMP:(0): processing KE payload. message ID = 0

000980: *Nov 5 12:47:36.562: ISAKMP:(0): processing NONCE payload. message ID = 0

000981: *Nov 5 12:47:36.562: ISAKMP:(0):found peer pre-shared key matching 94.56.66.60

000982: *Nov 5 12:47:36.562: ISAKMP:(2078): processing vendor id payload

000983: *Nov 5 12:47:36.562: ISAKMP:(2078): vendor ID is Unity

000984: *Nov 5 12:47:36.562: ISAKMP:(2078): processing vendor id payload

000985: *Nov 5 12:47:36.562: ISAKMP:(2078): vendor ID is DPD

000986: *Nov 5 12:47:36.562: ISAKMP:(2078): processing vendor id payload

000987: *Nov 5 12:47:36.562: ISAKMP:(2078): speaking to another IOS box!

000988: *Nov 5 12:47:36.562: ISAKMP:received payload type 20

000989: *Nov 5 12:47:36.562: ISAKMP (2078): NAT found, both nodes inside NAT

000990: *Nov 5 12:47:36.562: ISAKMP:received payload type 20

000991: *Nov 5 12:47:36.562: ISAKMP (2078): My hash no match - this node inside NAT

000992: *Nov 5 12:47:36.562: ISAKMP:(2078):Input = IKE_MESG_INTERNAL, IKE_PROCESS_MAIN_MODE

000993: *Nov 5 12:47:36.562: ISAKMP:(2078):Old State = IKE_I_MM4 New State = IKE_I_MM4

000994: *Nov 5 12:47:36.562: ISAKMP:(2078):Send initial contact

000995: *Nov 5 12:47:36.562: ISAKMP:(2078):SA is doing pre-shared key authentication using id type ID_IPV4_ADDR

000996: *Nov 5 12:47:36.562: ISAKMP (2078): ID payload

        next-payload : 8

        type         : 1

        address      : 192.168.15.96

        protocol     : 17

        port         : 0

        length       : 12

000997: *Nov 5 12:47:36.562: ISAKMP:(2078):Total payload length: 12

000998: *Nov 5 12:47:36.562: ISAKMP:(2078): sending packet to 94.56.66.60 my_port 4500 peer_port 4500 (I) MM_KEY_EXCH

000999: *Nov 5 12:47:36.562: ISAKMP:(2078):Sending an IKE IPv4 Packet.

001000: *Nov 5 12:47:36.566: ISAKMP:(2078):Input = IKE_MESG_INTERNAL, IKE_PROCESS_COMPLETE

001001: *Nov 5 12:47:36.566: ISAKMP:(2078):Old State = IKE_I_MM4 New State = IKE_I_MM5

001002: *Nov 5 12:47:46.526: ISAKMP (2078): received packet from 94.56.66.60 dport 500 sport 500 Global (I) MM_KEY_EXCH

001003: *Nov 5 12:47:46.526: ISAKMP:(2078): phase 1 packet is a duplicate of a previous packet.

001004: *Nov 5 12:47:46.526: ISAKMP:(2078): retransmitting due to retransmit phase 1

001005: *Nov 5 12:47:47.026: ISAKMP:(2078): retransmitting phase 1 MM_KEY_EXCH...

001006: *Nov 5 12:47:47.026: ISAKMP (2078): incrementing error counter on sa, attempt 1 of 5: retransmit phase 1

001007: *Nov 5 12:47:47.026: ISAKMP:(2078): retransmitting phase 1 MM_KEY_EXCH

001008: *Nov 5 12:47:47.026: ISAKMP:(2078): sending packet to 94.56.66.60 my_port 4500 peer_port 4500 (I) MM_KEY_EXCH

001009: *Nov 5 12:47:47.026: ISAKMP:(2078):Sending an IKE IPv4 Packet.

001010: *Nov 5 12:47:56.526: ISAKMP (2078): received packet from 94.56.66.60 dport 500 sport 500 Global (I) MM_KEY_EXCH

001011: *Nov 5 12:47:56.526: ISAKMP:(2078): phase 1 packet is a duplicate of a previous packet.

001012: *Nov 5 12:47:56.526: ISAKMP:(2078): retransmitting due to retransmit phase 1

001013: *Nov 5 12:47:57.022: ISAKMP:(2077):purging node 1354135933

001014: *Nov 5 12:47:57.022: ISAKMP:(2077):purging node -517948856

001015: *Nov 5 12:47:57.022: ISAKMP:(2077):purging node -1659841532

001016: *Nov 5 12:47:57.026: ISAKMP:(2078): retransmitting phase 1 MM_KEY_EXCH...

001017: *Nov 5 12:47:57.026: ISAKMP (2078): incrementing error counter on sa, attempt 2 of 5: retransmit phase 1

001018: *Nov 5 12:47:57.026: ISAKMP:(2078): retransmitting phase 1 MM_KEY_EXCH

001019: *Nov 5 12:47:57.026: ISAKMP:(2078): sending packet to 94.56.66.60 my_port 4500 peer_port 4500 (I) MM_KEY_EXCH

001020: *Nov 5 12:47:57.026: ISAKMP:(2078):Sending an IKE IPv4 Packet.

001021: *Nov 5 12:48:06.286: ISAKMP: set new node 0 to QM_IDLE

001022: *Nov 5 12:48:06.286: ISAKMP:(2078):SA is still budding. Attached new ipsec request to it. (local 192.168.15.96, remote 94.56.66.60)

001023: *Nov 5 12:48:06.286: ISAKMP: Error while processing SA request: Failed to initialize SA

001024: *Nov 5 12:48:06.286: ISAKMP: Error while processing KMI message 0, error 2.

001025: *Nov 5 12:48:06.526: ISAKMP (2078): received packet from 94.56.66.60 dport 500 sport 500 Global (I) MM_KEY_EXCH

001026: *Nov 5 12:48:06.526: ISAKMP:(2078): phase 1 packet is a duplicate of a previous packet.

001027: *Nov 5 12:48:06.526: ISAKMP:(2078): retransmitting due to retransmit phase 1

001028: *Nov 5 12:48:07.022: ISAKMP:(2077):purging SA., sa=87F98DE0, delme=87F98DE0

001029: *Nov 5 12:48:07.026: ISAKMP:(2078): retransmitting phase 1 MM_KEY_EXCH...

001030: *Nov 5 12:48:07.026: ISAKMP (2078): incrementing error counter on sa, attempt 3 of 5: retransmit phase 1

001031: *Nov 5 12:48:07.026: ISAKMP:(2078): retransmitting phase 1 MM_KEY_EXCH

001032: *Nov 5 12:48:07.026: ISAKMP:(2078): sending packet to 94.56.66.60 my_port 4500 peer_port 4500 (I) MM_KEY_EXCH

001033: *Nov 5 12:48:07.026: ISAKMP:(2078):Sending an IKE IPv4 Packet.

001034: *Nov 5 12:48:16.526: ISAKMP (2078): received packet from 94.56.66.60 dport 500 sport 500 Global (I) MM_KEY_EXCH

001035: *Nov 5 12:48:16.526: ISAKMP:(2078): phase 1 packet is a duplicate of a previous packet.

001036: *Nov 5 12:48:16.526: ISAKMP:(2078): retransmitting due to retransmit phase 1

001037: *Nov 5 12:48:17.026: ISAKMP:(2078): retransmitting phase 1 MM_KEY_EXCH...

001038: *Nov 5 12:48:17.026: ISAKMP (2078): incrementing error counter on sa, attempt 4 of 5: retransmit phase 1

001039: *Nov 5 12:48:17.026: ISAKMP:(2078): retransmitting phase 1 MM_KEY_EXCH

001040: *Nov 5 12:48:17.026: ISAKMP:(2078): sending packet to 94.56.66.60 my_port 4500 peer_port 4500 (I) MM_KEY_EXCH

001041: *Nov 5 12:48:17.026: ISAKMP:(2078):Sending an IKE IPv4 Packet.

001042: *Nov 5 12:48:26.526: ISAKMP (2078): received packet from 94.56.66.60 dport 500 sport 500 Global (I) MM_KEY_EXCH

001043: *Nov 5 12:48:26.526: ISAKMP:(2078): phase 1 packet is a duplicate of a previous packet.

001044: *Nov 5 12:48:26.526: ISAKMP:(2078): retransmitting due to retransmit phase 1

001045: *Nov 5 12:48:27.026: ISAKMP:(2078): retransmitting phase 1 MM_KEY_EXCH...

001046: *Nov 5 12:48:27.026: ISAKMP (2078): incrementing error counter on sa, attempt 5 of 5: retransmit phase 1

001047: *Nov 5 12:48:27.026: ISAKMP:(2078): retransmitting phase 1 MM_KEY_EXCH

001048: *Nov 5 12:48:27.026: ISAKMP:(2078): sending packet to 94.56.66.60 my_port 4500 peer_port 4500 (I) MM_KEY_EXCH

001049: *Nov 5 12:48:27.026: ISAKMP:(2078):Sending an IKE IPv4 Packet.

001050: *Nov 5 12:48:36.286: ISAKMP: set new node 0 to QM_IDLE

001051: *Nov 5 12:48:36.286: ISAKMP:(2078):SA is still budding. Attached new ipsec request to it. (local 192.168.15.96, remote 94.56.66.60)

001052: *Nov 5 12:48:36.286: ISAKMP: Error while processing SA request: Failed to initialize SA

001053: *Nov 5 12:48:36.286: ISAKMP: Error while processing KMI message 0, error 2.

001054: *Nov 5 12:48:37.026: ISAKMP:(2078): retransmitting phase 1 MM_KEY_EXCH...

001055: *Nov 5 12:48:37.026: ISAKMP:(2078):peer does not do paranoid keepalives.

001056: *Nov 5 12:48:37.026: ISAKMP:(2078):deleting SA reason "Death by retransmission P1" state (I) MM_KEY_EXCH (peer 94.56.66.60)

001057: *Nov 5 12:48:37.026: ISAKMP:(2078):deleting SA reason "Death by retransmission P1" state (I) MM_KEY_EXCH (peer 94.56.66.60)

001058: *Nov 5 12:48:37.026: ISAKMP: Unlocking peer struct 0x88F90CC4 for isadb_mark_sa_deleted(), count 0

001059: *Nov 5 12:48:37.026: ISAKMP: Deleting peer node by peer_reap for 94.56.66.60: 88F90CC4

001060: *Nov 5 12:48:37.026: ISAKMP:(2078):deleting node -2010197579 error FALSE reason "IKE deleted"

001061: *Nov 5 12:48:37.026: ISAKMP:(2078):deleting node -2000843173 error FALSE reason "IKE deleted"

001062: *Nov 5 12:48:37.026: ISAKMP:(2078):deleting node 1131329234 error FALSE reason "IKE deleted"

001063: *Nov 5 12:48:37.026: ISAKMP:(2078):Input = IKE_MESG_INTERNAL, IKE_PHASE1_DEL

001064: *Nov 5 12:48:37.026: ISAKMP:(2078):Old State = IKE_I_MM5 New State = IKE_DEST_SA

==================================================================

spoke config:(KEY and everything is matched )

IOS is -Uptime is 23 hours, 1 minute

System returned to ROM by reload at 13:19:19 UTC Wed Nov 4 2015

System image file is "flash:c880data-universalk9-mz.152-3.T.bin"

==============

crypto isakmp policy 10

encr 3des

authentication pre-share

group 2

crypto isakmp key xxxxx address 0.0.0.0

crypto isakmp keepalive 60 3

!

!

crypto ipsec transform-set 3des-sha esp-3des esp-sha-hmac

!

!

crypto ipsec profile dmvpn

set transform-set 3des-sha

!

!

!

!

!

bba-group pppoe global

!

!

interface Tunnel22

ip address 10.10.1.77 255.255.255.0

no ip redirects

ip mtu 1400

ip nhrp authentication xxxxx

ip nhrp map 10.10.1.1 94.56.66.60

ip nhrp map multicast 94.56.66.60

ip nhrp network-id 12

ip nhrp holdtime 60

ip nhrp nhs 10.10.1.1

ip ospf network broadcast

ip ospf priority 0

tunnel source FastEthernet4

tunnel mode gre multipoint

tunnel key 12

tunnel protection ipsec profile dmvpn shared

!

interface FastEthernet0

description +++LAN Ports+++

switchport access vlan 20

no ip address

!

interface FastEthernet1

description +++LAN Ports+++

switchport access vlan 20

no ip address

!

interface FastEthernet2

description +++LAN Ports+++

switchport access vlan 20

no ip address

!

interface FastEthernet3

description +++LAN Ports+++

switchport mode trunk

no ip address

!

interface FastEthernet4

description +++WAN Interface++++

ip address 192.168.15.96 255.255.255.0

ip helper-address 172.21.77.100

ip nat outside

ip virtual-reassembly in

duplex auto

speed auto

!

interface Vlan1

no ip address

!

interface Vlan10

no ip address

ip nat inside

ip virtual-reassembly in

!

interface Vlan20

description +++LAN Layer 3 Address+++

ip address 172.21.77.1 255.255.255.0

ip nat inside

ip virtual-reassembly in

!

interface Dialer1

no ip address

!

router ospf 1

router-id 172.21.77.1

network 10.10.1.0 0.0.0.255 area 1

network 172.21.77.0 0.0.0.255 area 1

!

no ip forward-protocol nd

ip http server

ip http access-class 1

ip http authentication local

ip http secure-server

!

!

ip nat inside source list 1 interface FastEthernet4 overload

ip route 0.0.0.0 0.0.0.0 192.168.15.7

ip route 172.21.62.0 255.255.255.0 172.21.77.100

ip route 192.168.15.0 255.255.255.0 192.168.15.7

!

access-list 1 permit 172.21.77.0 0.0.0.255

access-list 1 permit 172.21.62.0 0.0.0.255

access-list 1 permit 192.168.18.0 0.0.0.255

access-list 1 permit 192.168.2.0 0.0.0.255

dialer-list 1 protocol ip permit

Vahid Tavajjohi · ‎11-19-2015

im not sure, but add this:

crypto ipsec transform-set 3des-sha esp-3des esp-sha-hmac

mode transport

!

if its help

dnsroot13 · ‎12-06-2015

Hi Vahid,

IN other spokes which are working fine now its configurd as tunnel mode only

crypto ipsec transform-set 3des-sha esp-3des esp-sha-hmac
mode tunnel

I will also check HUB side debug .

dnsroot13 · ‎01-20-2016

Hello Guys

Sorry for the late reply.after upgrading the IOS its stable now .thanks for the inputs

Philip D'Ath · ‎12-06-2015

You can use transport mode if all nodes have a public IPv4 address. Tunnel mode works weather you have a public IP address or not on the outside. As long as all nodes are configured the same it is fine to be using tunnel mode.

Is the flapping happening at regular repeating intervals? if so, reasons I have seen for this happening are:

* Two spokes have the same IP address on their Tunnel interfaces (especially if there are two spokes flapping ...).

* Redistributing the "outside" addresses used for the internal in the interior routing protocol

* IOS Bug. Try comparing the IOS version of a working spoke with one that doesn't work. If they are different, try using the same IOS version as the spoke that works.

If the flapping is not occurring at regular repeating intervals (aka the tunnel actually works for a while and then breaks) then it is more likely to be a circuit issue.

It looks to me like the hub is not happy with this spoke, as there hub keeps retransmitting a packet. We would need a debug from the hub as well. However I bet one of the three things I have given above is the issue.

dnsroot13 · ‎12-06-2015

Hi P.dath The issue is, if the internet links (wan interface) if i shut down and no shut again ,the tunnel will take around 30 mins to comes up else need to reload the router . going to upgrade the IOS as well as there are bugs involved in nearby images.

CSCtx90299

Symptoms: The DMVPN IPsec sessions might get torn down and unable to re- establish themselves after experiencing link-flap events.

Conditions: In a scaled DMVPN environment, when physical-port link-state up/down events happen, there will be stormed IPSec events to tear down and/or re-negotiate the sessions; it might run into a bad state that it cannot establish new sessions. Hence, when those active sessions expire (by time period or volume based), it can no longer be re-created. After some period of time, no more active session remains on the router.

Workaround: Reload the router.

-------------------------------------------

HUB side debug will do after IOS upgrade

Philip D'Ath · ‎12-06-2015

Does your WAN interface have a dynamic IP address by chance? If it does, you'll need to add the below. Note it wont take effect with the current NHRP registration, but the next NHRP registration (you need to wait for the existing NHRP registration to time out so that the next one gets registered with the no-unique flag).

interface Tunnel 22

ip nhrp registration no-unique

I see lots of 'peer does not do paranoid keepalives', however you have keepalives configured in your spoke configuration. Make sure you have keepalives configured on the hub as well. Otherwise you could simply be having trouble with spi mis-matches on the hub side.

crypto isakmp keepalive 60 3

If it is neither of those two issues I vote on an IOS bug. If could be those issues being compounded by an IOS bug. Try going to the next "gold star" release. Especially try making sure you hub is on a good gold star release.

Philip D'Ath · ‎11-20-2015

Need the same debug from the head end to see what it is saying,

DMVPN SPOKE ISSUE flapping