can you post the relevant ASA

Edward856 · ‎11-27-2015

Hello everyone,

I have a problem with one of ours VPN Site-to-site tunnel on Cisco ASA 5515-X, can you take a look on this log:

I already work on this log, and i can see QM FSM ERROR, it seems to refer to crypto ACL but there are both correct, it's the same ACL

I always get Received non-routine Notify message: Invalid hash info (23), can anyone tell me what is the problem of this???

Finally, i get Received encrypted packet with no matching SA, dropping but i get the exact SA on both site. I don't get it...Can you just tell me where i have to search on my config? Thank you very much for your support!

Here's the log:

QM FSM error (P2 struct &0x00007fff2ac41340, mess id 0xce302ad7)!
Duplicate Phase 2 packet detected. Retransmitting last packet.
Received non-routine Notify message: Invalid hash info (23)
PHASE 2 COMPLETED (msgid=ce302ad7)
Initiator resending lost, last msg
Duplicate Phase 2 packet detected. Retransmitting last packet.
Received non-routine Notify message: Invalid hash info (23)
PHASE 2 COMPLETED (msgid=ce302ad7)
Initiator resending lost, last msg
Duplicate Phase 2 packet detected. Retransmitting last packet.
Received non-routine Notify message: Invalid hash info (23)
PHASE 2 COMPLETED (msgid=ce302ad7)
Initiator resending lost, last msg
Duplicate Phase 2 packet detected. Retransmitting last packet.
Received non-routine Notify message: Invalid hash info (23)
PHASE 2 COMPLETED (msgid=ce302ad7)
IPSEC: An inbound LAN-to-LAN SA (SPI= 0x426E840C) between y.y.y.yand x.x.x.x (user= x.x.x.x) has been created.
Group = x.x.x.x, IP = x.x.x.x, Security negotiation complete for LAN-to-LAN Group (x.x.x.x) Initiator, Inbound SPI = 0x426e840c, Outbound SPI = 0x15c976b8
IPSEC: An outbound LAN-to-LAN SA (SPI= 0x15C976B8) between y.y.y.yand x.x.x.x (user= x.x.x.x) has been created.
Group = x.x.x.x, IP = x.x.x.x, Responder forcing change of IPSec rekeying duration from 28800 to 1800 seconds
Group = x.x.x.x, IP = x.x.x.x, Responder forcing change of IKE rekeying duration from 86400 to 28800 seconds
Group = x.x.x.x, IP = x.x.x.x, Responder forcing change of IKE rekeying duration from 86400 to 28800 seconds
Group = x.x.x.x, IP = x.x.x.x, PHASE 1 COMPLETED
AAA retrieved default group policy (DfltGrpPolicy) for user = x.x.x.x
IP = x.x.x.x, IKE Initiator: New Phase 1, Intf inside, IKE Peer x.x.x.x local Proxy Address 10.136.193.0, remote Proxy Address 10.168.194.0, Crypto map (outside_map)
Local:y.y.y.y:500 Remote:x.x.x.x:500 Username:Unknown IKEv2 Received request to establish an IPsec tunnel; local traffic selector = Address Range: 10.136.193.40-10.135.192.40 Protocol: 0 Port Range: 0-65535; remote traffic selector = Address Range: 10.168.194.3-10.168.194.3 Protocol: 0 Port Range: 0-65535
IP = x.x.x.x, Received encrypted packet with no matching SA, dropping
Local:y.y.y.y:500 Remote:x.x.x.x:500 Username:Unknown IKEv2 Received request to establish an IPsec tunnel; local traffic selector = Address Range: 10.136.193.40-10.136.193.40 Protocol: 0 Port Range: 0-65535; remote traffic selector = Address Range: 10.168.194.3-10.168.194.3 Protocol: 0 Port Range: 0-65535
Group = x.x.x.x, Username = x.x.x.x, IP = x.x.x.x, Session disconnected. Session Type: LAN-to-LAN, Duration: 0h:00m:30s, Bytes xmt: 1200, Bytes rcv: 0, Reason: Lost Service
Group = x.x.x.x, IP = x.x.x.x, Session is being torn down. Reason: Lost Service
IPSEC: An inbound LAN-to-LAN SA (SPI= 0x8CF48106) between x.x.x.x and y.y.y.y(user= x.x.x.x) has been deleted.
IPSEC: An outbound LAN-to-LAN SA (SPI= 0x15C976B7) between y.y.y.yand x.x.x.x (user= x.x.x.x) has been deleted.
Group = x.x.x.x, IP = x.x.x.x, QM FSM error (P2 struct &0x00007fff2abed0d0, mess id 0xadf68908)!

Thank you

Dennis Mink · ‎11-29-2015

can you post the relevant ASA config. thanks

Please remember to rate useful posts, by clicking on the stars below.

swj · ‎12-01-2015

Hi Edward,

From the existing logs it's not clear what is the issue.

FSM error is an generic error. Also we see "Duplicate Phase 2 packet detected. Retransmitting last packet" this looks the previous packet is not received on the Remote end and it send the Qm1 again.

To better understand the issue take the condition debug for that particular site.

debug crypto condition peer x.x.x.x

debug crypto ikev1 255

debug crypto ipsec 255

And the captures on the outside interface. Add entire debug ouputs + captuers.

Edward856 · ‎12-02-2015

Hello, swj and thank you for your answer!!

With the debug crypto condition peer x.x.x.x, there's nothing happen..nothing to show you...

How do you debug outside interface?

I think i have a same crypto map for different tunnel, could be the reason of the problem?

Thank you

On the debug crypto ikev1 255

i can see this log:

ASA-CISCO# Dec 02 15:52:43 [IKEv1]IKE Receiver: Packet received on y.y.y.y:500 from x.x.x.x:500

IKEv1 Recv RAW packet dump
06 c7 e6 6d 18 0d d2 83 41 e8 19 01 b7 32 83 0a    | ...m....A....2..
08 10 20 01 18 15 9e e7 00 00 00 fc 67 02 3e 81    | .. .........g.>.
da 27 c7 84 69 82 7b 4b 17 50 8a 72 5c 01 89 a5    | .'..i.{K.P.r\...
56 96 3f 26 07 f1 93 61 7a f5 61 72 ab cd 2a 86    | V.?&...az.ar..*.
f1 27 ec 03 e5 9d 44 91 2f 9a e2 b5 02 6e 22 61    | .'....D./....n"a
d1 08 02 a4 bd 88 11 fd d0 ae 72 fc 07 75 ce 17    | ..........r..u..
cd 8c 22 4e d3 50 90 3e ab 78 fa 57 1e 04 5e 8a    | .."N.P.>.x.W..^.
86 c8 48 0f dd 5a 3a 29 22 9c 84 dc 6a 4c 43 18    | ..H..Z:)"...jLC.
b0 ab 2d d1 a3 59 36 f8 d9 10 38 21 4d 58 c1 56    | ..-..Y6...8!MX.V
14 20 dd 43 77 3c f5 5c 9b c3 e9 01 67 d1 10 cd    | . .Cw<.\....g...
d7 5d 07 18 d1 1d c8 00 92 db a8 80 ea a0 96 46    | .].............F
10 9b f0 13 e0 b7 d4 9d cd a8 0a 6e 13 1d 9b ab    | ...........n....
ed b1 1e 2d c6 61 71 ca 3b 42 69 bc 97 b6 a1 5c    | ...-.aq.;Bi....\
a9 55 6a e1 a8 29 43 71 7a 9b f1 ae ef 3c fb e0    | .Uj..)Cqz....<..
79 dc 08 fa 9c b9 cb ef 4c f6 76 04 df 1e ca dc    | y.......L.v.....
df cd d1 b9 d8 f2 e8 ae bb 49 00 ae                | .........I..

RECV PACKET from x.x.x.x
ISAKMP Header
Initiator COOKIE: 06 c7 e6 6d 18 0d d2 83
Responder COOKIE: 41 e8 19 01 b7 32 83 0a
Next Payload: Hash
Version: 1.0
Exchange Type: Quick Mode
Flags: (Encryption)
MessageID: 18159EE7
Length: 252
Dec 02 15:52:43 [IKEv1]Group = x.x.x.x, IP = x.x.x.x, Duplicate Phase 2 packet detected. Retransmitting last packet.
Dec 02 15:52:43 [IKEv1]Group = x.x.x.x, IP = x.x.x.x, Initiator resending lost, last msg
Dec 02 15:52:43 [IKEv1 DEBUG]Group = x.x.x.x, IP = x.x.x.x, Starting P2 rekey timer: 1515 seconds.
Dec 02 15:52:43 [IKEv1]Group = x.x.x.x, IP = x.x.x.x, PHASE 2 COMPLETED (msgid=18159ee7)
Dec 02 15:52:43 [IKEv1]IKE Receiver: Packet received on y.y.y.y:500 from x.x.x.x:500

IKEv1 Recv RAW packet dump
06 c7 e6 6d 18 0d d2 83 41 e8 19 01 b7 32 83 0a    | ...m....A....2..
08 10 05 01 ea 92 39 da 00 00 00 4c cc 0d 83 0f    | ......9....L....
05 48 2f 5a c4 d0 79 3c 8d 95 58 88 1a 87 16 9d    | .H/Z..y<..X.....
d8 97 c7 85 e2 b6 30 45 2d 47 59 c3 ae 5c 42 33    | ......0E-GY..\B3
ef 36 ce f2 18 67 d7 88 37 cd c8 62                | .6...g..7..b

RECV PACKET from x.x.x.x
ISAKMP Header
Initiator COOKIE: 06 c7 e6 6d 18 0d d2 83
Responder COOKIE: 41 e8 19 01 b7 32 83 0a
Next Payload: Hash
Version: 1.0
Exchange Type: Informational
Flags: (Encryption)
MessageID: EA9239DA
Length: 76

AFTER DECRYPTION
ISAKMP Header
Initiator COOKIE: 06 c7 e6 6d 18 0d d2 83
Responder COOKIE: 41 e8 19 01 b7 32 83 0a
Next Payload: Hash
Version: 1.0
Exchange Type: Informational
Flags: (Encryption)
MessageID: EA9239DA
Length: 76
Payload Hash
    Next Payload: Notification
    Reserved: 00
    Payload Length: 24
    Data:
      e8 33 14 61 6e e9 45 9a 93 0e 4c 67 da 14 95 c6
      10 05 e7 6d
Payload Notification
    Next Payload: None
    Reserved: 00
    Payload Length: 12
    DOI: IPsec
    Protocol-ID: PROTO_ISAKMP
    Spi Size: 0
    Notify Type: INVALID_HASH_INFO
Dec 02 15:52:43 [IKEv1]IP = x.x.x.x, IKE_DECODE RECEIVED Message (msgid=ea9239da) with payloads : HDR + HASH (8) + NOTIFY (11) + NONE (0) total length : 64
Dec 02 15:52:43 [IKEv1 DEBUG]Group = x.x.x.x, IP = x.x.x.x, processing hash payload
Dec 02 15:52:43 [IKEv1 DEBUG]Group = x.x.x.x, IP = x.x.x.x, processing notify payload
Dec 02 15:52:43 [IKEv1]Group = x.x.x.x, IP = x.x.x.x, Received non-routine Notify message: Invalid hash info (23)
no debug crypto ikev1 255

On debug crypto ipsec 255

ASA-CISCO# Dec 02 15:52:43 [IKEv1]IKE Receiver: Packet received on y.y.y.y:500 from x.x.x.x:500

IKEv1 Recv RAW packet dump
06 c7 e6 6d 18 0d d2 83 41 e8 19 01 b7 32 83 0a    | ...m....A....2..
08 10 20 01 18 15 9e e7 00 00 00 fc 67 02 3e 81    | .. .........g.>.
da 27 c7 84 69 82 7b 4b 17 50 8a 72 5c 01 89 a5    | .'..i.{K.P.r\...
56 96 3f 26 07 f1 93 61 7a f5 61 72 ab cd 2a 86    | V.?&...az.ar..*.
f1 27 ec 03 e5 9d 44 91 2f 9a e2 b5 02 6e 22 61    | .'....D./....n"a
d1 08 02 a4 bd 88 11 fd d0 ae 72 fc 07 75 ce 17    | ..........r..u..
cd 8c 22 4e d3 50 90 3e ab 78 fa 57 1e 04 5e 8a    | .."N.P.>.x.W..^.
86 c8 48 0f dd 5a 3a 29 22 9c 84 dc 6a 4c 43 18    | ..H..Z:)"...jLC.
b0 ab 2d d1 a3 59 36 f8 d9 10 38 21 4d 58 c1 56    | ..-..Y6...8!MX.V
14 20 dd 43 77 3c f5 5c 9b c3 e9 01 67 d1 10 cd    | . .Cw<.\....g...
d7 5d 07 18 d1 1d c8 00 92 db a8 80 ea a0 96 46    | .].............F
10 9b f0 13 e0 b7 d4 9d cd a8 0a 6e 13 1d 9b ab    | ...........n....
ed b1 1e 2d c6 61 71 ca 3b 42 69 bc 97 b6 a1 5c    | ...-.aq.;Bi....\
a9 55 6a e1 a8 29 43 71 7a 9b f1 ae ef 3c fb e0    | .Uj..)Cqz....<..
79 dc 08 fa 9c b9 cb ef 4c f6 76 04 df 1e ca dc    | y.......L.v.....
df cd d1 b9 d8 f2 e8 ae bb 49 00 ae                | .........I..

RECV PACKET from x.x.x.x
ISAKMP Header
Initiator COOKIE: 06 c7 e6 6d 18 0d d2 83
Responder COOKIE: 41 e8 19 01 b7 32 83 0a
Next Payload: Hash
Version: 1.0
Exchange Type: Quick Mode
Flags: (Encryption)
MessageID: 18159EE7
Length: 252
Dec 02 15:52:43 [IKEv1]Group = x.x.x.x, IP = x.x.x.x, Duplicate Phase 2 packet detected. Retransmitting last packet.
Dec 02 15:52:43 [IKEv1]Group = x.x.x.x, IP = x.x.x.x, Initiator resending lost, last msg
Dec 02 15:52:43 [IKEv1 DEBUG]Group = x.x.x.x, IP = x.x.x.x, Starting P2 rekey timer: 1515 seconds.
Dec 02 15:52:43 [IKEv1]Group = x.x.x.x, IP = x.x.x.x, PHASE 2 COMPLETED (msgid=18159ee7)
Dec 02 15:52:43 [IKEv1]IKE Receiver: Packet received on y.y.y.y:500 from x.x.x.x:500

IKEv1 Recv RAW packet dump
06 c7 e6 6d 18 0d d2 83 41 e8 19 01 b7 32 83 0a    | ...m....A....2..
08 10 05 01 ea 92 39 da 00 00 00 4c cc 0d 83 0f    | ......9....L....
05 48 2f 5a c4 d0 79 3c 8d 95 58 88 1a 87 16 9d    | .H/Z..y<..X.....
d8 97 c7 85 e2 b6 30 45 2d 47 59 c3 ae 5c 42 33    | ......0E-GY..\B3
ef 36 ce f2 18 67 d7 88 37 cd c8 62                | .6...g..7..b

RECV PACKET from x.x.x.x
ISAKMP Header
Initiator COOKIE: 06 c7 e6 6d 18 0d d2 83
Responder COOKIE: 41 e8 19 01 b7 32 83 0a
Next Payload: Hash
Version: 1.0
Exchange Type: Informational
Flags: (Encryption)
MessageID: EA9239DA
Length: 76

AFTER DECRYPTION
ISAKMP Header
Initiator COOKIE: 06 c7 e6 6d 18 0d d2 83
Responder COOKIE: 41 e8 19 01 b7 32 83 0a
Next Payload: Hash
Version: 1.0
Exchange Type: Informational
Flags: (Encryption)
MessageID: EA9239DA
Length: 76
Payload Hash
    Next Payload: Notification
    Reserved: 00
    Payload Length: 24
    Data:
      e8 33 14 61 6e e9 45 9a 93 0e 4c 67 da 14 95 c6
      10 05 e7 6d
Payload Notification
    Next Payload: None
    Reserved: 00
    Payload Length: 12
    DOI: IPsec
    Protocol-ID: PROTO_ISAKMP
    Spi Size: 0
    Notify Type: INVALID_HASH_INFO
Dec 02 15:52:43 [IKEv1]IP = x.x.x.x, IKE_DECODE RECEIVED Message (msgid=ea9239da) with payloads : HDR + HASH (8) + NOTIFY (11) + NONE (0) total length : 64
Dec 02 15:52:43 [IKEv1 DEBUG]Group = x.x.x.x, IP = x.x.x.x, processing hash payload
Dec 02 15:52:43 [IKEv1 DEBUG]Group = x.x.x.x, IP = x.x.x.x, processing notify payload
Dec 02 15:52:43 [IKEv1]Group = x.x.x.x, IP = x.x.x.x, Received non-routine Notify message: Invalid hash info (23)
no debug crypto ikev1 255

Problem with VPN Site-to-site on Cisco ASA