IPSEC tunnel issue between ISR4331 and Palo alto

[vithoba]

we have IPSEC tunnel between cisco ISR router ISR4331/K9 and Palo alto. Getting below messages. The Tunnel interface is up but users unable to browse internet/intranet. When we manually run clear crypto sa peer 161.202.161.11 after that it works and users able to use internet and other applications. This is happening repetitively now. 

000702: Sep 22 02:17:49.847 UTC: %CRYPTO-6-ISAKMP_MANUAL_DELETE: IKE SA manually deleted. Do 'clear crypto sa peer 161.202.161.11' to manually clear IPSec SA's covered by this IKE SA.
000721: Sep 22 03:44:04.633 UTC: %IOSXE-4-PLATFORM: SIP1: cpp_cp: QFP:0.0 Thread:001 TS:00000093734079972594 %CERM_DP-4-DP_TX_BW_LIMIT: Maximum Tx Bandwidth limit of 85000 Kbps reached for Crypto functionality with securityk9 technology package license.
000791: Sep 22 05:03:45.818 UTC: %CRYPTO-6-ISAKMP_MANUAL_DELETE: IKE SA manually deleted. Do 'clear crypto sa peer 161.202.161.11' to manually clear IPSec SA's covered by this IKE SA.
000871: Sep 22 05:04:45.866 UTC: %CRYPTO-6-ISAKMP_MANUAL_DELETE: IKE SA manually deleted. Do 'clear crypto sa peer 161.202.161.11' to manually clear IPSec SA's covered by this IKE SA.
000950: Sep 22 05:05:45.855 UTC: %CRYPTO-6-ISAKMP_MANUAL_DELETE: IKE SA manually deleted. Do 'clear crypto sa peer 161.202.161.11' to manually clear IPSec SA's covered by this IKE SA.
001030: Sep 22 05:06:45.855 UTC: %CRYPTO-6-ISAKMP_MANUAL_DELETE: IKE SA manually deleted. Do 'clear crypto sa peer 161.202.161.11' to manually clear IPSec SA's covered by this IKE SA.
001110: Sep 22 05:07:45.853 UTC: %CRYPTO-6-ISAKMP_MANUAL_DELETE: IKE SA manually deleted. Do 'clear crypto sa peer 161.202.161.11' to manually clear IPSec SA's covered by this IKE SA.
001191: Sep 22 05:08:45.873 UTC: %CRYPTO-6-ISAKMP_MANUAL_DELETE: IKE SA manually deleted. Do 'clear crypto sa peer 161.202.161.11' to manually clear IPSec SA's covered by this IKE SA.

crypto keyring SINGARING
pre-shared-key address 161.202.161.11 key McY2k16!
!
!
!
!
!
crypto isakmp policy 16
encr aes
authentication pre-share
group 2
crypto isakmp key McY2k16! address 161.202.161.11
crypto isakmp invalid-spi-recovery
crypto isakmp keepalive 30 5
crypto isakmp profile ISAKMP-PRO
keyring SINGARING
match identity address 161.202.161.11 255.255.255.255
!
crypto ipsec security-association lifetime seconds 86400
!
crypto ipsec transform-set TSET esp-aes esp-sha-hmac
mode tunnel
!
crypto ipsec profile NewPaloProfile
set transform-set TSET
set isakmp-profile ISAKMP-PRO
!
!

[paul driver]

Hello
Please review this here

[vithoba]

Thanks Paul for the reply. What about the other error?

[paul driver]

Hello
I evsiisage you need to procur a HSEC-K9 license for the rtr

[vithoba]

The phase 1 tunnel is not established, below is the current status.

#sh crypto isakmp sa detail
Codes: C - IKE configuration mode, D - Dead Peer Detection
K - Keepalives, N - NAT-traversal
T - cTCP encapsulation, X - IKE Extended Authentication
psk - Preshared key, rsig - RSA signature
renc - RSA encryption
IPv4 Crypto ISAKMP SA

C-id Local Remote I-VRF Status Encr Hash Auth DH Lifetime Cap.

1157 203.121.70.198 161.202.161.11 ACTIVE aes sha psk 2 0 D
Engine-id:Conn-id = ???
(deleted)

008767: Sep 23 08:36:18.647 UTC: %CRYPTO-6-ISAKMP_MANUAL_DELETE: IKE SA manually deleted. Do 'clear crypto sa peer 161.202.161.11' to manually clear IPSec SA's covered by this IKE SA.
008848: Sep 23 08:37:18.635 UTC: %CRYPTO-6-ISAKMP_MANUAL_DELETE: IKE SA manually deleted. Do 'clear crypto sa peer 161.202.161.11' to manually clear IPSec SA's covered by this IKE SA.
008930: Sep 23 08:38:18.625 UTC: %CRYPTO-6-ISAKMP_MANUAL_DELETE: IKE SA manually deleted. Do 'clear crypto sa peer 161.202.161.11' to manually clear IPSec SA's covered by this IKE SA.
009012: Sep 23 08:39:18.625 UTC: %CRYPTO-6-ISAKMP_MANUAL_DELETE: IKE SA manually deleted. Do 'clear crypto sa peer 161.202.161.11' to manually clear IPSec SA's covered by this IKE SA.
AP-MAL-KL-VPN#sh clo
08:39:52.222 UTC Fri Sep 23 2022

[MHM Cisco World]

keyring SINGARING <<- you use this keyring where is the config of it??

[johnlloyd_13]

hi,

can you post a 'show run' and mask/remove sensitive info (public IP, key string).

make sure you've configured this in your GRE tunnel:

interface Tunnelx
 ip mtu 1400
 ip tcp adjust-mss 1360

[vithoba]

Below is the IPSEC tunnel configuration.

crypto isakmp policy 16
encryption aes
hash sha
authentication pre-share
group 2
crypto isakmp key yyyyyyyy address x.x.x.x
crypto isakmp invalid-spi-recovery
crypto isakmp keepalive 30 5
crypto isakmp profile ISAKMP-PRO
keyring SINGARING
match identity address x.x.x.x 255.255.255.255
!
crypto ipsec security-association lifetime seconds 86400
!
crypto ipsec transform-set TSET esp-aes esp-sha-hmac
mode tunnel
!
crypto ipsec profile NewPaloProfile
set transform-set TSET
set isakmp-profile ISAKMP-PRO
!
!
!
!
!
!
!
!
!
!
interface Tunnel6
no ip address
!
interface Tunnel16
description SINGAPURE-IBM-PA-[TUNNEL13]
ip address 10.2.7.42 255.255.255.252
keepalive 2800 3
tunnel source GigabitEthernet0/0/1
tunnel mode ipsec ipv4
tunnel destination x.x.x.x
tunnel protection ipsec profile NewPaloProfile

[MHM Cisco World]

keyring SINGARING <<- you use this keyring where is the config of it??

but you have only key in global mode not under keyring ??

[vithoba]

Below is the IPSEC tunnel configuration.

crypto isakmp policy 16
encryption aes
hash sha
authentication pre-share
group 2
crypto isakmp key yyyyyyyy address x.x.x.x
crypto isakmp invalid-spi-recovery
crypto isakmp keepalive 30 5
crypto isakmp profile ISAKMP-PRO
keyring SINGARING
match identity address x.x.x.x 255.255.255.255
!
crypto ipsec security-association lifetime seconds 86400
!
crypto ipsec transform-set TSET esp-aes esp-sha-hmac
mode tunnel
!
crypto ipsec profile NewPaloProfile
set transform-set TSET
set isakmp-profile ISAKMP-PRO
!
!
!
!
!
!
!
!
!
!
interface Tunnel6
no ip address
!
interface Tunnel16
description SINGAPURE-IBM-PA-[TUNNEL13]
ip address 10.2.7.42 255.255.255.252
keepalive 2800 3
tunnel source GigabitEthernet0/0/1
tunnel mode ipsec ipv4
tunnel destination x.x.x.x
tunnel protection ipsec profile NewPaloProfile

[MHM Cisco World]

crypto keyring keyring1 <<<
  pre-shared-key address 192.168.0.2 key cisco <<<

crypto isakmp profile profile1
   keyring keyring1
   match identity address 192.168.0.102 255.255.255.255

 this what I not see in config 

[vithoba]

sorry i missed it.

crypto keyring SINGARING
pre-shared-key address x.x.x.x key yyyyyyyyy
pre-shared-key address 0.0.0.0 0.0.0.0 key yyyyyyyyy

[MHM Cisco World]

 show crypto ipsec sa

debug crypto isakmp

can you share the output 
Note:- disable debug after finish troublshooting 

[Georg Pauwen]

Hello,

the problem does not necessarily have to be the Cisco, it could also be on the Palo Alto side. Can you post the XML config from the PA ?

[vithoba]

Thanks all for your support. Issue resolved after upgrading router IOS to latest available.