GRE over IPSEC-Sending TS unacceptable notify-Cisco ASR1001-Palo Alto

[mkrishnan]

Hi

Platform

My end : Cisco ASR1001

Far end : Palo Alto

I am trying to establish  GRE over IPSEC tunnel with a customer using Palo Alto which fails when Palo Alto tries to initiate (role initiator) and Asr1001 is the responder. When the roles are switched (that is every time the tunnel goes down , the tunnel negotiation is initiated by tunnel reset at ASR1001) then tunnel comes up. Appreciate any help, Thank you

Debug logs shows:

Cisco end:

Nov 29 17:44:27.250: IKEv2:(SESSION ID = 64303,SA ID = 2):IPSec policy validate request sent for profile Paradise with psh index 2.

Nov 29 17:44:27.250: IKEv2:(SESSION ID = 64303,SA ID = 2):

Nov 29 17:44:27.254: IKEv2:(SESSION ID = 64303,SA ID = 2):(SA ID = 2):[IPsec -> IKEv2] Callback received for the validate proposal - FAILED.

Nov 29 17:44:27.255: IKEv2-ERROR:(SESSION ID = 64303,SA ID = 2):: There was no IPSEC policy found for received TSNov 29 17:44:27.255: IKEv2:(SESSION ID = 64303,SA ID = 2):Sending TS unacceptable notify

Palo Alto end:

023-12-06 15:16:58.127 -0400 [DEBG]: processing isakmp packet
2023-12-06 15:16:58.127 -0400 [DEBG]: ===
2023-12-06 15:16:58.127 -0400 [DEBG]: 137 bytes message received from 216.16X.XXX.5X
2023-12-06 15:16:58.127 -0400 [DEBG]: { 5: }: [IKE Initiator] response message_id 1 expected 1
2023-12-06 15:16:58.128 -0400 [PWRN]: { 5: }: 69.17.1xx.x0[500] - 216.16X.XXX.5X[500]:0x5607e608e760 vendor id payload ignored
2023-12-06 15:16:58.128 -0400 [PWRN]: { 5: }: 69.17.194.x0[500] - 216.16X.XXX.5X[500]:0x5607e608e760 received notify type NO_PROPOSAL_CHOSEN
2023-12-06 15:16:58.128 -0400 [INFO]: { 5: }: 69.1X.XXX.XX[500] - 216.16X.XXX.5X[500]:0x7fcbf4037610 authentication result: success
2023-12-06 15:16:58.128 -0400 [DEBG]: { 5: }: update response message_id 0x1
2023-12-06 15:16:58.128 -0400 [DEBG]: { 5: }: ikev2_process_child_notify(0x7fcbf4025018, 0x7fcc112a18b0), notify type NO_PROPOSAL_CHOSEN
2023-12-06 15:16:58.128 -0400 [PWRN]: { 5: }: 14 is not a child notify type
2023-12-06 15:16:58.128 -0400 [PERR]: { 5: }: 69.1X.XXX.XX[500] - 216.16X.XXX.5X[500]:0x7fcbf4037610 received Notify type NO_PROPOSAL_CHOSEN, failed establishing child_sa
2023-12-06 15:16:58.128 -0400 [PNTF]: { 5: }: ====> IKEv2 CHILD SA NEGOTIATION FAILED AS INITIATOR, non-rekey; gateway ike-vpn-fw02-ibasis-sig <====
====> Failed SA: 69.1X.XXX.XX[500]-216.16X.XXX.5X[500] message id:0x00000001 parent SN:2494 <==== Error code 19
2023-12-06 15:16:58.128 -0400 [DEBG]: { 5: }: SA established: state INI_IKE_AUTH_RCVD, caller initiator_ike_sa_auth_cont, attach 1
2023-12-06 15:16:58.128 -0400 [PNTF]: { 5: }: ====> IKEv2 IKE SA NEGOTIATION SUCCEEDED AS INITIATOR, non-rekey; gateway ike-vpn-fw02-ibasis-sig <====
====> Established SA: 69.1X.XXX.XX[500]-216.16X.XXX.5X[500] SPI:8dc026cee9b9e51d:5f7739109410fcd4 SN:2494 lifetime 86400 Sec <====

Attached config for Cisco ASR and palto Alto

[MHM Cisco World]

That why I ask you are you use palo in identity command (not match identity)?

You can see in your debug  the profile and policy not match.

Can you copy paste the command (show run) and hide the public IP.

Let me check all

Thanks 

MHM

[mkrishnan]

crypto ikev2 profile Parad
match identity remote address xx.xx.194.70 255.255.255.255
identity local address xxx.xxx.187.52
authentication remote pre-share
authentication local pre-share
keyring local Parad
ivrf IBASIS-PUBLIC
!

crypto ipsec profile Parad
set security-association lifetime seconds 28800
set transform-set Parad
set pfs group14
set ikev2-profile Parad
reverse-route static
!

crypto ipsec transform-set Parad esp-gcm
mode tunnel

Thanks

[MHM Cisco World]

That all config there is no config of policy?

Also tunnel use ivrf IBASIS-PUBLIC

There is no fvrf so

keyring local Parad <- this must without any fvrf 
ivrf IBASIS-PUBLIC <- this ivrf is correct but I never see anyone use it under ikev2 profile' remove it and try

MHM

[mkrishnan]

crypto ikev2 proposal Parad
encryption aes-gcm-128
prf sha256
group 14

crypto ikev2 policy POLICY1
proposal PROPOSAL1
proposal Parad
proposal
proposal

keyring local Parad <- this must without any fvrf

As per debug, psk is shared and authentication is successful , so this has no effect on the issue being reported by ASR1K
ivrf IBASIS-PUBLIC <- this ivrf is correct but I never see anyone use it under ikev2 profile' remove it and try

Again ,ivrf has no bearing  on the issue being reported by ASR1K as this is for our internal routing back to our end device

I am more curious to understand , why ASR1K is getting TS in the messages from Palo Alto when there is no encryption domain/ACL are defined on both ends to validate the interesting traffic as GREoIPSEC is not configured the way  IPSEC tunnels are setup (crypto iskamp)

Similarly when ASR1K is initiating , this TS issue is no longer there the tunnel is established ( what is influencing roles Initiator/responder) so this makes me think when ASR1K initiates its not sending any TS messages to be validated by PALO ALTO and tunnel is established

[MHM Cisco World]

Hi friend 
I try lab config (with out iVRF) and face issue the IKEv2 GREoIPSec not work at all 
show crypto ipsec sa 
show error pkt count increase when I ping from LAN to LAN over tunnel 
I think you face same issue 
and then clear all config and add it again and it work (same config)
so what maybe cause this issue 
1- you run crypto map under the tunnel interface, I read in cisco guide the IOS XE face issue if tunnel source use crypto map and tunnel use crypto profile 
2- you use VTI and then change to GREoIPSec using same tunnel config. 

how I troubleshooting my lab 
show crypto ike2 profile (same as your there is no remote identity !!)
show crypto ike2 session (this include more info that show crypto ikev2 sa)
show crypto ipsec sa 

R1

crypto ikev2 proposal prop
encryption 3des
integrity md5
group 14
!
crypto ikev2 policy pol
match address local 100.0.0.1
proposal prop
!
crypto ikev2 keyring key
peer R2
address 100.0.0.2
pre-shared-key mhm
!
!
!
crypto ikev2 profile prof
match identity remote address 100.0.0.2 255.255.255.255
identity local address 100.0.0.1
authentication remote pre-share
authentication local pre-share
keyring local key
dpd 10 2 periodic
!
!
!
crypto ipsec transform-set mhm esp-des
mode tunnel
!
crypto ipsec profile MHM
set transform-set mhm
set ikev2-profile prof
!
!
!
!
!
!
!
interface Tunnel0
ip address 5.0.0.2 255.255.255.0
tunnel source FastEthernet0/0
tunnel destination 100.0.0.2
tunnel protection ipsec profile MHM

R2

crypto ikev2 proposal prop
encryption 3des
integrity md5
group 14
!
crypto ikev2 policy pol
match address local 100.0.0.2
proposal prop
!
crypto ikev2 keyring key
peer R1
address 100.0.0.1
pre-shared-key mhm
!
!
!
crypto ikev2 profile prof
match identity remote address 100.0.0.1 255.255.255.255
identity local address 100.0.0.2
authentication remote pre-share
authentication local pre-share
keyring local key
dpd 10 2 periodic
!
!
!
crypto ipsec transform-set mhm esp-des
mode tunnel
!
crypto ipsec profile MHM
set transform-set mhm
set ikev2-profile prof
!
!
!
!
!
!
!
interface Tunnel0
ip address 5.0.0.2 255.255.255.0
tunnel source FastEthernet0/0
tunnel destination 100.0.0.1
tunnel protection ipsec profile MHM

[mkrishnan]

Hi

Thank you for the efforts and prompt feedback.

I also tested it in the lab(eveng) but  unable to replicate this issue. When I changed the GREoIPSEC to VTI (on the same config) the tunnel came up but tunnel interface ips were not reachable, the only thing I had to do was to clear the session  and after that ips were reachable.

Nothing conclusive as such from this exercise.

1- you run crypto map under the tunnel interface, I read in cisco guide the IOS XE face issue if tunnel source use crypto map and tunnel use crypto profile

But in this scenario , we are not doing policy based vpn from the very beginning of this setup as Palo Alto dont support Policy based VPN (only route based)

2- you use VTI and then change to GREoIPSec using same tunnel config.

Tested with cisco peers, couldn't replicate the issue in the lab (works straight away)

Thanks

[MHM Cisco World]

A-
1- you run crypto map under the tunnel interface, I read in cisco guide the IOS XE face issue if tunnel source use crypto map and tunnel use crypto profile

But in this scenario , we are not doing policy based vpn from the very beginning of this setup as Palo Alto dont support Policy based VPN (only route based)

I am talking about any Policy based VPN you run not need toward Palo, can you confirm that you dont run any Policy based VPN

B-
can you share the SPI of both side IOS XE and Palo

thanks alot 
MHM 

[mkrishnan]

Hi

we have multiple customers with Policy based vpn sessions running in combination with Route based vpns but never had any issues.

I also tested in the lab with policy based, VTI and GREoIPSEC ASR1K peers all comes up straight away

inbound esp sas:
spi: 0x31426196(826433942)
transform: esp-gcm ,
in use settings ={Tunnel, }
conn id: 18664, flow_id: HW:16664, sibling_flags FFFFFFFF80000048, crypto map: Tunnel203-head-0
sa timing: remaining key lifetime (k/sec): (4607858/2554)
IV size: 8 bytes
replay detection support: Y
Status: ACTIVE(ACTIVE)

inbound ah sas:

inbound pcp sas:

outbound esp sas:
spi: 0xA471921D(2758906397)
transform: esp-gcm ,
in use settings ={Tunnel, }
conn id: 18663, flow_id: HW:16663, sibling_flags FFFFFFFF80000048, crypto map: Tunnel203-head-0
sa timing: remaining key lifetime (k/sec): (4607860/2554)
IV size: 8 bytes
replay detection support: Y
Status: ACTIVE(ACTIVE)

outbound ah sas:

outbound pcp sas:

Thanks

[mkrishnan]

Also ikev2 session detailed:

Session-id:176, Status:UP-IDLE, IKE count:7, CHILD count:0

Tunnel-id Local Remote fvrf/ivrf Status
11 xxx.xxx.187.52/500 xx.xx.194.70/500 none/IBASIS-PUBLIC READY
Encr: AES-GCM, keysize: 128, PRF: SHA256, Hash: None, DH Grp:14, Auth sign: PSK, Auth verify: PSK
Life/Active Time: 86400/62 sec
CE id: 81043, Session-id: 176
Status Description: Negotiation done
Local spi: F52602556D605640 Remote spi: 401AB0C5AFAEBE00
Local id: xxx.xxx.187.52
Remote id: xx.xx.194.70
Local req msg id: 0 Remote req msg id: 2
Local next msg id: 1 Remote next msg id: 2
Local req queued: 0 Remote req queued: 2
Local window: 1 Remote window: 1
DPD configured for 0 seconds, retry 0
Fragmentation not configured.
Dynamic Route Update: disabled
Extended Authentication not configured.
NAT-T is not detected
Cisco Trust Security SGT is disabled
Initiator of SA : No ----> initiator Palo Alto

Session-id:176, Status:UP-ACTIVE, IKE count:2, CHILD count:1

Tunnel-id Local Remote fvrf/ivrf Status
1 xxx.xxx.187.52/500 xx.xx.194.70/500 none/IBASIS-PUBLIC READY
Encr: AES-GCM, keysize: 128, PRF: SHA256, Hash: None, DH Grp:14, Auth sign: PSK, Auth verify: PSK
Life/Active Time: 86400/6 sec
CE id: 81108, Session-id: 176
Status Description: Negotiation done
Local spi: D852F36BED034BD6 Remote spi: E01334A3AA95E4F2
Local id: xxx.xxx.187.52
Remote id: xx.xx.194.70
Local req msg id: 2 Remote req msg id: 1
Local next msg id: 2 Remote next msg id: 1
Local req queued: 2 Remote req queued: 1
Local window: 5 Remote window: 1
DPD configured for 0 seconds, retry 0
Fragmentation not configured.
Dynamic Route Update: disabled
Extended Authentication not configured.
NAT-T is not detected
Cisco Trust Security SGT is disabled
Initiator of SA : Yes -- ASR1K

Tu203 up up Parad

[MHM Cisco World]

the ASR dont accept Palo as initiator and even child SA no build 
add under crypto ipsec 
responder-only 

let check if ASR accept child SA from Palo

MHM 

[mzab]

Hi!

I'm from the palo alto side in this configuration we are having issues with.

I believe we already tried setting responder-only on the ASR and the outcome was the same.

[MHM Cisco World]

Router1#show crypto session

when you do show crypto session did you see any nego session between the Palo and ASR1K? (sure I am talking about the not working case)
MHM