IKEv2 Policy Mismatch when both Remote Access and Site to Site VPN

[mustafa.chapal]

Hi,

I have two Cisco ISR 897VA routers with advanced IP services IOS on each site. Both the routers have one WAN/Outside interface with only one IP address assigned. Both routers are connected through IKEv2 Site to Site VPN tunnel and one of these routers have IKEv2 Remote Access VPN configured on them.

Site to Site and Remote Access VPN both work fine when configured/enabled individually but stop working when both are configured/enabled simultaneously. Meaning when I remove the ikev2 policy for Remote Access, Site to Site VPN starts working fine and vice versa. I even tried merging both the ikev2 policy in one but the issue persists.

I would appreciate if you guys can let me know a workaround so both VPNs can work simultaneously.

Attached are the config and debug for crypto ikev2.

aaa new-model
!
!
aaa authentication login default local
aaa authorization exec default local 
aaa authorization network FlexVPN local

crypto pki server FlexVPN-CA
 no database archive
 grant auto
 eku server-auth client-auth 
 shutdown
!
crypto pki trustpoint FlexVPN-CA
 revocation-check crl
 rsakeypair FlexVPN-CA
!
crypto pki trustpoint FlexVPN
 enrollment url http://96.65.7.4:80
 subject-name cn=example.net
 revocation-check none
 rsakeypair FlexVPN
!
!
!
crypto pki certificate map FlexVPN 10
 issuer-name co cn = flexvpn-ca

crypto ikev2 authorization policy FlexVPN 
 pool FlexVPN
 dns 8.8.8.8 8.8.4.4
 netmask 255.255.255.0
 def-domain example.net
!
crypto ikev2 proposal FlexVPN 
 encryption aes-cbc-128 aes-cbc-256 aes-cbc-192
 integrity sha256
 group 19
no crypto ikev2 proposal default
crypto ikev2 proposal ikev2proposal 
 encryption aes-gcm-128
 prf sha256
 group 19
!
crypto ikev2 policy FlexVPN 
 proposal FlexVPN
no crypto ikev2 policy default
crypto ikev2 policy ikev2policy 
 proposal ikev2proposal
!
crypto ikev2 keyring ikev2keyring
 peer TEST
  address 203.130.1.2
  pre-shared-key local Testing123
  pre-shared-key remote Testing123

crypto ikev2 profile FlexVPN
 match identity remote key-id *$AnyConnectClient$*
 authentication local rsa-sig
 authentication remote anyconnect-eap aggregate
 pki trustpoint FlexVPN
 aaa authentication anyconnect-eap default
 aaa authorization group anyconnect-eap list FlexVPN FlexVPN
 aaa authorization user anyconnect-eap cached
 virtual-template 10
!
crypto ikev2 profile ikev2profile
 match identity remote fqdn 2.example.net
 identity local fqdn 1.example.net
 authentication remote pre-share
 authentication local pre-share
 keyring local ikev2keyring
!
no crypto ikev2 http-url cert

crypto ipsec transform-set ESP-GCM esp-gcm 
 mode tunnel
crypto ipsec transform-set FlexVPN esp-aes 256 esp-sha256-hmac 
 mode tunnel
!
crypto ipsec profile FlexVPN
 set transform-set FlexVPN 
 set ikev2-profile FlexVPN
!
no crypto ipsec profile default
!
crypto ipsec profile ipsecprofile
 set transform-set ESP-GCM 
 set ikev2-profile ikev2profile

interface Tunnel0
 bandwidth 10000000
 ip unnumbered Vlan10
 no ip proxy-arp
 ip nat inside
 ip tcp adjust-mss 1360
 tunnel source GigabitEthernet8
 tunnel mode ipsec ipv4
 tunnel destination 203.130.1.2
 tunnel path-mtu-discovery
 tunnel bandwidth transmit 10000000
 tunnel bandwidth receive 10000000
 tunnel protection ipsec profile ipsecprofile

interface Virtual-Template10 type tunnel
 ip unnumbered Vlan10
 ip nat inside
 ip tcp adjust-mss 1360
 tunnel mode ipsec ipv4
 tunnel path-mtu-discovery
 tunnel protection ipsec profile FlexVPN

376476: Dec 14 00:18:13.871 Chicago: IKEv2:Received Packet [From 203.130.1.2:500/To 96.65.7.4:500/VRF i0:f0] 
Initiator SPI : DC838A76CB5993D2 - Responder SPI : 0000000000000000 Message id: 0
IKEv2 IKE_SA_INIT Exchange REQUEST 
Payload contents: 
 SA KE N VID VID VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP) 

376477: Dec 14 00:18:13.872 Chicago: IKEv2:(SESSION ID = 358580,SA ID = 1):Verify SA init message
376478: Dec 14 00:18:13.873 Chicago: IKEv2:(SESSION ID = 358580,SA ID = 1):Insert SA
376479: Dec 14 00:18:13.873 Chicago: IKEv2:Searching Policy with fvrf 0, local address 96.65.7.4
376480: Dec 14 00:18:13.873 Chicago: IKEv2:Found Policy 'FlexVPN'
376481: Dec 14 00:18:13.873 Chicago: IKEv2:(SESSION ID = 358580,SA ID = 1):Processing IKE_SA_INIT message
376482: Dec 14 00:18:13.876 Chicago: IKEv2-ERROR:(SESSION ID = 358580,SA ID = 1):Received Policies: : Failed to find a matching policyProposal 1:  AES-GCM-128 SHA256 DH_GROUP_256_ECP/Group 19
376483: Dec 14 00:18:13.876 Chicago: 
376484: Dec 14 00:18:13.876 Chicago: 
376485: Dec 14 00:18:13.876 Chicago: IKEv2-ERROR:(SESSION ID = 358580,SA ID = 1):Expected Policies: : Failed to find a matching policyProposal 1:  AES-CBC-128 AES-CBC-256 SHA256 SHA256 DH_GROUP_256_ECP/Group 19
376486: Dec 14 00:18:13.877 Chicago: 
376487: Dec 14 00:18:13.877 Chicago: 
376488: Dec 14 00:18:13.877 Chicago: IKEv2-ERROR:(SESSION ID = 358580,SA ID = 1):: Failed to find a matching policy
376489: Dec 14 00:18:13.877 Chicago: IKEv2:(SESSION ID = 358580,SA ID = 1):Sending no proposal chosen notify 

376490: Dec 14 00:18:13.877 Chicago: IKEv2:(SESSION ID = 358580,SA ID = 1):Sending Packet [To 203.130.1.2:500/From 96.65.7.4:500/VRF i0:f0] 
Initiator SPI : DC838A76CB5993D2 - Responder SPI : 50E9ECBF1C0D0DD6 Message id: 0
IKEv2 IKE_SA_INIT Exchange RESPONSE 
Payload contents: 
 NOTIFY(NO_PROPOSAL_CHOSEN) 

376491: Dec 14 00:18:13.878 Chicago: IKEv2:(SESSION ID = 358580,SA ID = 1):Failed SA init exchange
376492: Dec 14 00:18:13.878 Chicago: IKEv2-ERROR:(SESSION ID = 358580,SA ID = 1):Initial exchange failed: Initial exchange failed
376493: Dec 14 00:18:13.878 Chicago: IKEv2:(SESSION ID = 358580,SA ID = 1):Abort exchange
376494: Dec 14 00:18:13.878 Chicago: IKEv2:(SESSION ID = 358580,SA ID = 1):Deleting SA

[MHM Cisco World]

why you not try config identity under the proposal not only under the profile.

[mustafa.chapal]

I do not understand what you are trying to say as there is no option for identity under proposal

[MHM Cisco World]

crypto ikev2 policy ikev2policy 
match address local x.x.x.x <<<-- 
proposal ikev2pooposal 

the IKEv2 phase1 start the router dont know which proposal use, so we add to proposal under policy the identity, 
this make router use policy for L2L different than that for FlexVPN remote access. 

that what I mean and ask you to try. 

[mustafa.chapal]

I configured match address local under ikev2policy policy but the behavior is the same. Site to site vpn is working but remote access vpn is not.

[MHM Cisco World]

just to update you I run lab and use not same but same step you use and get same result, 
I will try to find result and update you soon.

[tvotna]

Try:

no crypto ikev2 policy FlexVPN 
crypto ikev2 policy ikev2policy 
proposal ikev2proposal
proposal FlexVPN

[mustafa.chapal]

Mentioned in my initial message that I have tried this already.

[MHM Cisco World]

@tvotnagive you good suggest, and I try it and I can pass policy failed (please check my lab screenshot).

NO crypto ikev2 policy ikev2policy 
crypto ikev2 policy MHM 
proposal Flex 
proposal ikev2proposal 

[mustafa.chapal]

@MHM Cisco World @tvotna following is the debug crypto ikev2 if i merge the ikev2 policy in one. the site to site vpn works fine but the remote access is unable to establish for which output is below. also it is considering the incorrect GCM transform set instead of CBC

683294: Dec 18 07:15:57.859 Chicago: IKEv2:Received Packet [From 110.93.2.2:60760/To 96.65.7.4:500/VRF i0:f0] 
Initiator SPI : 3A0F0A43102E3F98 - Responder SPI : 0000000000000000 Message id: 0
IKEv2 IKE_SA_INIT Exchange REQUEST 
Payload contents: 
 SA KE N VID VID VID VID VID VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP) VID CFG NOTIFY(REDIRECT_SUPPORTED) 

683295: Dec 18 07:15:57.859 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Verify SA init message
683296: Dec 18 07:15:57.859 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Insert SA
683297: Dec 18 07:15:57.859 Chicago: IKEv2:Searching Policy with fvrf 0, local address 96.65.7.4
683298: Dec 18 07:15:57.859 Chicago: IKEv2:Found Policy 'FlexVPN'
683299: Dec 18 07:15:57.859 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Processing IKE_SA_INIT message
683300: Dec 18 07:15:57.859 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Received valid config mode data
683301: Dec 18 07:15:57.859 Chicago: IKEv2:Config data recieved:
683302: Dec 18 07:15:57.859 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Config-type: Config-request 
683303: Dec 18 07:15:57.859 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Attrib type: unknown, length: 2, data: 0x2 0x40
683304: Dec 18 07:15:57.859 Chicago: IKEv2:IKEv2 responder - ignoring config data received in IKE_SA_INIT exch
683305: Dec 18 07:15:57.859 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Set received config mode data
683306: Dec 18 07:15:57.859 Chicago: IKEv2:(SA ID = 2):[IKEv2 -> PKI] Retrieve configured trustpoint(s)
683307: Dec 18 07:15:57.859 Chicago: IKEv2:(SA ID = 2):[PKI -> IKEv2] Retrieved trustpoint(s): 'FlexVPN'   'FlexVPN-CA'   'TP-self-signed-653483565'   
683308: Dec 18 07:15:57.863 Chicago: IKEv2:(SA ID = 2):[IKEv2 -> PKI] Get Public Key Hashes of trustpoints
683309: Dec 18 07:15:57.863 Chicago: IKEv2:(SA ID = 2):[PKI -> IKEv2] Getting of Public Key Hashes of trustpoints PASSED
683310: Dec 18 07:15:57.863 Chicago: IKEv2:(SA ID = 2):[IKEv2 -> PKI] Start PKI Session
683311: Dec 18 07:15:57.863 Chicago: IKEv2:(SA ID = 2):[PKI -> IKEv2] Starting of PKI Session PASSED
683312: Dec 18 07:15:57.863 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):[IKEv2 -> Crypto Engine] Computing DH public key, DH Group 19
683313: Dec 18 07:15:57.863 Chicago: IKEv2:(SA ID = 2):[Crypto Engine -> IKEv2] DH key Computation PASSED
683314: Dec 18 07:15:57.863 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Request queued for computation of DH key
683315: Dec 18 07:15:57.863 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):[IKEv2 -> Crypto Engine] Computing DH secret key, DH Group 19
683316: Dec 18 07:15:57.875 Chicago: IKEv2:(SA ID = 2):[Crypto Engine -> IKEv2] DH key Computation PASSED
683317: Dec 18 07:15:57.875 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Request queued for computation of DH secret
683318: Dec 18 07:15:57.875 Chicago: IKEv2:(SA ID = 2):[IKEv2 -> Crypto Engine] Calculate SKEYSEED and create rekeyed IKEv2 SA
683319: Dec 18 07:15:57.875 Chicago: IKEv2:(SA ID = 2):[Crypto Engine -> IKEv2] SKEYSEED calculation and creation of rekeyed IKEv2 SA PASSED
683320: Dec 18 07:15:57.875 Chicago: IKEv2:IKEv2 responder - no config data to send in IKE_SA_INIT exch
683321: Dec 18 07:15:57.875 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Generating IKE_SA_INIT message
683322: Dec 18 07:15:57.875 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):IKE Proposal: 1, SPI size: 0 (initial negotiation), 
Num. transforms: 3
   AES-GCM   SHA256   DH_GROUP_256_ECP/Group 19
683323: Dec 18 07:15:57.875 Chicago: IKEv2:(SA ID = 2):[IKEv2 -> PKI] Retrieve configured trustpoint(s)
683324: Dec 18 07:15:57.875 Chicago: IKEv2:(SA ID = 2):[PKI -> IKEv2] Retrieved trustpoint(s): 'FlexVPN'   'FlexVPN-CA'   'TP-self-signed-653483565'   
683325: Dec 18 07:15:57.875 Chicago: IKEv2:(SA ID = 2):[IKEv2 -> PKI] Get Public Key Hashes of trustpoints
683326: Dec 18 07:15:57.875 Chicago: IKEv2:(SA ID = 2):[PKI -> IKEv2] Getting of Public Key Hashes of trustpoints PASSED 

683327: Dec 18 07:15:57.875 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Sending Packet [To 110.93.2.2:60760/From 96.65.7.4:500/VRF i0:f0] 
Initiator SPI : 3A0F0A43102E3F98 - Responder SPI : 634C4A15C0E0033D Message id: 0
IKEv2 IKE_SA_INIT Exchange RESPONSE 
Payload contents: 
 SA KE N VID VID VID VID VID VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP) CERTREQ 

683328: Dec 18 07:15:57.875 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Completed SA init exchange
683329: Dec 18 07:15:57.875 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Starting timer (30 sec) to wait for auth message 

683330: Dec 18 07:15:58.239 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Received Packet [From 110.93.2.2:53688/To 96.65.7.4:500/VRF i0:f0] 
Initiator SPI : 3A0F0A43102E3F98 - Responder SPI : 634C4A15C0E0033D Message id: 1
IKEv2 IKE_AUTH Exchange REQUEST 
Payload contents: 
 VID IDi CERTREQ CFG SA NOTIFY(IPCOMP_SUPPORTED) TSi TSr NOTIFY(INITIAL_CONTACT) NOTIFY(ESP_TFC_NO_SUPPORT) NOTIFY(NON_FIRST_FRAGS) 

683331: Dec 18 07:15:58.239 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Stopping timer to wait for auth message
683332: Dec 18 07:15:58.239 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Checking NAT discovery
683333: Dec 18 07:15:58.239 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):NAT OUTSIDE found
683334: Dec 18 07:15:58.239 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):NAT detected float to init port 53688, resp port 4500
683335: Dec 18 07:15:58.239 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Searching policy based on peer's identity '*$AnyConnectClient$*' of type 'key ID'
683336: Dec 18 07:15:58.239 Chicago: IKEv2:found matching IKEv2 profile 'FlexVPN'
683337: Dec 18 07:15:58.239 Chicago: IKEv2:Searching Policy with fvrf 0, local address 96.65.7.4
683338: Dec 18 07:15:58.239 Chicago: IKEv2:Found Policy 'FlexVPN'
683339: Dec 18 07:15:58.239 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):not a VPN-SIP session
683340: Dec 18 07:15:58.239 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Verify peer's policy
683341: Dec 18 07:15:58.239 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Peer's policy verified
683342: Dec 18 07:15:58.239 Chicago: IKEv2:(SA ID = 2):[IKEv2 -> PKI] Retrieving trustpoint(s) from received certificate hash(es)
683343: Dec 18 07:15:58.239 Chicago: IKEv2:(SA ID = 2):[PKI -> IKEv2] Retrieved trustpoint(s): NONE
683344: Dec 18 07:15:58.239 Chicago: IKEv2:% Received cert hash is invalid, using configured trustpoints from profile for signing

683345: Dec 18 07:15:58.239 Chicago: IKEv2:(SA ID = 2):[IKEv2 -> PKI] Getting cert chain for the trustpoint FlexVPN
683346: Dec 18 07:15:58.243 Chicago: IKEv2:(SA ID = 2):[PKI -> IKEv2] Getting of cert chain for the trustpoint PASSED
683347: Dec 18 07:15:58.243 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Check for EAP exchange
683348: Dec 18 07:15:58.243 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Check for EAP exchange
683349: Dec 18 07:15:58.243 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Generate my authentication data
683350: Dec 18 07:15:58.243 Chicago: IKEv2:[IKEv2 -> Crypto Engine] Generate IKEv2 authentication data
683351: Dec 18 07:15:58.243 Chicago: IKEv2:[Crypto Engine -> IKEv2] IKEv2 authentication data generation PASSED
683352: Dec 18 07:15:58.243 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Get my authentication method
683353: Dec 18 07:15:58.243 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):My authentication method is 'RSA'
683354: Dec 18 07:15:58.243 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Sign authentication data
683355: Dec 18 07:15:58.243 Chicago: IKEv2:(SA ID = 2):[IKEv2 -> PKI] Getting private key
683356: Dec 18 07:15:58.243 Chicago: IKEv2:(SA ID = 2):[PKI -> IKEv2] Getting of private key PASSED
683357: Dec 18 07:15:58.243 Chicago: IKEv2:(SA ID = 2):[IKEv2 -> Crypto Engine] Sign authentication data
683358: Dec 18 07:15:58.243 Chicago: IKEv2:(SA ID = 2):[Crypto Engine -> IKEv2] Signing of authenticaiton data PASSED
683359: Dec 18 07:15:58.263 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Authentication material has been sucessfully signed
683360: Dec 18 07:15:58.263 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Generating AnyConnect EAP request
683361: Dec 18 07:15:58.263 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Sending AnyConnect EAP 'hello' request
683362: Dec 18 07:15:58.263 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Constructing IDr payload: '96.65.7.4' of type 'IPv4 address'
683363: Dec 18 07:15:58.263 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Building packet for encryption.  
Payload contents: 
 VID IDr CERT CERT AUTH EAP 

683364: Dec 18 07:15:58.267 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Sending Packet [To 110.93.2.2:53688/From 96.65.7.4:4500/VRF i0:f0] 
Initiator SPI : 3A0F0A43102E3F98 - Responder SPI : 634C4A15C0E0033D Message id: 1
IKEv2 IKE_AUTH Exchange RESPONSE 
Payload contents: 
 ENCR 

683365: Dec 18 07:15:58.267 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Starting timer (90 sec) to wait for auth message
683366: Dec 18 07:15:59.803 Chicago: IKEv2-ERROR:(SESSION ID = 1,SA ID = 1):: Maximum number of retransmissions reached
683367: Dec 18 07:15:59.803 Chicago: IKEv2:(SESSION ID = 1,SA ID = 1):Failed SA init exchange
683368: Dec 18 07:15:59.803 Chicago: IKEv2-ERROR:(SESSION ID = 1,SA ID = 1):Initial exchange failed: Initial exchange failed
683369: Dec 18 07:15:59.803 Chicago: IKEv2:(SESSION ID = 1,SA ID = 1):Abort exchange
683370: Dec 18 07:15:59.803 Chicago: IKEv2:(SESSION ID = 1,SA ID = 1):Deleting SA
683371: Dec 18 07:16:00.303 Chicago: %SYS-5-CONFIG_I: Configured from console by  on vty1 (EEM:update-tunnel-destination) 

683372: Dec 18 07:16:02.659 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Received Packet [From 110.93.2.2:53688/To 96.65.7.4:4500/VRF i0:f0] 
Initiator SPI : 3A0F0A43102E3F98 - Responder SPI : 634C4A15C0E0033D Message id: 2
IKEv2 IKE_AUTH Exchange REQUEST 
Payload contents: 
 EAP 

683373: Dec 18 07:16:02.659 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Stopping timer to wait for auth message
683374: Dec 18 07:16:02.659 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Processing AnyConnect EAP response
683375: Dec 18 07:16:02.659 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Checking for Dual Auth
683376: Dec 18 07:16:02.659 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Generating AnyConnect EAP AUTH request
683377: Dec 18 07:16:02.659 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Sending AnyConnect EAP 'auth-request'
683378: Dec 18 07:16:02.659 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Building packet for encryption.  
Payload contents: 
 EAP 

683379: Dec 18 07:16:02.659 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Sending Packet [To 110.93.2.2:53688/From 96.65.7.4:4500/VRF i0:f0] 
Initiator SPI : 3A0F0A43102E3F98 - Responder SPI : 634C4A15C0E0033D Message id: 2
IKEv2 IKE_AUTH Exchange RESPONSE 
Payload contents: 
 ENCR 

683380: Dec 18 07:16:02.659 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Starting timer (90 sec) to wait for auth message 

683381: Dec 18 07:16:11.075 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Received Packet [From 110.93.2.2:53688/To 96.65.7.4:4500/VRF i0:f0] 
Initiator SPI : 3A0F0A43102E3F98 - Responder SPI : 634C4A15C0E0033D Message id: 3
IKEv2 IKE_AUTH Exchange REQUEST 
Payload contents: 
 EAP 

683382: Dec 18 07:16:11.075 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Stopping timer to wait for auth message
683383: Dec 18 07:16:11.075 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Processing AnyConnect EAP response
683384: Dec 18 07:16:11.075 Chicago: IKEv2:Using authentication method list default

683385: Dec 18 07:16:11.075 Chicago: IKEv2:(SA ID = 2):[IKEv2 -> AAA] Authentication request sent
683386: Dec 18 07:16:11.079 Chicago: IKEv2-ERROR:AnyConnect EAP - failed to get author list
683387: Dec 18 07:16:11.079 Chicago: IKEv2:Received response from aaa for AnyConnect EAP
683388: Dec 18 07:16:11.079 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Generating AnyConnect EAP VERIFY request
683389: Dec 18 07:16:11.079 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Sending AnyConnect EAP 'VERIFY' request
683390: Dec 18 07:16:11.079 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Building packet for encryption.  
Payload contents: 
 EAP 

683391: Dec 18 07:16:11.079 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Sending Packet [To 110.93.2.2:53688/From 96.65.7.4:4500/VRF i0:f0] 
Initiator SPI : 3A0F0A43102E3F98 - Responder SPI : 634C4A15C0E0033D Message id: 3
IKEv2 IKE_AUTH Exchange RESPONSE 
Payload contents: 
 ENCR 

683392: Dec 18 07:16:11.079 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Starting timer (90 sec) to wait for auth message 

683393: Dec 18 07:16:11.447 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Received Packet [From 110.93.2.2:53688/To 96.65.7.4:4500/VRF i0:f0] 
Initiator SPI : 3A0F0A43102E3F98 - Responder SPI : 634C4A15C0E0033D Message id: 4
IKEv2 IKE_AUTH Exchange REQUEST 
Payload contents: 
 EAP 

683394: Dec 18 07:16:11.447 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Stopping timer to wait for auth message
683395: Dec 18 07:16:11.447 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Processing AnyConnect EAP ack response
683396: Dec 18 07:16:11.447 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Generating AnyConnect EAP success request
683397: Dec 18 07:16:11.447 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Sending AnyConnect EAP success status message
683398: Dec 18 07:16:11.447 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Building packet for encryption.  
Payload contents: 
 EAP 

683399: Dec 18 07:16:11.447 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Sending Packet [To 110.93.2.2:53688/From 96.65.7.4:4500/VRF i0:f0] 
Initiator SPI : 3A0F0A43102E3F98 - Responder SPI : 634C4A15C0E0033D Message id: 4
IKEv2 IKE_AUTH Exchange RESPONSE 
Payload contents: 
 ENCR 

683400: Dec 18 07:16:11.447 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Starting timer (90 sec) to wait for auth message 

683401: Dec 18 07:16:11.743 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Received Packet [From 110.93.2.2:53688/To 96.65.7.4:4500/VRF i0:f0] 
Initiator SPI : 3A0F0A43102E3F98 - Responder SPI : 634C4A15C0E0033D Message id: 5
IKEv2 IKE_AUTH Exchange REQUEST 
Payload contents: 
 AUTH 

683402: Dec 18 07:16:11.743 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Stopping timer to wait for auth message
683403: Dec 18 07:16:11.743 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Send AUTH, to verify peer after EAP exchange
683404: Dec 18 07:16:11.743 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Verification of peer's authentication data FAILED
683405: Dec 18 07:16:11.743 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Sending authentication failure notify
683406: Dec 18 07:16:11.743 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Building packet for encryption.  
Payload contents: 
 NOTIFY(AUTHENTICATION_FAILED) 

683407: Dec 18 07:16:11.743 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Sending Packet [To 110.93.2.2:53688/From 96.65.7.4:4500/VRF i0:f0] 
Initiator SPI : 3A0F0A43102E3F98 - Responder SPI : 634C4A15C0E0033D Message id: 5
IKEv2 IKE_AUTH Exchange RESPONSE 
Payload contents: 
 ENCR 

683408: Dec 18 07:16:11.743 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Auth exchange failed
683409: Dec 18 07:16:11.743 Chicago: IKEv2-ERROR:(SESSION ID = 3309,SA ID = 2):: Auth exchange failed
683410: Dec 18 07:16:11.743 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Abort exchange
683411: Dec 18 07:16:11.743 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Deleting SA
683412: Dec 18 07:16:11.743 Chicago: IKEv2:(SA ID = 2):[IKEv2 -> PKI] Close PKI Session
683413: Dec 18 07:16:11.743 Chicago: IKEv2:(SA ID = 2):[PKI -> IKEv2] Closing of PKI Session PASSED
cr1#
683414: Dec 18 07:16:23.791 Chicago: IKEv2:(SESSION ID = 4,SA ID = 4):Retransmitting packet 

683415: Dec 18 07:16:23.791 Chicago: IKEv2:(SESSION ID = 4,SA ID = 4):Sending Packet [To 110.93.2.2:500/From 96.65.7.4:500/VRF i0:f0] 
Initiator SPI : BD2FAD59FE486437 - Responder SPI : 0000000000000000 Message id: 0
IKEv2 IKE_SA_INIT Exchange REQUEST 
Payload contents: 
 SA KE N VID VID VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP) 

683416: Dec 18 07:16:28.587 Chicago: IKEv2-ERROR:(SESSION ID = 4,SA ID = 4):: Maximum number of retransmissions reached
683417: Dec 18 07:16:28.587 Chicago: IKEv2:(SESSION ID = 4,SA ID = 4):Failed SA init exchange
683418: Dec 18 07:16:28.587 Chicago: IKEv2-ERROR:(SESSION ID = 4,SA ID = 4):Initial exchange failed: Initial exchange failed
683419: Dec 18 07:16:28.587 Chicago: IKEv2:(SESSION ID = 4,SA ID = 4):Abort exchange
683420: Dec 18 07:16:28.587 Chicago: IKEv2:(SESSION ID = 4,SA ID = 4):Deleting SA

[MHM Cisco World]

As I see there are two issue 
1- policy which I hope we solve it by merge two proposal under one policy 
2- selection of profile, 
debug you share indicate that profile select is FlexVPN for L2L?
if yes are you sure anyconnect using key-id as identify ?
are you sure that L2L use fqdn ?

[mustafa.chapal]

The last debug I shared is the output for remote access and not L2L. L2L is connected fine using fqdn but I am not sure if anyconnect being used to remote access is using key-id or no.

[tvotna]

Profile selection by "key-id" works fine as well as EAP authentication. I believe you're prompted to enter username and password on the client and it seems the password is verified by the router local EAP. It sends EAP Success. Exchange fails after that:

683403: Dec 18 07:16:11.743 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Send AUTH, to verify peer after EAP exchange
683404: Dec 18 07:16:11.743 Chicago: IKEv2:(SESSION ID = 3309,SA ID = 2):Verification of peer's authentication data FAILED

Something wrong with certs?

[mustafa.chapal]

I am not sure if anything is wrong with the certs because when I remove the L2L ikev2policy, anyconnect connects completely fine but due to this L2L stops working.

[MHM Cisco World]

I think about your issue from Yesterday, 
we agree how to solve IKEv2 policy 
now about the anyconnect flexvpn, 
can you check that you can access CA and/or AAA for auth the anyconnect when the L2L tunnel UP. I meaning can you reachable the AAA/CA if the tunnel UP.