VPN Tunnel (is inactive due to Internal Error)

Gerald74 · ‎03-12-2021

Hello,

I have three FTD 6.6.1 managed by FMC 6.6.1, all three are the mesh topology.

I got the following error message:

Tunnel Manager failed to dispatch a KEY_ACQUIRE message.
Probable mis-configuration of the crypto map or tunnel-group. Map Tag = unknown. Map Sequence Number = 0.

Tunnel Manager has failed to establish an L2L SA. All configured IKE versions failed to establish the tunnel. Map Tag= CSM_outside_map. Map Sequence Number = 2.

The tunnel goes down then nothing works properly in the network.

I tried all IKE2 version nothing works.

Thank you.

balaji.bandi · ‎03-12-2021

check the common errors while building the tunnels :

https://www.cisco.com/c/en/us/support/docs/security/asa-5500-x-series-next-generation-firewalls/81824-common-ipsec-trouble.html#error752006

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

Spooster IT Services · ‎03-12-2021

Hello @Gerald74

Checkout if your OS is affected by the following bug (CSCvu92056):

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvu92056/?rfs=iqvred

Spooster IT Services Team

Gerald74 · ‎03-12-2021

Thank you but all errors are for ASA!

Gerald74 · ‎03-12-2021

Thank you. No is not affected.

balaji.bandi · ‎03-12-2021

Firepower is LINA(ASA) + IPS (SNORT)

so most the code same.

Also give us more information - what is other side of the device ? Same FTD ?

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

Gerald74 · ‎03-12-2021

Hello,

SiteA is FTD 1140 V6.6.1.91 - SiteB is FTD 1010 V6.6.1.91

Model : Cisco Firepower 1010 Threat Defense (78) Version 6.6.1 (Build 91)

Cisco Adaptive Security Appliance Software Version 9.14(1)150
SSP Operating System Version 2.8(1.129)

Model : Cisco Firepower 1140 Threat Defense (78) Version 6.6.1 (Build 91)

Cisco Adaptive Security Appliance Software Version 9.14(1)150
SSP Operating System Version 2.8(1.129)

have been implemented in production since March 1, from March 8 I got this error than the tunnel goes down.

2021-03-11 07:16:13 Error Tunnel Manager has failed to establish an L2L SA. All configured IKE versions failed to establish the tunnel. Map Tag= CSM_outside_map. Map Sequence Number = 2. IKE and IPsec
firepower.A
Local:IP:4500 Remote:IP:4500 Username:DefaultL2LGroup IKEv2 No pre-shared key or trustpoint configured for self in tunnel group DefaultL2LGroup

seems to be port 4500 blocked?!!!

Now I use:

IKE2 : Policy aes-gcm-null-sha-latest

pre-shared key manual

IPSec aes-gcm

Thank you

balaji.bandi · ‎03-12-2021

have been implemented in production since March 1, from March 8 I got this error than the tunnel goes down.

Does this mean when it was set up originally it was all working, suddenly broken on march 8th onwards?

what is changed ? is the FTD behind NAT of other devices or directly connect to Internet?

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

Gerald74 · ‎03-12-2021

I added another location to the mesh.

Georg Pauwen · ‎03-12-2021

Hello,

most likely there is a misconfiguration in the access lists used in the peering. If possible, post the full configs of the devices involved...

Gerald74 · ‎03-12-2021

: Hardware: FPR-1140, 5279 MB RAM, CPU Atom C3000 series 2000 MHz, 1 CPU (16 cores)
:
NGFW Version 6.6.1
!

object-group network FMC_INLINE_src_rule_268440596

description Auto Generated by FMC from dst of UnifiedNGFWRule# 3 (MLP/mandatory)

access-list CSM_FW_ACL_ remark rule-id 268440596: L7 RULE: VPN-S2S

!
tcp-map UM_STATIC_TCP_MAP
tcp-options range 6 7 allow
tcp-options range 9 18 allow
tcp-options range 20 255 allow
urgent-flag allow
!
no pager
logging list MANAGER_VPN_EVENT_LIST level errors class auth
logging list MANAGER_VPN_EVENT_LIST level errors class vpn
logging list MANAGER_VPN_EVENT_LIST level errors class vpnc
logging list MANAGER_VPN_EVENT_LIST level errors class vpnfo
logging list MANAGER_VPN_EVENT_LIST level errors class vpnlb
logging list MANAGER_VPN_EVENT_LIST level errors class webfo
logging list MANAGER_VPN_EVENT_LIST level errors class webvpn
logging list MANAGER_VPN_EVENT_LIST level errors class ca
logging list MANAGER_VPN_EVENT_LIST level errors class svc
logging list MANAGER_VPN_EVENT_LIST level errors class ssl
logging list MANAGER_VPN_EVENT_LIST level errors class dap
logging list MANAGER_VPN_EVENT_LIST level errors class ipaa
logging FMC MANAGER_VPN_EVENT_LIST
no logging message 106015
no logging message 313001
no logging message 313008
no logging message 106023
no logging message 710003
no logging message 106100
no logging message 302015
no logging message 302014
no logging message 302013
no logging message 302018
no logging message 302017
no logging message 302016
no logging message 302021
no logging message 302020
mtu outside 1500
mtu INSIDE 1500
mtu VLAN150 1500
mtu DMZ-TK 1500
mtu INSIDE2 1500
mtu diagnostic 1500
ip verify reverse-path interface outside
no failover
no monitor-interface VLAN150
no monitor-interface service-module
icmp unreachable rate-limit 1 burst-size 1
icmp permit any INSIDE
no asdm history enable
arp timeout 14400
no arp permit-nonconnected
arp rate-limit 16384

!
object network NET-RA-VPN
nat (outside,outside) dynamic interface
object network all-ipv4
nat (INSIDE,outside) dynamic interface dns
object network NET-VLAN150
nat (VLAN150,outside) dynamic interface
access-group CSM_FW_ACL_ global
route outside 0.0.0.0 0.0.0.0 IP GW ISP 1
timeout xlate 3:00:00
timeout pat-xlate 0:00:30
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 sctp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
timeout tcp-proxy-reassembly 0:00:30
timeout floating-conn 0:00:00
timeout conn-holddown 0:00:15
timeout igp stale-route 0:01:10
aaa-server CiscoAD protocol ldap
max-failed-attempts 4
realm-id 3
aaa-server CiscoAD host 192.168.1.11
server-port 389
ldap-base-dn DC=praxis,DC=local
ldap-group-base-dn DC=praxis,DC=local
ldap-scope subtree
ldap-naming-attribute samaccountname
ldap-login-password *****
ldap-login-dn
server-type microsoft
aaa proxy-limit disable
aaa authentication login-history
http server enable
http 192.168.1.253 255.255.255.255 INSIDE
http 192.168.1.253 255.255.255.255 VLAN150
no snmp-server location
no snmp-server contact
no snmp-server enable traps snmp authentication linkup linkdown coldstart warmstart
crypto ipsec ikev2 ipsec-proposal CSM_IP_1
protocol esp encryption aes-gcm-256 aes-gcm-192 aes-gcm
protocol esp integrity null
crypto ipsec ikev2 ipsec-proposal CSM_IP_2
protocol esp encryption aes-256 aes-192 aes
protocol esp integrity sha-512 sha-384 sha-256 sha-1
crypto ipsec ikev2 ipsec-proposal CSM_IP_3
protocol esp encryption aes-gcm-256
protocol esp integrity sha-1
crypto ipsec security-association pmtu-aging infinite
crypto dynamic-map CSM_outside_map_dynamic 2 set ikev2 ipsec-proposal CSM_IP_1
crypto dynamic-map CSM_outside_map_dynamic 2 set reverse-route
crypto map CSM_outside_map 1 set ikev2 ipsec-proposal CSM_IP_2
crypto map CSM_outside_map 1 set reverse-route
crypto map CSM_outside_map 2 match address CSM_IPSEC_ACL_3
crypto map CSM_outside_map 2 set pfs
crypto map CSM_outside_map 2 set peer IP Site B
crypto map CSM_outside_map 2 set ikev2 ipsec-proposal CSM_IP_1
crypto map CSM_outside_map 2 set reverse-route
crypto map CSM_outside_map 3 match address CSM_IPSEC_ACL_4
crypto map CSM_outside_map 3 set pfs
crypto map CSM_outside_map 3 set peer IP Site C
crypto map CSM_outside_map 3 set ikev2 ipsec-proposal CSM_IP_1
crypto map CSM_outside_map 3 set reverse-route
crypto map CSM_outside_map 4 set ikev2 ipsec-proposal CSM_IP_1
crypto map CSM_outside_map 4 set reverse-route
crypto map CSM_outside_map 5 set ikev2 ipsec-proposal CSM_IP_3
crypto map CSM_outside_map 5 set reverse-route
crypto map CSM_outside_map 30001 ipsec-isakmp dynamic CSM_outside_map_dynamic
crypto map CSM_outside_map interface outside

webvpn
enable outside
http-headers
hsts-server
enable
max-age 31536000
include-sub-domains
no preload
hsts-client
enable
x-content-type-options
x-xss-protection
content-security-policy
anyconnect enable
tunnel-group-list enable
cache
no disable
error-recovery disable
group-policy DfltGrpPolicy attributes
vpn-tunnel-protocol ikev2 ssl-client
user-authentication-idle-timeout none
webvpn
anyconnect keep-installer none
anyconnect modules value dart
anyconnect ask none default anyconnect
http-comp none
activex-relay disable
file-entry disable
file-browsing disable
url-entry disable
deny-message none
group-policy .DefaultS2SGroupPolicy internal
group-policy .DefaultS2SGroupPolicy attributes
vpn-idle-timeout 30
vpn-idle-timeout alert-interval 1
vpn-session-timeout none
vpn-session-timeout alert-interval 1
vpn-filter none
vpn-tunnel-protocol ikev1 ikev2
group-policy AnyConnectRemoteAccess internal
group-policy AnyConnectRemoteAccess attributes
banner none
wins-server none
dns-server value 212.121.128.10 8.8.8.8
dhcp-network-scope none
vpn-simultaneous-logins 3
vpn-idle-timeout 30
vpn-idle-timeout alert-interval 1
vpn-session-timeout none
vpn-session-timeout alert-interval 1
vpn-filter none
vpn-tunnel-protocol ikev2 ssl-client
split-tunnel-policy tunnelspecified
ipv6-split-tunnel-policy tunnelall
split-tunnel-network-list value SPLIT_TUNNEL
default-domain none
split-dns none
split-tunnel-all-dns enable
client-bypass-protocol disable
vlan none
address-pools value RA-VPN-POOL
webvpn
anyconnect ssl dtls enable
anyconnect mtu 1406
anyconnect firewall-rule client-interface public none
anyconnect firewall-rule client-interface private none
anyconnect ssl keepalive 20
anyconnect ssl rekey time none
anyconnect ssl rekey method none
anyconnect dpd-interval client 30
anyconnect dpd-interval gateway 30
anyconnect ssl compression none
anyconnect dtls compression none
anyconnect modules value dart
anyconnect profiles value AA type user
anyconnect ask none default anyconnect
anyconnect ssl df-bit-ignore disable
dynamic-access-policy-record DfltAccessPolicy
tunnel-group IP Site B type ipsec-l2l
tunnel-group IP Site B general-attributes
default-group-policy .DefaultS2SGroupPolicy
tunnel-group IP Site B ipsec-attributes
ikev2 remote-authentication pre-shared-key *****
ikev2 local-authentication pre-shared-key *****
tunnel-group IP Site C type ipsec-l2l
tunnel-group IP Site C general-attributes
default-group-policy .DefaultS2SGroupPolicy
tunnel-group IP Site C ipsec-attributes
ikev2 remote-authentication pre-shared-key *****
ikev2 local-authentication pre-shared-key *****
tunnel-group AnyConnectRemoteAccess type remote-access
tunnel-group AnyConnectRemoteAccess general-attributes
address-pool RA-VPN-POOL
authentication-server-group CiscoAD
default-group-policy AnyConnectRemoteAccess
tunnel-group AnyConnectRemoteAccess webvpn-attributes
group-alias AnyConnectRemoteAccess enable
!
class-map inspection_default
match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
parameters
message-length maximum client auto
message-length maximum 512
no tcp-inspection
policy-map global_policy
class inspection_default
inspect dns preset_dns_map
inspect ftp
inspect h323 h225
inspect h323 ras
inspect rsh
inspect rtsp
inspect sqlnet
inspect skinny
inspect sunrpc
inspect xdmcp
inspect sip
inspect netbios
inspect tftp
inspect ip-options
inspect icmp
inspect icmp error
inspect snmp
class class-default
set connection advanced-options UM_STATIC_TCP_MAP
policy-map type inspect ip-options UM_STATIC_IP_OPTIONS_MAP
parameters
eool action allow
nop action allow
router-alert action allow
!
service-policy global_policy global
prompt hostname context
call-home
profile CiscoTAC-1
no active
destination address http https://tools.cisco.com/its/service/oddce/services/DDCEService
destination address email callhome@cisco.com
destination transport-method http
subscribe-to-alert-group diagnostic
subscribe-to-alert-group environment
subscribe-to-alert-group inventory periodic monthly
subscribe-to-alert-group configuration periodic monthly
subscribe-to-alert-group telemetry periodic daily
app-agent heartbeat interval 1000 retry-count 3
snort preserve-connection
Cryptochecksum:f0388b3ddbbd33706554871a85f90c6e

balaji.bandi · ‎03-12-2021

can you post both the side of the devices to check they are ok.

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

Gerald74 · ‎03-12-2021

SITE B

: Hardware: FPR-1010, 2935 MB RAM, CPU Atom C3000 series 2200 MHz, 1 CPU (4 co res)
:
NGFW Version 6.6.1
!
hostname FTD-MVZ
enable password ***** encrypted
service-module 0 keepalive-timeout 4
service-module 0 keepalive-counter 6
names
no mac-address auto

8.1.0 255.255.255.0
!
tcp-map UM_STATIC_TCP_MAP
tcp-options range 6 7 allow
tcp-options range 9 18 allow
tcp-options range 20 255 allow
urgent-flag allow
!
no pager
logging list MANAGER_VPN_EVENT_LIST level errors class auth
logging list MANAGER_VPN_EVENT_LIST level errors class vpn
logging list MANAGER_VPN_EVENT_LIST level errors class vpnc
logging list MANAGER_VPN_EVENT_LIST level errors class vpnfo
logging list MANAGER_VPN_EVENT_LIST level errors class vpnlb
logging list MANAGER_VPN_EVENT_LIST level errors class webfo
logging list MANAGER_VPN_EVENT_LIST level errors class webvpn
logging list MANAGER_VPN_EVENT_LIST level errors class ca
logging list MANAGER_VPN_EVENT_LIST level errors class svc
logging list MANAGER_VPN_EVENT_LIST level errors class ssl
logging list MANAGER_VPN_EVENT_LIST level errors class dap
logging list MANAGER_VPN_EVENT_LIST level errors class ipaa
logging FMC MANAGER_VPN_EVENT_LIST
no logging message 106015
no logging message 313001
no logging message 313008
no logging message 106023
no logging message 710003
no logging message 106100
no logging message 302015
no logging message 302014
no logging message 302013
no logging message 302018
no logging message 302017
no logging message 302016
no logging message 302021
no logging message 302020
mtu outside 1500
mtu inside 1500
mtu VLAN188 1500
mtu diagnostic 1500
no failover
no monitor-interface VLAN188
no monitor-interface service-module
icmp unreachable rate-limit 1 burst-size 1
icmp permit any outside
no asdm history enable
arp timeout 14400
no arp permit-nonconnected
arp rate-limit 16384
t NET-Illert no-proxy-arp route-lookup
!
object network all-ipv4
nat (any,outside) dynamic interface dns
access-group CSM_FW_ACL_ global
route outside 0.0.0.0 0.0.0.0 92.50.105.153 1
timeout xlate 3:00:00
timeout pat-xlate 0:00:30
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 sctp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
timeout tcp-proxy-reassembly 0:00:30
timeout floating-conn 0:00:00
timeout conn-holddown 0:00:15
timeout igp stale-route 0:01:10
aaa proxy-limit disable
aaa authentication login-history
http server enable
http 192.168.88.0 255.255.255.0 inside
http 192.168.88.0 255.255.255.0 VLAN188
no snmp-server location
no snmp-server contact
no snmp-server enable traps snmp authentication linkup linkdown coldstart warmst art
crypto ipsec ikev2 ipsec-proposal CSM_IP_1
protocol esp encryption aes-gcm-256 aes-gcm-192 aes-gcm
protocol esp integrity null
crypto ipsec ikev2 ipsec-proposal CSM_IP_2
protocol esp encryption aes-256 aes-192 aes
protocol esp integrity sha-512 sha-384 sha-256 sha-1
crypto ipsec ikev2 ipsec-proposal CSM_IP_3
protocol esp encryption aes-gcm-256
protocol esp integrity sha-1
crypto ipsec security-association pmtu-aging infinite
crypto map CSM_outside_map 1 set ikev2 ipsec-proposal CSM_IP_2
crypto map CSM_outside_map 1 set reverse-route
crypto map CSM_outside_map 2 match address CSM_IPSEC_ACL_3
crypto map CSM_outside_map 2 set pfs
crypto map CSM_outside_map 2 set peer IP SITE A
crypto map CSM_outside_map 2 set ikev2 ipsec-proposal CSM_IP_1
crypto map CSM_outside_map 2 set reverse-route
crypto map CSM_outside_map 3 match address CSM_IPSEC_ACL_2
crypto map CSM_outside_map 3 set pfs
crypto map CSM_outside_map 3 set peer IP SITE C
crypto map CSM_outside_map 3 set ikev2 ipsec-proposal CSM_IP_1
crypto map CSM_outside_map 3 set reverse-route
crypto map CSM_outside_map 4 set ikev2 ipsec-proposal CSM_IP_1
crypto map CSM_outside_map 4 set reverse-route
crypto map CSM_outside_map 5 set ikev2 ipsec-proposal CSM_IP_3
crypto map CSM_outside_map 5 set reverse-route
crypto map CSM_outside_map interface outside
crypto ca trustpoint RA-VPN-1.1
enrollment self
subject-name CN=firepower.mlp
ip-address 192.168.1.1
keypair <Default-RSA-Key>
crl configure
crypto ca trustpool policy
crypto ca certificate chain RA-VPN-1.1
certificate 00f6d10f60

crypto ikev2 policy 1
encryption aes-gcm-256
integrity null
group 14
prf sha256
lifetime seconds 86400
crypto ikev2 enable outside
crypto ikev1 am-disable
telnet timeout 5
console timeout 0
dhcpd dns 192.168.88.253 8.8.8.8
!
dhcpd address 192.168.88.180-192.168.88.199 inside
dhcpd enable inside
!
threat-detection basic-threat
threat-detection statistics access-list
no threat-detection statistics tcp-intercept
group-policy .DefaultS2SGroupPolicy internal
group-policy .DefaultS2SGroupPolicy attributes
vpn-idle-timeout 30
vpn-idle-timeout alert-interval 1
vpn-session-timeout none
vpn-session-timeout alert-interval 1
vpn-filter none
vpn-tunnel-protocol ikev1 ikev2
dynamic-access-policy-record DfltAccessPolicy
tunnel-group IP SITE A type ipsec-l2l
tunnel-group IP SITE A general-attributes
default-group-policy .DefaultS2SGroupPolicy
tunnel-group IP SITE A ipsec-attributes
ikev2 remote-authentication pre-shared-key *****
ikev2 local-authentication pre-shared-key *****
tunnel-group IP SITE C type ipsec-l2l
tunnel-group IP SITE C general-attributes
default-group-policy .DefaultS2SGroupPolicy
tunnel-group IP SITE C ipsec-attributes
ikev2 remote-authentication pre-shared-key *****
ikev2 local-authentication pre-shared-key *****
!
class-map inspection_default
match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
parameters
message-length maximum client auto
message-length maximum 512
no tcp-inspection
policy-map global_policy
class inspection_default
inspect dns preset_dns_map
inspect ftp
inspect h323 h225
inspect h323 ras
inspect rsh
inspect rtsp
inspect sqlnet
inspect skinny
inspect sunrpc
inspect xdmcp
inspect sip
inspect netbios
inspect tftp
inspect ip-options
inspect icmp
inspect icmp error
inspect snmp
class class-default
set connection advanced-options UM_STATIC_TCP_MAP
policy-map type inspect ip-options UM_STATIC_IP_OPTIONS_MAP
parameters
eool action allow
nop action allow
router-alert action allow
!
service-policy global_policy global
prompt hostname context
call-home
profile CiscoTAC-1
no active
destination address http https://tools.cisco.com/its/service/oddce/services/DD CEService
destination address email callhome@cisco.com
destination transport-method http
subscribe-to-alert-group diagnostic
subscribe-to-alert-group environment
subscribe-to-alert-group inventory periodic monthly
subscribe-to-alert-group configuration periodic monthly
subscribe-to-alert-group telemetry periodic daily
app-agent heartbeat interval 1000 retry-count 3
snort preserve-connection
Cryptochecksum:767f136714f4911533f5f3e85cc90926
: end
>

Gerald74 · ‎03-12-2021

Scott Leport · ‎03-17-2021

Hi there,

If you run the Packet Tracer command on either FTD, what are the results you get?

I didn't see any NAT exemption in the configs you've supplied, unless the NAT exemption is somehow being covered by this line at Site A:

object network NET-RA-VPN
nat (outside,outside) dynamic interface

Not how I would write it TBH. In any case, that line is not configured at Site B, so suspect there is still a bit of work to do here.