Hello,
I have asa 5515 (os: 9.5(2)6) , 3 vpn s2s to branches (asa5506) and strange problem with one prefix which asa5515 can't send via ipsec tunnel. Other prefixes, to other branches are send without problem. Also other prefixes via the same tunnel can go without problem.
I did some debug: I see this packets on capture test as incoming packets, there is correct entry in routing table, there is crypto acl but for this prefix counter shows zero or low number. There is cryptomap on correct interface. There is no ipsec sa for this prefix or there is sa but shows only few packets in one directions. ACL no overlaps eachother, i put entry high, even i tested on first position at crypto acl on both sides. Big thanks for any help or advice how debug further
config asa 5515:
ciscoasa/MAIN# sh running-config crypto
crypto ipsec ikev2 ipsec-proposal DES
protocol esp encryption des
protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal 3DES
protocol esp encryption 3des
protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal AES
protocol esp encryption aes
protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal AES192
protocol esp encryption aes-192
protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal AES256
protocol esp encryption aes-256
protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal AES256_SHA2
protocol esp encryption aes-256
protocol esp integrity sha-256
crypto ipsec security-association pmtu-aging infinite
crypto map CRYPTO_MAP 10 match address branches_1
crypto map CRYPTO_MAP 10 set pfs group14
crypto map CRYPTO_MAP 10 set peer 10.10.83.249
crypto map CRYPTO_MAP 10 set ikev2 ipsec-proposal AES256_SHA2
crypto map CRYPTO_MAP 10 set security-association lifetime seconds 3600
crypto map CRYPTO_MAP interface branch_1
crypto map branch_2_map3 1 match address branch_2_cryptomap
crypto map branch_2_map3 1 set pfs group14
crypto map branch_2_map3 1 set peer 10.10.83.254
crypto map branch_2_map3 1 set ikev2 ipsec-proposal AES256_SHA2
crypto map branch_2_map3 1 set security-association lifetime seconds 3600
crypto map branch_2_map3 interface branch_2
crypto map branch_3_map3 1 match address branch_3_cryptomap
crypto map branch_3_map3 1 set pfs group14
crypto map branch_3_map3 1 set peer 10.10.83.245
crypto map branch_3_map3 1 set ikev2 ipsec-proposal AES256_SHA2
crypto map branch_3_map3 1 set security-association lifetime seconds 3600
crypto map branch_3_map3 interface branch_3
crypto isakmp disconnect-notify
crypto ikev2 policy 10
encryption aes-256
integrity sha256
group 14
prf sha256
lifetime seconds 86400
crypto ikev2 enable branch_1
crypto ikev2 enable branch_3
crypto ikev2 enable branch_2
ciscoasa/MAIN# show access-list branch_2_cryptomap
access-list branch_2_cryptomap;
access-list branch_2_cryptomap line 1 extended permit object-group DM_INLINE_SERVICE_1 interface branch_2 object branch_2_ASA (hitcnt=114) 0xa16fc6aa
access-list branch_2_cryptomap line 1 extended permit ospf interface branch_2 host 10.10.83.254 (hitcnt=22)
access-list branch_2_cryptomap line 1 extended permit udp interface branch_2 host 10.10.83.254 eq isakmp (hitcnt=92)
access-list branch_2_cryptomap line 2 extended permit ip host 10.10.83.205 101.61.11.128 255.255.255.128 (hitcnt=1)
access-list branch_2_cryptomap line 3 extended permit object-group DM_INLINE_PROTOCOL_2 object fw_core 102.65.9.0 255.255.255.248 (hitcnt=10)
access-list branch_2_cryptomap line 3 extended permit ip 20.80.16.0 255.255.255.248 102.65.9.0 255.255.255.248 (hitcnt=10)
access-list branch_2_cryptomap line 3 extended permit icmp 20.80.16.0 255.255.255.248 102.65.9.0 255.255.255.248 (hitcnt=0)
ping from 10.10.83.205 to 101.61.11.141 i see on asa5515:
1: 18:47:57.142952 802.1Q vlan#23 P0 10.10.83.205 > 101.61.11.141: icmp: echo request
7: 18:48:02.142952 802.1Q vlan#23 P0 10.10.83.205 > 101.61.11.141: icmp: echo request
ciscoasa/MAIN(config)# show route 101.61.11.141
Routing Table: MAIN
* 10.10.83.254, from 10.10.83.1, 2:39:28 ago, via branch_2
Route metric is 20, traffic share count is 1
in logs i see connection are immediately deleted/destroyed:
Sep 12 2016 18:53:12: %ASA-7-609002: Teardown local-host branch_2:101.61.11.141 duration 0:00:00
Sep 12 2016 18:53:12: %ASA-7-609001: Built local-host branch_2:101.61.11.141
Sep 12 2016 18:53:12: %ASA-7-609002: Teardown local-host branch_2:101.61.11.141 duration 0:00:00
Sep 12 2016 18:53:17: %ASA-7-609001: Built local-host branch_2:101.61.11.141
Sep 12 2016 18:53:17: %ASA-7-609002: Teardown local-host branch_2:101.61.11.141 duration 0:00:00
Sep 12 2016 18:53:17: %ASA-7-609001: Built local-host branch_2:101.61.11.141
Sep 12 2016 18:53:17: %ASA-7-609002: Teardown local-host branch_2:101.61.11.141 duration 0:00:00
Sep 12 2016 18:53:17: %ASA-7-609001: Built local-host branch_2:101.61.11.141
Sep 12 2016 18:53:17: %ASA-7-609002: Teardown local-host branch_2:101.61.11.141 duration 0:00:00
Sep 12 2016 18:53:18: %ASA-7-609001: Built local-host branch_2:101.61.11.141
Sep 12 2016 18:52:42: %ASA-7-609001: Built local-host HQ:10.10.83.205
Sep 12 2016 18:52:42: %ASA-7-609002: Teardown local-host HQ:10.10.83.205 duration 0:00:00
Sep 12 2016 18:52:42: %ASA-7-609001: Built local-host HQ:10.10.83.205
Sep 12 2016 18:52:42: %ASA-7-609002: Teardown local-host HQ:10.10.83.205 duration 0:00:00
Sep 12 2016 18:52:42: %ASA-7-609001: Built local-host HQ:10.10.83.205
Sep 12 2016 18:52:42: %ASA-7-609002: Teardown local-host HQ:10.10.83.205 duration 0:00:00
Sep 12 2016 18:52:42: %ASA-7-609001: Built local-host HQ:10.10.83.205
Sep 12 2016 18:52:42: %ASA-7-609002: Teardown local-host HQ:10.10.83.205 duration 0:00:00
Sep 12 2016 18:52:42: %ASA-7-609001: Built local-host HQ:10.10.83.205
Sep 12 2016 18:52:42: %ASA-7-609002: Teardown local-host HQ:10.10.83.205 duration 0:00:00
sh crypto ipsec sa peer 10.10.83.254:
Crypto map tag: branch_2_map3, seq num: 1, local addr: 10.10.83.253
access-list branch_2_cryptomap extended permit ip host 10.10.83.205 101.61.11.128 255.255.255.128
local ident (addr/mask/prot/port): (10.10.83.205/255.255.255.255/0/0)
remote ident (addr/mask/prot/port): (101.61.11.128/255.255.255.128/0/0)
current_peer: 10.10.83.254
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 2, #pkts decrypt: 2, #pkts verify: 2
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts comp failed: 0, #pkts decomp failed: 0
#pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
#PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
#TFC rcvd: 0, #TFC sent: 0
#Valid ICMP Errors rcvd: 0, #Invalid ICMP Errors rcvd: 0
#send errors: 0, #recv errors: 0
local crypto endpt.: 10.10.83.253/500, remote crypto endpt.: 10.10.83.254/500
path mtu 1500, ipsec overhead 78(44), media mtu 1500
PMTU time remaining (sec): 0, DF policy: copy-df
ICMP error validation: disabled, TFC packets: disabled
current outbound spi: F308B4A1
current inbound spi : B9D44B3B
inbound esp sas:
spi: 0xB9D44B3B (3117697851)
transform: esp-aes-256 esp-sha-256-hmac no compression
in use settings ={L2L, Tunnel, PFS Group 14, IKEv2, }
slot: 0, conn_id: 13635584, crypto-map: branch_2_map3
sa timing: remaining key lifetime (kB/sec): (4193279/3274)
IV size: 16 bytes
replay detection support: Y
Anti replay bitmap:
0x00000000 0x00000007
outbound esp sas:
spi: 0xF308B4A1 (4077434017)
transform: esp-aes-256 esp-sha-256-hmac no compression
in use settings ={L2L, Tunnel, PFS Group 14, IKEv2, }
slot: 0, conn_id: 13635584, crypto-map: branch_2_map3
sa timing: remaining key lifetime (kB/sec): (3962880/3271)
IV size: 16 bytes
replay detection support: Y
Anti replay bitmap:
0x00000000 0x00000001
debug crypto ipsec:
IPSEC: Completed host OBSA update, SPI 0xF308B4A1
IPSEC: Creating outbound VPN context, SPI 0xF308B4A1
Flags: 0x00000085
SA : 0x00007ff368f8c0c0
SPI : 0xF308B4A1
MTU : 1500 bytes
VCID : 0x00000005
Peer : 0x00000000
SCB : 0x03317487
Channel: 0x00007ff34d14ee80
IPSEC: Increment SA NP ref counter for outbound SPI 0xF308B4A1, old value: 0, new value: 1, (ctm_ipsec_create_vpn_context:7147)
IPSEC: Completed outbound VPN context, SPI 0xF308B4A1
VPN handle: 0x0000000001fe625c
IPSEC: New outbound encrypt rule, SPI 0xF308B4A1
Src addr: 10.10.83.205
Src mask: 255.255.255.255
Dst addr: 101.61.11.128
Dst mask: 255.255.255.128
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 0
Use protocol: false
SPI: 0x00000000
Use SPI: false
IPSEC: Increment SA NP ref counter for outbound SPI 0xF308B4A1, old value: 1, new value: 2, (ctm_ipsec_create_acl_entry:6129)
IPSEC: Completed outbound encrypt rule, SPI 0xF308B4A1
Rule ID: 0x00007ff37a5a6680
IPSEC: Decrement SA NP ref counter for outbound SPI 0xF308B4A1, old value: 2, new value: 1, (ctm_ipsec_create_acl_cb:5231)
IPSEC: New outbound permit rule, SPI 0xF308B4A1
Src addr: 10.10.83.253
Src mask: 255.255.255.255
Dst addr: 10.10.83.254
Dst mask: 255.255.255.255
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 50
Use protocol: true
SPI: 0xF308B4A1
Use SPI: true
IPSEC: Increment SA NP ref counter for outbound SPI 0xF308B4A1, old value: 1, new value: 2, (ctm_ipsec_create_acl_entry:6279)
IPSEC: Completed outbound permit rule, SPI 0xF308B4A1
Rule ID: 0x00007ff36128f2a0
IPSEC: Decrement SA NP ref counter for outbound SPI 0xF308B4A1, old value: 2, new value: 1, (ctm_ipsec_create_acl_cb:5231)
IPSEC: Decrement SA NP ref counter for outbound SPI 0xF308B4A1, old value: 1, new value: 0, (ctm_np_vpn_context_cb:10960)
IPSEC: Increment SA HW ref counter for outbound SPI 0xF308B4A1, old value: 0, new value: 1, (ctm_nlite_ipsec_create_hw_obsa:1242)
IPSEC: Updating the inbound SA, SPI: 0xB9D44B3B
IPSEC: New embryonic SA created @ 0x00007ff36a4b46b0,
SCB: 0x6128D930,
Direction: inbound
SPI : 0xB9D44B3B
Session ID: 0x00D01000
VPIF num : 0x00050004
Tunnel type: l2l
Protocol : esp
Lifetime : 240 seconds
IPSEC: Completed host IBSA update, SPI 0xB9D44B3B
IPSEC: Creating inbound VPN context, SPI 0xB9D44B3B
Flags: 0x00000086
SA : 0x00007ff36a4b46b0
SPI : 0xB9D44B3B
MTU : 0 bytes
VCID : 0x00000005
Peer : 0x01FE625C
SCB : 0x033059B9
Channel: 0x00007ff34d14ee80
IPSEC: Increment SA NP ref counter for inbound SPI 0xB9D44B3B, old value: 0, new value: 1, (ctm_ipsec_create_vpn_context:7061)
IPSEC: Completed inbound VPN context, SPI 0xB9D44B3B
VPN handle: 0x0000000001fe8a14
IPSEC: Updating outbound VPN context 0x01FE625C, SPI 0xF308B4A1
Flags: 0x00000085
SA : 0x00007ff368f8c0c0
SPI : 0xF308B4A1
MTU : 1500 bytes
VCID : 0x00000005
Peer : 0x01FE8A14
SCB : 0x03317487
Channel: 0x00007ff34d14ee80
IPSEC: Increment SA NP ref counter for outbound SPI 0xF308B4A1, old value: 0, new value: 1, (ctm_ipsec_create_vpn_context:7134)
IPSEC: Completed outbound VPN context, SPI 0xF308B4A1
VPN handle: 0x0000000001fe625c
IPSEC: Completed outbound inner rule, SPI 0xF308B4A1
Rule ID: 0x00007ff37a5a6680
IPSEC: Completed outbound outer SPD rule, SPI 0xF308B4A1
Rule ID: 0x00007ff36128f2a0
IPSEC: Decrement SA NP ref counter for outbound SPI 0xF308B4A1, old value: 1, new value: 0, (ctm_np_vpn_context_cb:10960)
debug engine:
CTM: ipsec session with normal priority allocated @ 0x00007ff3659e43a0
CTM: Session 0x00007ff3659e43a0 uses a npx (Nitrox PX) as its hardware engine
CTM: ipsec context allocated for session 0x00007ff3659e43a0
CTM: dh session with no priority allocated @ 0x00007ff361968420
CTM: Session 0x00007ff361968420 uses a npx (Nitrox PX) as its hardware engine
CTM: dh context allocated for session 0x00007ff361968420
CTM: Cached key request received for DH group 14
CTM: DH cache entry removed, DH group: 14, Valid ctr: 31, Low water mark: 28
CTM: Key pair taken from DH group 14's cache, keyset pointer: 0x00002aaab5315970
CTM: ipsec session with normal priority allocated @ 0x00007ff361968420
CTM: Session 0x00007ff361968420 uses a npx (Nitrox PX) as its hardware engine
CTM: ipsec context allocated for session 0x00007ff361968420
