Hi everyone,
I've just encounter a tricky issue about VPN on ASA5515 and wondering if someone's got clue on it.
First of all, Let's detail the context :
- I'm working on ASA 5515 device on 9.4(2)IOS version
- I've 3 peers(A,B,C). each one's got a L2L VPN mounted with others.
- Last firewall's reboot happened a year ago on each firewall.
- "sysopt permit vpn" option is disabled on each FW.
- logs are enabled from notification (5) level
- we use inspect icmp. We assume that FW is statefull even with icmp flow.
- Each firewall is clustered in multiple mode
The first occurrence of the issue :
when I reloaded peer "A", the following situation appears :
-the vpn remains mounted :
- B-C works fine
- A-C works fine
- A-B, works only in one way: from A to B.
-closer look at A-B issue, Using a flow from A LAN to B LAN :
LAN WAN LAN
LAN A -- in| A |out ======== out| B |in -- LAN B
-On firewall "A":
VPN looks normal except that nothing comes from Firewall B:
"show crypto ikev2 sa/ipsec sa" ... return a mounted VPN
"show crypto ipsec stats" is funnier :
Outbound is increasing however there is no Inbound packets
inside(LAN) ACL is hit
nothing comes in outside(WAN) ACL
there is no VPN log nor error counter increasing (show asp drop)
-On firewall "B":
VPN looks normal except that nothing go to FW A:
"show crypto ikev2 sa/ipsec sa" ... return a mounted VPN
"show crypto ipsec stats" return a mirror of the A one :
Inbound packet counter is increasing whereas there is no Outbound packets
outside(WAN) ACL is hit
inside (LAN) ACL is hit in a weird way :
for example let's assume there is 5 pings coming from a device in A LAN to B LAN,
the ACL is triggered only once (when the flow is launched).
there is no VPN log
but the "Expired VPN context counter" increases (show asp drop)
At this point, I was pretty convinced that firewall B was bugged(or at least in an erratic state); Something like Bug (ref CSCtd36473) evenif the known affected releases is only 8.0/8.2(2)
(http://www.cisco.com/c/en/us/support/docs/security/asa-5500-x-series-next-generation-firewalls/81824-common-ipsec-trouble.html#outboundinbound / https://quickview.cloudapps.cisco.com/quickview/bug/CSCtd36473
The decision was made to reload B Firewall.
After this reload, the issue move :
-the vpn still mounted :
- B-C down
- A-C still works fine
- A-B, works fine.
Exact same symptoms than the previous one. with a slight difference, there was an other counter increasing in B>C on C FW :
vpn-overlap-conflict counter increases
This was solved by clearing the "crypto sa"
but we still got the first problem; B-C only work from "B" to "C", nothings come from C.
Same decision was made for firewall C.
After this last reload everything seems alright.
Does someone encounter this issue or similar ?
Do you have any explanation about this behavior ?
Is there any other way to avoid this issue, or getting out from it without reloading ?
Bonus : In fact, I’ve 2 flows per VPN tunnel, and only one is impact as described.
There is also some other tunnels which aren't affected.
