06-01-2012 08:33 PM - edited 02-21-2020 06:06 PM
Helloo All,
This has to be the most weirdest issue I have seen since the past year on my ASA.
I have an ASA 5540 running the 8.4(2) code without any issues until I stumbled upon this problem last week and I have spent sleepless nights with no resolution! So, take a deep breath and here is a brief description of my setup and the problem:
A Simple IPSEC tunnel between my ASA 5540 8.4(2) and a Juniper SSG 140 screen OS 6.3.0r9.0(route based VPN)
The tunnel comes up without any issues but the ASA refuses to encrypt the traffic but decrypts it with GLORY!
below are some debug outputs, show outputs and a packet tracer output which also has an explanation of my WEIRD NAT issue:
my setup - ( I wont get into the tunnel encryption details as my tunnel negotiations are perfect and comes up right off the bat when the ASA is configured as answer only)
CISCO ASA - IPSec networking details
LOCAL NETWORK - 10.2.4.0/28
REMOTE NETWORK - 192.168.171.8/32
JUNIPER SSG 140 - IPSec networking details
PROXY ID:
LOCAL NETWORK - 192.168.171.8/32
REMOTE NETWORK - 10.2.4.0/28
HOSTNAME# sh cry ipsec sa peer <JUNIPER SSG PEER>
peer address: <JUNIPER SSG PEER>
Crypto map tag: outside_map, seq num: 5, local addr: <local peer>
access-list outside_cryptomap_4 extended permit ip 10.2.4.0 255.255.255.240 host 192.168.171.8
local ident (addr/mask/prot/port): (10.2.4.0/255.255.255.240/0/0)
remote ident (addr/mask/prot/port): (192.168.171.8/255.255.255.255/0/0)
current_peer: <JUNIPER SSG PEER>
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 72, #pkts decrypt: 72, #pkts verify: 72
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts comp failed: 0, #pkts decomp failed: 0
#pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
#PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
#send errors: 0, #recv errors: 0
local crypto endpt.: <local peer>/0, remote crypto endpt.: <juniper remote peer>/0
path mtu 1500, ipsec overhead 58, media mtu 1500
current outbound spi: 5041C19F
current inbound spi : 0EC13558
inbound esp sas:
spi: 0x0EC13558 (247543128)
transform: esp-3des esp-sha-hmac no compression
in use settings ={L2L, Tunnel, }
slot: 0, conn_id: 22040576, crypto-map: outside_map
sa timing: remaining key lifetime (sec): 3232
IV size: 8 bytes
replay detection support: Y
Anti replay bitmap:
0xFFFFFFFF 0xFFFFFFFF
outbound esp sas:
spi: 0x5041C19F (1346486687)
transform: esp-3des esp-sha-hmac no compression
in use settings ={L2L, Tunnel, }
slot: 0, conn_id: 22040576, crypto-map: outside_map
sa timing: remaining key lifetime (sec): 3232
IV size: 8 bytes
replay detection support: Y
Anti replay bitmap:
0x00000000 0x00000001
VPN CONTEXTS for this IPSEC tunnel:
HOSTNAME# sh asp table vpn-context det
VPN CTX = 0x0742E6BC
Peer IP = 192.168.171.8
Pointer = 0x78C94BF8
State = UP
Flags = ENCR+ESP
SA = 0x9C28B633
SPI = 0x5041C19D
Group = 0
Pkts = 0
Bad Pkts = 0
Bad SPI = 0
Spoof = 0
Bad Crypto = 0
Rekey Pkt = 0
Rekey Call = 0
VPN Filter = <none>
VPN CTX = 0x07430D3C
Peer IP = 192.168.1.8
Pointer = 0x78F62018
State = UP
Flags = DECR+ESP
SA = 0x9C286E3D
SPI = 0x9B6910C5
Group = 1
Pkts = 297
Bad Pkts = 0
Bad SPI = 0
Spoof = 0
Bad Crypto = 0
Rekey Pkt = 0
Rekey Call = 0
VPN Filter = <none>
access-list outside_cryptomap_4 extended permit ip 10.2.4.0 255.255.255.240 host 192.168.171.8
nat (inside,outside) source static Ren-vers Ren-vers destination static Peer-Host Peer-Host no-proxy-arp route-lookup
object network Ren-vers
subnet 10.2.4.0 255.255.255.240
object network Peer-Host
host 192.168.171.8
sh cry ipsec sa
IKE Peer: <remote peer>
Type : L2L Role : responder
Rekey : no State : MM_ACTIVE
packet tracer output extract for a packet being sent from the 10.2.4.0/28 network to 192.168.171.8 host
Phase: 7
Type: VPN
Subtype: encrypt
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
out id=0x7789d788, priority=70, domain=encrypt, deny=false
hits=2, user_data=0x742e6bc, cs_id=0x7ba38680, reverse, flags=0x0, protocol=0
src ip/id=10.2.4.0, mask=255.255.255.240, port=0
dst ip/id=192.168.171.8, mask=255.255.255.255, port=0, dscp=0x0
input_ifc=any, output_ifc=outside
THE VPN contexts match for the encrytpion+encapsulation and the hits here increment only when I run a packet tracer test from my inside host to the remote peer.
A Full packet tracer output for a packet from the 10.2.4.1 255.255.255.255 network to host 192.168.171.8:
Phase: 1
Type: ACCESS-LIST
Subtype:
Result: ALLOW
Config:
Implicit Rule
Additional Information:
Forward Flow based lookup yields rule:
in id=0x77ebd1b0, priority=1, domain=permit, deny=false
hits=3037156, user_data=0x0, cs_id=0x0, l3_type=0x8
src mac=0000.0000.0000, mask=0000.0000.0000
dst mac=0000.0000.0000, mask=0100.0000.0000
input_ifc=inside, output_ifc=any
Phase: 2
Type: ROUTE-LOOKUP
Subtype: input
Result: ALLOW
Config:
Additional Information:
in 192.168.171.0 255.255.255.0 outside
Phase: 3
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x77ec1030, priority=0, domain=inspect-ip-options, deny=true
hits=212950, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0
input_ifc=inside, output_ifc=any
Phase: 4
Type:
Subtype:
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7c12cb18, priority=18, domain=flow-export, deny=false
hits=172188, user_data=0x78b1f438, cs_id=0x0, use_real_addr, flags=0x0,
src ip/id=0.0.0.0, mask=0.0.0.0, port=0
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0
input_ifc=inside, output_ifc=any
Phase: 5
Type: NAT
Subtype:
Result: ALLOW
Config:
nat (inside,outside) source static Ren-vers Ren-vers destination static Peer-Host Peer-Host no-proxy-arp route-lookup
Additional Information:
Static translate 10.2.4.1/2700 to 10.2.4.1/2700
Forward Flow based lookup yields rule:
in id=0x77e0a878, priority=6, domain=nat, deny=false
hits=9, user_data=0x7b7360a8, cs_id=0x0, use_real_addr, flags=0x0, proto
src ip/id=10.2.4.1, mask=255.255.255.240, port=0
dst ip/id=192.168.171.8, mask=255.255.255.255, port=0, dscp=0x0
input_ifc=inside, output_ifc=outside
(this is the weird NAT issue I am seeing. I see the hits count is incrementing only when I run the packet tracer even thugh I have constant pings(traffic) from the 192.168.171.8 host to the 10.2.4.1/28) - please see the packet capture that i have pasted after this section)
Phase: 6
Type: VPN
Subtype: encrypt
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
out id=0x7b8751f8, priority=70, domain=encrypt, deny=false
hits=3, user_data=0x7432b74, cs_id=0x7ba38680, reverse, flags=0x0, proto
src ip/id=10.2.4.1, mask=255.255.255.240, port=0
dst ip/id=192.168.171.8, mask=255.255.255.255, port=0, dscp=0x0
input_ifc=any, output_ifc=outside
Phase: 7
Type: VPN
Subtype: ipsec-tunnel-flow
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in id=0x78b0c280, priority=69, domain=ipsec-tunnel-flow, deny=false
hits=154, user_data=0x7435f94, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=192.168.171.8, mask=255.255.255.255, port=0
dst ip/id=10.2.4.1, mask=255.255.255.240, port=0, dscp=0x0
input_ifc=outside, output_ifc=any
Phase: 8
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in id=0x77e7a510, priority=0, domain=inspect-ip-options, deny=true
hits=184556, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0
input_ifc=outside, output_ifc=any
Phase: 9
Type: FLOW-CREATION
Subtype:
Result: ALLOW
Config:
Additional Information:
New flow created with id 119880921, packet dispatched to next module
Module information for forward flow ...
snp_fp_tracer_drop
snp_fp_inspect_ip_options
snp_fp_tcp_normalizer
snp_fp_translate
snp_fp_adjacency
snp_fp_encrypt
snp_fp_fragment
snp_ifc_stat
Module information for reverse flow ...
snp_fp_tracer_drop
snp_fp_inspect_ip_options
snp_fp_ipsec_tunnel_flow
snp_fp_translate
snp_fp_tcp_normalizer
snp_fp_adjacency
snp_fp_fragment
snp_ifc_stat
Result:
input-interface: inside
input-status: up
input-line-status: up
output-interface: outside
output-status: up
output-line-status: up
Action: allow
HOSTNAME# sh cap cap1
8 packets captured
1: 12:26:53.376033 192.168.10.252 > 10.2.4.1: icmp: echo request
2: 12:26:53.376597 10.2.4.1 > 192.168.10.252: icmp: echo reply
3: 12:26:56.487905 192.168.171.8 > 10.2.4.1: icmp: echo request
4: 12:27:01.489217 192.168.171.8 > 10.2.4.1: icmp: echo request
5: 12:27:03.378245 192.168.10.252 > 10.2.4.1: icmp: echo request
6: 12:27:03.378825 10.2.4.1 > 192.168.10.252: icmp: echo reply
7: 12:27:06.491597 192.168.171.8 > 10.2.4.1: icmp: echo request
8: 12:27:11.491856 192.168.171.8 > 10.2.4.1: icmp: echo request
8 packets shown
As you can see, there is no echo reply packet at all as the packet is not being encapsulated while it is being sent back.
I have been going madddd with this. Also, this is a live production multitenant firewall with no issues at all apart from this ipsec tunnel to a juniper!!
Also, the 192.168.10.0/24 is another IPSec tunnel remote network to this 10.2.4.0/28 network and this IPSEC tunnel has a similar Juniper SSG 140 screen os 6.3.0r9.0 at the remote end and this woks like a charm without any issues, but the 171 is not being encrypted by the ASA at all.
If anybody could help me out, that would be greatt and greatly appreciated!!
Thanks heaps.!
Solved! Go to Solution.
06-02-2012 02:17 AM
no you can't..
proxy id should be the peer public IP
06-02-2012 02:21 AM
Okay, thanks.
Looks like I have hit a road block then!
I am unable to answer this question though.. I have another IPSec tunnel with the same configs on the ASA and the juniper ssg 140 and that works perfectly fine, but just this. I was under the impression that I knew the IPSec tunneling on ASA's down pat, reckon I was mistaken. The ASA is making me sweat...
Just can't seem to think anything else.
Any other ideas Jen??
Thanks.
06-02-2012 02:39 AM
this has to be a bitter pill to swallow:
after a frantic 5 min phone call, i bounced my ASA a while back and guess what!
IT
STILL
DOESN'T
WORK
maybe it is just the way the juniper ssg is the initiator, the ASA being the asnwer only is not suitable for this IPSEC tunnel. even though it works on the other juniper ssg 140, it just doesn't like it in this instance.
I shall resort to route this traffic from another ipsec hop!
For now, this question remains unanswered just like the rest of the others out there on the internet who had the same issue!
PS: I am still open to try something else out if someone has any other suggesstions.
Thanks.
06-02-2012 03:16 AM
Do you mind sharing the full show run output?
Maybe we can spot something..
06-02-2012 05:11 AM
Hi Jen,
I shall send you a PM with my extracted sh run.
let me know if that is okay. Thanks.
06-02-2012 06:36 AM
Just re-reading all the post again, and here is my conclusion:
1) Packet tracer shows that it is going through just fine, meaning that it is not a configuration error on the ASA.
2) Packet capture shows that there is no echo reply, meaning that the return traffic is not even hitting the ASA, hence nothing will be encrypted because there is nothing arriving on the ASA.
3) Management network is on 192.168.171.0/24, meaning there is a possibility (I don't know how your mgmt network is connected), the return traffic is going to your management network, instead of back towards the ASA inside interface.
06-02-2012 06:48 AM
thanks a lot Jen!
The problem is related to my L3 switch.. something must have gone wrong with the control plane on the active one and it is not routing right and it hasn't failed over yet..
i changed the default gateway on my inside peer to point to the inside interface of the ASA and it works with full glory. oh sweet baby jesuss.. feels soo good..i knew i had the asa down pat! thanks a lot for your time Jen!
A good way to spend a gloomy weekend indoors!
Cheers
06-02-2012 06:51 AM
Perfect!! Now you have to find something else indoors to do tomorrow --> weather forecasting rain again
Please kindly mark the post as answered so others can learn from it. Thanks.
06-02-2012 06:56 AM
haha..being from mel, kinda immune to the wet cold weather!
bring on more asa realted issues.so much funn to play around with packets and thinkk.. i so love security and these boards!!
06-03-2012 05:42 PM
sorted all of this out now.
my L3 switch was buggy, made it failover and it was all sorted.
Also, as a quick heads up, for an IPSec tunnel between an ASA and a Juniper, it is best to have the ASA in an answer only mode and use proxy id's on the Juniper with VPN monitoring.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide