Re: Asa Troubleshooting IPSEC traffic

blwegrzyn · ‎01-09-2018

I have a IPsec tunnet to amazon VPC client.

The tunnel is up and the VPC side can get access to my resources but I cannot get access to VPC side.

The client claims that inbound security rules are setup to allow my subnet.

How can I troubleshoot if my packet to his network leave the outside interface through the tunnel.

I see the packet increment via show crypto ipsec but how can I be sure that they were sent to the client?

I also see in the packet capture over port 4500 that there is communication between IPsec tunnel pair.

How can I be 100% that my icmp packet left via tunnel.

I see communication over port 4500 but how can i know if this is related to my ICMP request?

Is there a way to check if asa is drooping those in case correct reply was sent via the tunnel?

PCVST-ASA# capture b type raw-data interface outside match udp 1.1.1.1 255.255.255.255 2.2.2.2 255.255.255.255 eq 4500
PCVST-ASA# ping inside 172.31.48.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.31.48.1, timeout is 2 seconds:
?????
Success rate is 0 percent (0/5)
PCVST-ASA# show cap b

20 packets captured

1: 23:37:04.081157 1.1.1.1.4500 > 2.2.2.2.4500: udp 148
2: 23:37:06.076106 1.1.1.1.4500 > 2.2.2.2.4500: udp 148
3: 23:37:07.914823 2.2.2.2.4500 > 1.1.1.1.4500: udp 1
4: 23:37:07.914884 2.2.2.2.4500 > 1.1.1.1.4500: udp 1
5: 23:37:08.076183 1.1.1.1.4500 > 2.2.2.2.4500: udp 148
6: 23:37:09.917249 2.2.2.2.4500 > 1.1.1.1.4500: udp 96
7: 23:37:09.918485 1.1.1.1.4500 > 2.2.2.2.4500: udp 96
8: 23:37:10.076152 1.1.1.1.4500 > 2.2.2.2.4500: udp 148
9: 23:37:12.076091 1.1.1.1.4500 > 2.2.2.2.4500: udp 148
10: 23:37:15.934644 2.2.2.2.4500 > 1.1.1.1.4500: udp 400
11: 23:37:15.937741 1.1.1.1.4500 > 2.2.2.2.4500: udp 448
12: 23:37:16.499272 2.2.2.2.4500 > 1.1.1.1.4500: udp 100
13: 23:37:16.499974 1.1.1.1.4500 > 2.2.2.2.4500: udp 100
14: 23:37:16.510242 2.2.2.2.4500 > 1.1.1.1.4500: udp 100
15: 23:37:16.510319 2.2.2.2.4500 > 1.1.1.1.4500: udp 292
16: 23:37:16.510975 1.1.1.1.4500 > 2.2.2.2.4500: udp 100
17: 23:37:16.511799 1.1.1.1.4500 > 2.2.2.2.4500: udp 356
18: 23:37:16.521503 2.2.2.2.4500 > 1.1.1.1.4500: udp 100
19: 23:37:16.522281 1.1.1.1.4500 > 2.2.2.2.4500: udp 660
20: 23:37:16.532351 2.2.2.2.4500 > 1.1.1.1.4500: udp 100
20 packets shown
PCVST-ASA#

GioGonza · ‎01-10-2018

Hello @blwegrzyn,

Can you test with a packet-tracer and verify what is happening to the traffic once it reaches the ASA?.

packet-tracer input inside icmp <IP IN Network> 8 0 172.31.48.1 detail

HTH

Gio

mls577 · ‎01-10-2018

I concur with the suggestion for packet tracer, that should tell you if it is actually hitting the crypto acl and going to be sent over the tunnel. You could also send traffic and make sure you see encaps / de-encaps on show crypto ipsec sa. If you see it getting encapsulated, then you know it at least is getting to their side. If you don't see any de-encaps for the return traffic you know the issue is on their side.

blwegrzyn · ‎01-10-2018

The traffic work when client initiates the connections.

He is on amazon VPC, and he claims that all acls are ok.

When i initiate the connection i get encap but no decap.

Is seeing encap enough to prove to the client that i am sending the traffic to him?

GioGonza · ‎01-10-2018

Hello @blwegrzyn,

One thing you need to keep in mind, VPN tunnels with Amazon support only 1 ACL on the crypto map that´s why they recommend to use "any" as source of the traffic.

Another thing will be to check the configuration on your side but if you can see encaps that means the traffic is being sent to them and they should be able to see it.

HTH

Gio

blwegrzyn · ‎01-11-2018

yes amazon says in their guide:

"

! This access list should contain a static route corresponding to your VPC CIDR and allow traffic from any subnet.
! If you do not wish to use the "any" source, you must use a single access-list entry for accessing the VPC range.
! If you specify more than one entry for this ACL without using "any" as the source, the VPN will function erratically.

"

So in my case i have:

access-list acl-aws extended permit ip object AWS-Src-10.213.0.0 object AWS-Dest-172.31.48.0

where:

object network AWS-Dest-172.31.48.0
subnet 172.31.48.0 255.255.240.0

object network AWS-Src-10.213.0.0
subnet 10.213.0.0 255.255.0.0

So that works fine.

It must be the other end security rule.

PCVST-ASA# packet-tracer input inside icmp 10.213.3.12 8 0 172.31.48.1

Phase: 1
Type: ACCESS-LIST
Subtype:
Result: ALLOW
Config:
Implicit Rule
Additional Information:
MAC Access list

Phase: 2
Type: UN-NAT
Subtype: static
Result: ALLOW
Config:
nat (inside,outside) source static AWS-Src-10.213.0.0 AWS-Src-10.213.0.0 destination static AWS-Dest-172.31.48.0 AWS-Dest-172.31.48.0
Additional Information:
NAT divert to egress interface outside
Untranslate 172.31.48.1/0 to 172.31.48.1/0

Phase: 3
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group FROM_INSIDE in interface inside
access-list FROM_INSIDE extended permit icmp any any object-group TRACEroute
object-group icmp-type TRACEroute
icmp-object echo-reply
icmp-object unreachable
icmp-object time-exceeded
icmp-object echo
Additional Information:

Phase: 4
Type: NAT
Subtype:
Result: ALLOW
Config:
nat (inside,outside) source static AWS-Src-10.213.0.0 AWS-Src-10.213.0.0 destination static AWS-Dest-172.31.48.0 AWS-Dest-172.31.48.0
Additional Information:
Static translate 10.213.3.12/0 to 10.213.3.12/0

Phase: 5
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:

Phase: 6
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:

Phase: 7
Type: SFR
Subtype:
Result: ALLOW
Config:
class-map FIREPOWER
match access-list FIREPOWER
policy-map global_policy
class FIREPOWER
sfr fail-open
service-policy global_policy global
Additional Information:

Phase: 8
Type: INSPECT
Subtype: np-inspect
Result: ALLOW
Config:
class-map inspection_default
match default-inspection-traffic
policy-map global_policy
class inspection_default
inspect icmp
service-policy global_policy global
Additional Information:

Phase: 9
Type: INSPECT
Subtype: np-inspect
Result: ALLOW
Config:
Additional Information:

Phase: 10
Type: FLOW-EXPORT
Subtype:
Result: ALLOW
Config:
Additional Information:

Phase: 11
Type: VPN
Subtype: encrypt
Result: ALLOW
Config:
Additional Information:

Phase: 12
Type: NAT
Subtype: rpf-check
Result: ALLOW
Config:
nat (inside,outside) source static AWS-Src-10.213.0.0 AWS-Src-10.213.0.0 destination static AWS-Dest-172.31.48.0 AWS-Dest-172.31.48.0
Additional Information:

Phase: 13
Type: USER-STATISTICS
Subtype: user-statistics
Result: ALLOW
Config:
Additional Information:

Phase: 14
Type: VPN
Subtype: ipsec-tunnel-flow
Result: ALLOW
Config:
Additional Information:

Phase: 15
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:

Phase: 16
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:

Phase: 17
Type: USER-STATISTICS
Subtype: user-statistics
Result: ALLOW
Config:
Additional Information:

Phase: 18
Type: FLOW-CREATION
Subtype:
Result: ALLOW
Config:
Additional Information:
New flow created with id 36549835, packet dispatched to next module

Result:
input-interface: inside
input-status: up
input-line-status: up
output-interface: outside
output-status: up
output-line-status: up
Action: allow

GioGonza · ‎01-11-2018

Hello @blwegrzyn,

You are sending the traffic so you need to follow up with AWS in order to verify what´s happening to your traffic since the ASA is doing what he is suppposed to do.

HTH

Gio

blwegrzyn · ‎01-10-2018

Yes my case is exactly where traffic hits the tunnel and I see encap but no decap.

blwegrzyn · ‎03-09-2018

i also found that below commands is helpful:

# capture asp type asp-drop all circular buffer 33554432

Florin Barhala · ‎03-12-2018

And how did you use it? What did it show for you in this case?

blwegrzyn · ‎03-09-2018

I also found that below command can be helpful:

# capture asp type asp-drop all circular buffer 33554432