L2L VPN established but no traffic

[jobrien929]

I have set up a VPN between a local ASA and Azure. The tunnel is established and I can ping in both directions but that's all I can do. No other traffic is getting passed the ASA. I did a packet trace from a local machine to one in Azure on port 139 and got this result:

Phase: 9
Type: ACCESS-LIST
Subtype: filter-aaa
Result: DROP
Config:
Additional Information:
Forward Flow based lookup yields rule:
out id=0x7f46072448a0, priority=13, domain=filter-aaa, deny=true
hits=1, user_data=0x7f4609f2b580, filter_id=0x27(VPN-FILTER-SMBBlock), protocol=6
src ip=0.0.0.0, mask=0.0.0.0, port=139
dst ip=0.0.0.0, mask=0.0.0.0, port=0

A little searching around told me that filter-aaa relates to vpn filters but I didn't apply one. Here are the results of a few show commands:

act# sh vpn-sessiondb l2l filter ipaddress x.x.x.x

Session Type: LAN-to-LAN

Connection : x.x.x.x
Index : 60729 IP Addr : x.x.x.x
Protocol : IKEv2 IPsec
Encryption : IKEv2: (1)AES256 IPsec: (1)AES256
Hashing : IKEv2: (1)SHA1 IPsec: (1)SHA256
Bytes Tx : 15791985 Bytes Rx : 15767697
Login Time : 09:37:37 EDT Mon Jul 29 2019
Duration : 3d 5h:52m:24s

sh run tunnel-group x,x,x,x
tunnel-group x.x.x.x type ipsec-l2l
tunnel-group x.x.x.x general-attributes
default-group-policy AZURE-DEV-GROUP-POLICY
tunnel-group x.x.x.x ipsec-attributes
peer-id-validate nocheck
ikev2 remote-authentication pre-shared-key *****
ikev2 local-authentication pre-shared-key *****

sh run group-policy AZURE-DEV-GROUP-POLICY
group-policy AZURE-DEV-GROUP-POLICY internal
group-policy AZURE-DEV-GROUP-POLICY attributes
vpn-tunnel-protocol ikev2

Let me know what other information would be helpful.

Thanks

[Rob Ingram]

Hi,
Do you have a NAT exemption rule in place, in order not to NAT traffic over the tunnel?
Does "show crypto ipsec sa" confirm encaps|decaps are increasing?
Just to confirm does "show vpn-sessiondb detail l2l" confirm nothing under "Filter-Name"?

Can you provide the full output of the packet-tracer test, including the syntax used (change IP addresses if needs be)

[jobrien929]

Here's the full packet-tracer output:

Phase: 1
Type: ROUTE-LOOKUP
Subtype: Resolve Egress Interface
Result: ALLOW
Config:
Additional Information:
found next-hop 169.254.225.6 using egress ifc AZURE-PROD-VTI20

Phase: 2
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group NextGen-ProdDb_access_in in interface NextGen-ProdDb
access-list NextGen-ProdDb_access_in extended permit ip object-group DM_INLINE_NETWORK_169 object Azure_Exams_Prod
object-group network DM_INLINE_NETWORK_169
network-object 10.129.20.0 255.255.255.0
network-object 10.129.25.0 255.255.255.0
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7f4606460c60, priority=13, domain=permit, deny=false
hits=4, user_data=0x7f4609f25ac0, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
src ip/id=10.129.25.0, mask=255.255.255.0, port=0, tag=any
dst ip/id=10.229.0.0, mask=255.255.0.0, port=0, tag=any, dscp=0x0
input_ifc=NextGen-ProdDb, output_ifc=any

Phase: 3
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7f4605f4e8f0, priority=1, domain=nat-per-session, deny=true
hits=1197839516, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=any, output_ifc=any

Phase: 4
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7f46060e7880, priority=0, domain=inspect-ip-options, deny=true
hits=28590288, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=NextGen-ProdDb, output_ifc=any

Phase: 5
Type: SFR
Subtype:
Result: ALLOW
Config:
class-map sfr
match access-list sfr_redirect
policy-map global_policy
class sfr
sfr fail-open
service-policy global_policy global
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7f460b148f40, priority=71, domain=sfr, deny=false
hits=3402596, user_data=0x7f460adb9960, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=NextGen-ProdDb, output_ifc=any

Phase: 6
Type: INSPECT
Subtype: inspect-ftp
Result: ALLOW
Config:
class-map strasz-ftp
match access-list strasz-ftp-acl
policy-map global_policy
class strasz-ftp
inspect ftp
service-policy global_policy global
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7f460b0cc880, priority=70, domain=inspect-ftp, deny=false
hits=3402102, user_data=0x7f4608fd3d80, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=NextGen-ProdDb, output_ifc=any

Phase: 7
Type: FOVER
Subtype: standby-update
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x7f46060dbf10, priority=20, domain=lu, deny=false
hits=1542329, user_data=0x0, cs_id=0x0, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=NextGen-ProdDb, output_ifc=any

Phase: 8
Type: VPN
Subtype: encrypt
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
out id=0x7f4607216200, priority=70, domain=encrypt, deny=false
hits=46, user_data=0x127df7e4, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=any, output_ifc=AZURE-PROD-VTI20

Phase: 9
Type: ACCESS-LIST
Subtype: filter-aaa
Result: DROP
Config:
Additional Information:
Forward Flow based lookup yields rule:
out id=0x7f46072448a0, priority=13, domain=filter-aaa, deny=true
hits=5, user_data=0x7f4609f2b580, filter_id=0x27(VPN-FILTER-SMBBlock), protocol=6
src ip=0.0.0.0, mask=0.0.0.0, port=139
dst ip=0.0.0.0, mask=0.0.0.0, port=0

Result:
input-interface: NextGen-ProdDb
input-status: up
input-line-status: up
output-interface: AZURE-PROD-VTI20
output-status: up
output-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule

The tunnel endpoint on this side is a VTI with no NAT defined.

Thanks

[Rahul Govindan]

Do you have a filter under DefaultGroupPolicy? If yes, the applied group-policy may have inherited from there. 

[jobrien929]

We have 2 VPNs set up going to separate virtual networks with different address spaces in Azure. The configurations are identical except for the endpoint addresses and routes. Only one of the tunnels has this problem. If a filter was being inherited, wouldn't I see the same issue on both?

[Rahul Govindan]

Ideally it should affect either both or none of the tunnels if the configuration is identical. Can you share the full sanitized config in this thread?

Also, there have been bugs where traffic has been dropped by the filter rules incorrectly. Example 

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvd97319/?rfs=iqvred

Also, looks like VPN-FILTER-SMBBlock is the ACL applied to this session. Can you check where this is applied? 

[Afolarin Omole]

From the packet tracer output , it clearly shows what is dropping the traffic , which is the VPN filter attached to the tunnel group policy. Please check the group policy in place for the tunnel and the filter attached to it (VPN-FILTER-SMBBlock).

Also remember VPN filter do not work like normal ACL , it is there to restrict access from outside traffic towards us not the other way round.

try and disable that rule , and post the packet tracer output , if all works fine then you need to work on the VPN Filter configuration.

try : sh run group-policy | I POLICY NAME  or

sh run all group-policy POLICY NAME

[JMiraD1966]

Had this same problem: L2L VPN established but no traffic from local LAN to remote LAN.

show vpn-sessiondb detail l2l command output, identified the cause: The Group-policy attributes had a vpn-filter applied.

Removing the vpn-filter solved it.

Thanks