Firepower VTI to Azure no traffic flow

jsalmond · ‎08-10-2022

Afternoon,

I have set up route based VPN between our on-premise Firepower appliance (Using FMC) and Azure, the VPN is up and BGP is advertising routes.

The issue we face is that we are unable to ping or RDP to a VM in Azure from the inside networks. ACL are in place and according to Packet Tracer they traffic is allowed but then gets dropped with the reason unexpected-packet.

Result:
input-interface: Inside(vrfid:0)
input-status: up
input-line-status: up
output-interface: VTI_MBtoAzure(vrfid:0)
output-status: up
output-line-status: up
Action: drop
Drop-reason: (unexpected-packet) Unexpected packet, Drop-location: frame 0x000000aaacc88868 flow (NA)/NA

I can ping the VM from the FTD's CLI using "ping tcp <AzureVM IP> 3389" but not if i specify the source and use an IP address within the inside security zone.

I thought that the unexpected-packet may be related to NAT rules configured but I have tied the following rules with no luck

nat (any,any) source static InsideSubnets InsideSubnets destination static AzureSubnets AzureSubnets NetworkGroup_vnet-hub-uks-01_Subnets

nat (Inside,any) source static InsideSubnets InsideSubnets destination static AzureSubnets AzureSubnets NetworkGroup_vnet-hub-uks-01_Subnets route-lookup

Regards

Moh Shakhatreh · ‎08-10-2022

Hello ,

please try the below :

remove (any, any) and try to use specific names in the nat rules

apply pre-filter on the traffic and see if issue is related to the NAP or snort,

jsalmond · ‎08-10-2022

Hi @Moh Shakhatreh

Thank you for the reply pre-filter are in place and packet tracer shows it as allowed.

FMC doesn't have an option to specify the VTI interface and if I use Inside to Outside the traffic is then not routed out the VTI and over VPN.

Regards

James

MHM Cisco World · ‎08-10-2022

do same packet-tracer but with detail may be it give us some hint where the packet is drop

jsalmond · ‎08-10-2022

Hi @MHM Cisco World

Thank you for the reply, the results show 12 Phases with result as Allow it not until the results we see Action: drop
Drop-reason: (unexpected-packet) Unexpected packet, Drop-location: frame 0x000000aaacc88868 flow (NA)/NA

Result:
input-interface: Inside(vrfid:0)
input-status: up
input-line-status: up
output-interface: VTI_MBtoAzure(vrfid:0)
output-status: up
output-line-status: up
Action: drop
Drop-reason: (unexpected-packet) Unexpected packet, Drop-location: frame 0x000000aaacc88868 flow (NA)/NA

MHM Cisco World · ‎08-10-2022

Share packet-tracer

jsalmond · ‎08-10-2022

Phase: 1
Type: INPUT-ROUTE-LOOKUP
Subtype: Resolve Egress Interface
Result: ALLOW
Config:
Additional Information:
Found next-hop 10.253.1.2 using egress ifc VTI_MBtoAzure(vrfid:0)

Phase: 2
Type: UN-NAT
Subtype: static
Result: ALLOW
Config:
nat (any,any) source static NetworkGroup_MBInsideVLANS NetworkGroup_MBInsideVLANS destination static NetworkGroup_vnet-hub-uks-01_Subnets NetworkGroup_vnet-hub-uks-01_Subnets
Additional Information:
NAT divert to egress interface VTI_MBtoAzure(vrfid:0)
Untranslate 10.40.0.52/3389 to 10.40.0.52/3389

Phase: 3
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group CSM_FW_ACL_ global
access-list CSM_FW_ACL_ advanced trust ip ifc Inside any object-group FMC_INLINE_dst_rule_268457984 rule-id 268457984 event-log flow-end
access-list CSM_FW_ACL_ remark rule-id 268457984: PREFILTER POLICY: HSD-Prefilter-MB
access-list CSM_FW_ACL_ remark rule-id 268457984: RULE: InsudeDMZ_to_Azure
object-group network FMC_INLINE_dst_rule_268457984
description: Auto Generated by FMC from dst of PrefilterRule# 10 (HSD-Prefilter-MB/mandatory)
network-object 10.40.0.52 255.255.255.255
network-object object Network_vnet-hub-uks-01_subnet
group-object NetworkGroup_vnet-hub-uks-01_Subnets
Additional Information:
Forward Flow based lookup yields rule:
in id=0xffe5ee0fa0, priority=12, domain=permit, trust
hits=49, user_data=0x5569fe9900, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, ifc=Inside(vrfid:0)
dst ip/id=10.40.0.52, mask=255.255.255.255, port=0, tag=any, ifc=any, vlan=0, dscp=0x0, nsg_id=none
input_ifc=any, output_ifc=any

Phase: 4
Type: CONN-SETTINGS
Subtype:
Result: ALLOW
Config:
class-map class-default
match any
policy-map global_policy
class class-default
set connection advanced-options UM_STATIC_TCP_MAP
service-policy global_policy global
Additional Information:
Forward Flow based lookup yields rule:
in id=0xffe43152d0, priority=7, domain=conn-set, deny=false
hits=62946262, user_data=0xffe4312eb0, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=Inside(vrfid:0), output_ifc=any

Phase: 5
Type: NAT
Subtype:
Result: ALLOW
Config:
nat (any,any) source static NetworkGroup_MBInsideVLANS NetworkGroup_MBInsideVLANS destination static NetworkGroup_vnet-hub-uks-01_Subnets NetworkGroup_vnet-hub-uks-01_Subnets
Additional Information:
Static translate 192.168.3.33/3389 to 192.168.3.33/3389
Forward Flow based lookup yields rule:
in id=0xffe82a5c40, priority=6, domain=nat, deny=false
hits=0, user_data=0x5562b69870, cs_id=0x0, flags=0x0, protocol=0
src ip/id=192.168.3.0, mask=255.255.255.0, port=0, tag=any
dst ip/id=10.40.0.48, mask=255.255.255.240, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=any, output_ifc=any

Phase: 6
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x5578648f40, priority=0, domain=nat-per-session, deny=false
hits=14161433, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=any, output_ifc=any

Phase: 7
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x557d59cfa0, priority=0, domain=inspect-ip-options, deny=true
hits=63597554, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=Inside(vrfid:0), output_ifc=any

Phase: 8
Type: NAT
Subtype: rpf-check
Result: ALLOW
Config:
nat (any,any) source static NetworkGroup_MBInsideVLANS NetworkGroup_MBInsideVLANS destination static NetworkGroup_vnet-hub-uks-01_Subnets NetworkGroup_vnet-hub-uks-01_Subnets
Additional Information:
Forward Flow based lookup yields rule:
out id=0xffe8496de0, priority=6, domain=nat-reverse, deny=false
hits=1, user_data=0x5563c726e0, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
src ip/id=192.168.3.0, mask=255.255.255.0, port=0, tag=any
dst ip/id=10.40.0.48, mask=255.255.255.240, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=any, output_ifc=any

Phase: 9
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in id=0x5578648f40, priority=0, domain=nat-per-session, deny=false
hits=14161435, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=any, output_ifc=any

Phase: 10
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in id=0xffe5eb3320, priority=0, domain=inspect-ip-options, deny=true
hits=2603, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=VTI_MBtoAzure(vrfid:0), output_ifc=any

Phase: 11
Type: FLOW-CREATION
Subtype:
Result: ALLOW
Config:
Additional Information:
New flow created with id 64083484, packet dispatched to next module
Module information for forward flow ...
snp_fp_inspect_ip_options
snp_fp_tcp_normalizer
snp_fp_translate
snp_fp_adjacency
snp_fp_fragment
snp_fp_tracer_drop
snp_ifc_stat

Module information for reverse flow ...
snp_fp_inspect_ip_options
snp_fp_translate
snp_fp_tcp_normalizer
snp_fp_adjacency
snp_fp_fragment
snp_fp_tracer_drop
snp_ifc_stat

Phase: 12
Type: INPUT-ROUTE-LOOKUP-FROM-OUTPUT-ROUTE-LOOKUP
Subtype: Resolve Preferred Egress interface
Result: ALLOW
Config:
Additional Information:
Found next-hop 10.253.1.2 using egress ifc VTI_MBtoAzure(vrfid:0)

Result:
input-interface: Inside(vrfid:0)
input-status: up
input-line-status: up
output-interface: VTI_MBtoAzure(vrfid:0)
output-status: up
output-line-status: up
Action: drop
Drop-reason: (unexpected-packet) Unexpected packet, Drop-location: frame 0x000000aaacc88868 flow (NA)/NA

MHM Cisco World · ‎08-10-2022

nat (any,any) source static NetworkGroup_MBInsideVLANS NetworkGroup_MBInsideVLANS destination static NetworkGroup_vnet-hub-uks-01_Subnets NetworkGroup_vnet-hub-uks-01_Subnets route-lookup

jsalmond · ‎08-16-2022

Managed to get this issue resolved, problem looked to be with OSPF and BGP redistribution. Removed this element for now to get connectivity to Azure.

@MHM Cisco World Thank you again for your help, unfortunately the FMC wouldn't allow the use of route-lookup when using (any,any). In the end NAT rule used was nat (Inside,Outside) source static NetworkGroup_MBInsideVLANS NetworkGroup_MBInsideVLANS destination static NetworkGroup_vnet-hub-uks-01_Subnets NetworkGroup_vnet-hub-uks-01_Subnets no-proxy-arp route-lookup.