dynamic NAT dual ISP on ASA 9.8

[El Rondo]

I ran dual ISP on ASA firepower 5525-x v9.8. One go to ADSL PPPOE line and second go to lease line ISP2. I need 3 vlans to be routed to the specific line accordingly such vlan wifi to pppoe line and the rest vlans Level_4 and Level_6 to ISP2 line.

I managed to made the pppoe connection up and working fine for wifi vlan users to the internet by using dynamic IP provided by the pppoe setroute interface. In the meantime I also managed to use PBR to route the rest vlan level_4 and level_6 to the lease line ISP2. Both vlan level_4 and level_6 were able to access internet with dynamic PAT which been provided by the ISP. I discovered that vlan level_4 and level_6 dynamic PAT was not working instead it used dynamic ip from pppoe setroute interface. I verified this from the internet browser public ip address location. However command sh route-map shows that internal LAN level_4 and level_6 was NAT to the defeault next hop 10.90.80.73. I was still confuse is that anything that I miss to make sure my inside LAN was properly NAT using ISP2 public address. Here is my sanitized config.

ASA Version 9.8(2) 
!
hostname FW-ASA5525X
domain-name abccompany.com
enable password xxxxxxxxxxxxxxxxxx
names

!
interface GigabitEthernet0/0
 description # connected to ISP2 line
 nameif WAN
 security-level 0
 ip address 10.90.80.74 255.255.255.224 
!
interface GigabitEthernet0/1
 description # connected to inside LAN
 nameif LAN
 security-level 100
 ip address 172.16.4.253 255.255.254.0 
 policy-route route-map ISP2_LINE
!
interface GigabitEthernet0/2
 nameif DMZ
 security-level 100
 ip address 10.90.81.65 255.255.255.224 
!
interface GigabitEthernet0/3.1
 description # connected to fiber modem
 vlan 500
 nameif UNIFI
 security-level 0
 pppoe client vpdn group GROUP_UNIFI
 ip address pppoe setroute 
!
interface Management0/0
 management-only
 nameif management
 security-level 100
 no ip address
!
dns domain-lookup LAN
dns server-group DefaultDNS
 name-server 1.9.1.9 
 name-server 172.16.4.5 
 name-server 172.16.4.9 
 domain-name abccompany.com
same-security-traffic permit inter-interface
same-security-traffic permit intra-interface
object network WIFI
 subnet 172.17.7.0 255.255.255.0
 description WIFI Vlan
object network LEVEL_6
 subnet 172.17.6.0 255.255.255.0
 description # connected to user level 6
object network LEVEL_4
 subnet 172.17.4.0 255.255.255.0
 description # connected to user level 4
object-group network ALL_ISP2_VLAN
 description # All vlan subjected to ISP2 line
 network-object object LEVEL_4
 network-object object LEVEL_6
access-list ACL_ISP2 extended permit ip object-group ALL_ISP2_VLAN any 
pager lines 24
logging asdm informational
mtu WAN 1500
mtu LAN 1500
mtu DMZ 1500
mtu UNIFI 1492
mtu management 1500
no failover
no monitor-interface service-module 
icmp unreachable rate-limit 1 burst-size 1
asdm image disk0:/asdm-782.bin
no asdm history enable
arp timeout 14400
no arp permit-nonconnected
arp rate-limit 16384
!
object network WIFI
 nat (LAN,UNIFI) dynamic interface
object network LEVEL_6
 nat (LAN,WAN) dynamic 10.90.80.75
object network LEVEL_4
 nat (LAN,WAN) dynamic 10.90.80.76
!
route-map ISP2_LINE permit 10
 match ip address ACL_ISP2
 set ip default next-hop 10.90.80.73
!
route WAN 0.0.0.0 0.0.0.0 10.90.80.73 2
route LAN 172.17.4.0 255.255.255.0 172.16.4.254 1
route LAN 172.17.6.0 255.255.255.255 172.16.4.254 1
route LAN 172.17.7.0 255.255.255.255 172.16.4.254 1
!
vpdn group GROUP_UNIFI request dialout pppoe
vpdn group GROUP_UNIFI localname abccompany@unifi
vpdn group GROUP_UNIFI ppp authentication pap
vpdn username abccompany@unifi password ***** 
!
class-map inspection_default
 match default-inspection-traffic
!
policy-map type inspect dns preset_dns_map
 parameters
  message-length maximum client auto
  message-length maximum 512
  no tcp-inspection
policy-map global_policy
 class inspection_default
  inspect dns preset_dns_map 
  inspect ftp 
  inspect h323 h225 
  inspect h323 ras 
  inspect rsh 
  inspect rtsp 
  inspect sqlnet 
  inspect skinny  
  inspect sunrpc 
  inspect xdmcp 
  inspect sip  
  inspect netbios 
  inspect tftp 
  inspect ip-options 
!

[MHM Cisco World]

problem is set default next-hop 
this make PBR check the default route in routing table which select set route and hence the PBR not work 
you need 
set next-hop "without default"

[El Rondo]

I already remove the default becoming this

route-map ISP2_LINE permit 10
 match ip address ACL_ISP2
 set ip next-hop 10.90.80.73

but result still the same, NAT not working. it still been using dynamic ip given from pppoe interface 
this is my sh route that might help give you some information

S*       0.0.0.0 0.0.0.0 [1/0] via 10.233.65.32, UNIFI
C        10.90.80.72 255.255.255.224 is directly connected, WAN
L        10.90.80.74 255.255.255.255 is directly connected, WAN
C        172.16.4.0 255.255.254.0 is directly connected, LAN
S        172.16.4.0 255.255.255.0 [1/0] via 172.16.4.254, LAN
L        172.16.4.253 255.255.255.255 is directly connected, LAN
S        172.17.4.0 255.255.255.0 [1/0] via 172.16.4.254, LAN
S        172.17.6.0 255.255.255.0 [1/0] via 172.16.4.254, LAN

[MHM Cisco World]

OK, 
just clear Conn and Xlate, 
and check again. 

[El Rondo]

Just did that

Unfortunately I got the same outcome.
I would probably thinking like to reboot the ASA and see what happen.

[MHM Cisco World]

debug policy-route

[El Rondo]

This is my snapshot for command debug policy-route

pbr: route map ISP2_LINE, sequence 10, permit; proceed with policy routing
pbr: evaluating next-hop 10.90.80.73
pbr: policy based routing applied; egress_ifc = WAN : next_hop = 10.90.80.73
pbr: policy based route lookup called for 172.17.7.49/60632 to 10.90.80.82/443 proto 6 sub_proto 0 received on interface LAN
pbr: First matching rule from ACL(2)
bpbr: route map ISP2_LINE, sequence 10, permit; proceed with policy routing
pbr: evaluating next-hop 10.90.80.73
pbr: policy based routing applied; egress_ifc = WAN : next_hop = 10.90.80.73
ug pbr: policy based route lookup called for 172.17.7.36/49812 to 118.26.120.35/80 proto 6 sub_proto 0 received on interface LAN

this is my tracert result from one of my client ip address 172.17.7.49/24

# tracert 8.8.8.8

Tracing route to dns.google [8.8.8.8]
over a maximum of 30 hops:

  1     7 ms     2 ms     2 ms  172.17.7.254  (core switch)
  2     8 ms     *        3 ms  172.16.4.253  (ASA)
  3     *        8 ms     4 ms  10.90.80.73   (router)
  4     5 ms     3 ms     3 ms  10.226.153.77
  5    15 ms    17 ms    14 ms  10.251.97.33
  6    16 ms    17 ms    15 ms  10.251.97.34
  7    17 ms    15 ms    15 ms  10.251.97.9
  8    17 ms    15 ms    15 ms  58.26.96.137
  9    17 ms    17 ms    16 ms  203.106.131.0
 10     *       24 ms    22 ms  10.55.48.66
 11    29 ms    24 ms    28 ms  72.14.197.66
 12    25 ms    23 ms    25 ms  108.170.250.1
 13     *       25 ms    25 ms  108.170.230.85
 14    25 ms    24 ms    24 ms  dns.google [8.8.8.8]

seems like trace route passed the packet flow to 8.8.8.8
is there anything I should look into coreswitch?

[El Rondo]

I have checked on my core switch configuration. FYI, my core switch currently been configured to route to two (2) firewall at the same time. One is my existing ASA firewall v8.3 serve for another ISP and another one is new ASA v9.8 which will be replacing the old one. PBR being used to route different vlans to different firewalls accordingly. 

I found that existing ospf table only recognized one neighbor that was segmented to old firewall vlan. There was no new neighbor in the ospf table. Could this be the root cause?

Neighbor ID     Pri   State           Dead Time   Address         Interface
172.16.255.254    1   FULL/DR         00:00:38    172.16.255.254  Vlan5

I need to reconfigure it to add the new neighbor in (in this case new firewall vlan) in order for the core understand to broadcast. I will give it a try and let you know the result.

[El Rondo]

Sorry the late feedback since I have been in involved in another project. However I already checked my core switch and was able to match the ospf routing table for both new and old firewall into my routing coreswitch.

I suspected that there were asymmetric routing occurred in my firewall. 

[El Rondo · Accepted solution]

Sorry for taking some time to reply on this issue.
I had been away for few weeks and had the opportunity to report a status on this matter.

My dynamic NAT issue has been solved. I had not given enough information on the problem statement. Actually my firewall was not directly connected to ISP however it was connected to a private WAN. My config for dynamic NAT was properly fine and worked at the private WAN cloud. They have verified. My routing OSPF was also being rectified and works like a charm.

Thanks for all your feedbacks and hopefully I can deliver a better picture for my case in future.

Regards