Dual ISP, Cisco ASRs with Palo Alto Firewalls / WAN Failover

[bmiddleton@taconiccap.com]

Hi,

Looking for some guidance on our setup. I am looking to establish pure ISP failover without having to take action on my / my team's side. Presently when there is an outage, we need to do manual intervention to get connectivity back up.

Here is an overview of our network, internet facing.

ISP A (/30) -> Cisco ASR Router 1 (I control) (/24 ASN eBGP established to ISP A) WAN Interface -> ASR Router (LAN Interface - Public IP in same /24) -> DMZ Switch Stack (VLAN 5 - WAN Facing)

ISP B (/30) -> Cisco ASR Router 2 (I control) (/24 ASN eBGP established to ISP B) WAN Interface -> ASR Router (LAN Interface - Public IP in same /24) -> DMZ Switch Stack (VLAN 5 - WAN Facing)

ISP A = 1Gb

ISP B = 500Mb

Cisco Router 1 - No prepend, default route to ISP carrier

Cisco Router 2 - Prepend, default route to ISP carrier + ip route x.x.x.x /24 null0

*** (I have found if I take away the prepend and null0 loopback, packets going out cannot route back in)

PA (Palo Alto) 3020 x2 (Active/Passive) (E1/1) -> DMZ Switch Stack (VLAN 5)

E1/1 - WAN IP in the same /24 block above

NAT from the PA is dynamic-ip-and-port with the E1/1 Interface IP from untrust to trust zone

No PBF but x1 VR with default route, with traffic going to Cisco Router 1 LAN IP for next hop

- I have tried putting in route monitoring in the VR default route to the Cisco Router 2 LAN IP, removing the prepend on Cisco Router 2 and null route and internet stops working from behind the PA.

- I have tried doing PBF with ping monitoring and running through the same scenario, no go.

When ISP A goes down, we need to remove the prepend and remove the null route, change the route manually on PA and clear NAT sessions. Not ideal ...

Can anyone offer any suggestions or thoughts on how to improve the setup? Changing setup, connections, hardware, etc... is all open and fine.

[paul driver]

Hello bmiddleton@taconiccap.com 
Conditional failover should be applicable, Would you be able to to provide a topology diagram, so we could visualise your network (remove/replace all necessary public addressing)

[bmiddleton@taconiccap.com]

Hi,

Sure please see attached, thanks.

Brian

[Seb Rupik]

Hi there,

Is there a burning desire to keep the ASRs as edge routers? I have completed plenty of deployments using Palos at the very edge of the network running multi-homed BGP and they work great.

I've labbed this up in EVE-NG, albeit without the Palo HA pair, the topology looks like:

Connect your two ISP connections directly to the DMZ switch stack, and decommision the ASRs. Configure the Palo cluster to run eBGP towards both ISPs.

The DMZ subnet will be routed just by the Palo Alto, we also need to configre a loopback interface which uses a single IP from that /24:

Make sure that you create a BGP import policy for both ISPs, giving ISP-A a higher local preference than ISP-B.

set network virtual-router default protocol bgp policy import rules make_isp-b_second action allow update community none
set network virtual-router default protocol bgp policy import rules make_isp-b_second action allow update extended-community none
set network virtual-router default protocol bgp policy import rules make_isp-b_second action allow update local-preference 50
set network virtual-router default protocol bgp policy import rules make_isp-b_second action allow update as-path none
set network virtual-router default protocol bgp policy import rules make_isp-b_second match from-peer ISP-B-01
set network virtual-router default protocol bgp policy import rules make_isp-b_second match route-table unicast
set network virtual-router default protocol bgp policy import rules make_isp-b_second used-by ISP-B
set network virtual-router default protocol bgp policy import rules make_isp-b_second enable yes

set network virtual-router default protocol bgp policy import rules make_isp-a_pri action allow update local-preference 200
set network virtual-router default protocol bgp policy import rules make_isp-a_pri action allow update as-path none
set network virtual-router default protocol bgp policy import rules make_isp-a_pri action allow update community none
set network virtual-router default protocol bgp policy import rules make_isp-a_pri action allow update extended-community none
set network virtual-router default protocol bgp policy import rules make_isp-a_pri match from-peer ISP-A-01
set network virtual-router default protocol bgp policy import rules make_isp-a_pri match route-table unicast
set network virtual-router default protocol bgp policy import rules make_isp-a_pri used-by ISP-A
set network virtual-router default protocol bgp policy import rules make_isp-a_pri enable yes

You can configure that NAT policy in two ways. You could NAT behind the WAN IPs, but if one of the links to the DMZ switch fails then NAT state table will hold invalid infomation and new NAT sessions will need to be created to go via the Eth1/2 interface. Argubaly you could use an Aggreagte-Ethernet bundle to reduce this risk.

My preference would be to use the loopback interface as the NAT source therefore mitigating any WAN interface failures on the Palo Alto.

cheers,

Seb.