Re: HA failover between WAN sites not working

John Hinckley · ‎06-05-2019

Here's the scenario:

I have 2 separate corporate sites, Site A and Site B, that each have their own internet connection and use a static default route to reach the ASA. These 2 sites plus 4 additional branch sites make up the total network. All sites are connected via MPLS cloud but only sites A and B provide internet to the branches. Right now, Site A (primary path) provides the default route to all the branches. Like Site A, Site B should always use its own internet connection unless it fails, then it should use Site A (high availability)

What I need to happen is this, when Site A goes down, Site B should distribute its default route to the other 5 sites including Site A. I configured Site B to prepend the default route so it will not win out when Site A is active.

The problem I have is this, since Site A and Site B are both announcing the 0.0.0.0 prefix through BGP, when either site fails, it can't receive the default route from the other site. The only way to make it work is to remove the 0/0 prefix from BGP but that is not a fix because then the branches don't receive the default route.

I have attached a topo WAN diagram for visual aid. Here is the relevant config:

Site A

interface GigabitEthernet0/1 (WAN)
ip address 172.25.25.42 255.255.255.252

interface GigabitEthernet0/0 (LAN)

ip address 10.10.10.1 255.255.255.0

router bgp 65535
bgp log-neighbor-changes
neighbor 172.25.25.41 remote-as 65005
neighbor 172.25.25.41 update-source GigabitEthernet0/1
neighbor 172.25.25.41 version 4
!
address-family ipv4
network 0.0.0.0
network 10.10.10.0 mask 255.255.252.0
network 10.15.15.0 mask 255.255.255.0
network 10.15.50.0 mask 255.255.255.0
neighbor 172.25.25.41 activate
neighbor 172.25.25.41 allowas-in
neighbor 172.25.25.41 soft-reconfiguration inbound
neighbor 172.25.25.41 route-map default-backup out
exit-address-family

ip sla 1
icmp-echo 4.2.2.1 source-ip 10.10.10.1
ip sla schedule 1 life forever start-time now

track 1 ip sla 1 reachability

ip route 4.2.2.1 255.255.255.255 10.10.10.254
ip route 0.0.0.0 0.0.0.0 10.10.10.254 track 1
ip route 0.0.0.0 0.0.0.0 172.25.25.41 10

Site B:

interface GigabitEthernet0/1 (WAN)
ip address 172.25.25.38 255.255.255.252

interface GigabitEthernet0/0 (LAN)

ip address 10.20.20.1 255.255.255.0

router bgp 65535
bgp log-neighbor-changes
neighbor 172.25.25.37 remote-as 65005
neighbor 172.25.25.37 update-source GigabitEthernet0/1
neighbor 172.25.25.37 version 4
!
address-family ipv4
network 0.0.0.0
network 10.20.20.0 mask 255.255.252.0
network 10.25.25.0 mask 255.255.255.0
network 10.25.50.0 mask 255.255.255.0
neighbor 172.25.25.37 activate
neighbor 172.25.25.37 allowas-in
neighbor 172.25.25.37 soft-reconfiguration inbound
neighbor 172.25.25.37 route-map default-backup out
exit-address-family

ip sla 1
icmp-echo 8.26.56.26 source-ip 10.20.20.1
ip sla schedule 1 life forever start-time now

track 1 ip sla 1 reachability

ip route 8.26.56.26 255.255.255.255 10.20.20.254
ip route 0.0.0.0 0.0.0.0 10.20.20.254 track 1
ip route 0.0.0.0 0.0.0.0 172.25.25.37 10

ip prefix-list 10 description Seconary Route
ip prefix-list 10 seq 5 permit 0.0.0.0/0

route-map default-backup permit 5
match ip address prefix-list 10
set as-path prepend 65535

What am I doing wrong?

Thanks in advance,

-John

Hector Gustavo Serrano Gutierrez · ‎06-05-2019

Hi John,

Once Site A stops pinging 4.2.2.1 it should remove its static default route which points to the ASA (ip route 0.0.0.0 0.0.0.0 10.10.10.254 track 1) from its Routing Table and since it will not longer have a 0.0.0.0/0 in its Routing Table it should stop advertising it via BGP to the MPLS.

Site A can then receive the default route via BGP from the MPLS.

So you do not really need that floating static route pointing to the MPLS (ip route 0.0.0.0 0.0.0.0 172.25.25.41 10), why you have it there? This is what keeps your Router advertise a default route to the MPLS even after your local Internet connection is lost and you should receive it instead.

Another thing, you are using as-path prepend on Site B to suggest the MPLS that since Site A and Site B are both advertising a default route, the preferred path is Site A. I am assuming that you had to put allowas-in in place under BGP's config to receive Site B's default route otherwise it will drop the received route in BGP due to duplicated AS.

The usual way to make sure Site A's default route is preferred by the Branch sites over Site B is by advertising the Default Route appending a BGP COMMUNITY that the ISP recognizes. Without going too deep, by BGP COMMUNITY the ISP knows which site is preferred when both are advertising the same route, in your case the default route. You should obtain from your ISP what BGP COMMUNITY to advertise them along your default route to accomplish this.

As a final note, I suggest you to add the permanent keyword to the static route you have correctly put in place to make sure the IP SLA test is successful only to the local Internet. Otherwise that static route will be removed from the Routing Table if the LAN cable gets disconnected. In that case the IP SLA may come up if the IP SLA hits 4.2.2.1 using Site B's Internet.

Site A
ip route 4.2.2.1 255.255.255.255 10.10.10.254 permanent

Same observations apply for Site B.

DISCLAIMER:

The configurations discussed in this post can be merely templates and may not be final configurations that can be just copied & pasted to any network device in a production environment. It is responsibility of whoever follows this suggestions to review, evaluate and modify the configurations at convenience. Ensure that you understand the potential impact of any command. In all cases, make sure not to lose remote management access to the device. It is highly suggested to introduce changes to live networks only during maintenance windows. The author of this post is not responsible of unintended consequences by failing to follow this disclaimer note.

Francesco Molino · ‎06-05-2019

Hi

I understand the default route towards asa and even the tracking that forces a delete depending on sla status.
However, why are you putting a default route towards your mpls?
I mean if you can advertise a default route and your provider doesn't filter it, there's no need to add a static route.

When you say they're not receiving the default route, are you taking about the rib or bgp table? It's important because issues are different if in bgp table and not into rib than no default route info bgp table.

On site A, can you run the following command to see if the default is sent:
sh ip bgp neig 172.25.25.41 received-route

Do the same on site B (just replace the ip of the previous command by using 172.25.25.37.

Share the outputs please.

Thanks
Francesco
PS: Please don't forget to rate and select as validated answer if this answered your question

Giuseppe Larosa · ‎06-06-2019

Hello John,

in addition to the interesting notes by Hector and Francesco I would like to point out the following:

The command

neighbor 172.25.25.37 allowas-in

means only one presence of own AS number in AS path. In order to receive and accept a prepended AS path you need to use

neighbor 172.25.25.37 allowas-in 2

The command has a default of 1, but a greater number can be specified.

If you want to go on with the AS path prepending approach for SiteB backup default route all other sites need to use allowas-in 2.

Hope to help

Giuseppe

John Hinckley · ‎06-06-2019

@Giuseppe Larosa wrote:
Hello John,
in addition to the interesting notes by Hector and Francesco I would like to point out the following:

The command
neighbor 172.25.25.37 allowas-in

means only one presence of own AS number in AS path. In order to receive and accept a prepended AS path you need to use
neighbor 172.25.25.37 allowas-in 2

The command has a default of 1, but a greater number can be specified.

If you want to go on with the AS path prepending approach for SiteB backup default route all other sites need to use allowas-in 2.

Hope to help
Giuseppe

Thanks for the tip. I looked it up and default value is actually 3, so I think I'm ok there. I can always bump it if I need to.

-J

Giuseppe Larosa · ‎06-07-2019

Hello John,

the default value was 1 in the past.

Can you tell us what happens if:

a) You remove the floating default static route to MPLS on Site A (this is an error as explained by other colleagues it does not allow Site A to withdraw the BGP default route when primary route to ASA fails, this is why you need to manually change BGP configuration)

b) You emulated a fault of primary route on Site A

c) are the branch site receiving the backup default route from Site B. Is Site A receiving the backup default route from Site A now ?

Hope to help

Giuseppe