LB (ECMP) of Internet traffic between two ASR routers and two ISP's

[elon1505]

Hello Community Members,

I need little help on Load balance the Internet traffic.I have the simple topology where L3 routers is at distribution Layer and ASR 1002 routers are core layers. I have two ISP's on two diff core routers and have an eBGP and getting the full routing table. On Southbound I have Dist L3 routers and Core ASRP routers (HSRP IP) in same L3 segment, no routing protocol, dist Routers has next-hop Core for 0/0 route. 

Problem Statement:- Due to only 1 router being HSRP master all traffic from Southbound to Northbound is handled by router 1. Router 1 have no choice sending traffic to connected ISP1. or lets say if failover occurs then Router2 will be HSRP master and all outbound traffic is sent over ISP2 so at a time only 1 ISP is being used.

 I want to achieve full Load balance of internet traffic on both ISP, either one router being active or both core routers being active (removing HSRP), similar like spine-leaf architecture.  I thought of doing Multipath but I can't do as I have EBGP routes from ISP and  IBGP routes from peer device. Multi path won't work on this . Any ideas please how to achieve that?

[Karsten Iwen]

There are a couple of ways to achieve this. In your particular situation I would consider the following the easiest:

• Configure a second HSRP group that is active on the ISP2-router
• Use HSRP1 virtual address on the left distribution as a default next hop
• Use HSRP2 virtual address on the right distribution as default

With this the traffic that goes to the to distribution routers will automatically load balance. The next question is *if* the user traffic will hit both distribution routers or not. If only one device is the next hop for the LAN you have the same problem. Here you could also use two HSRP groups or move this functionality to GLBP.

If that is all too complex, running a routing protocol with ECMP between ASRs and Distribution could be the other solution. Or migrate the ASRs to GLBP instead of HSRP.

[Joseph W. Doherty]

BTW, keep in mind, GLBP (I recall) uses source MACs for balancing, i.e. traffic coming from a single host (in this case a router or L3 switch) will all go to just a single GLBP gateway.  I.e. GLBP wouldn't be a suitable solution.

Using a second HSRPv2 gateway, though, will work just fine.

[Karsten Iwen]

Good catch! 

[Joseph W. Doherty]

Thank you - generally often due to personal experience - which often means, already made that mistake, myself.  ; )

[MHM Cisco World]

[Joseph W. Doherty]

I believe you can do multipath via iBGP (or eBGP), but it does require additional config, as by default, Cisco's BGP doesn't do it.  (BTW, this assuming iBGP also running behind your ISP routers, but since you're taking two full Internet route tables, I presume that's not the case.  Otherwise, [which is what you're really describing], eBGP takes precedence over an equal route via iBGP, although a "better" path can go via one ASR to the other ASR via iBGP.)

What @Karsten Iwen suggested, using another HSRPv2 gateway, is probably your easiest solution.

BTW, in my experience, using full Internet tables, with multiple ISPs, usually offers little benefit, because although one ISP may have a shorter AS hop path, is the path truly better?  Taking just your ISP's one hop AS routes, from a BGP perspective, is often a reasonable approach.

Further, if you really want the best possible ISP LB, you need to use something like Cisco's PfR, which will dynamically LB your links and can optionally find the best performing path, to any destination you're sending traffic to.  If fact, in my experience, it does that so well, you can just use a default route to each ISP and let PfR make any adjustments, if it needs to, to obtain the best possible Internet performance.

[Joseph W. Doherty]

Oh, although I described what I believe is the best possible approach (and have used it, myself), and have said @Karsten Iwen suggestion to use a second HSRPv2 gateway might be the easiest solution (for what OP is already doing), Karsten's other suggestion to do ECMP to the ASRs, would be what I would prefer over using HSRP.  Reason being, when doing routing, between routers, I prefer to use routing protocols, not L2 client FHRPs.  You could have your ASRs inject just a dynamic default route to your interior network, which, ideally, is being dynamically provided from your two ISPs.  (The latter also deals with not sending traffic to the ASR which has lost its BGP connection to your ISP - which is "messy" when using HSRP, although possible tracking via SLA.)

[paul driver]

Hello
An alternative would be to remove HSRP from the wan rtrs and append an IGP (ospf) between those rtrs and the L3 cores switches which would introduce ECMP, then advertise a just default route into ospf and redistribute OSPF into BGP ( minus the default)