ASK THE EXPERT - Border Gateway Protocol Multi-homing

[ciscomoderator]

With Manigandan Ganesan

Welcome to the Cisco Networking Professionals Ask the Expert conversation. This is an opportunity to get an update on design and troubleshooting BGP with Cisco expert Manigandan Ganesan. Mani is an engineer working with the Routing Protocols team in the Cisco Technical Assistance Center in Bangalore. There he configures and troubleshoots various routing protocols like Enhanced Interior Gateway Routing Protocol, Open Shortest Path First protocol, Border Gateway Protocol, and Protocol Independent Multicast. He also focuses on filing technical and documentation bugs in these areas, and delivers training sessions on these technologies to other teams in Cisco. Mani holds a bachelor’s degree in electrical and electronics engineering from Anna University, Chennai. He also holds CCIE certification #27200 in Routing and Switching.

Remember to use the rating system to let Mani know if you have received an adequate response.

You can review the Live Webcast Video where Mani gave a presentation on this topic. You can also read the questions he answered during the live event in this FAQ Document.

Mani might not be able to answer each question due to the volume expected during this event. Remember that you can continue the conversation on the Security discussion forums shortly after the event. This event lasts through February 25, 2011. Visit this forum often to view responses to your questions and the questions of other community members.

[ahmedchohan]

Hi Mani

Saw your "ATE" on cisco virtual and was informative. Thanks !

1.     Can u please elaborate on the role of iBGP on the CE routers. Would the CE routers need the complete routing table for it to make use of the AS path for the closest destination and other tweaking ?

2.      In case of both the CE routers getting the default route, how can we load balance ?

3.     What would be role of hsrp in both of the above scenarios.

Thanks

Ahmed

.

[Ibrahim Jamil]

Hi Manigandan

Thanks and it was an  informative session (I Prefer FfR Included in the Presantation) for me and  I had been waiting for long for this Presentation,okay now what the configuration Looks Like if we have dual ASA involved in the scenario Like ahmedchohan's Post ,and these dual ASA must be active/active coz we have own ASN/Address Block from RiR,so what is the recommend configuration on the ASA's ,Pls Bear in Mind that we  have servers tiers presented to Internet required continuous Internet connectivity

our configuration on the Border Router similar to the attached file,Pls Have a look to it while the overall diagram its the Exactly the same like ahmedchohan's diagram

Thank You

[Mani Ganesan]

Hi Ibrahim,

Glad you liked the session.

As far as ASA configuration goes, both of them need to have a default route pointing to thier respective upstream devices either static or IGP ( if any ).

The important point to take care when it comes to ASA is asymmetric routing. If we send traffic to one upstream link, but receive the return traffic on the other one  ( or ) send end traffic via one ASA, but receive the return traffic on the other ASA would be a problem. FW would drop the return traffic. To avoid this, we have to make sure that the IGP is set up in such a way that for traffic from a specific block of the network would leave and reach ASA on the same link (or) go through and come back via the same ASA.

If there is any thing specific that you want to get clarified on the ASA part with BGP, Please let me know.

Best Regards,

Mani

[Ibrahim Jamil]

HI Mani

i think the asr-group does solving the Asymetric routing,I have been diving in cisco.com,but i didnt find this type of configuration with bgp multihoming to 2 different ISP with the present of two ASAs running as Active/Active,do u have such kind of this configuration for whole design configuation with ASAs active/active

[Mani Ganesan]

Hi Ibrahim,

There is nothing specific needed on the ASA group running in Active / Active state for BGP multihoming, as the ASA group we create is meant for the security appliance failover. As we always do, we need to make sure both the ASAs are configured to allow traffic betwen the core and the border routers and stuffs like permitting port 179, if we have IBGP sessions going across FWs.

The below document talks about the overall design when we have a multihoming set-up and IBGP sessions running across the FWs

http://www.cisco.com/en/US/tech/tk365/technologies_configuration_example09186a008009487d.shtml

For active / active part on ASAs, configuration steps are explained below.  It is all about creating a logical group and dividing the security contexts on the device into failover groups,

http://www.cisco.com/en/US/products/ps6120/products_configuration_example09186a0080834058.shtml#act1

Best Regards,

Mani

[Ibrahim Jamil]

Hi ahmedchohan

I have exactly the same as ur Topology,would Please Share the configuration with me,however Put Fake IP address Instead

[Mani Ganesan]

Hi Ahmed,

Glad that you liked the session.

Please see my answers inline :

1.   a)  Can u please elaborate on the role of iBGP on the CE routers.

There are multiple useful roles that IBGP between the border routers play. I will mention the important ones.

      If BGP with the primary ISP fails, but your core  devices keep sending traffic to router 1 ( because first router is still  running and your HSRP / IGP stay intact ), if we have an IBGP connection between two routers, R1 can send traffic to R2 over IBGP and  then to ISP 2.

       Another reason is setting of route preference. As we know, Local Preference is an attribute in BGP which is used to influence the way we route outbound packets. When we set Local preference on one router, it is shared with all the routers running BGP with in the SAME AS, which would mean all IBGP peers. So coming to the point, when we run IBGP, LP can be shared and both routers would be aware of the prefrence we set.

         For Example, If we are getting full routing table from both ISPs and on first router, you match half the routes and set a higher local preference. In this case, if you run IBGP with the second router, this LP value is propagated and the second router would be aware of the fact that, for the first half of the routes, router one is the preferred one.

     b) Would the CE routers need the complete routing table for it to make use  of the AS path for the closest destination and other tweaking ?

Go for complete routing table if you are really specific about the path it takes for every destination ( which is not the case most of the times ) and if your Routers have the memory / CPU for 350K routes. From my personal experience, a default and routes from the Provider's directly connected AS's are good enough to do good optimal routing. For the rest of the routes in internet, taking any of the ISPs would be fine.

2.      In case of both the CE routers getting the default route, how can we load balance ?

In this case, do not set any LP. So the question boils down to your IGP / GLBP / MHSRP load balancing. If RTR 1 gets the traffic from core, it would take it own default and the same case for RTR 2. If we have an IGP running, we can make sure that we have equal cost path defaukt routes to the both the routers from the core, so that we send out packets to internet via both the ISPs.

3.     What would be role of hsrp in both of the above scenarios.

HSRP would be important, if we are running BGP in fail over mode and we do not have IGP running between our core and the border routers. For instance, if we have a Flat L2 network, we can configure HSRP's vitual IP as the gateway and track the WAN interface. If the WAN interface ( and hence BGP ) goes down, HSRP will fail over to R2 and it will take over in BGP as well. We can also use HSRP's VIP as the next hop, if we have a static default route configured on the core.

I would be glad to answer if there are any further queries.

Regards,

Mani

[ahmedchohan]

Hi Mani

Thanks for the detailed reply

Just a few more doubts , if you please :

"      If  BGP with the primary ISP fails, but your core  devices keep sending  traffic to router 1 ( because first router is still  running and your  HSRP / IGP  stay intact ), if we have an IBGP connection between two  routers, R1 can send traffic to R2 over IBGP and  then to ISP 2. "

In the above scenario, Would R1's "IP redirect" come into play , and then alltraffice from HSRP would go to R2 ? or all traffic would always go to R1 and then go to R2 ( in case of R1 bgp failure (until unless we use advaced tracking))

Thanks

Ahmed

[Mani Ganesan]

Hi Ahmed,

That s a very good question.

Though redirects are being supported in HSRP now a days, Redirects to passive HSRP routers are not  permitted ( Redirects can be done in HSRP only if the next hop that is suggested in the ICMP redirect packet is that of another active router or a router not running HSRP ) . Redundancy may be lost if hosts learn the real IP addresses  of HSRP routers.

For more details on this -- http://www.cisco.com/en/US/docs/ios/12_1t/12_1t3/feature/guide/dt_hsrpi.html

So, Packets would traverse through R1( HSRP Active )  to R2 and then the ISP.

We can always track a route learned from BGP to track the availability of the ISP and change the HSRP state accordingly.

Please let me know if there are any more questions on this.

Regards,

Mani

[travis-dennis_2]

Hello Manigandan,

In a configuration with a single router with 2 different ISPs on it but each ISP will not advertise routes for the other (Low end broadband connections)  What are my options for inbound traffic failover?  There are a few devices out there that claim to be able to still route the inbound traffic over ISP B if ISP A fails.  If it takes 2 routers to make something work that is something that can be done.

Thakns in advance!

[Mani Ganesan]

Hello Travis,

When you say single router with 2 different ISPs, I assume you are ruuning BGP with both of them and not routing through static default.

To do inbound failoverin this scenario, you can advertise address blcocks assigned by both the ISPs to both of your BGP peers. The trick here is when you advertise the second's ISPs address block to the 1 st BGP peer, do it with AS-path prepending. Similary advertsie the first ISP's address block to your 2nd BGP peer, but with  AS-path prepending.

When both ISPs are up and running, traffic coming back to 1 st ISP's address block would come via ISP 1 and the same case for traffic destined to 2 nd block. If one of them fails, say ISP1,  we would be still be receiving traffic for block 1 via ISP 2, since we are advertising both the blocks to both of the ISPs.

Hope this helps.

Regards,

Mani

[unclerico]

Hi Mani,

Prefix length aside, is there any way that I can do BGP multi-homing with this topology??

My concern is forming the iBGP peering relationship between the CE router in HQ and the CE router in Colo with NAT involved. I'm using RFC1918 addresses inside the firewalls and globally routable addresses on the outside of the firewalls. Thank you very much for your assistance!!!

[Mani Ganesan]

Hi,

We can acheive perfect load balance / failover in this multihoming set-up.

If the CE routers are running different AS numbers (as they are peering with two different ISPs )  IBGP between the CE routers in not an option, we can still configure EBGP between the two CEs. In this scenario, each of the CE would have two EBGP peers ( 1 for the ISP and one for the other CE ),

- assign a higher local preference for the ISP peer and

-  Configure AS path prepending on the other CE peer for the networks you advertise

By doing this we can send and receive traffic through the local ISP at each site, as long as they are available. If one of them fails, CE would start routing traffic through the other CE. So fail over would be acheived seamlessly.

If both CE routers are running the same AS number, IBGP can be configured and the same logic explained above can be done for this as well.

If it is going to be IBGP, we just need ip conenctivity to the peer address and port 179 needs to be open through the FW.

If it is EBGP, we need neighbor ebgp-multihop  < TTL value > along with the above conditions.

OSPF as an IBGP is fine , we can inject a default route from each of the CEs in OSPF using ' default information originate ', which would inject only if it has the BGP default route locally available. so the inside devices would automatically start to send traffic to the other CE, if BGP fails on one of the CE.

When both routers inject a default route, we need to make sure we give preference to the local CE, not the other one.

Hope this helps.

Best regards,

Mani

[unclerico]

Hi Mani,

Thanks so much for responding. I have a followup question in regards to this:

"If it is going to be IBGP, we just need ip conenctivity to the peer address and port 179 needs to be open through the FW."

What would this look like with NAT involved on the fiewalls?? If there were globally routable IP's on both sides of the firewall I could see how to do it, but with RFC1918 IP's on the inside I don't know how I would get this to work. Maybe I'm overanalyzing it. Thank you again.