BGP multiple sites internet access

[amit bhatnagar]

hello ,

we have attached config with two sites primary and secondary data centre 1 as primary data centre 2 as secondary .

We were expecting for internet access it should have used primary site but whats happening is internet is going from both datacentre primary and secondary not sure why .

Thanks for help

[Jon Marshall]

Can you provide more details -

1) what is the destination IP you are testing to ?

2) for that destination IP what do the routing tables and BGP table show for the network on both routers.

3) where are you testing from ie. how are the DCs connected to each other, do you have L3 switches in each DC etc..

4) Your configs per router are showing 2 EBGP peers and 2 IBGP peers. What are these exactly.

From your initial description it's very difficult to say why it is routing the way it is.

Why are you expecting traffic to go via the primary DC ? Is that because you are influencing routing to send it all out via that link ? What about the secondary DC, is it's traffic meant to go out via the primary DC or use it's own links ?

Jon

[amit bhatnagar]

Hello John thanks for reply please find reply inline

1) what is the destination IP you are testing to ?Amit -we used source as lan machine in DC1 and destination as public ip 4.2.2.2, when we do trace from internet it goes via the primary DC , also from inside we are able to reach internet via primary DC , but when Big  traffic is generated example FTP transfer we see it coming to our network via secondary data centre links .

2) for that destination IP what do the routing tables and BGP table show for the network on both routers. Amit- my understanding is it should go via default route

3) where are you testing from ie. how are the DCs connected to each other, do you have L3 switches in each DC etc..Amit :

I am testing from DC1 both DC are connected by layer 3 6509 switch , iBGP is running between them

4) Your configs per router are showing 2 EBGP peers and 2 IBGP peers. What are these exactly.

Amit :- iBGP is for communication between the 2 DC and eBGP is used to connect to service provider

From your initial description it's very difficult to say why it is routing the way it is.

Why are you expecting traffic to go via the primary DC ? Is that because you are influencing routing to send it all out via that link ? What about the secondary DC, is it's traffic meant to go out via the primary DC or use it's own links ?

Amit :we want traffic to always go via the primary link as per current configration we have not done any route influencing but if any one can suggest how can I force trafic via primary link and when primary is not avaiabl eit can go via secondary

Amit

[Jon Marshall]

Amit

Just to clarify.  DCs are interconnected via a 6500 switch. Is this switch runing IBGP as well as the routers ?

If you want all traffic to go via primary then you could -

1) if running IBGP on 6500 use local preference to favour the routes received from primary router

2) if not running IBGP are you redistributing into an IGP on the BGP routers ? If so you could influence the metrics so that the primary router is always favoured.

3) If you are not running IBGP on 6500 or redistributing from BGP to an IGP you could use HSRP between the LAN intefaces of the BGP routers and make the primary HSRP active.

With all of the above though what you need to understand is that you are running IBGP between your BGP routers. If they are receiving more than the default route ie. full or partial routes when the packet gets to the primary router it may see a better path via the secondary router due to the IBGP route it received.   It's difficult to say for sure as it's not clear from your posts what routes your BGP routers are receiving.

All of the above addresses outbound traffic. For traffic coming inbound you can use AS prepending to advertise your DC routes out so that the primary link is favoured over your secondary link.

Does the above make sense ?

What is your primary concern ie. outbound traffic from the DC, inbound or both ?

Edit - just rereaf your last post. It would appear you are more concerned with return traffic. If that is the case most of the above is not relevant, only the AS prepending bit.

Jon

[amit bhatnagar]

Thanks John for detailed reply really appreciate that ,

We are using 6500 as Layer 3 and end device on which iBGP is also running both DC are connected via iBGP .

iBGP is is running using peer ip DC1( 101.82.70.9&101.82.70.41) ) DC2( 101.82.70.9&101.82.70.42)

My concern is both incoming and outgoing traffic DC 1 should be used as primary link and when it is not available DC2 can be used . do advise .

[Jon Marshall]

Amit

We are using 6500 as Layer 3 and end device on which iBGP is also running both DC are connected via iBGP .

What routes are you receiving for the internet ? Is it just the default ?

It's not clear if there are remote sites that use the DC for internet connectivity and how they connect to the DCs ?

If there are remote sites and they come to the DC for internet how does the traffic flow ie. do they go via the 6500 or not ?

It's also not clear how you advertise your public IPs to the ISP for internet connectivity. What public addressing do you have ?

Jon

[amit bhatnagar]

Hello Jon ,

Please find reply inline

What routes are you receiving for the internet ? Is it just the default ?   - Amit  its just the default route we are recieveing from internet

It's not clear if there are remote sites that use the DC for internet connectivity and how they connect to the DCs ? Amit- there is no other site which use our 2 DC for the internet 

If there are remote sites and they come to the DC for internet how does the traffic flow ie. do they go via the 6500 or not ? - no remote  site comes to DC

It's also not clear how you advertise your public IPs to the ISP for internet connectivity. What public addressing do you have ?Amit : DC1 public subnet 110.30.140.0/24 , DC 2 :110.30.141.0/24 .

[Jon Marshall · Accepted solution]

Amit

Thanks for the repiles.

So there are no remote sites using the DC for any type of traffic and you only receive a default route for the internet on both routers ?

In terms of inbound traffic from the internet if you only advertise your DC1 public IP subnet from the DC1 router then you should be fine.

Outbound traffic you have the options i described above. It all depends on exactly what the setup is. I'm not sure why you are running IBGP for just a default route. The advantage of running it would be if internet traffic could end up at the secondary router. Using local preference you would be able to then send that traffic back across to the primary router and out to the internet.

If traffic always has to go via the 6500 though you could still use local pref (6500 running IBGP), or if it is a single 6500 you could use BGP weight to again favour the primary router.

An alternative as already stated would be to just have a default route on the 6500 and run HSRP between the WAN routers and make the primary active. You wouldn't then run IBGP on the 6500.

I am still not entirely sure i fully understand the setup because i am wondering why you are running IBGP between the WAN routers and the single 6500 ? Is there a reason for this ?

Jon

[amit bhatnagar]

thanks John that was really helpful