12-20-2013 05:14 AM - edited 03-04-2019 09:55 PM
Hello,
I'm having an issue where we want to implement a 2 router in two site configuration with one BGP peer in each site. While we move off our provider space, they've agreed to allow their IP space to be originated from our new AS. We've also added a wrinkle where we're doing HSRP for both our old provider space and our new IP space on different routers in case of LAN issues. If I run 1 ISP everything works as it's going to the 1 ISP. If I activate both, Traffic local to the HSRP master will go out the directly connected ISP (for the most part). Just wondering what maybe going on. Configs and testing below.
A little background
Site 1
Router interfaces
gi0/0 v.v.v.v/30 connection to provider 1
gi0/1 w.w.w.253/23 New PI space for our AS HSRP STANDBY
gi0/2 192.168.50.10/24 iBGP interface
gi0/3 x.x.x.253/24 old provider space HSRP MASTER x.x.x.1
router bgp fff
bgp log-neighbor-changes
network w.w.w.0 mask 255.255.254.0
network x.x.x.0
neighbor a.a.a.a remote-as ccc
neighbor a.a.a.a ebgp-multihop 255
neighbor a.a.a.a prefix-list OUT out
neighbor 192.168.50.20 remote-as fff
neighbor 192.168.50.20 next-hop-self
ip prefix-list OUT seq 5 permit w.w.w.0/23
ip prefix-list OUT seq 10 permit x.x.x.0/24
Site 2
Router interfaces
gi0/0 y.y.y.y/30 connection to provider 2
gi0/1 w.w.w.254/23 New PI space for our AS HSRP MASTER w.w.w.1
gi0/2 192.168.50.20/24 iBGP interface
gi0/3 x.x.x.254/24 old provider space HSRP STANDBY
router bgp fff
bgp log-neighbor-changes
network w.w.w.0 mask 255.255.254.0
network x.x.x.0
neighbor b.b.b.b remote-as eee
neighbor b.b.b.b prefix-list OUT out
neighbor 192.168.50.10 remote-as fff
neighbor 192.168.50.10 next-hop-self
ip prefix-list OUT seq 5 permit w.w.w.0/23
ip prefix-list OUT seq 10 permit x.x.x.0/24
When running 1 ISP everything works swimmingly
When making both ISPs active the following occurs:
If my source IP is w.w.w.200/23 I seem to go through provider2 connected to Site2 unless the network is directly connected to provider1 (Ie their website)
If my source IP is x.x.x.x.2/24 I go through provider1 connected to Site1 unless the network is directly connected to provider2.
Sh bgp summ from both routers:
Site 2
BGP table version is 49039450, main routing table version 49039450
476858 network entries using 70574984 bytes of memory
925522 path entries using 59233408 bytes of memory
148030/75445 BGP path/bestpath attribute entries using 20132080 bytes of memory
133842 BGP AS-PATH entries using 5494338 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 155434810 total bytes of memory
BGP activity 7731420/7254550 prefixes, 16909007/15983485 paths, scan interval 60 secs
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
b.b.b.b 4 eee 79431 37 49039360 0 0 00:29:31 475485
192.168.50.10 4 fff 676380 432984 49039450 0 0 5d05h 450035
Site1
BGP table version is 5290586, main routing table version 5290586
470195 network entries using 69588860 bytes of memory
470197 path entries using 30092608 bytes of memory
75405/75391 BGP path/bestpath attribute entries using 10255080 bytes of memory
68490 BGP AS-PATH entries using 2764238 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 112700786 total bytes of memory
BGP activity 6293079/5822878 prefixes, 13887691/13417494 paths, scan interval 60 secs
a.a.a.a 4 ccc 81320 30 5250229 0 0 00:21:36 470446
192.168.50.20 4 fff 432991 676380 5250230 0 0 5d05h 227807
Here's what I see when I do a show IP bgp from both routers to say 8.8.8.8 (which both ISPs peer with)
Site 2
show ip bgp 8.8.8.8
BGP routing table entry for 8.8.8.0/24, version 48394969
Paths: (2 available, best #2, table default)
Advertised to update-groups:
31
Refresh Epoch 1
fff 15169
192.168.50.10 from 192.168.50.10 (x.x.x.253)
Origin IGP, metric 0, localpref 100, valid, internal
Refresh Epoch 1
eee 15169
y.y.y.y from y.y.y.y (64.235.192.12)
Origin IGP, localpref 100, valid, external, best
Site 1
show ip bgp 8.8.8.8
BGP routing table entry for 8.8.8.0/24, version 4817315
Paths: (2 available, best #1, table default)
Advertised to update-groups:
17
Refresh Epoch 1
fff 15169
a.a.a.a from a.a.a.a (68.67.63.67)
Origin IGP, metric 0, localpref 100, valid, external, best
Refresh Epoch 1
eee 15169
192.168.50.20 from 192.168.50.20 (x.x.x.254)
Origin IGP, metric 0, localpref 100, valid, internal
Is there something I'm missing? If I do sh ip bgp I see routes on both routers showing origins on both provider ASes. Just confused as to what could be going on. The end game for me would be to have both table smeshed so that traffic flows to both ISPs based on AS PATH. Is this too good to be true? Is there anything I should be doing to force traffic through certain ISPs?
Thank you for any insight,
David
Solved! Go to Solution.
12-20-2013 05:54 AM
David
I am a bit confused as to the issue ie.
If my source IP is w.w.w.200/23 I seem to go through provider2 connected to Site2 unless the network is directly connected to provider1 (Ie their website)
If my source IP is x.x.x.x.2/24 I go through provider1 connected to Site1 unless the network is directly connected to provider2.
This is what i would have expected because of where the HSRP active gateways are ?
Jon
12-20-2013 05:54 AM
David
I am a bit confused as to the issue ie.
If my source IP is w.w.w.200/23 I seem to go through provider2 connected to Site2 unless the network is directly connected to provider1 (Ie their website)
If my source IP is x.x.x.x.2/24 I go through provider1 connected to Site1 unless the network is directly connected to provider2.
This is what i would have expected because of where the HSRP active gateways are ?
Jon
12-20-2013 06:00 AM
Hi Jon,
I'm not sure. I always assumed BGP would give the best route no matter on which router it was. If it just default routes (for the most part) to the local router where the interface is... doesn't it defeat the purpose of having it multi-router? May as well just have a default route out.
Please correct me if i'm wrong, but wouldn't the routing tables converge and chose the best route between the two routers and always send traffic to the best route independent of what router the egress is?
Thanks
David
12-20-2013 06:08 AM
David
I always assumed BGP would give the best route no matter on which router it was
But it has ie. on each router the best path is via the EBGP connection as you can see from your BGP output. So the thing that decides which router to use in your setup is HSRP.
Unless i am missing something ?
Jon
12-20-2013 06:03 AM
Hi David,
This is normal behavior since each BGP router will select the ebgp learnt prefixes if the as paths are equal on either side.
Regards
12-20-2013 06:30 AM
Hello Harold,
This is normal behavior since each BGP router will select the ebgp learnt prefixes if the as paths are equal on either side.
Ok, should I put the HSRP master for both interfaces on the same router to make sure they go over the same link? It's just weird that when I have both links going in this configuration I'm able to get to some sites and others not so much.
I was hoping to get some kind of routing table "magic" where the routers would receive the BGP prefixes from the ISPs, merge the two into some global route list and then the routers would send the 'LAN' traffic to the shortest AS-PATH to the Internet at large regardless that it's internal or external. I'm now seeing that's not the case.
Have I got this right?
Thanks again for your help
David
12-20-2013 06:47 AM
David
I'm sure Harold will answer this and my have other options but it all depends on how you want to use your routers.
I was hoping to get some kind of routing table "magic" where the routers would receive the BGP prefixes from the ISPs, merge the two into some global route list and then the routers would send the 'LAN' traffic to the shortest AS-PATH to the Internet at large regardless that it's internal or external.
That is what your routers have done. They have exchanged the routes learnt from the ISPs via IBGP but IBGP will not modify the AS path so it depends entirely on the AS path received from the ISPs.
They will use the shortest AS path and that is what is happening. If the HSRP standby router had received a route to 8.8.8.8 with a shorter AS path than the HSRP active then the HSRP active would have sent it to the standby router to be sent out to the internet.
Note the above assumes that all other attributes that have preference over the AS path length in the BGP best path selection were equal.
So what you want is actually happening.
Unless you want something different to happen ? If so could you perhaps clarify.
Jon
12-20-2013 10:41 AM
Hi Jon,
That is what your routers have done. They have exchanged the routes learnt from the ISPs via IBGP but IBGP will not modify the AS path so it depends entirely on the AS path received from the ISPs.
I think this is the crux of my misunderstanding. For equal AS path routes it hits the path out on the local router and that's fine. I would need to test a bit more with my providers as to why when I activate both ISPs I seem to not be able to reach certain sites that I was able to reach with only one ISP.
I'm just wondering as well in regards to the show bgp summ why there's such a discrepancy in the amount of prefixes received on both routers iBGP peers.
IE:
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
b.b.b.b 4 eee 79431 37 49039360 0 0 00:29:31 475485
192.168.50.10 4 fff 676380 432984 49039450 0 0 5d05h 450035
and
a.a.a.a 4 ccc 81320 30 5250229 0 0 00:21:36 470446
192.168.50.20 4 fff 432991 676380 5250230 0 0 5d05h 227807
Thanks again for your replies,
--David
12-20-2013 07:53 AM
Hi David,
> I was hoping to get some kind of routing table "magic" where the routers would receive the BGP
> prefixes from the ISPs, merge the two into some global route list and then the routers would send
> the 'LAN' traffic to the shortest AS-PATH to the Internet at large regardless that it's internal or external.
This would be the case if the AS path on one side was shorter than the other but in the case you listed (8.8.8.8) the AS path is equal (so are the other BGP attributes), which leads to each router preferring the ebgp learnt prefixes over the ibgp learnt prefixes.
The current scenario will provide some load sharing, as some traffic will use one rouetr as HSRP primary and some other traffic will use the other. Is this what you are looking for?
Regards
12-20-2013 10:44 AM
Hi Harold,
I was looking for a load sharing scenario where some traffic would go one way and others would go the other. I just wasn't sure on the details. The previous replies have set me straight as to what to expect.
I'm just having trouble reaching some sites with both feeds running compared to when I just have one. I will work with my ISPs to figure out what might be going on when trying to reach these problematic sites.
Thanks again,
David
12-20-2013 09:23 PM
Hi David,
Please have a Traceroute for the sites which are working and for the one which is not working.
In Tunning BGP you can use AS prepend an AS Path ACL depends on the output you need.
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: