Using BGP to add routes

christopher_hall · ‎03-06-2014

I have posted previously about an issue with BGP and still need help. I have new information below. For background information, here's the old thread:

https://supportforums.cisco.com/message/4134769#4134769.

Overview:

We have ATMs that communicate over our network to our host, then out through our internet service (via our vendor's router using vpn) to our vendor. Each location learns the route through bgp. We also have a dr location with the same router setup. When the host location goes down, the locations should then go to the dr site. I've attached configs for the switches involved, but here's the shorthand version of how we've configured bgp:

Operations switch:

router bgp 2388

network 10.15.1.0 mask 255.255.255.0

ip route 10.15.1.0 255.255.255.0 10.20.102.125 track 1

We're using IPSLA to drop the route on the operations switch.

On the dr switch, we have a similar configuration, except we have the static route as such:

ip route 10.15.1.0 255.255.255.0 10.20.128.16 250

Our thinking is that in normal operations, all locations will go to the host site (operations), including the dr site. When the host site goes down, bgp drops the route at operations and the dr route will be used. It's not working.

Any ideas?

Jon Marshall · ‎03-06-2014

Chris

How far did you get with the last post ie. did you try to implement it, did it not work etc.

If you would rather i didn't get involved in this new thread then please just say, i won't be at all offended

Jon

christopher_hall · ‎03-06-2014

Jon,

Kick an old friend off this conversation? Me? Noooooo..

Previously, we thought the original route wasn't getting dropped, but it is. We also dropped the ipsla from the dr site (w columbia), as it wasn't really needed (I think you had pointed that out). We had added the static route (and assigned it a weight of 250), but now realized we hadn't added it to the route-map. Here's our route-map config:

route-map DefaultRoute permit 10

match ip address 5 10 20 25 15 30

set as-path prepend 2386 2386 2386

So, we'd just drop that static route in there?

Chris

Jon Marshall · ‎03-06-2014

Chris

I need to get a cup of coffee and have a good read of the old thread to catch up where we are

Presumably the setup is the same ?

If someone else doesn't jump in give me a while to catch up and i'll get back to you.

Jon

Jon Marshall · ‎03-06-2014

Chris

Backup to speed now. To answer your last question -

route-map DefaultRoute permit 10

match ip address 5 10 20 25 15 30

set as-path prepend 2386 2386 2386

So, we'd just drop that static route in there?

Yes, just add 35 to the match ip address line.

However that route on the W Columbia switch should not have an AD of 250 because -

1) you want local W Columbia traffic to go via JConnect2

If you use an AD of 250 then the 10.15.1.0/24 route advertised by the Ops router and received via BGP on the W Columbia router will then be passed to the W Columbia switch and will take preference over the static route because it has a better AD.

2) you want the W Columbia switch to be advertising the 10.15.1.0/24 network via BGP to the W Columbia router so it advertised to the AVPN cloud but with a longer AS PATH so the Ops route is preferred.

So just make it a normal static route with no AD.

That should be the only route for 10.15.1.0/24 on the W Columbia switch.

In terms of tracking this route you do need to do this for local W Columbia clients using the 10.15.1.0/24 network. If you don't track it then if the JConnect2 link fails it will not be removed so local clients will be cut off. If you track it and it fails then it is removed and then the 10.15.1.0/24 route learnt via the AVPN cloud will be used.

After you have done this you will need to do a "clear ip bgp 10.20.128.110 soft out" on the W Columbia switch.

The way to test this is all working, apart from traceroutes obviously, is -

1) on the W Columbia router make sure it is being advertised out ie.

"sh ip bgp neigh x.x.x.x advertised-routes" <--- where x.x.x.x is the AVPN peer IP address

2) on a remote site login to the BGP router and do a "sh ip bgp | include 10.15.1.0 255.255.255.0".

You should see two routes for 10.15.1.0/24 but only one selected as the best path and it should be the one with the shorter AS PATH ie. the Ops route.

Let me know if there is anything that is not clear and let me know how it goes.

Jon

christopher_hall · ‎03-10-2014

Jon,

I've been out for the past three days, so pardon the late response...

You state: "That should be the only route for 10.15.1.0/24 on the W Columbia switch."

How would the W Columbia switch NOT receive a route to 10.15.1.0 from BGP? Or are you just saying that we shouldn't have another state route configured?

More later...

christopher_hall · ‎03-10-2014

Okay, after reading your second post below, I think it's more clear. What I hear is....

1. We add/modify the static route (w/o weight), then include that in the route map as such..

route-map DefaultRoute permit 10

match ip address 5 10 20 25 15 30 35

set as-path prepend 2386 2386 2386

2. Because of the pre-pending, the local clients will see the route to jconnect2 as preferred due to the AS distance (?).

3. Other clients will connect to jconnect1 until an outage, then due to tracking, the route to jconnect1 will drop and they will use the route to jconnect2.

4. We need to have tracking on wcolumbia switch, so if jconnect2 goes down, they will get access to jconnect1.

Does this sound about right?

Questions:

a) After the tracking determines that the route to jconnect1 is no longer usable, how long should it take for that route to drop from bgp?

b) After we make the changes above, should I see routes to both jconnect1 and jconnect2 in bgp (sh ip route)?

c) If I don't see the route to jconnect2, what should be our next step?

Thanks,

Chris

Jon Marshall · ‎03-11-2014

Chris

Apologies for delay in replying. This new site is not working as it should so i'm not even sure you will see this answer but i'll respond anyway. Assuming your username is also your e-mail address i'll copy this there as well.

In answer to your points first -

1) yes your config looks good

2) not because of the prepending but because of the AD ie. static has better AD than BGP

3) yes, if the tracked JConnect1 route is removed then BGP no longer advertises it so traffic should go to the W Columbia site

4) yes for local W Columbia clients

In answer to your questions -

a) pretty quickly. Once the route is removed from the routing table the Ops switch should send a BGP withdraw message for that route to the Ops router and it should then stop advertising it.

b) depends on where you look at it. The best way to test is tracerouting from a remote site to make sure it goes via the Ops site

then if possible remove the static route on the Ops switch and try the traceroute again and it should go via W Columbia

When you make the change both sites should be advertising the same route to the AVPN cloud but obviously the W Columbia route will have a longer AS PATH. On both the Ops router and the W Columbia router do a -

"sh ip bgp x.x.x.x advertised-routes" where x.x.x.x is the AVPN peer for each router and you should see the route being advertised.

You can also do a "sh ip bgp 10.15.1.0 255.255.255.0" to see which advertisements it gets but remember for a remote site the AVPN peer is the same.

Like i say use the advertised-routes command to make sure both routers are advertising the route and use traceroute from a remote site or sites to check the actual path being taken.

Jon

christopher_hall · ‎03-12-2014

Jon,

Response received.

We'll be testing in a little over an hour. I'll post the results. One thing I'm curious about is that since we advertise the route on the w columbia switch, shouldn't that route always show in bgp along with the route advertised at the operations switch? It doesn't. I just see the operations route.

Chris

Jon Marshall · ‎03-12-2014

Chris

Not sure i understand where.

On the W Columbia switch you should see the route in the BGP table and on the W Columbia router you should see it as well. In fact you should see both routes and it may be,, as discussed, that the W Columbia router prefers the AVPN cloud route from the Ops router but this shouldn't be a problem.

The Ops router will also receive the W Columbia route but because of the prepending will prefer the route received from the Ops switch.

The W Columbia switch will always prefer it's static route no matter what it receives via BGP.

When you test can you make sure you capture the output of "sh ip bgp 10.15.1.0 255.255.255.0" from -

1) the Ops router

2) the W Columbia router

3) a remote site router

Jon

christopher_hall · ‎03-12-2014

I didn't think about sh ip bgp for a specific route...Here's what I currently see:

WColumbia switch:

SW11-3560-01#sh ip bgp 10.15.1.0 255.255.255.0
BGP routing table entry for 10.15.1.0/24, version 1072
Paths: (2 available, best #1, table Default-IP-Routing-Table)
Advertised to update-groups:
1 2
Local
10.20.128.16 from 0.0.0.0 (172.20.128.1)
Origin IGP, metric 0, localpref 100, weight 32768, valid, sourced, local, best
65213 13979 65200 2388
10.20.128.110 from 10.20.128.110 (192.168.0.98)
Origin IGP, localpref 100, valid, external
SW11-3560-01#

WColumbia router:

RT11-2811-02#sh ip bgp 10.15.1.0 255.255.255.0
BGP routing table entry for 10.15.1.0/24, version 1068
Paths: (2 available, best #2, table Default-IP-Routing-Table)
Advertised to update-groups:
1
2386 2386 2386 2386
10.20.128.16 from 10.20.128.1 (172.20.128.1)
Origin IGP, metric 0, localpref 100, valid, external
13979 65200 2388
192.168.0.97 from 192.168.0.97 (12.123.71.107)
Origin IGP, localpref 100, valid, external, best
RT11-2811-02#

Ops router:

RT00-2811-02#sh ip bgp 10.15.1.0 255.255.255.0
BGP routing table entry for 10.15.1.0/24, version 2012
Paths: (1 available, best #1, table Default-IP-Routing-Table)
Advertised to update-groups:
1
2388
10.20.200.1 from 10.20.200.1 (172.20.102.1)
Origin IGP, metric 0, localpref 100, valid, external, best
RT00-2811-02#

So, I'm assuming that once I drop the ops route, I will see the wcolumbia route on the operations router.

I'll post back results.

Thanks Jon, Chris

Jon Marshall · ‎03-12-2014

Chris

Yes, once the route is dropped on the Ops switch the Ops router should see the route via the W Columbia router.

A better way to view the routes my be to do -

"sh ip bgp | include "10.15.1.0"

that way you just see a summary of it which can be easier to read.

Jon

christopher_hall · ‎03-12-2014

Jon,

We've completed the test and I have some information below. The ATMs did not come back online, so the test failed. What's odd is that in doing tracert from remote locations, the traffic would die at 192.168.0.98, which is the serial side of the W Columbia router.

From the W Columbia router:

RT11-2811-02#traceroute 10.15.1.64

Type escape sequence to abort.
Tracing the route to 10.15.1.64

1 10.20.128.16 0 msec 0 msec 0 msec
2 * * *
3 *
RT11-2811-02#sh ip bgp 10.15.1.0 255.255.255.0
BGP routing table entry for 10.15.1.0/24, version 1093
Paths: (1 available, best #1, table Default-IP-Routing-Table)
Flag: 0x820
Advertised to update-groups:
1
2386 2386 2386 2386
10.20.128.16 from 10.20.128.1 (172.20.128.1)
Origin IGP, metric 0, localpref 100, valid, external, best
RT11-2811-02#

BGP table version is 1093, local router ID is 192.168.0.98
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path
*> 0.0.0.0 10.20.128.195 0 2386 2386 2386 2386 7018 i
*> 10.15.1.0/24 10.20.128.16 0 0 2386 2386 2386 2386 i
*> 10.20.10.0/24 192.168.0.97 0 13979 65200 2388 i
*> 10.20.11.0/24 10.20.128.195 0 0 2386 2386 2386 2386 i
*> 10.20.100.0/24 192.168.0.97 0 13979 65200 2388 i
*> 10.20.102.0/24 192.168.0.97 0 13979 65200 2388 i
*> 10.20.103.0/24 192.168.0.97 0 13979 65200 2388 i
*> 10.20.104.0/24 192.168.0.97 0 13979 65200 2388 i
*> 10.20.105.0/24 192.168.0.97 0 13979 65201 i
*> 10.20.110.0/24 192.168.0.97 0 13979 65204 i
*> 10.20.112.0/24 192.168.0.97 0 13979 65209 i
*> 10.20.114.0/24 192.168.0.97 0 13979 65202 i
*> 10.20.118.0/24 192.168.0.97 0 13979 65205 i
*> 10.20.122.0/24 192.168.0.97 0 13979 65210 i
--More--

From the Operations router:

RT00-2811-02#sh ip bgp 10.15.1.64 255.255.255.0
BGP routing table entry for 10.15.1.0/24, version 2028
Paths: (1 available, best #1, table Default-IP-Routing-Table)
Advertised to update-groups:
1
13979 65213 2386 2386 2386 2386
192.168.0.122 from 192.168.0.122 (12.123.71.47)
Origin IGP, localpref 100, valid, external, best
RT00-2811-02#
RT00-2811-02#
RT00-2811-02#trace
RT00-2811-02#traceroute 10.15.1.64

Type escape sequence to abort.
Tracing the route to 10.15.1.64

1 192.168.0.122 4 msec 8 msec 4 msec
2 192.168.0.97 [AS 13979] [MPLS: Label 18646 Exp 3] 16 msec 20 msec 16 msec
3 192.168.0.98 [AS 13979] 12 msec 12 msec 12 msec
4 * * *
5
RT00-2811-02#

BGP table version is 2028, local router ID is 192.168.0.121
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path
*> 0.0.0.0 10.20.200.1 0 2388 2386 7018 i
*> 10.15.1.0/24 192.168.0.122 0 13979 65213 2386 2386 2386 2386 i
*> 10.20.10.0/24 10.20.200.1 0 0 2388 i
*> 10.20.11.0/24 192.168.0.122 0 13979 65213 2386 2386 2386 2386 i
*> 10.20.100.0/24 10.20.200.1 0 0 2388 i
*> 10.20.102.0/24 10.20.200.1 0 0 2388 i
*> 10.20.103.0/24 10.20.200.1 0 0 2388 i
*> 10.20.104.0/24 10.20.200.1 0 0 2388 i
*> 10.20.105.0/24 192.168.0.122 0 13979 65201 i
*> 10.20.110.0/24 192.168.0.122 0 13979 65204 i
*> 10.20.112.0/24 192.168.0.122 0 13979 65209 i
*> 10.20.114.0/24 192.168.0.122 0 13979 65202 i
*> 10.20.118.0/24 192.168.0.122 0 13979 65205 i
*> 10.20.122.0/24 192.168.0.122 0 13979 65210 i
*> 10.20.124.0/24 192.168.0.122 0 13979 65203 i

The address 192.168.0.122 will be the ATT network (serial side that connects to the operations router).

Oddly, see the tracert from the Operations switch:

Operations switch:
sw00-3560-01#sh ip bgp 10.15.1.64 255.255.255.0
BGP routing table entry for 10.15.1.0/24, version 17949
Paths: (1 available, best #1, table Default-IP-Routing-Table)
Not advertised to any peer
65200 13979 65213 2386 2386 2386 2386
10.20.200.2 from 10.20.200.2 (192.168.0.121)
Origin IGP, localpref 100, valid, external, best
sw00-3560-01#
sw00-3560-01#
sw00-3560-01#tra
sw00-3560-01#traceroute 10.15.1.64

Type escape sequence to abort.
Tracing the route to 10.15.1.64

1 10.20.200.2 8 msec 0 msec 0 msec
2 192.168.0.122 [AS 65200] 8 msec 9 msec 8 msec
3 192.168.0.97 [AS 13979] 17 msec 17 msec 25 msec
4 192.168.0.98 [AS 13979] 9 msec 8 msec 8 msec
5 10.20.128.16 [AS 65213] 17 msec 17 msec 17 msec
6 * * *
7
sw00-3560-01#

Chris

Jon Marshall · ‎03-12-2014

Chris

Unfortunately the new site has removed the jpg you posted in the old thread.

It looks like the BGP advertisements are working in terms of the Ops router and the Ops switch seeing the route via the W Columbia router.

The issue seems to be with the W Columbia side of things. There seem to be missing routes (not the 10.15.1.0/24) network in the IP routing tables.

Is 10.20.128.16 the W Columbia switch ?

It's important to understand that just because it is in the BGP table does not mean it is in the IP routing table.

Because of the way the current site is performing it may be better if you simply e-mail the relevant outputs etc. and i can look over them.

If you could post the jpg to me i can then ask for outputs of the BGP and IP routing tables.

When you test you need to know what the source IP is so we can work out whether there are missing routes.

I think to progress i need the diagram and then we can troubleshoot via e-mail.

Like i say the BGP part seems to be working from what you have posted.

My e-mail is jms.123@hotmail.co.uk and i will be around for a while so if you want to troubleshoot just send me the diagram and we can go from there.

Jon

Jon Marshall · ‎03-13-2014

Just a quick update for anyone else reading this.

The BGP configuration discussed worked fine. The remote sites failed over correctly to the W Columbia site.

The connectivity to the ATMs did not work via the backup site but it looks like this is a vendor equipment issue rather than the BGP routing.

Jon