Upgrade Internet router with new ISP for NEW BGP as a secondary

[byme88]

I have an existing Internet router Cisco 3945 that have an existing BGP connection, I am planning to replace this router with a new Cisco 4331, adding an additional Gigabit interface to peering with a NEW ISP for another BGP peering.

The goal is to terminate the existing BGP and swing the traffic to NEW BGP ISP.

What is the best practice to do something like this scenario?

I am thinking to replace the existing 3945 router with new 4331 with additional BGP configuration for the NEW BGP, and make it a secondary Internet access, then remove the existing Primary BGP.

The question is which way is the best way to do this with minimum downtime???

Any comment, suggestion, advice is much appreciated!

I uploaded a simple drawing for current 3945 router and projection of replacement 4331 router. Also, I posted the current configuration of the existing 3945 router.

Thank you in advance

byme88

[balaji.bandi]

Not sure what your network looks like - so please post a network diagram,

Replacing the Router with a new Router you need downtime.

Since you intend to move to a new ISP, Build a New Router with a new ISP and conduct all the tests with the new ISP, BGP peering.

at this stage in the network you have both the routers working as expected, Then in the maintenance window, do the cut-over steering the traffic to the new ISP, if all is working, shut down the old BGP session.

above steps thinking you have 2 different ISP and different AS ?

is the above steps works?

[byme88]

Hi Balaji,

Thanks for a quick post, your suggestion is the prefer way, I post the current configuration and diagram. Current configuration has 1 BGP peering, New router will have 2 BGP peering, but I would like to keep the old BGP as primary and New BGP as secondary and soft swing the old BGP to Secondary during a maintenance window. Does this sound feasible? and what is the best method to do this with the current configuration of BGP? I have upload both diagram and config of the existing BGP router.

Thanks,

Byme88 

[byme88]

To answer your question below:

"above steps thinking you have 2 different ISP and different AS ?

is the above steps works??

the answers is YES, and YES.

but, how would I manage to swing the traffic from old ISP BGP to NEW ISP BGP without the new BGP taking over first???

[Richard Burts]

Byme88

There are complications in bringing up a new ISP connection. Instead of bringing up the new router with both and then using a maintenance window to switch over I would suggest bringing up the new router with only the existing ISP and then using a maintenance window to bring up the new ISP and switchover.

In considering bringing up a new ISP you should think about both outgoing traffic and incoming traffic. Outgoing traffic is  relatively easy to control. One thing to consider is whether you will be receiving the same advertisements from both ISP (are both advertising just a default route, or both advertising full Internet route table, or one or both advertising partial Internet table). If both are advertising the same advertisements then you could use either weight or local preference to prefer the original ISP and then to switch to the new ISP. If one or both are advertising a partial Internet table then you can not control the outbound traffic and that is one reason to make the introduction of new ISP and transition in a maintenance window. To try to control inbound traffic you can use prepend on your advertisements to the new ISP to influence inbound traffic to use the original ISP. But even with prepending it is possible that some Internet traffic to you will use the new ISP, which is another reason to use a maintenance window to introduce the new ISP and to change over.

[byme88]

Hi Richard,

Thanks for the suggestion post, in my case here, both ISPs are advertising the default route for BGP peering!

Should I just use the maintenance widow to swap the router and use only new ISP to avoid complications results from BGP’s return traffic?

Truly appreciated your thoughts here! Thanks again!

byme88

[balaji.bandi]

yes, the above steps work as expected - BGP Preference can be set as below mentioned :

https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/13753-25.html

[paul driver]

Hello

suggest :

1-bring up the new wan rtr2 and create an ibgp session between your existing wan rtr1 and apply next-hopself on both sides of this peering 

2-apply default local preference on your exiting wan rtr1 bgp stanza so this rtr1 will still be preferred rtr for all site egress traffic as when you make the new rtr2 ebgp peering active towards new isp

3- wan rtr2 append as-path prepend attribute for all local  routes  via a outbound route-map 

4- create and append on both wan rtrs 1-2 an as-path filter to filter out and not advertise all as-path traffic towards old/new isp - this would be to negate your site from ever becoming a transit path for either ISP and allow you to just advertise local networks

5- wan rtr2 bring up new ebgp peering to new isp as/when this becomes established your traffic egree/ingress should still traverse vis wan rtr1 

6-test your resilience by disconnecting wan rtr 1 from isp - traffic should fall over to via wan rtr 2 due to the default behaviour of bgp next hop tracking and external fast fall over 

[byme88]

Hi Paul

thanks for your suggestion. I read and reread your post and it make sense with the steps in your post. But I am really not a BGP guru and not sure if I can perform these configuration during production without affecting the incoming and outgoing of both BGPs peers.

Since I will have to plan for a downtime with maintenance windows anyway so I am beginning to think if I can just testing the new new router with new ISP BGP first, once confirmed the BGP establishment I will mirror the rest of the configuration of old router to new router, then schedule a maintenance window and cutover! Do you think it would work and if there’s any “gotcha” I should be awared off by doing this way?

again, thanks for your suggestion.

byme88

[Richard Burts]

byme88

It is helpful to know that both ISP advertise only the default route. This means that for outbound traffic you would be able to set up an effective primary/secondary relationship with the ISPs so that all of your outbound traffic could go to ISP1 and then when you are ready can switch over and all outbound traffic would go to ISP2.

However you potentially have an issue about inbound traffic. Your config shows that your BGP is doing redistribute connected. So you are advertising some networks to the ISPs. Paul suggests using AS prepending to make the path through ISP2 less appealing. And that is a good suggestion. It could get most of the incoming traffic to you to use ISP1. But there is a very real chance that some traffic may come to you from ISP2. This could create a situation where a device sends something to the Internet which goes out ISP1. But the response from the Internet might come back through ISP2. If it happens would that be a problem? In some networks it is not a problem, and in some networks it is a problem. We do not know enough about your situation to know if it would be a problem or not. What do you think about this?

[byme88]

Hi Richard, 

thanks for your response, to answer your question regarding returning traffic, it will be a problem! Your thought?

thanks again

byme88 

[Richard Burts · Accepted solution]

byme88

If traffic going out using ISP1 and response comes back using ISP2 is a problem then I think it is pretty clear that you can not implement a primary/secondary relationship to run both ISP at the same time. When you introduce ISP2 you need to remove ISP1 at that time.

Perhaps it might help to think about it in this way: assume that you have implemented the steps suggested by Paul and are using both ISP in the primary/secondary relationship. Assume that ISP2 has a customer who has network x.y.z.0. A device in your network sends a request to a server that is in x.y.z.0. The request goes out using ISP1. ISP1 forwards the request and it gets to the server. The server generates a response and sends it to ISP2. Now ISP2 has to make a forwarding decision about how to reach your network. It has an advertisement that goes through multiple AS to get to ISP1 or it can forward directly to you. If you use prepending there will not be much traffic that is asymmetric but I do not believe that you could say there will be no traffic that is asymmetric. If you really do need all traffic to be symmetric then you need a single ISP at a time.

[byme88]

Thank you Richard! To avoid unexplained complications, I think I will have to cut-over using just the new ISP BGP with default route out, and hope for the best! I should however test out thoroughly the NEW ISP BGP first, correct?

Should I just configure the new router with basic BGP configuration with nothing advertised out and test so I won’t mess up the production BGP traffic? Will this work? Thanks 

[Richard Burts · Accepted solution]

There might be some things about your network that we do not know and which might impact our advice (for example is there space to install the new router, and to connect its Ethernet to ISP2 without impacting the current installation). But in general I believe that it should be quite possible to bring up the new router with its interface to ISP2 configured, and to configure BGP between the new router and ISP2 (just do not use redistribute connected or any network statements in the BGP config). You can test the ISP connection, can verify that the BGP neighbor relationship is negotiated, and verify that you are receiving the default route from ISP2.

I am guessing that you will want to use the same IP addresses on the inside interfaces of the new router as are being used on the old router. So you probably do not want to connect the inside interfaces of the new router until you are ready to switch over.

[byme88]

Appreciated your response and recommendations! I will go with that solution!

thanks again!

byme88