iBGP peers not communicating

[Paul Harris]

I've got a bit of an odd BGP problem that I'm hoping someone might be able to advise on. I've attached a basic diagram to make it easier to describe the problem.

Router A has a BGP session to ISP A

Router B has a BGP session to ISP B

All three routers have BGP sessions between each other

The loopback addresses I have marked are also the BGP router IDs

You will probably have already spotted a problem, but I'll get back to that . First a bit of background.

Yesterday, due to problems with ISP B, we shut down the BGP session to ISP B on router B. This had the effect that router B was unable to route traffic because the BGP routing table is now empty. This should not be the case, because it should have picked up alternative routes from routers A and C. Forgot C for a moment...

Router B is unable to peer with Router A and I think this is because the router ID is the same (192.168.4.1). However, this doesn't make sense because previously they were happily exchanging updates with no problem. Looking at the idle time, it appears that the connection was broken 1 week and 6 days ago.... which just so happens to be when the router was restarted due to a power cut. Now A and B can peer with C but not each other.

This doesn't really make much sense though because I've never changed the router ID on any of them in the 7 months I've been in this job. The only change to BGP config I have made the entire time is to configure router A to give a higher weight to routes originating from ISP A. I can't see any way that could be related to the issue but is it possible that it is?

It's difficult to diagnose as I can't go back in time to the point where it was working correctly so I can only speculate. It's important to note that none of the routers had been rebooted for at least a year until the power incident. I think that my predecessor had changed the configuration in some way that affects the router ID but not restarted the internal BGP session, leaving the previous (working) config running until the power cut and reboot. Just wondering what other people think of this hypothesis?

But my second problem is with router C. It is not advertising routes from A to B and routes from B to A. Obviously, if it was, we wouldn't have had the issue yesterday when we shut down the session to ISP B. Is it possible that this is related to the fact that routers A and B have the same router ID? In which case, manually changing the router ID on one of them should fix both problems.

Thanks for reading through my wall of text, hope it all makes sense

[Jon Marshall · Accepted solution]

Paul

Not sure about the issue between A and B. Could it be that the B was using a different BGP router-id and then it was rebooted and ended up with the same one. Either way, unless you need the same loopback it would be a good idea to change it.

As for C, an IBGP router will not advertise a route learned from a BGP router to another BGP router. So you have 2 choices -

1) make C a route reflector for A and B and then it can advertise the routes

2) use an IGP such as EIGRP/OSPF between A, B and C. You could then redistribute BGP routes on A and B into the IGP. But this does depend on how many routes you are receiving from your ISPs.

Is there a reason you are running IBGP on C. Do you need to influence which way the traffic goes ?

If not then why not simply have A and B send default-routes to C using an IGP and not run BGP on C at all. This is assuming your ISPs are sending A and B default-routes.

Jon

[mandeepku]

Definely route learned by C through A will not pass to B and vice-versa because of BGP-split horizon.

Making route reflector for a small network in not a feasible solution.

Change ip loopback address of one of the router and kindly share the config of both router A , B and C for furthur diagnosis.

[Paul Harris]

I'm not exactly sure why iBGP needs to be configured on router c as it was done before my time. Equally, there doesn't seem to be any particularly compelling reason to remove it - once the problem with communication between A and B is resolved it shouldn't be an issue.

Anyway, I have changed the loopback address on router B. However, it hasn't updated so if I issue a show ip bgp summary it is displaying the old loopback address as router ID. What is the least disruptive way that I can get this to update?

[Jon Marshall]

Paul

You are going to have to reset the bgp connections so it will need to be done out of hours or you could just rely on router A to route traffic out.

Regarding C, it will not act as a transit between A and B so you need to get that IBGP session up between A and B. You say there is no reason to remove it and that's fine but personally if it's not doing anything i wouldn't run BGP on it, i would simply send it 2 default-routes.

However like you say it's not doing any harm as is.

Jon

[Paul Harris]

Jon,

Would I need to clear all bgp connections on the router or just the session with A? For example, would the command; clear ip bgp x.x.x.x work? If so, is it not the case that since this session is currently down anyway, it wouldn't make any difference?

I'm still a bit of a bgp n00b so I'm trying to make sure I fully understand the implication of any config changes before committing

Paul

[Jon Marshall]

Paul

Actually when you manually change the router-id using the "bgp router-id x.x.x.x" command all BGP sessions are automatically reset so you shouldn't need to do a "clear ip bgp x.x.x.x" for any sessions.

Jon

[Paul Harris]

All working now thanks!