Re: why do I need iBGP Multihop?

ChrisMott30064 · ‎08-21-2023

I have two C9300s acting as Core Routers for my Site, with multiple VRFs. I'm migrating routing from OSPF to BGP, with OSPF acting as the Underlay for a VRF and BGP performing inter-VRF routing, between the two Cores, a Palo FW, and a Cisco SDWAN router. The two Cores and the SDWAN router use iBGP, while Core to FW is eBGP.

The BGP Peers from either Core to the FW (eBGP) and SDWAN (iBGP) are up and active using "normal" BGP settings. However, the iBGP Peer between the two Cores would not come up, even though each Core could PING the other Core's Loopback address.

A peer of mine is battling a similar issue between two SDWAN routers running iBGP between them where the iBGP Peer will not come up. He opened a TAC case and was told to try the command "neighbor X.X.X.X ebgp-multihop".

I happen to be staging a Site and could test this, and what do you know ... the iBGP Peer comes up using "ebgp-multihop".

Can someone explain to me why an iBGP Peer would only come up using an eBGP command that should have ZERO relevance to iBGP?

Giuseppe Larosa · ‎08-21-2023

Hello @ChrisMott30064 ,

>> He opened a TAC case and was told to try the command "neighbor X.X.X.X ebgp-multihop".

>> Can someone explain to me why an iBGP Peer would only come up using an eBGP command that should have ZERO relevance to iBGP?

You haven't provided the details on how the two core swiches are interconnected. I suppose they have a direct link with a L2 or L3 port channel with OSPF running on one SVI or directly on L3 port-channel.

However, I guess the TAC's suggestion is a workaround to force a TTL=255 in the BGP packets to make possible to establish the session. You are right that in normal conditions an iBGP session should use a TTL=255 by default and in fact we configure iBGP sessions on loopbacks as a best practice with no additional commands.

So I suppose there is a software bug, that the workaround allows to handle by manually enforcing the TTL value.

But, this is is just a first sight impression.

To see if this is true you should capture the iBGP packets exchanged between peers with and without the workaround.

Just to be conservative, you can check that TTL is set to 255 with the workaround applied.

Hope to help

Giuseppe

ChrisMott30064 · ‎08-29-2023

Hi Giueseppe,

Thanks for the reply.

There was a BGP piece I was missing here (thank you to Cisco TAC):
https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/13761-39.html

In this scenario, my top-level BGP Process considered the per-VRF iBGP Peer I built as an eBGP Peer, so "ebgp multi-hop" is a valid, and in my case necessary, command for the Peer to come up.