05-05-2022 07:54 AM
We are having an issue with our Internet edge routers looping some IP blocks between them. Curious if anyone has any input. Below is a super simple diagram of the setup.
[ISP A] [ISP B]
| EBGP | EBGP
[Router A] <-- IBGP --> [Router B]
When we traceroute to 23.78.173.46 it hits Router B, forwards to Router A, when then forwards back to Router B and the cycle continues. What is so bizare is Router B from what I can tell, has no reason to do this, while Router A has the best route to be present on Router B. Here is what I would presume is the relevant output from both routers. I've removed private IPs / ASs from the output.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! show ip bgp 23.78.173.46 !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! Router A !
BGP routing table entry for 23.78.160.0/19, version 498429827
Paths: (2 available, best #1, table default)
Advertised to update-groups:
56
Refresh Epoch 1
<AS ISP B> 7843 20940 16625
<ROUTER B> from <ROUTER B> (<ISP B>)
Origin IGP, metric 0, localpref 100, valid, internal, best
rx pathid: 0, tx pathid: 0x0
Refresh Epoch 1
<AS ISP A> 174 7018 20940 16625, (received & used)
206.40.127.169 from <ISP A> (<ISP A's Peer>)
Origin IGP, localpref 100, valid, external
rx pathid: 0, tx pathid: 0
! Router B !
BGP routing table entry for 23.78.160.0/19, version 23546084
Paths: (1 available, best #1, table default)
Advertised to update-groups:
2 3
Refresh Epoch 1
<AS ISP B> 7843 20940 16625
<ISP B> from <ISP B> (<ISP B's Peer>)
Origin IGP, localpref 100, valid, external, best
Community: 514001009 514001350 514001468 514003149
rx pathid: 0, tx pathid: 0x0
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! show ip route 23.78.173.46 !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! Router A !
Routing entry for 23.78.160.0/19
Known via "bgp <AS INTERNAL>", distance 200, metric 0
Tag 10796, type internal
Last update from <ROUTER B> 2d06h ago
Routing Descriptor Blocks:
* <ROUTER B>, from <ROUTER B>, 2d06h ago
Route metric is 0, traffic share count is 1
AS Hops 4
Route tag 10796
MPLS label: none
| Router B|
Routing entry for 23.78.160.0/19
Known via "bgp <AS INTERNAL>", distance 20, metric 0
Tag 10796, type external
Last update from <ISP B> 2d06h ago
Routing Descriptor Blocks:
* <ISP B>, from <ISP B>, 2d06h ago
Route metric is 0, traffic share count is 1
AS Hops 4
Route tag 10796
MPLS label: none
Example traceroute so no one thinks I'm crazy:
Tracing route to 23.78.173.46 over a maximum of 30 hops
1 <1 ms <1 ms <1 ms 1.1.1.1
2 <1 ms <1 ms <1 ms 2.2.2.2
3 1 ms 1 ms <1 ms 3.3.3.3
4 1 ms 1 ms 1 ms <Router B>
5 1 ms 1 ms 1 ms <Router A>
6 1 ms 1 ms 1 ms <Router B>
7 1 ms 1 ms 1 ms <Router A>
This all looks normal, however when you traceroute you hit Router B, which immediately forwards it to Router A, and then as mentioned above the loop starts. I'm obviously missing something but it makes no sense to me that Router B is ever forwarding to Router A. Inaddition there are plenty of other routes that Router A sees better through Router B that do not have the looping issue. At this point, all the IPs appear to be Akamai, not that that should have any bearing on it.
Any input or point in the right direction would be greatly appreciated, thanks!
05-05-2022 08:06 AM - edited 05-05-2022 08:59 AM
iBGP is not direct connect it connect via IGP and this make LOOP.
the LINK explain your issue
also can you confirm what is the IGP you use?
05-05-2022 11:27 AM
We are simply using BGP between them to exchange the routes from the 2 ISPs. Our firewall has a static route to an virtual IP Router A and Router B share via HSRP. The firewall is on the same external IP block, not sure if this will help clarify.
[Router A] (10.0.0.3) <-- 10.0.0.1 (HSRP IP) --> [Router B] (10.0.0.2)
|
| Static Route to 10.0.0.1
!
[Firewall]
The routers and firewall are on the same switching plane and communicate there. I will take a look at the article you linked, thank you for that. I am confused as to how the loop is so selective on what is looping, and it's not more prominent. We've had this configuration for at least 5+ years and never had a single problem.
05-05-2022 01:40 PM
can you share the following
show ip route <ISP A IP address>
show ip route <ISP B IP address>
05-06-2022 01:01 AM - edited 05-06-2022 01:03 AM
Hello
Ona side note for each ibgp/ebgp rtr append the next-hop-self command to each ibgp peering and also negate those rtrs from becoming a transit ASN for yor the ISPs.
Both rtrs
ip as-path access-list 10 permit ^$
router bgp xx
neighbour <ibgp neighbor> next-hop-self
neighbor <ebgp peering> filter-list 10 out
Can you post the following:
sh ip protocols
sh ip int brief | in up
sh standby
sh run | sec router
05-06-2022 01:12 AM
Hello,
post the full running configs of both routers, so we can lab this up.
05-06-2022 03:02 AM
I really appreciate all the responses. We reloaded the Router B and the routing loops have been resolved. If the situation reoccurs I'll update the post with the configuration and the other information requested. Or if I get frisky later I'll sanitize the output and throw it in here. Once again, appreciate the entire communities help on this.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide