cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1117
Views
20
Helpful
6
Replies

2 isp, 2 routers, /23 with 1 x /24 advertised to 1 isp and other 1 x /24 advertised to other isp. Second advert not working

mn-sysadmin
Level 1
Level 1

I have 2 routers with each connected to a different ISP. Between my two local routers is ibgp.

 

I went ahead and enabled "synchronization" because before I enabled that each router was advertising the adjacent routers /24 to the local ebgp peer via the ibgp learned route. After i enabled synchronization the ibgp learned route went away and the first router is now just advertising it's /24 and the second router is now just advertising the other /24...but now I cannot reach the second /24 from the internet (138.43.255.0/24)

 

Im not quite sure why even though the 138.43.255.0/24 is advertised out ISP2/Router2 that its not working. There is a live HSRP address at 138.43.255.254 that should be reachable. The corresponding 138.43.254.254 is also live on ISP1/Router1 and is reachable from the internet

 

When I access looking-glasses it seems that the 138.43.255.0/24 does not even show as a subnet in the table, but 138.43.254.0/24 does for ISP1

 

The configs are essentially mirrors of each other so it is unclear why the first /24 is fine but the second is not.

 

Configs below of both routers

 

local router 1:

 

router bgp 397304
synchronization
bgp log-neighbor-changes
bgp bestpath as-path multipath-relax
network 138.43.254.0 mask 255.255.255.0
neighbor External-PeerGroup peer-group
neighbor External-PeerGroup remote-as 577
neighbor External-PeerGroup password 7 xxxxxxxxxxxxxx
neighbor External-PeerGroup ebgp-multihop 2
neighbor External-PeerGroup update-source TenGigabitEthernet0/0/1
neighbor External-PeerGroup version 4
neighbor External-PeerGroup soft-reconfiguration inbound
neighbor External-PeerGroup filter-list 1 out
neighbor Internal-PeerGroup peer-group
neighbor Internal-PeerGroup remote-as 397304
neighbor Internal-PeerGroup update-source Loopback0
neighbor Internal-PeerGroup version 4
neighbor Internal-PeerGroup next-hop-self
neighbor Internal-PeerGroup soft-reconfiguration inbound
neighbor 10.98.2.229 peer-group External-PeerGroup
neighbor 216.208.172.174 peer-group Internal-PeerGroup
!
ip as-path access-list 1 permit ^$

-

#show ip bgp neighbors 10.98.2.229 advertised-routes
BGP table version is 16713661, local router ID is 216.208.172.173
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
t secondary path,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

Network Next Hop Metric LocPrf Weight Path
*> 138.43.254.0/24 0.0.0.0 0 32768 i

Total number of prefixes 1
mtor1-ifs-rte01#show ip route static
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
a - application route
+ - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

10.0.0.0/30 is subnetted, 1 subnets
S 10.98.2.228 [1/0] via 216.208.172.161
138.43.0.0/16 is variably subnetted, 78 subnets, 6 masks
S 138.43.254.0/24 is directly connected, BDI254
!

interface BDI254
ip address 138.43.254.251 255.255.254.0
standby 1 ip 138.43.254.254
standby 1 priority 202
standby 1 preempt
standby 2 ip 138.43.255.254
standby 2 priority 201
standby 2 preempt
!

 

#####################################

 

local router 2:

router bgp 397304
synchronization
bgp log-neighbor-changes
bgp bestpath as-path multipath-relax
network 138.43.255.0 mask 255.255.255.0
neighbor External-PeerGroup peer-group
neighbor External-PeerGroup remote-as 852
neighbor External-PeerGroup password 7 xxxxxxxxx
neighbor External-PeerGroup version 4
neighbor External-PeerGroup soft-reconfiguration inbound
neighbor External-PeerGroup filter-list 1 out
neighbor Internal-PeerGroup peer-group
neighbor Internal-PeerGroup remote-as 397304
neighbor Internal-PeerGroup update-source Loopback0
neighbor Internal-PeerGroup version 4
neighbor Internal-PeerGroup next-hop-self
neighbor Internal-PeerGroup soft-reconfiguration inbound
neighbor 209.202.125.25 peer-group External-PeerGroup
neighbor 216.208.172.173 peer-group Internal-PeerGroup

!
ip as-path access-list 1 permit ^$

-

show ip bgp neighbors 209.202.125.25 advertised-routes
BGP table version is 5159590, local router ID is 216.208.172.174
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
t secondary path,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

Network Next Hop Metric LocPrf Weight Path
*> 138.43.255.0/24 0.0.0.0 0 32768 i

Total number of prefixes 1



show ip route static
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
a - application route
+ - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

138.43.0.0/16 is variably subnetted, 78 subnets, 6 masks
S 138.43.255.0/24 is directly connected, BDI254



interface BDI254
ip address 138.43.255.252 255.255.254.0
standby 1 ip 138.43.254.254
standby 1 priority 201
standby 1 preempt
standby 2 ip 138.43.255.254
standby 2 priority 202
standby 2 preempt
!
6 Replies 6

gachilleas
Level 1
Level 1

Hello,

 

You should check with your ISPs and the Internet database (ARIN).

Issues like subnet advertising and routes/inetnums assignment mismatches may appear if not configured properly in Internet databse.

 

Also check whether this subnet (/23) is a PA or PI because your ISPs should treat it differently.

Hey,

 

As I see the registration of your prefix is correct. 

 

Source Registry
ARIN
Net Range
138.43.254.0 - 138.43.255.255
CIDR
138.43.254.0/23
Name
MEDIA-566
Handle
NET-138-43-254-0-1
Parent
NET-138-0-0-0-0
Net Type
DIRECT ASSIGNMENT
Origin AS
not provided
Registration
Tue, 19 Feb 2019 21:55:11 GMT (Tue Feb 19 2019 local time)
Last Changed
Tue, 19 Feb 2019 21:55:11 GMT (Tue Feb 19 2019 local time)
Self
https://rdap.arin.net/registry/ip/138.43.254.0
Alternate
https://whois.arin.net/rest/net/NET-138-43-254-0-1
Port 43 Whois
whois.arin.net
Related Entities 1 Entity

 

And I've checked on an Infowest Looking glass and it is truly not visible

 

Router: InfoWest D3C Border
Command: show ip bgp 138.43.254.0


Thu May 16 02:18:33.368 MDT
BGP routing table entry for 138.43.254.0/24
Versions:
Process bRIB/RIB SendTblVer
Speaker 414879695 414879695
Last Modified: May 14 01:14:40.669 for 2d01h
Paths: (3 available, best #3)
Advertised to update-groups (with more than one peer):
0.2 0.8
Advertised to peers (in unique update groups):
208.76.14.223 10.16.0.229 209.33.231.254
Path #1: Received by speaker 0
Not advertised to any peer
174 577 397304
10.250.10.101 (metric 1) from 10.250.10.101 (10.250.10.101)
Origin IGP, metric 75, localpref 76, valid, internal, group-best
Received Path ID 0, Local Path ID 0, version 0
Community: 174:21001 174:22003 11071:300
Path #2: Received by speaker 0
Not advertised to any peer
6461 577 397304
10.250.10.233 (metric 1) from 10.250.10.233 (10.250.10.233)
Origin IGP, metric 100, localpref 76, valid, internal, group-best
Received Path ID 0, Local Path ID 0, version 0
Community: 6461:5997 11071:300
Extended community: VALIDITY:1
Origin-AS validity: not-found (iBGP signalled)
Path #3: Received by speaker 0
Advertised to update-groups (with more than one peer):
0.2 0.8
Advertised to peers (in unique update groups):
208.76.14.223 10.16.0.229 209.33.231.254
209 577 397304
63.234.254.141 from 63.234.254.141 (205.171.200.178)
Origin IGP, metric 79, localpref 76, valid, external, best, group-best
Received Path ID 0, Local Path ID 0, version 414879695
Community: 209:888 11071:300
Origin-AS validity: not-found

 

outer: InfoWest D3C Border 
Command: show ip bgp 138.43.255.0 <<<<<<

 

Thu May 16 02:21:05.104 MDT
% Network not in table

According to your config, it looks good. However it is now on your ISP to check their Internet router, and within their AS what is happening with your advertisements. In my opinion, they should advertise your /24 prefixes without any further do, so definitely raise a case with them and ask them to check it. 

 

It is easy for them to validate, as they can see if they receive your advertisement, and they can see how their router handles it.

 

Keep us posted about this case's progress.

 

Was my comment helpful? If so please leave feedback.

On most ISPs (some do not check) prefix advertisements and ARIN registrations should match.
Additionally it depends if this prefix allocation is PA or PI.

I raised the issue with the ISP and yes, they were somehow overriding my advertisements and just advertising the /23. (However, I didn't even see the /23 in the internet table, but maybe it created a mismatch and then just nothing was advertised)

 

One of the main reasons we have a PI /23 is to do multi-homed HA with traffic engineering on a subnet basis. It is just surprising that the ISP would essentially limit the very basis for having BGP...to advertise subnets.

 

They made some adjustments and assured me that they will now rely on my advertisements in any combination whatsoever of the /23 and /24's.

 

Now I am just going to switch the /24s back and try some different combinations of advertisement to make sure there are no other hidden constraints.

 

I want to be able to guarantee 100% that a specific /24 is only using ISP 1 and 100% that the other /24 is only using ISP 2. Even if ISP 1 is Tier1 with 12 AS peers and ISP 2 is Tier 2 with less AS peers.

 

advertising separate isolates /24's seems to be the only way to be 100% on this. If I advert the overlapping /23 to ISP 1 then ISP 1 gets almost all the traffic since its Tier1.  I don't want to settle for prepends or other partial measures.

 

So I am going to need to use Conditional Route Injection for failover of the subnets between the ISP's. I don't see any other solution and have hard partitions between ISP's and subnets

 

 

 

 

 

 

 

Hello ,

nice to hear you have contacted ISP2 for this issue.

 

>>

advertising separate isolates /24's seems to be the only way to be 100% on this. If I advert the overlapping /23 to ISP 1 then ISP 1 gets almost all the traffic since its Tier1. I don't want to settle for prepends or other partial measures.

 

So I am going to need to use Conditional Route Injection for failover of the subnets between the ISP's. I don't see any other solution and have hard partitions between ISP's and subnets

 

Yes I agree that conditional Route advertisement can be helpful be aware of the possible side effects of having enabled BGP synchronization as you have written in the initial post of this thread.

 

Hope to help

Giuseppe

 

Giuseppe Larosa
Hall of Fame
Hall of Fame

Hello,

I have checked in a looking glass and I can confirm what you have seen

prefix 138.43.254.0/24 is advertised and present in BGP table, prefix 138.43.255.0/24 is not present in the table.

 

>>When I access looking-glasses it seems that the 138.43.255.0/24 does not even show as a subnet in the table, but 138.43.254.0/24 does for ISP1

 

I have tried to check on ARIN registry the entry that is present is for the whole 138.43.254.0/23, the record has been modified recently Feb 19 2019 and no source AS is registered.

If you own the prefix you may need to complete the registration adding this info in the ARIN database.

www.arin.net

 

These lines

Net Type
DIRECT ASSIGNMENT
Origin AS
>> not provided
Registration
Tue, 19 Feb 2019 21:55:11 GMT (Tue Feb 19 2019 local time)
Last Changed
Tue, 19 Feb 2019 21:55:11 GMT (Tue Feb 19 2019 local time)

Hope to help

Giuseppe

 

Review Cisco Networking products for a $25 gift card