topic Re: Routing/BGP Issue in Routing and SD-WAN

Routing/BGP Issue

jimmlegs — Wed, 27 Apr 2022 12:53:21 GMT

I have an issue trying to route from my PC at 192.168.150.202 successfully to 62.10.10.66. The 192.168.150/24 and 192.168.68.0/24 networks are connected via cross-connected sites using the 172.30.254.0/29 network to communicate.

62.10.10.66 and 192.168.68.43 are two interfaces on the same server, the private IP has a BGP neighborship with 192.168.68.253 advertising the 62.10.10.66/28 network. This is what my router sees:

B 62.10.10.10 255.255.255.240
[20/1] via 192.168.68.43, 5w1d

A traceroute gets me to the private IP of the correct server but dies there. I do not think that the other router is aware of the path back but I have not been able to figure out how to fix this.

tracert 62.10.10.66

Tracing route to 62.10.10.66 over a maximum of 30 hops

1 4 ms * 3 ms 172.30.254.2
2 3 ms 3 ms 3 ms 192.168.68.43
3 * * * Request timed out.
4 * * * Request timed out.
5 * * * Request timed out.
6 * * * Request timed out.
7 * * * Request timed out.
8 * * * Request timed out.

BGP on the remote side looks like this:

Network Next Hop Metric LocPrf Weight Path
192.168.68.253 0 0 65518 i
* 172.30.0.0 192.168.68.253 0 0 65518 i
* 172.30.254.0/28 192.168.68.253 0 0 65518 i
* 192.168.150.0 192.168.68.253 0 0 65518 i

192.168.68.252 is an L3 switch and the default gateway on the 192.168.68.0/24 network. 192.168.68.253 is the router and gateway of last resort for the L3 switch. I've tried to explicitly add a route for 172.30.254.0/28 over the lan2-if but I get a message saying: "ERROR: Cannot add route, connected route exists"

Any help would be much appreciated.

Relevent Configurations:

interface GigabitEthernet1/2.68
vlan 68
nameif inside-68
security-level 100
ip address 192.168.68.253 255.255.255.0 standby 192.168.68.254

interface GigabitEthernet1/2.254
vlan 254
nameif lan2-if
security-level 100
ip address 172.30.254.4 255.255.255.240
!

router bgp 65518
bgp log-neighbor-changes
bgp router-id x.x.x.x
address-family ipv4 unicast
neighbor 192.168.68.42 remote-as 15518
neighbor 192.168.68.42 activate
neighbor 192.168.68.43 remote-as 15518
neighbor 192.168.68.43 activate
network 192.168.150.0
network 192.168.152.0
network 192.168.160.0
network 192.168.162.0
network 172.30.254.0 mask 255.255.255.240
auto-summary
synchronization
exit-address-family
!

route outside 0.0.0.0 0.0.0.0 x.x.x.x 1
route inside-68 62.10.10.66 255.255.255.255 192.168.68.43 1
route lan2-if 192.168.150.0 255.255.255.0 172.30.254.2 1
route lan2-if 192.168.152.0 255.255.255.0 172.30.254.2 1
route lan2-if 192.168.160.0 255.255.255.0 172.30.254.2 1
route lan2-if 192.168.162.0 255.255.255.0 172.30.254.2 1

Re: Routing/BGP Issue

Jon Marshall — Wed, 27 Apr 2022 13:13:27 GMT

The configuration for your 192.168.68.253 router looks more like a firewall configuration ?

If it is that could cause an issue as your traffic is asymmetric because the ping to the server does not go via that router but direct from the L3 switch as that has an interface in 192.168.68.x but the return traffic points back to that router.

Not really sure what the L3 switch is meant to be doing in the setup.

Jon

Re: Routing/BGP Issue

jimmlegs — Wed, 27 Apr 2022 13:21:40 GMT

Yes, it is an ASA device performing the routing. I believe the switch was supposed to be handling the bulk of the routing however BGP was a requirement from the vendor and the switch does not support it. What are my options to remedy asymmetric routing?

Re: Routing/BGP Issue

Jon Marshall — Wed, 27 Apr 2022 13:39:40 GMT

The main issue is the L3 switch ie. if was not doing any routing then the ASA and the router on the top right could simply route traffic between them on the 172.30.24.0/29 subnet and all traffic would be symmetric.

This would also match your description of 172.30.24.0/29 being the subnet that connects your two sites.

But at the moment that description is not strictly accurate because of the L3 switch but it may not be possible to turn off routing as that may effect the rest of your network.

You could just shut down the 192.168.68.x interface on the L3 switch but again you would then need to make sure the routing still worked ie. the L3 switch would then need to know how to reach 192.168.68.x via the ASA firewall.

Jon

Re: Routing/BGP Issue

Harold Ritter — Wed, 26 Oct 2022 10:31:09 GMT

Hi @jimmlegs ,

It looks like the following routes are learnt via BGP, but none of them is selected as the best path.

Network Next Hop Metric LocPrf Weight Path
192.168.68.253 0 0 65518 i
* 172.30.0.0 192.168.68.253 0 0 65518 i
* 172.30.254.0/28 192.168.68.253 0 0 65518 i
* 192.168.150.0 192.168.68.253 0 0 65518 i

It is probably because the next hop is not reachable. You should do a

show bgp ipv4 uni 192.168.150.0

to get more information.

Regards,

Re: Routing/BGP Issue

jimmlegs — Wed, 27 Apr 2022 14:49:23 GMT

Here are the results.

show bgp ipv4 uni 192.168.150.0

BGP routing table entry for 192.168.150.0/24, version 3827
Paths: (1 available, best #1, table default)
  Advertised to update-groups:
              1
  Local
    172.30.254.2 from 0 (x.x.x.x)
      Origin IGP, metric 0, localpref 100, weight 32768, valid, sourced, local, best

Thank you,

Re: Routing/BGP Issue

jimmlegs — Wed, 27 Apr 2022 15:54:10 GMT

Being that 192.168.68.252 (switch IP) is the default gateway of the servers on the network, would "TCP State Bypass" be an option as a temporary work-around until the gateways can be updated to the point towards ASA and I can remove the IP from the switch?

Thanks

Re: Routing/BGP Issue

Jon Marshall — Wed, 27 Apr 2022 16:31:59 GMT

I was going to mention that as an option but didn't as it basically circumvents the firewall.

It could be but first maybe have a look at the issue Harold raised as that could solve it for you.

Jon

Re: Routing/BGP Issue

jimmlegs — Wed, 26 Oct 2022 10:34:34 GMT

I don't understand why 172.30.254.2 is showing as preferred in the

show bgp ipv4 uni 192.168.150.0

results but BGP is advertising the path to "192.168.68.253". Can I force BGP to use the 172.30.254.2 address?

I thought perhaps the routes weren't matching exactly but the statement (in BGP context) "network 192.168.150.0 mask 255.255.255.0" still only shows "network 192.168.150.0" in BGP, I guess the subnet is implied?

Thanks

Re: Routing/BGP Issue

Jon Marshall — Wed, 26 Oct 2022 10:36:21 GMT

Is that

sh ip bgp 192.168.150.0

from the server or the router ?

If the server I would expect the

next hop

to be 192.168.68.253 as that is the BGP peer IP address.

Jon

Re: Routing/BGP Issue

jimmlegs — Wed, 26 Oct 2022 10:38:06 GMT

That is from my ASA. I do not have access to the remote device but from what they had provided me previously they are looking at 192.168.68.253 as the

next-hop

This was what they had provided:

192.168.150.0 192.168.68.253 0 0 65518 i

I will request the output for the specific route as you requested previously, apologies that I did not understand the device you were referring to.

Thanks

Re: Routing/BGP Issue

Harold Ritter — Wed, 26 Oct 2022 10:54:27 GMT

Hi @jimmlegs ,

> I don't understand why 172.30.254.2 is showing as preferred in the

show bgp ipv4 uni 192.168.150.0

This is because the BGP path is originated from a static route with 172.30.254.2 as the

next hop

You can configure

neighbor 192.168.68.42 next-hop-self

on the FW side to advertise its peering address as the

next hop.

> "network 192.168.150.0" in BGP, I guess the subnet is implied?

Yes, the classful subnet (class C) is implied.

Regards,

Re: Routing/BGP Issue

Jon Marshall — Wed, 26 Oct 2022 10:40:36 GMT

Okay, in your original post when you said posted the

sh ip bgp

from the remote side, was that from the server ?

I think I may be confusing the issue because Harold is saying the

 next hop

is not reachable from that output which I have been assuming is from the remote server.

Jon

Re: Routing/BGP Issue

jimmlegs — Wed, 27 Apr 2022 18:13:14 GMT

Thank you, I will give this a shot tonight after close of business.

Re: Routing/BGP Issue

jimmlegs — Wed, 26 Oct 2022 10:42:27 GMT

Yes, "192.168.150.0 192.168.68.253 0 0 65518 i" is from the BGP neighbor

My ASA shows

show bgp ipv4 uni 192.168.150.0

BGP routing table entry for 192.168.150.0/24, version 3827
Paths: (1 available, best #1, table default)
  Advertised to update-groups:
              1
  Local
    172.30.254.2 from 0 (x.x.x.x)
      Origin IGP, metric 0, localpref 100, weight 32768, valid, sourced, local, best

I'm going to try

neighbor 192.168.68.42 next-hop-self

as Mr. Ritter advised and will update the thread with the status.

Thanks

Re: Routing/BGP Issue

Jon Marshall — Wed, 26 Oct 2022 10:44:11 GMT

Okay, not sure what 192.168.68.42 is as I thought the neighbor was 192.168.68.43 ?

Also my BGP must be getting rusty because I don't see how that fixes anything ie. the server is seeing the correct

next hop IP

and

next hop

self is usually an IBGP thing but you are peering with EBGP so again not clear how that helps.

That said Harold is way sharper than me so I assume I am just not understanding this fully.

Jon

Re: Routing/BGP Issue

jimmlegs — Wed, 26 Oct 2022 10:56:33 GMT

Just for completeness I wanted to post the results of

sho bgp ipv4 uni 192.168.150.0

from the remote device

sho bgp ipv4 uni 192.168.150.0
BGP routing table entry for 192.168.150.0/24, version 840
Paths: (4 available, best #3, table default)
Multipath: eBGP
Advertised to update-groups:
2 
Refresh Epoch 1
65518
142.3.109.201 from 142.3.109.201 (142.2.107.169)
Origin IGP, metric 0, localpref 100, valid, internal
Community: 17063:298 17063:3590
rx pathid: 0, tx pathid: 0
Refresh Epoch 1
65518, (received-only)
142.3.109.201 from 142.3.109.201 (142.2.107.169)
Origin IGP, metric 0, localpref 100, valid, internal
Community: 17063:298
rx pathid: 0, tx pathid: 0
Refresh Epoch 1
65518
192.168.68.253 from 192.168.68.253 (149.197.17.6)
Origin IGP, metric 0, localpref 100, valid, external, best
Community: 17063:298
rx pathid: 0, tx pathid: 0x0
Refresh Epoch 1
65518, (received-only)
192.168.68.253 from 192.168.68.253 (149.197.17.6)
Origin IGP, metric 0, localpref 100, valid, external
rx pathid: 0, tx pathid: 0

Thanks again for all of the assistance, thus far.

Re: Routing/BGP Issue

Harold Ritter — Wed, 27 Apr 2022 18:30:38 GMT

Hi @Jon Marshall ,

From the config that was provided, I see two neighbors.

neighbor 192.168.68.42 remote-as 15518
neighbor 192.168.68.42 activate
neighbor 192.168.68.43 remote-as 15518
neighbor 192.168.68.43 activate

Next-hop-seld would need to be applied to both, obviously.

Regards,

Re: Routing/BGP Issue

Jon Marshall — Wed, 26 Oct 2022 10:46:20 GMT

Hi Harold

Sorry but still not following.

The server has the correct

next hop IP

of 192.168.68.253 which is right as far as I can see because it is an EBGP peering with the firewall.

So why do you need

next hop self

ie. what is it achieving as far as the return path from the server is concerned ?

Jon

Re: Routing/BGP Issue

Harold Ritter — Wed, 26 Oct 2022 10:49:14 GMT

Hi @jimmlegs ,

192.168.150.0 192.168.68.253 0 0 65518 i

In this context, 192.168.68.253 is the neighbor address, not necessarily the

next hop

The

show bgp ipv4 unicast 192.168.150.0

will give you the

next hop address

Regards,