Hi networkers!
Requirements:
- 2 locations (OFFICE, DC) in the same town
- each having two active WAN connections (carrying individual routing domains): The default Any2Any WAN (where several other locations are connected to) and a client specific MC WAN.
- There is a high speed "metro" connection between the locations
- Targets of MC WAN must only be available from a dedicated "MC LAN" network segment
- The default route of "MC LAN" is into Any2Any. Some specific routes coming from MC WAN will overrule A2A routes
- By default, all locally generated traffic should leave into the local WAN links
- In case of a local fault, the locally generated traffic should go via "metro" link into the remote WAN links.
- Traffic between office and DC has to use the metro link.
Hardware: Cat 4500X in VSS configuration at both locations acting as router.
The challenge is with the "MC LAN" that should be fully integrated into A2A routing (communicating locally with devices in other LAN segments and remotely with other sites) but it should also communicate with some special targets of the MC WAN that all other LAN segments must not see.
The general solution that I found is to set the "MC LAN segment" into the GRT but apply "ip vrf receive VRF_MC" and "set vrf VRF_MC" as PBR for targets that should be reached via MC-WAN. It is makes me a little unhappy, that I have to configure a static PBR "routing" because the MC routes are already available by BGP within VRF_MC. But I have tested several other solutions (route leackage e.g.). But they did not work (route leakage for example is not possible on-device between VLANs but only between physical ports).
I put in here only the OFFICE part of the configuration. At the DC there is no "MC LAN" only "MC WAN" which is fully isolated by VRF.
We create two transfer networks at each side. One for the Metro and one for the WAN and start BGP sessions with the neighbors. Failover is guaranteed by longer AS-PATH:
vrf definition VRF_MC
description MC routing domain
rd 65500:1
!
address-family ipv4
exit-address-family
!
interface Vlan3
description MC Office
ip vrf receive VRF_MC
ip address 1.40.1.1 255.255.255.0
no ip redirects
no ip proxy-arp
ip policy route-map MC_PBR_VRF
!
interface Vlan30
description WAN A2A transfer (partner 2.2.2.18 // remote-as 65293 - local AS 65502)
ip address 2.2.2.21 255.255.255.240
!
interface Vlan31
description WAN MC(partner 2.2.2.50 // remote-as 65293 - local AS 65502)
vrf forwarding VRF_MC
ip address 2.2.2.53 255.255.255.240
!
interface Vlan34
description Metro A2A transfer (partner 3.3.3.69 remote-as 65503)
ip address 3.3.3.66 255.255.255.240
!
interface Vlan36
description Metro MC transfer (partner 3.3.3.85 remote-as 65503)
vrf forwarding VRF_MC
ip address 3.3.3.82 255.255.255.240
!
router bgp 65502
bgp always-compare-med
bgp log-neighbor-changes
network 1.40.1.0 mask 255.255.255.0 <-- MC LAN
network 1.1.192.0 mask 255.255.248.0 <-- other Office LAN segments below
network 1.1.200.0 mask 255.255.248.0
network 1.1.208.0 mask 255.255.248.0
neighbor 2.2.2.18 remote-as 65293
neighbor 2.2.2.18 description to_A2A_WAN
neighbor 2.2.2.18 version 4
neighbor 2.2.2.18 remove-private-as
neighbor 2.2.2.18 soft-reconfiguration inbound
neighbor 2.2.2.18 prefix-list BGP_A2A_out out
neighbor 3.3.3.69 remote-as 65503
neighbor 3.3.3.69 description A2A_Metro_to_DC
neighbor 3.3.3.69 update-source Vlan34
neighbor 3.3.3.69 version 4
neighbor 3.3.3.69 soft-reconfiguration inbound
!
address-family ipv4 vrf VRF_MC
network 1.40.1.0 mask 255.255.255.0 <-- MC LAN
neighbor 2.2.2.50 remote-as 65293
neighbor 2.2.2.50 description to_MC_WAN
neighbor 2.2.2.50 version 4
neighbor 2.2.2.50 activate
neighbor 2.2.2.50 remove-private-as
neighbor 2.2.2.50 soft-reconfiguration inbound
neighbor 2.2.2.50 prefix-list BGP_MC_out out
neighbor 3.3.3.85 remote-as 65503
neighbor 3.3.3.85 description MC_Metro_to_DC
neighbor 3.3.3.85 update-source Vlan36
neighbor 3.3.3.85 activate
neighbor 3.3.3.85 soft-reconfiguration inbound
exit-address-family
!
route-map MC_PBR_VRF permit 10
match ip address MC_PBR_ROUTE
set vrf VRF_MC
!
! control BGP
ip prefix-list BGP_A2A_out seq 10 permit 1.1.192.0/21 le 32
ip prefix-list BGP_A2A_out seq 20 permit 1.1.200.0/21 le 32
ip prefix-list BGP_A2A_out seq 30 permit 1.1.208.0/21 le 32
ip prefix-list BGP_A2A_out seq 40 permit 1.40.1.0/24 le 32
!
! control BGP
ip prefix-list BGP_MC_out seq 10 permit 1.40.1.0/24 le 32
!
ip access-list extended MC_PBR_ROUTE
permit ip any 2.2.2.48 0.0.0.15
permit ip any 3.3.3.80 0.0.0.15
permit ip any 7.87.208.0 0.0.15.255
permit ip any 55.55.0.0 0.0.0.255
permit ip any host 93.93.93.93
!
That's all.
What is possible:
- traceroute into MC WAN from Office LAN router "traceroute vrf VRF_MC 55.55.0.83"
1 2.2.2.50 [AS 65276] 8 msec 0 msec 0 msec
2 10.10.21.189 [AS 65276] 4 msec 0 msec 4 msec
3 10.10.41.74 [AS 65276] 12 msec 8 msec 16 msec
- MC LAN is fully reachable from A2A WAN
- Metro link is used for backup and "city" traffic between office and DC.
What does not work:
- A device connected to MC LAN cannot reach any target in MC WAN. Example:
C:\Users\me>tracert -d 55.55.0.83
1 2 ms 1 ms 1 ms 2.2.2.53 <- IP local VLAN31 MC-WAN transfer net (belonging to VRF_MC)
2 <1 ms <1 ms <1 ms 2.2.2.18 <- jump back into the GTR (A2A WAN router IP)
3 1 ms 1 ms 1 ms 5.5.5.5 <- A2A WAN
What is missing?? Is my solution itself a no-go?
Additional question: There is a backup metro link with a smaller bandwidth that should be used only in case of main metro link is down. I installed a route-map to "set local-preference 20" for all routes received via this backup metro link. Is this the recommended way to implement such backup link.
Best regards
