01-06-2012 12:04 PM
I have a Hub and Spoke MPLS VPN topology, meaning spoke to spoke traffic MUST pass through the hub. : I need to have 2 Hubs: HUB1 and HUB2 HUB1 will be primary for spoke-A, secondary for spoke-B HUB2 will be primary for spoke-B, secondary for spoke-A HUB1 and HUB2 will have an eBGP peering link external to the MPLS cloud for failover purposes. All sites will have unique AS #'s. : HUB1 will connect to PE1 HUB2 will connect to PE2 Spoke-A and Spoke-B will connect to PE3 : In a Hub and Spoke topology I know I need an ingress & egress vrf on each HUB CE-PE peering pair as this is what forces spoke-to-spoke traffic to flow through the hub. : If I want PRIMARY traffic flows of Spoke-A to HUB1 & Spoke-B to HUB2, do I need one (1) additional vrf (total of 3 vrf's) on the HUB CE-PE peering pair to support the "Load Sharing" topology or 2 additional vrfs (total of 4) per peering pair (HUB PE-CE) or can I do this with just the two vrfs (1 ingress vrf & 1 egress vrf) already configured? : Examples would be great!!!! :) Thanks again! Frank
Solved! Go to Solution.
01-07-2012 01:34 AM
Hi Frank
This is a very interesting scenario of Hub and Spoke MPLS VPN requirement along with the the need to provide the HUB servcies in alternate redundant way..
I would put forward my understanding which can be a possible solution to this scenario. Lets look at the two requirements separately:
1. HUB & Spoke Services
We will only require 2 VRFs on the HUB-PEs viz one Hub_Ingress VRF to collect the routes from the remote spoke PE and send it towards the HUB and one Hub_Egress VRF to send the Hub_Routes as well as remote Spoke Routes from the HUB back to the MP-iBGP cloud. Having said that we would need two logical L3 peering between HUB PE-CE . One peering to be part of Hub_Ingress VRF and other Peering to be part of Hub_Egress VRF..We will have two eBGP neighbourship between HUB PE-CE ie one per VRF..
Also an important point to consider here is that because of default AS-Path Loop Prevention mechanism of BGP on the HUB-PEs under HUB_Egress VRF BGP Neighbourship config we need to add " allowas-in " command so that the spokes routes sent via HUB_Ingress VRF are not dropped when coming back to the HUB PE.
2. Alternate Redundant HUB Services to Spokes
To achive the deisgn such that Spoke 1 is services by HUB1 as Primary and HUB2 as Secondary and vice-versa for Spoke 2 we would need to design the RTs in such a way that on the remote spoke PE we can macth on the unique RT value and change the local preference of the HUB_PE1 routes under Spoke1_VRF to Higher
Value so that Spoke1 prefers HUB1 PE as Primary and similarly we need to macth on the unique RT value and change the local preference of the HUB_PE1 routes under Spoke1_VRF to Higher Value so that Spoke1 prefers HUB1 PE as Primary
So from the above points the VRF Design would be as below
HUB-PE1
VRF: HUB1_Ingress
rd 64513:1
route-target import 64513:200
route-target import 64513:300
VRF: HUB1_Egress
rd 64513:2
route-target export 64513:100
HUB-PE2
VRF: HUB2_Ingress
rd 64513:3
route-target import 64513:200
route-target import 64513:300
VRF: HUB1_Egress
rd 64513:4
route-target export 64513:400
Spoke-PE
VRF: Spoke1
rd 64513:5
route-target import 64513:100
route-target import 64513:400
route-target export 64513:200
VRF: Spoke2
rd 64513:6
route-target import 64513:100
route-target import 64513:400
route-target export 64513:300
Also on Spoke PE we need to apply inbound import-map to change the LP to higher value as discussed above
SPoke PE
ip vrf Spoke1
import map HUB1
!
ip vrf Spoke2
import map HUB2
!
route-map HUB1 permit 10
match extcommunity HUB1
set local-preference 500
!
route-map HUB1 permit 20
!
route-map HUB2 permit 10
match extcommunity HUB2
set local-preference 500
!
route-map HUB2 permit 20
!
ip extcommunity-list standard HUB1 permit rt 64513:100
ip extcommunity-list standard HUB2 permit rt 64513:400
Hope this provides some insight in your traffic requirements.
Regards
Varma
01-07-2012 01:34 AM
Hi Frank
This is a very interesting scenario of Hub and Spoke MPLS VPN requirement along with the the need to provide the HUB servcies in alternate redundant way..
I would put forward my understanding which can be a possible solution to this scenario. Lets look at the two requirements separately:
1. HUB & Spoke Services
We will only require 2 VRFs on the HUB-PEs viz one Hub_Ingress VRF to collect the routes from the remote spoke PE and send it towards the HUB and one Hub_Egress VRF to send the Hub_Routes as well as remote Spoke Routes from the HUB back to the MP-iBGP cloud. Having said that we would need two logical L3 peering between HUB PE-CE . One peering to be part of Hub_Ingress VRF and other Peering to be part of Hub_Egress VRF..We will have two eBGP neighbourship between HUB PE-CE ie one per VRF..
Also an important point to consider here is that because of default AS-Path Loop Prevention mechanism of BGP on the HUB-PEs under HUB_Egress VRF BGP Neighbourship config we need to add " allowas-in " command so that the spokes routes sent via HUB_Ingress VRF are not dropped when coming back to the HUB PE.
2. Alternate Redundant HUB Services to Spokes
To achive the deisgn such that Spoke 1 is services by HUB1 as Primary and HUB2 as Secondary and vice-versa for Spoke 2 we would need to design the RTs in such a way that on the remote spoke PE we can macth on the unique RT value and change the local preference of the HUB_PE1 routes under Spoke1_VRF to Higher
Value so that Spoke1 prefers HUB1 PE as Primary and similarly we need to macth on the unique RT value and change the local preference of the HUB_PE1 routes under Spoke1_VRF to Higher Value so that Spoke1 prefers HUB1 PE as Primary
So from the above points the VRF Design would be as below
HUB-PE1
VRF: HUB1_Ingress
rd 64513:1
route-target import 64513:200
route-target import 64513:300
VRF: HUB1_Egress
rd 64513:2
route-target export 64513:100
HUB-PE2
VRF: HUB2_Ingress
rd 64513:3
route-target import 64513:200
route-target import 64513:300
VRF: HUB1_Egress
rd 64513:4
route-target export 64513:400
Spoke-PE
VRF: Spoke1
rd 64513:5
route-target import 64513:100
route-target import 64513:400
route-target export 64513:200
VRF: Spoke2
rd 64513:6
route-target import 64513:100
route-target import 64513:400
route-target export 64513:300
Also on Spoke PE we need to apply inbound import-map to change the LP to higher value as discussed above
SPoke PE
ip vrf Spoke1
import map HUB1
!
ip vrf Spoke2
import map HUB2
!
route-map HUB1 permit 10
match extcommunity HUB1
set local-preference 500
!
route-map HUB1 permit 20
!
route-map HUB2 permit 10
match extcommunity HUB2
set local-preference 500
!
route-map HUB2 permit 20
!
ip extcommunity-list standard HUB1 permit rt 64513:100
ip extcommunity-list standard HUB2 permit rt 64513:400
Hope this provides some insight in your traffic requirements.
Regards
Varma
01-09-2012 08:15 AM
Hi Vaibhava,
Thank you for your guidence, it is MUST appreciated
After I applied your suggestions, the spoke routes are now flapping in/out of the routing tables every 20 seconds or so.
Also on HUB1 I do not get any routes from HUB2 and HUB2 does not see any routers from HUB1.
Wondering if I need import/export between Hub1 & Hub2?
Tks
Frank
01-09-2012 08:35 AM
Hi Frank
Yes if we need routes exchange between the two HUB Sites then we would need to import the HUB_Egress VRF Export RTs into each other's HUB_Ingress VRF..
When the spoke routes are flapping on the HUB sites is it just the route-flap or also the BGP session flap ?
Can you share the Network Toplogy Diagram at the HUB sites to check for any possible feedback routing ?
Can you share the sh ip bgp output for a particular spoke route before and after flap ?
Regards
Varma
01-09-2012 09:36 AM
Hi Varma,
:Sorry about the formatting, Ciscos site is not very friendly, I have pasted and repasted and still jumbles my text into one big mess.
:
No bgp neighbors are flapping at on any CE, PE or P routers.
My backup link between CE-HUB1 and CE-HUB2 is manually shut at this time to prevent additional confusion. I can send the topology map (Visio) and configs for Hubs, spokes and PE1, PE2 and PE3 (.txt) but cannot send til after 6pm EST, but here are the sh ip bgp nei between Hub1 and PE1 and futher down is the sh ip bgp before and after flap.
:
I think I may need to implement soo to prevent loops but not sure where to implement.
:
As reference:
HUB1 (WAN1) is 172.16.10 and 172.16.1
HUB2 (WAN2) is 172.16.20 and 172.16.2
Spoke1 WAN3) is 172.16.30 and 172.16.3.
Spoke2 (WAN4) is 172.16.40 and 172.16.4.
:
:
:
Here I have 5 less routes as compared with next output, about 30 seconds later
:
WAN1#sh ip bgp nei 172.17.10.2 received-routes
BGP table version is 391772, local router ID is 172.16.10.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 172.16.2.0/24 172.17.10.2 0 65535 20 ?
*> 172.16.20.0/24 172.17.10.2 0 65535 20 i
*> 172.16.20.1/32 172.17.10.2 0 65535 20 ?
*> 172.16.20.2/32 172.17.10.2 0 65535 20 ?
*> 172.16.20.252/30 172.17.10.2 0 65535 20 ?
* 172.17.10.0/30 172.17.10.2 0 0 65535 ?
*> 172.17.20.0/30 172.17.10.2 0 65535 20 ?
*> 172.17.26.0/30 172.17.10.2 0 65535 ?
*> 172.17.30.0/30 172.17.10.2 0 65535 ?
*> 172.17.40.0/30 172.17.10.2 0 65535 ?
*> 172.18.250.0/30 172.17.10.2 0 65535 20 ?
Total number of prefixes 11
:
:
:
Now I have 5 additional routes
:
WAN1#sh ip bgp nei 172.17.10.2 received-routes
BGP table version is 392283, local router ID is 172.16.10.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 172.16.2.0/24 172.17.10.2 0 65535 20 ?
*> 172.16.3.0/24 172.17.10.2 0 65535 30 ?
*> 172.16.4.0/24 172.17.10.2 0 65535 20 65535 40 ?
*> 172.16.20.0/24 172.17.10.2 0 65535 20 i
*> 172.16.20.1/32 172.17.10.2 0 65535 20 ?
*> 172.16.20.2/32 172.17.10.2 0 65535 20 ?
*> 172.16.20.252/30 172.17.10.2 0 65535 20 ?
*> 172.16.30.2/32 172.17.10.2 0 65535 30 ?
*> 172.16.30.3/32 172.17.10.2 0 65535 30 ?
*> 172.16.40.2/32 172.17.10.2 0 65535 20 65535 40 ?
* 172.17.10.0/30 172.17.10.2 0 0 65535 ?
*> 172.17.20.0/30 172.17.10.2 0 65535 20 ?
*> 172.17.26.0/30 172.17.10.2 0 65535 ?
*> 172.17.30.0/30 172.17.10.2 0 65535 ?
*> 172.17.40.0/30 172.17.10.2 0 65535 ?
*> 172.18.250.0/30 172.17.10.2 0 65535 20 ?
Total number of prefixes 16
:
:
And here is SPOKE3 to PE3 bgp received routes :
WAN3#sh ip bgp sum
BGP router identifier 172.16.30.2, local AS number 30
BGP table version is 70541, main routing table version 70541
23 network entries using 2691 bytes of memory
23 path entries using 1196 bytes of memory
16/9 BGP path/bestpath attribute entries using 1984 bytes of memory
7 BGP AS-PATH entries using 168 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
9 BGP filter-list cache entries using 108 bytes of memory
BGP using 6147 total bytes of memory
BGP activity 192/148 prefixes, 34763/34718 paths, scan interval 60 secs
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
172.17.30.2 4 65535 115057 4393 70541 0 0 00:37:26 20
:
:
Here shows Spoke3 (WAN3) just received two additional routes
:
:
WAN3#sh ip bgp sum
BGP router identifier 172.16.30.2, local AS number 30
BGP table version is 70763, main routing table version 70763
25 network entries using 2925 bytes of memory
25 path entries using 1300 bytes of memory
16/9 BGP path/bestpath attribute entries using 1984 bytes of memory
7 BGP AS-PATH entries using 168 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
9 BGP filter-list cache entries using 108 bytes of memory
BGP using 6485 total bytes of memory
BGP activity 192/148 prefixes, 34875/34828 paths, scan interval 60 secs
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
172.17.30.2 4 65535 115281 4393 70541 0 0 00:37:32 22
:
:
WAN3#sh ip bgp
BGP table version is 70923, local router ID is 172.16.30.2
Network Next Hop Metric LocPrf Weight Path
*> 0.0.0.0 172.17.30.2 0 65535 20 i
*> 172.16.1.0/24 172.17.30.2 0 65535 20 65535 10 ?
*> 172.16.2.0/24 172.17.30.2 0 65535 20 ?
*> 172.16.3.0/24 0.0.0.0 0 32768 ?
*> 172.16.4.0/24 172.17.30.2 0 65535 20 65535 40 ?
*> 172.16.10.0/24 172.17.30.2 0 65535 20 65535 10 i
*> 172.16.10.1/32 172.17.30.2 0 65535 20 65535 10 ?
*> 172.16.10.2/32 172.17.30.2 0 65535 20 65535 10 ?
*> 172.16.10.3/32 172.17.30.2 0 65535 20 65535 10 ?
*> 172.16.10.252/30 172.17.30.2 0 65535 20 65535 10 ?
*> 172.16.20.0/24 172.17.30.2 0 65535 20 i
*> 172.16.20.1/32 172.17.30.2 0 65535 20 ?
*> 172.16.20.2/32 172.17.30.2 0 65535 20 ?
*> 172.16.20.252/30 172.17.30.2 0 65535 20 ?
*> 172.16.30.2/32 0.0.0.0 0 32768 ?
*> 172.16.30.3/32 0.0.0.0 0 32768 ?
*> 172.17.10.0/30 172.17.30.2 0 65535 20 65535 10 ?
*> 172.17.16.0/30 172.17.30.2 0 65535 20 65535 ?
*> 172.17.20.0/30 172.17.30.2 0 65535 20 ?
*> 172.17.26.0/30 172.17.30.2 0 65535 ?
r> 172.17.30.0/30 172.17.30.2 0 0 65535 ?
*> 172.17.40.0/30 172.17.30.2 0 65535 20 65535 ?
*> 172.18.250.0/30 172.17.30.2 0 65535 20 ?
r> 192.168.1.0 172.17.30.2 0 65535 20 ?
:
:
:
Now there are gone again, about 30 seconds later
:
WAN3#sh ip bgp
Network Next Hop Metric LocPrf Weight Path
*> 0.0.0.0 172.17.30.2 0 65535 20 i
*> 172.16.1.0/24 172.17.30.2 0 65535 20 65535 10 ?
*> 172.16.2.0/24 172.17.30.2 0 65535 20 ?
*> 172.16.3.0/24 0.0.0.0 0 32768 ?
*> 172.16.10.0/24 172.17.30.2 0 65535 20 65535 10 i
*> 172.16.10.1/32 172.17.30.2 0 65535 20 65535 10 ?
*> 172.16.10.2/32 172.17.30.2 0 65535 20 65535 10 ?
*> 172.16.10.3/32 172.17.30.2 0 65535 20 65535 10 ?
*> 172.16.10.252/30 172.17.30.2 0 65535 20 65535 10 ?
*> 172.16.20.0/24 172.17.30.2 0 65535 20 i
*> 172.16.20.1/32 172.17.30.2 0 65535 20 ?
*> 172.16.20.2/32 172.17.30.2 0 65535 20 ?
*> 172.16.20.252/30 172.17.30.2 0 65535 20 ?
*> 172.16.30.2/32 0.0.0.0 0 32768 ?
*> 172.16.30.3/32 0.0.0.0 0 32768 ?
*> 172.17.10.0/30 172.17.30.2 0 65535 20 65535 10 ?
*> 172.17.16.0/30 172.17.30.2 0 65535 20 65535 ?
*> 172.17.20.0/30 172.17.30.2 0 65535 20 ?
*> 172.17.26.0/30 172.17.30.2 0 65535 ?
r> 172.17.30.0/30 172.17.30.2 0 0 65535 ?
*> 172.17.40.0/30 172.17.30.2 0 65535 20 65535 ?
*> 172.18.250.0/30 172.17.30.2 0 65535 20 ?
r> 192.168.1.0 172.17.30.2 0 65535 20 ?
WAN3#
01-09-2012 10:13 AM
Hi Frank
Thanks for sharing the outputs. It seems the spokes routes which are learnt across mp-ibgp cloud are being lost at HUB Location and at Spoke 1, the Spoke 2 routes are flapping which are again learnt via HUB across MP-iBGP..
SoO would have to be considered only after looking at the network topology but in my view since at the HUB we have the dual homed CEs it needs to be applied there only but as the HUB_Ingress VRF is not exporting anything out and also there are two separate VRFs on the HUBs for Ingress/Egress without any route-leaking there should not be an issue. We can analyze more on the backup link between the the CE-HUB1 and CE-HUB2 as potential feedback routing point and can apply SoO at the two HUBs Egress VRF BGP neighbourship..
But even when backdoor link is shut there is route-flap happening..I have experenced similar issues of route-flap in test lab with overloaded BGP process killing the route cpu and causing route-flaps but at that time I did notice BGP session flap also sometimes..
Regards
Varma
01-09-2012 11:30 AM
01-09-2012 11:38 AM
Hi Varma,
As you may or may not be aware, I have a dual stacked environment.
It gets complicated pretty quickly. But never-the-less it's strange that IPv4 is having an
issue and IPv6 ALWAYS works flawlessly everytime, end-to-end.
Not a single problem with IPv6. For some strange reason IPv4 is failing.
Here are the configs for WAN1, WAN3 and WAN4, PE1, PE3 and P1.
Also, if you need the other configs on WAN2, PE2 and/or P2 Please let me know.
:
:
:
Here are PE1 cpu levels
:
PE1#sh proc cpu | e 0.00%
CPU utilization for five seconds: 76%/6%; one minute: 39%; five minutes: 32%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
91 419316 800553 523 8.71% 4.40% 3.57% 0 BGP I/O
113 688 1381679 0 1.03% 0.87% 0.84% 0 Ethernet Msec Ti
135 3648 339340 10 0.23% 0.19% 0.18% 0 IPAM Manager
139 378092 616470 613 7.83% 3.94% 3.20% 0 IP Input
181 952 41367 23 0.07% 0.06% 0.07% 0 TCP Timer
186 712312 371930 1915 17.59% 8.46% 6.88% 0 IP RIB Update
224 169056 345685 489 3.75% 2.09% 1.81% 0 BGP Scanner
323 827348 1183355 699 21.03% 10.33% 8.30% 0 BGP Router
356 624 43647 14 0.07% 0.03% 0.02% 0 e1t1 Framer back
389 354060 623692 567 8.55% 4.46% 3.71% 0 BGP Task
PE1#
:
PE1#sh proc cpu | e 0.00%
CPU utilization for five seconds: 3%/0%; one minute: 37%; five minutes: 32%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
2 408 2185 186 0.07% 0.03% 0.02% 0 Load Meter
113 688 1383069 0 0.71% 0.86% 0.84% 0 Ethernet Msec Ti
135 3648 339677 10 0.15% 0.19% 0.18% 0 IPAM Manager
150 14640 1560 9384 1.51% 0.28% 0.07% 0 Exec
181 952 41401 22 0.07% 0.06% 0.07% 0 TCP Timer
380 56 109115 0 0.07% 0.06% 0.07% 0 FR Broadcast Out
PE1#
:
:
See attached notepad file for configs - This input function on Cisco's web jumbles all the text into a big unreatable mess.
Regards
Frank
01-09-2012 01:30 PM
Hey Varma,
I think I have solved this issue.
:
I noticed on PE3, the local spoke router WAN4, 172.16.4.0/24 prefix owner was flapping between PE1 and WAN4. Every 30 seconds, the prefix would get updated by either PE1 or WAN4. Each time a new source (owner) of the prefix was provided, either PE1 (10.0.0.1) or WAN4 (172.17.40.1).
:
:
Simple solution was to set the eBGP prefixes received from WAN4 (on PE3) to a local preference higher than the prefixes coming in from HUB1 and HUB2.
:
:
EXAMPLE:
router bgp 65535
:
add ipv4 vrf REG4
nei 172.17.40.1 route-map SPOKE4 in
exit
:
ip access-list standard SPOKE4
permit 172.16.4.0
:
route-map SPOKE4 permit 10
match ip address SPOKE4
set local-preference 700
:
:
NOTE: Even through I did not add a local preference for WAN3 (172.16.3.0/24) this route is now also stable.
No idea why, nor no idea why IPv6 never had an issue.
Thanks again for your support!!!!!!
Frank
01-09-2012 08:49 PM
Hi Frank
Gr8 to hear that you found a fix for the route-flap issue..Did you implement the IPv6 Toplogy same as the IPv4 ie Hub and Spoke with redundant HUBs. I can see from your notes that its a Dual Stack Network for IPv6..Did you use 6PE or 6VPE for IPv6 network ?
I did think over what might have caused the route-flap and my understanding is as below:
Lets take the Subnet 172.16.4.0/24 being advertised from Spoke 2 to PE3. PE3 accepts this as an eBGP update best route under Spoke2_VRF and further readvertises it to the MP-iBGP cloud and being received at PE2 and selected as best route on the HUB_Ingress VRF and sent towards HUB2. Now HUB2 accepts this as an eBGP best route and further readvertises it to PE2 which again accepts is on HUB_Egress VRF becuase of the "allowas-in" feature being enabled and readvertises it to the MP-iBGP cloud. Now PE3 receives this update under Spoke2_VRF and set the Local Preference to 500 because of the import map we used under Spoke2_VRF "HUB2". Now since this MP-iBGP route had higher LP it removes the eBGP route learnt from Spoke2 and consequently the MP-iBGP loses that and then PE2 and the chain kicks off and this Subnet goes in a loop and starts flapping.
To solve this issue your solution is perfect on per prefix basis. What I was thinking on the VRF config level that we change the import MAPs in Spoke1_VRF and Spoke2_VRF to make the LP of the HUB1 routes on Spoke1 being learnt from PE2 to be lower and same ways on PE1 set the LP of the HUB2 routes being learnt from PE1 to be lower value.
This way on PE3 the local routes would always be preferred being eBGP and not contended based on LP value.
Modified route-maps would be:
SPoke PE
ip vrf Spoke1
import map HUB1
!
ip vrf Spoke2
import map HUB2
!
route-map HUB1 permit 10
match extcommunity HUB2
set local-preference 50
!
route-map HUB1 permit 20
!
route-map HUB2 permit 10
match extcommunity HUB1
set local-preference 50
!
route-map HUB2 permit 20
!
ip extcommunity-list standard HUB1 permit rt 64513:100
ip extcommunity-list standard HUB2 permit rt 64513:400
Regards
Varma
01-10-2012 05:23 AM
Hey Varma,
Yes I like your idea better as it is a cleaner solution, setting the looping prefixes to a lower LP, thus will prevent the looping in the first place.
:
As far as IPv6, yes everything is the same except for the addresses of course eg. ":" instead of ".".
:
I now think I understand why IPv6 is not flapping. When I run the command "sh ipv6 route" on any of the dual stacked boxes, I just happen to notice (just yesterday) none of the IPv6 routes have dates or timestamps associated with them as IPv4 does. Purly an oversite on my part - guess I was focusing on too many other things!
:
Best Regards
Frank
:
:
:
Hi Varma,
UPDATE . . . . . . .several hours after configurations updated and implemented
:
:
Turns out, selecting a lower LP on PE3 for spoke routes coming from the hubs (MP-iBGP) causes the original effect of not load sharing the spokes-to-spoke traffic between the hubs. So I am back to setting the LP for local routes from spokes to 700 in ingress (MP-eBGP) and the spoke routes from the hubs (MP-iBGP) to 500. This works very well and is totally stable. No flapping.
:
Also, before playing around with the LP for IPv4 prefixes, as I mentioned a while ago, IPv6 never had problems, BUT now that I changed the LP for IPv4, IPv6 decided to choose a single hub for all traffic. After implementing a very similar fix for IPv6 routes, all is good!! My guess is it's a special IOS feature!!!
:
This is how the good traffic is flowing: (there is no bad traffic)
WAN3 uses HUB2 as the primary path to reach WAN4
WAN4 used HUB1 as the primary path to reach WAN3
WAN3 uses HUB2 for 0/0 and ::0
WAN4 uses HUB1 for 0/0 and ::0
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide