I have made one or two topics a few weeks ago on how to get this topology working, and finally I have completed it ( yet I think so.. ).
There are 2 Networks on my spokes, DATA and INTERNET.
DATA network is always most important, and INTERNET is least.
What I have achieved is , DATA will always use HUB1 as its primary Hub, because it's more reliable and I wanted to centralize each network within one Hub instead of load balancing between two Hub's.Because ISP2 is less reliable, I made it so DATA will always use ISP1 if possible, while INTERNET always goes through ISP2 if possible to reduce the load on ISP1.
I need some recommendation from other people too because I haven't seen anyone else with the same or similar configuration to get this working.Thats why i'm not so sure if this configuration should work fine when it comes to production.
Note : This was all simulated in my Lab environment , and it SHOULD work the same way in production, since its the same topology there too.I have also changed all the IP addresses , so the configs might not be suitable for Copy Paste and needs editing.
Thanks all, and I appreciate everyones contribute.
Here's a screenshot of my Lab:
DATA : 220.127.116.11/24
ISP1: 192.168.28.18/30 - gw 192.168.28.17.
ISP2: 18.104.22.168/24 - no gw needed because its Layer 2 VPN.
T0 : 100.100.100.1/24
T1 : 100.100.101.1/24
ISP1: 192.168.28.249/29 - gw 192.168.28.254.
ISP2: 22.214.171.124 - no gw needed because its Layer 2 VPN.
T0 : 100.100.200.1/24
T1 : 100.100.201.1/24
ISP1 : 192.168.28.250/29 - gw 192.168.28.254
ISP2 : 126.96.36.199/24 - no gw needed because its Layer 2 VPN.
Simulated networks on Vlan interfaces:
Your config looks good in the fact you have redundancy setup correctly with the four tunnels. I have a couple of things that perhaps you should take a look at.
1) You mention that you have DATA and INTERNET subnets and you want to keep them separated and the routing to be different for each of them. This way you can have computers on the INTERNET subnet use the backup link first while the DATA is using the primary.
The concern I have here is how you plan to do that. The router will forward traffic based on the destination and not the source, so even though you have 4 paths to your Data Center only one path will be used.
Are you planning to use acls or perhaps vrf's do separate the traffic.
2) I don't see how the INTERNET traffic will flow specifically to HUB2 when it is accessing the internet. I don't see you advertising a default route down to the spokes. Perhaps there is some confusion on what you mean when you say DATA and INTERNET.
Thank you for your time to read this and contribute.
I have separated DATA from INTERNET traffic using offset-list at the spoke level, and at the Hub's.However, I couldn't think of a way how to remove the unnecessary routes on Hub1 for the INTERNET subnet, it would have been better if Hub1 had only DATA routes.That doesn't cause problems, not yet..but its not what I wanted to achieve.
A little nice feature i've added was adding offset-list on the stack switch, to advertise my Data Center subnets down to the spokes , I have created two acl's , and whenever I need to advertise a new network from my Data Center, I just have to add them to the corresponding acl on the stack switch.
Hub advertises summary routes down to the spokes & the stack switch with different metrics.Hub1 advertises DATA subnet with a metric of 1, and INTERNET subnet with a metric of 110, that way it will stay a backup Hub for INTERNET subnet, and so the other way around goes with Hub2.
You can see the summary routes on each Tunnel interface at the Hub level, and at the interface connected to the stack switch , so the stack switch will only have summary routes. ( if this can be achieved in a easier/shorter way, that would be great because its making my configs a bit long ).
Here's a quick description of my Tunnel interfaces:
Tu0 up up *** Primary for DATA via HUB1 - ISP1 ***
Tu1 up up *** Secondary for DATA via HUB1 - ISP2 ***
Tu2 up up *** Primary for INTERNET via HUB2 - ISP2 ***
Tu3 up up *** Secondary for INTERNET via HUB2 - ISP1 ***
In your original post you mentioned that you had two subnets behind each and every spoke DATA and INTERNET. From your descriptions it seems that instead the DATA subnets and the INTERNET routes are actually behind your two hub routers (hub1 and hub2). Further more you have adjusted the path costs such that the tunnels to hub1 are the best path for the DATA and the tunnels to the hub2 are the best path for the INTERNET. If that is correct I believe this should work well for you.