Hi Uhmazing,
Your newer design looks better than the original. It does cover a variety of use-cases.
The one recommendation I have for you is that I usually don't design data centers to have TLOC extension links. Instead, I terminate the WAN transports with some other hardware and then connect the vEdges to that. For example, get a /29 subnet from the internet provider, then physically terminate the link from the provider with a layer 2 switch to split the circuit to each vEdge. Give each vEdge internet interface its own IP in the /29 range. For MPLS, terminate the link with a non-sd-wan layer-3 device like an ASR and run BGP between the ASR and the MPLS PE Router. Then, on the vEdge MPLS interfaces, configure static default routes pointing at the ASR. This has worked well for me. What I've described is similar to your site id 1 design. What you haven't captured at site 1 is the circuit-terminating devices like the L2 switch and the L3 ASR to terminate the circuits, but the reality of it is that the provider is very unlikely to give you two physical connections for the same circuit at a site, so you'll need those. It's also unlikely that anyone would have two internet circuits and two MPLS circuits at a remote site or even at the data center.
The reason I don't like TLOC extension in the data center is basically because:
1) Its not as "clean" as dedicated TLOC links. TLOC-extension requires reliance on the other router being up to work, whereas direct connections to the WAN transports from each router remove that reliance.
2) You will have a hard time performing horizontal scaling with TLOC extension links in the future. Meaning that if the network scales higher than the SD-WAN router in the data center can manage, then you will have a hard time inserting another pair of headend SD-WAN routers in the data center if you are using TLOC extension.
Hope it helps!
Steve
