SD Access campus topology and Fusion routers?

[andrew.butterworth]

We are in the process of building a SD Access lab.  Based on some sample topologies we've been provided with this is what our lab looks like.

The top two switches are our Fusion devices (Cat 3750Xs), the middle two are Cat 9300 Border nodes and the two at the bottom are Cat 9300 Edge nodes.  There are dual GigE links between each Fusion and Border nodes (this is based on sample topology diagrams we have been provided with).  However I don't think dual connections are specifically required as we could just use single interfaces configured as .1q trunks.

I 'think' the logic is to have the Underlay physically separated from the Overlay networks - i.e. G1/0/43 on the Fusion devices configured as routed /30 interfaces in the GRT with the equivalent on the Borders.  G1/0/44 on the Fusions would then be a .1q trunk with VLANs and corresponding SVIs in each of the Overlay VRFs that extend to the Fusion devices.

The 'shared services' will be VLANs/SVIs in the GRT on the Cat 3750x's simulating our data centre.  DNA-C, ISE, Windows AD, DNS, DHCP and NTP will be available here as well as a Cat 9800 WLC.

Have I understood this correctly - i.e. separation of the Underlay and Overlay networks onto different physical links.  And if I have what is the logic behind this?

Cheers

Andy

[balaji.bandi]

I am also part of the journey - (everyone is learning new technology part of the journey)

yes, you are correct, there is no requirement for 2 Links, but Underlay is very important in DNAC, so better resilience is more stability (i know it is Lab) still exploring the best option to fit into the real role out.

you can use Layer 3 Port-channel with dual Links (rather rely on the single link ?)   - is this make sense (or my understanding is wrong here ?)

shared service always out of Fabric.

[Scott Hodgdon]

Andrew,

Here are my comments / answers:

1. You always want to design triangles if possible, not boxes. Our best practice is to have both borders connected to both Fusion devices. This allows for better routing protocol convergence if needed. You will hand the same VNs to each Fusion from each Border, just using different SVIs on each end (for now ... planning to remove the need for so many VLANs in a future DNA Center release).

2. L3 handoffs from the Borders to the Fusions will be VRF-lite, and this will include the GRT represented as the INFRA_VN when doing the L3 handoff as part of the Border workflow in DNA Center. We do not need a separate handoff just for the GRT / INFRA_VN. 

3. The link between the Borders can be a trunk to support both Overlay (iBGP) and Underlay (IGP) redundancy. No need for separate links for one vs the other.

Thanks for the nice diagram. It always makes answering questions easier.

Cheers,
Scott Hodgdon

Senior Technical Marketing Engineer

Enterprise Networking and Cloud Group

[andrew.butterworth]

Thanks for the reply Scott.

After some conversations today we have re-jigged the lab to look like this:

Or at least it will be once I get the DC guys to add the four additional cables between the Fusions and the Borders.  The Black lines represent the Underlay and the Brown the Overlay.  We have created a Shared-Services VRF on the c3750x's and moved all the interfaces northbound (except the link to the firewall) into this VRF.  We are leaking routes between the GRT and the Shared-Services VRF so there is full connectivity both ways.

I 'think' that's the Fusion and Shared-Services sorted so we can now move on to the Borders to get the Underlay links up....

[Scott Hodgdon]

Andrew,

Looks much better .

I will note that you do not need the separate black links between the Borders and Fusions as you can add the INFRA_VN to the L3 Handoff on the Brown links and this will carry your GRT to the Fusions. Likewise, you do not need the black link between the Borders. For Underlay resilience,  you can configure SVIs with your IGP on each border and add that to the trunk being used for Overlay resilience. 

Cheers,
Scott Hodgdon

Senior Technical Marketing Engineer

Enterprise Networking and Cloud Group

[andrew.butterworth]

Hi Scott, 

Is there an argument to have physical separation between the Underlay & Overlay?  I am just thinking that potentially the interfaces the Overlay VLANs are running over could get congested due to misconfiguration or just traffic volume or some weird STP loop.  If this happens it could create instability for the Underlay?

Cheers

Andy

[Scott Hodgdon]

Andrew.

You raise interesting points, but I have never seen such a design in my 5+ years doing SDA designs. Nor have I seen any issues with congestion or STP on those interfaces in deployments. 

I have certainly seen cases where different VNs are handed off on different interfaces to different upstream devices, but I have never seen Overlay and Underlay VNs handed off to the same devices on different links.

Cheers,
Scott Hodgdon

Senior Technical Marketing Engineer

Enterprise Networking and Cloud Group

[balaji.bandi]

Good going, thank you for the update.

I know it is only Lab for testing, In terms of FW - I see only 1 path, if that border/device fails - the traffic will be blackholed.

Is the border play a dedicated role External and Internal?

[andrew.butterworth]

Yes, the single firewall is a single point of failure.  Ideally it would be a pair connected into both c3750x's (fusions).  I could only find one spare though...

I'm not too worried about this as this is just a lab.

"Is the border play a dedicated role External and Internal?" - I'm not sure I understand the question?

[balaji.bandi]

your first answer overrides my other question.  sure as I was also made a comment since it is Lab we are not much worried about single or dual-homed, but worth testing dual home (that was my suggestion)- so the same can be replicated when we move to Live. ( thank you again sharing your views).