Re: SD-Access greenfield deployment design questions

techno.it · ‎10-29-2021

I am working on a SD-access design and green field deployment for our client I have attached a diagram to illustrate the design.

Firewall would connect outside to fabric borders which has connectivity to Internet, WAN and DMZs. In addition, those firewalls are used for East-West traffic between servers in server farm as well.

Here are some technical questions prior to finalizing the low level design.

1- At first place, is it a valid design? I would love to have your valuable inputs and recommendations.

2- For now, there is no plan for micro-segmentation using ISE and SGTs by customer. That said, macro-segmentation is way to go in the fabric for segmenating traffic between Corporate users, IoTs, Guest etc. VNs

In the design, I will use data center distribution switch for L3 handoff to handle communication between separate VN’s or VRF or from VN/VRF to Shared services residing at the Data Center. I want to ensure internet/unknown traffic originating from campus users is routed directly to firewalls.

What is recommended approach to accomplish it?

3- How should routing be configured when North-South traffic from clients to servers when some servers have network segment behind firewalls? I am guessing I have to creates VRFs on Data center switch then import them Campus VNs!

4- There would be full mesh connectivity between Border nodes and Fusion devices and cross-links between redundant border devices. What routing protocols and configuration will be needed ensure no traffic is disrupted if any link or device fails?

5- I have some IoTs devices for Building Management Systems (BMS) like HVAC, Campus Security and their servers are located in data center block, however these devices should have L2 adjacency with the server ? What is the optimal solution since all the links in campus fabric is L3.

Hoping for valuable suggestions from the great experts. Thanks in advance.

Scott Hodgdon · ‎10-29-2021

@techno.it ,

There is no diagram attached, unless I am missing it.

Cheers,
Scott Hodgdon

Senior Technical Marketing Engineer

Enterprise Networking and Cloud Group

techno.it · ‎10-29-2021

@Scott Hodgdon

I am sorry, my bad I have attached now.

Scott Hodgdon · ‎10-29-2021

@techno.it ,

Let me address each point individually ...

At a high level it looks valid. Where are your Fabric Control Planes ? Will they be colocated on the Fabric Borders ?
You can have multiple L3 handoffs. If the Data Center Block is just customer-controlled subnets and the only path is through the Firewalls, then pointing a default to the firewalls for each VN would route any Internet traffic in that direction. Again here we prefer BGP, so hopefully the Firewalls support that. Still, a simpler design would be to just have L3 handoffs to one set of devices (DC switches in this case) and let them decide where traffic goes (DC or Internet). Since the DC switches are connected to the firewalls in the first place, what is the issue with not using them as the only handoff for the Fabric Borders ?
This also lends itself to my suggestion to just use the DC switches as the single point of handoff for the Fabric Borders. If you hand off Campus VNs to the DC switches, you can more easily control which routes are advertised from their upstream devices and steer traffic accordingly. Otherwise, you would just advertise the server subnets via the firewall-to-Fabric Border L3 handoff so that traffic leaving the fabric knows to take that link and not the link to the DC switch.
Depending on when this will be implemented, cross links will not be needed. A new capability in DNA Center / SDA 2.2.3.3 (see Locator/ID Separation Protocol Publish/Subscribe and Dynamic Default Border sections at https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/release_notes/b_cisco_dna_center_rn_2_2_3.html#Cisco_Reference.dita_7521b123-0502-4dd8-a15d-b7beead502da) allows for dynamic traffic steering in case of failure outside the fabric, and so fabric sites built with these capabilities do not require cross-links or iBGP between the borders. If you have a full mesh between the Fabric Borders and peer devices, then a peer device or Fabric Border failure should not cause more than sub-second traffic disruption as traffic adjusts within the ECMP links between Fabric Edges and Fabric Borders.
Do these IOT devices need an IP gateway outside the fabric, or can they have an IP gateway on the Fabric Edge nodes ?

Cheers,
Scott Hodgdon

Senior Technical Marketing Engineer

Enterprise Networking and Cloud Group

techno.it · ‎10-29-2021

@Scott Hodgdon

Thank you for detailed explanation, highly appreciated.

1. Those borders nodes are hosting both CP + FB
2. I agree with you to use the DC switches to single handedly manage the L3 handoff to route any VN traffic destined to Data center subnets, however for internet traffic from VNs I am intending to direct them to firewalls VN>Border<Firewalls.

3. Servers subnets would be segmented behind firewalls to control the east-west traffic between them. How they can talk to campus VNs if L3 supposedly configured on DC switches ? I have just snippet the diagram and zoomed in to depict better. Please see attached.

4. Implementation is scheduled after a week, so please suggest the configuration considering the current DNAC software release

5. IoTs devices gateway will be on fabric borders.Out of curiosity since you mentioned, how to configure the IP gateway outside the fabric for a VLAN ?

Scott Hodgdon · ‎11-01-2021

@techno.it ,

OK. This was not clear in the original diagram, so I was just checking. I figured this was the case, but needed to make sure .
I understand this, but my question is "why do it this way and just not send all traffic through one L3 handoff and let the DC Switch then route to FW for Internet ?" I know this adds an extra hop, but it will be simpler to deal with one hand off and not two. The routing will be simpler as well, as you won't need to leak defaults from the firewall in two directions (DC Switch and Border).
If the DC switch has the same VRFs as the Border and FW, the server traffic will enter the VRF on the Firewall and simply transit the DC to the border. Again this adds a potential extra hop, but it gives you a single peer with the FW and not 2. This should make troubelshooting and management easier.
2.2.3.3 with IOS XE17.6.1 (needed for new features) is available now, but it is not the recommended release (see https://www.cisco.com/c/dam/en/us/td/docs/Website/enterprise/sda_compatibility_matrix/index.html ). If the customer decides to go with 2.2.2.6 and IOS XE 17.3.4 (the recommended releases), the you will still need the iBGP between the borders for resilience. We are planning a migration to the Dynamic Default Border function I mentioned, but we do not yet know in which release that will happen.
If you add an IP Pool for these IOT devices to the VN, then the gateways will be on the Fabric Edges and not the Borders. We have the ability to create an L2-Only service in fabric whereby the IP gateway can be outside the Border. This requires multicast in the underlay to transport the L2 traffic , and this feature currently requires Cisco Approval to run as it is what we call Limited Availability. We have many customers running it in production since 2.1.2.x, but given your deployment timeline you probably would not be able to get approval in that short amount of time.

Cheers,
Scott Hodgdon

Senior Technical Marketing Engineer

Enterprise Networking and Cloud Group

techno.it · ‎11-05-2021

Thank you @Scott Hodgdon and sorry for my late response

For second point, from security and standard network architecture perspective, would be valid design to route the campus internet traffic passing through Data center switches

For last L2 IoT issue, how to resolve this issue?

Scott Hodgdon · ‎11-10-2021

@techno.it ,

The only way to do what you want (no IP Gateway on the Edge Nodes) is to have the gateways outside the fabric and use a mechanism such as L2VNI to transport traffic to those gateways.

Is there any SD-Access Wireless involved here with these IOT clients, or are they all wired clients ?

Cheers,
Scott Hodgdon

Senior Technical Marketing Engineer

Enterprise Networking and Cloud Group

techno.it · ‎11-10-2021

@Scott Hodgdon

we have both wired and wireless IoT clients. Extending the L2VN to DC, wouldn't be a terrible?

Scott Hodgdon · ‎11-10-2021

@techno.it ,

It wouldn't have to extend to the DC, just to a gateway outside the fabric.

Do any of the IOT clients have overlapping IP spaces ? If not, there is another way that could work, but that also requires Cisco Approval as it is a non-standard design.

Is the plan to use SD-Access Wireless, or will wireless be Over The Top (OTT) and outside the fabric ?

Cheers,
Scott Hodgdon

Senior Technical Marketing Engineer

Enterprise Networking and Cloud Group