Introduction

dbellamk · ‎04-11-2019

Introduction
What is a Fabric Site ?
What is a Fabric Domain ?
What is a Transit ?
Why IP Based Transit ?
SD-Access Distributed Campus Fabric IP as Transit
Fusion Device
SGT Propagation between sites
Cisco SD-Access Network Requirements
Resources

Introduction

Cisco DNA Center gives us the flexibility to configure multiple fabric sites and connect them using different Transits. In this paper, we are going to talk about how we can connect multiple fabric sites using IP as a transit and how can we carry SGTs over IP Transit.

What is a Fabric Site ?

A “Fabric site” is a portion of the fabric which has its own set of Control Plane Nodes, Border Nodes and Edge Nodes. Different levels of redundancy and scale can be designed per Site
by including local resources: DHCP, AAA, DNS, Internet, etc. A Fabric Site may cover a single physical location, multiple locations, or just a subset of a location.

Single Location Branch, Campus or Metro Campus
Multiple Locations Metro Campus + Multiple Branches
Subset of a Location Building or Area within a Campus

What is a Fabric Domain ?

SD-Access fabric domain is hierarchal representation of fabric sites managed by Cisco DNAC SD-Access fabric domain can consist of multiple fabric sites and each site has its own devices which helps us achieve the benefits such as scale, resiliency and survivability. A Fabric domain should be able to scale horizontally by having state specific information within each site.

What is a Transit ?

Multiple Fabric sites in a single Fabric domain will be interconnected using a Transit.

The SD-Access transit consists of Control plane nodes which helps interconnect multiple fabric sites.

There are two types of Transits:

SD-Access Transit - Enables a native SD-Access (LISP,VXLAN,CTS) fabric, with a domain-wide Control Plane node for inter-site communication.

IP- Based Transit - Leverages a traditional IP-based (VRF-LITE, MPLS) network, which requires remapping of VRFs and SGTs between sites.

IP Transit :

Traffic between sites will use control and data plane of the transit network/WAN.
The SD-Access Border will hand off the traffic to the directly connected WAN/External router and the traffic is delivered across the Wan/external domain to other site. No Lisp lookup with the transit control node is performed here as the border will hand off traffic to the WAN/External router using VRF-LITE.
If SD-WAN is deployed in the transit network/WAN then performance based routing, Encryption etc. in the WAN is possible.
End to end policy can only be maintained by Manual configuration.

Screen Shot 2019-04-11 at 6.38.05 PM.png

Why IP Based Transit ?

IP Based Transit can be used if customers are already using existing WAN or have adapted SD-WAN. Tends to be many remote branch offices connected via traditional IP WAN/MPLS or SD-WAN. Also when there is a requirement for site-to-site encryption and traffic engineering and policy based routing.

Typical use cases

Internet Handoff
P2P IPSEC encryption
Policy Based Routing
WAN Accelerators
Traffic engineering
Mobile Backhaul LTE

SD-Access Distributed Campus Fabric IP as Transit

IP transits provides IP connectivity without native Cisco SD-Access encapsulation (SD-Access transit provides).Using IP based transits we can connect the SD-Access fabric to external networks using VRF lite for ip connectivity. We need to make sure that SGT information is also carried between sites using below methods in “SGT Propagation between sites” section.

Important to note that each fabric site has its own set of control plane nodes, edge nodes, border nodes, WLC. Cisco DNAC automates the BGP configuration for all the VRFs on the border nodes and it can be seen in the UI workflow below.

When client gets authenticated and gets assigned an SGT, that packet is traversed to the destination in data plane using VXLAN encapsulation which carries the SGT in its header and when the client is trying to reach to destination outside of the fabric then the packet is de encapsulated at the border from VXLAN header where SGT preservation is also gone. The same applies for returning traffic as well which enters the border switch and as border does not know about SGT for that source or destination.

In a SD-Access distributed deployment, you can have ISE PSNs dedicated to each of the fabric sites or for larger deployments where you have more than 2 PSNs at a site, we can have PSNs centrally located at a data center behind a load balancer and point the VIP of load balancer from DNAC settings

Screen Shot 2019-04-11 at 6.38.12 PM.png

Supported Platforms for Border (as of Cisco DNAC 1.2.10)

Cisco SD-Access Border Node	Cisco SD-Access Transit	IP Transit
C9K	YES	YES
ASR1K/ISR4K	YES	YES
C6K	NO	YES
N7K	NO	YES

Fusion Device

As Cisco SD-Access achieves macro segmentation using vrfs, Users in those vrfs would want to talk to shared services residing out of the fabric which is in global routing table and we use a fusion devices which can be either router/switch/firewall to do route leaking leveraging the L3 handoff on the Border.

Screen Shot 2019-04-11 at 6.38.21 PM.png

If the fusion device is a cisco firewall then ASA supports both SXP and inline tagging. Cisco FTD supports pxgrid and inline tagging and it will know about SGTs via pxgrid from ise. Now ,we will look at how we can propagate SGTs.

Steps on Configuring Fusion router

https://www.cisco.com/c/en/us/support/docs/cloud-systems-management/dna-center/213525-sda-steps-to-configure-fusion-router.html#anc11

SGT Propagation between sites

Note: Assuming Cisco ISE is deployed in the network and end users are authenticating to Cisco Ise via 802.1x or any other method. Cisco ISE will assign Security Group Tag (SGT) once they authenticate to your wired/wireless network.

There are two ways in which we can make sure that SGTs are propagated across Fabric Sites.

Device Inline Tagging.
Using SXP or Scalable group tag exchange Protocol

Device Inline Tagging

Inline tagging is the process where the Security Group Tag is carried within a special field known as CMD (Cisco Meta Data) that can be inserted in an L2 header, whether that be Ethernet, IPsec or Tunnel as in GRE/DMVPN. EtherType:0x8909 16-Bit SGT encapsulated within Cisco Meta Data (CMD) payload. We have to make sure all the devices in the path support Inline tagging. Cisco TrustSec-capable devices have built-in hardware capabilities than can send and receive packets with SGT embedded in the MAC (L2) layer. It allows Ethernet interfaces on the device to be enabled for L2-SGT imposition so that the device can insert an SGT in the packet to be carried to its next hop Ethernet neighbor. SGT-over-Ethernet is a method of hop-by-hop propagation of SGT embedded in Ethernet packets. The inline identity propagation is scalable, provides near line-rate performance and avoids control plane overhead.

Configs that need to be enabled on each and every device in the path on both ingress/egress interface is :

Border# configure terminal

Border(config)# interface gigabitethernet 1/0/1

Border(config-if)# cts manual

Border(config-if-cts-manual)# propagate sgt

Border(config-if-cts-manual)# policy static sgt 54 trusted

Enable this on all the sub interfaces on routers and physical interfaces on switches on the interface facing outside of the fabric from Border.

SGT Platform Compatible Matrix

https://www.cisco.com/c/dam/en/us/solutions/collateral/enterprise-networks/trustsec/software-platform-capability-matrix.pdf

Using SXP or Scalable group tag exchange Protocol

SXP or Scalable group tag exchange Protocol is a TCP-based, control plane protocol used to exchange IP to SGT bindings that have been created on a network device through either dynamic assignment (802.1X, MAB, WebAuth) or statically via CLI with peer networking devices. SXP is supported on most Cisco Routers, Switches, Firewalls, and the Cisco Identity Services Engine or ISE. IP-to-SGT binding exchange over 64999/TCP

SXP is typically used in scenarios where a network devices do not support inline tagging.

SXP is used to propagate the IP-SGT information over the above mentioned network.

Here's how we create SXP sessions between border switches and ISE. The SXP tunnel should be created individually for each VN. By using this you have to build SXP for each vrf onto the border device. Alternatively, we can create sxp domains on ise which allows you to put certain mappings in each domain and only send once you want to each VN using ISE SXP Domain Filters.

An SXP Domain is a collection of SXP Devices and the administrator can decide which domain to send IP-to-SGT mappings to. This is not mandatory as a Default Domain exists and this is used by default for all SXP Devices and all IP-to-SGT mappings. If using SXP Domains to control the distribution of mappings, add the required Domains from Work Centers > TrustSec > SXP > SXP Devices:

Each domain is a unique table of IP/SGT bindings which can be peered independently from other domains and you can chose to have each domain pointing to each VN or Site.

You can set ISE SXP Domain filters from Work Centers > TrustSec > SXP > All SXP Mappings > Manage SXP Domain FIlters

SXP isn’t orchestrated from Cisco DNAC today so we can use template editor in DNAC to configure SXP tunnel. Once the destination SGTs are known to Border via SXP and source SGTs are already known, we can do enforcement on the border manually by entering cts role based enforcement for those vlans on the trunk interface egressing towards the WAN.

Configuring SXP between Border and ISE:

Check to make sure SXP is enabled in ISE. Administration > System > Deployment > (ISE Hostname)

Screen Shot 2019-04-11 at 6.38.41 PM.png

Now, configure SXP on the border for respective VNs.

cts sxp enable

cts sxp default password <password>

cts sxp connection peer <> source <> password default mode local listener vrf <>

Now go to ISE, to the "Work Center > TrustSec > Settings > SXP Settings" and

Make sure the password is matched on ISE as well

Screen Shot 2019-04-11 at 6.38.48 PM.png

Now go to "Work Center > Trustsec > SXP > SXP Devices" and add a session each for each VN on each border.

Screen Shot 2019-04-11 at 6.38.53 PM.png

We can check the status of SXP connection using below commands:

show cts sxp connections vrf Campus

show cts sxp connections brief

We can perform Policy enforcement at the Firewall as well by building SXP from ISE to Firewall and sending the mappings to the firewall and sending SGTs from border to firewall via inline tagging as ASA firewall supports both types of SGT propagation.

Which Border to Pick ?

Screen Shot 2019-07-12 at 12.43.15 PM.png

Cisco SD-Access Network Requirements

Latency Requirements (RTT)

In Summary, device latency should be around 100 msec RTT, you can go up to 200 msec RTT but there could be a performance hit. Anything beyond 200 msec is not recommended by Cisco at this time

The RTT (round-trip time) between Cisco DNA Center and network devices should be taken into consideration. The optimal RTT should be less than 100 milliseconds to achieve optimal performance

for base automation and assurance. When RTT is between 100 milliseconds and 200 milliseconds, longer execution time could be experienced for certain events including Inventory Collection, Fabric Provision and Image Update, ranging from a few minutes to tens of minutes. Cisco does not recommend RTT more than 200 milliseconds.

Screen Shot 2019-07-19 at 3.06.39 PM.png