TrustSec deployment in large environment

[john5]

Hi,

I have a large implementation of TrustSec micro-segmentation using ISE in a distributed deployment with 2 ISEs for PAN and 2 for MnT and centralized PSNs in multiple regions which will cover alot of branches.

I still need to understand more about the enforcement of TrustSec

Q1:

if there an employee in BR1 and another one in BR2 and my policy says that employee tag can't talk to another employee tag "regardless of their IPs".

so my question if we assume that we let the core switches at every BR do the enforcement , how does this switch know that the destination tag is for employee , is it will be IP-SGT mapping enforced by ISE ? if yes what if I have a large number of branches "more that 1000" should i send all employees IP-SGT mapping to each core to enforce the policy ? is there any other solution ?

Q2:

Is there any tool to calculate the WAN traffic needed between PSNs and the switches in the branches ?

Q3:

Does anyone know if Viptela SD-WAN solution will allow propagation for SGT from branch to another one without any problem ? "I know it will but i want to make sure if someone face that scenario"

Thanks in advance.

[jeaves@cisco.com · Accepted solution]

Hi,

Q1: in order to enforce, an enforcement point must be told to enforce (cts role-based enforcement), it must have an IP:SGT mapping for the source, must have an IP:SGT mapping for the destination and it must have a policy to enforce from source to destination.

So, your question should relate to the source as well i.e. how does the core learn of the destination tag and the source tag in order to enforce.

Firstly, let's talk about the way the technology was designed. It is designed for egress enforcement, for good reason. The switch at egress needs to protect the mappings/groups that it knows about (directly connected). It already knows the destination mappings (as they are directly connected using static or dynamic classification) so it only needs to download policy for those directly connected groups. This is how we scale, policy is only downloaded for mappings that are known about and only when needed.

So, in your case it would be best to enforce at egress on the access switches because then all you would need to do is propagate the source SGT from one switch to another. This could be achieved using inline tagging for example where the SGT is carried in the L2 frame of every packet originated from the source.

If you want to enforce on the core for one reason or another then you still need to propagate the source SGT from BR1 to the core. This could be achieved using inline tagging, SXP from BR1 or SXP from ISE. You also need to propagate the destination SGT from BR2 to the core and this could be achieved using SXP from BR2 or SXP from ISE. You're right, sending mappings from ISE will not scale for 1000 branches so the recommendation would be to 'template' the configuration for SXP on each branch (to make config rollout easier), and send mappings from the access switches to the branch core.

Q2: There is a bandwidth calculator linked below but this doesn't actually include Bandwidth required for general RADIUS auth and accounting traffic. RADIUS traffic is generally less significant than PSN-PSN traffic and the actual requirement is highly contingent on multiple factors including total active endpoints, reauth intervals, and the authentication protocols used. To read the general message about bandwidth, see here: ISE Latency and Bandwidth Calculators

Q3: Yes, SD-WAN will transport the SGT but today you'll need to use DMVPN and insert the SGT into the DMVPN header.

The teams are looking at ways to integrate more fully but today you need to use DMVPN.

Regards, Jonothan.

[john5]

Thanks a lot Jonothan.

it's clear now except for the point ( 'template' the configuration for SXP on each branch ) , I need an example or a document that illustrate this point "if available"

[jeaves@cisco.com]

I just meant that SXP will need to be configured on ~1000 branches, from access switches to core. But apart from IP addresses etc the config will be the same on each branch. So, you could create some sort of script that substitutes IP addresses etc into the config needed to add SXP and use that to help in provisioning:

cts sxp enable

cts sxp default source-ip xx

cts sxp default password xx

cts sxp connection peer xx source xx password default mode local speaker

[Damien Miller]

Just going to drop a note here.  I have had a very frustrating time trying to scale SXP even with dedicated ASR hardware "reflectors".  I'm not sure how far along you are in the planning your enforcement strategy but i'm certainly open to discussing some real world scenarios with you. 

[john5]

Thanks a lot Damien, actually I'm struggling right now with my 1000 branches.

I was thinking that I can scale SXP using ISE as a listener and speaker to other branches but I was wrong after I found that ISE can only have maximum 200 SXP connection with v2.1 in release 6.3 system bulletin.

so please if you have solutions it will be appreciated to share with me

[kthumula]

John, Make sure to enable path length filtering on the routers if you are planning to go with the SXP reflectors route. I would recommend configuring the maximum paths as 2.

cts sxp limit export peer-sequence-nodes 2

cts sxp limit import peer-sequence-nodes 2

[kthumula]

Damien, I totally understand your frustration. I hope things are fine after the path length filtering you enabled on those SXP reflectors. Let me know if you have any other scaling issues with SXP.

[Damien Miller]

Lets discuss offline if you're around at Cisco Live, we have had some developments since I talked to you at the security sevt. 

[kthumula]

Sure Damien. We can discuss offline. I will be there at Cisco Live.

[khook@cisco.com]

Hey Karthik and Damien, please include me in the mtg/discussion @ CiscoLive.

Thanks,

Ken

[ngriffin@bankozarks.com]

I am in the process of designing a similar solution with a mix of nexus 7700/F3/8.1.1 and Cisco catalyst 9300's at the access layer. There will be some basic enforcement from the branch to the data center, but largely the enforcement will be between devices at the location and between locations with enforcement being on the 9300. I'm looking for some of details regarding scaling you are mentioning. I'm considering a reflector approach, with the reflection being done on either a couple nexus 7700's or a couple ASR 1002x's. Thanks

[Damien Miller]

How many sites and SGT mappings are you looking at? It could be beneficial to leverage ISE as the SXP reflector (or simply the speaker), rather than dedicating some expensive hardware to it. We managed ~130 bidirectional connections on dedicated ASR's with approximately 40k IP-SGT mappings. The CPU's on both ASR's were 90%+ at this point. This was the point where we stopped adding SXP connections, enforcement across the WAN from that point forward required an overlay and inline tagging. SXP across the WAN was a temporary measure here.

I would try to stay away from designing anything at scale with bidirectional SXP connections as they are extremely resource intensive. I would actually avoid SXP in any form at scale since it just isn't a great protocol. It works great in limited deployment, point to point such as WLC to core, or access to access spanning a wireless bridge. When you start adding connections and mappings together to cross the WAN it just isn't efficient.

I would strongly urge an overlay and inline tagging across the WAN. That's not to say SXP won't work for you, but it adds a scaling component. I would start that discussion with endpoint counts and and number sites/connections required.

[ngriffin@bankozarks.com]

Therein lies the rub. I agree with you. +/- 260 sites. We will have somewhere in the neighborhood of 400-500 9300 remote edge switches participating. We are also actively designing Cisco SD-wan so we will be able to handle the tagging natively long-term. The problem is the short term segmentation deadline. I’ve done my homework on SXP and ISE and the amount of compute necessary to get to the max of 800 sessions is massive and not doable. What am I planning doing is seeding the ip to sgt mappings from ISE to either a set of nexus switches are a set of ASR routers, and then I have listen only sxp sessions to the remote 9300 switches from the n7k/a1k. The nexus switches learn about the dynamic sessions from auth policies via ISE/SXP, and then the 9300’s learn the static mappings from the SXP reflectors. I’m leaning towards the A1K for SXP reflection since it seems to have the “cts sxp limit” commands that are recommended.

[Damien Miller]

You could simplify this by having ISE be the central speaker of all IP-SGT mappings and no return SXP connections. ISE will already be the source of truth for IP-SGT mappings since it knows them/learns them during authentication, you do not need to communicate them back to ISE.

Building SXP connections from reflectors to the access layer isn't a requirement if you are willing to utilize inline tagging in the LAN. Take this as an example of what may help it scale better.

You send these IP-SGT mappings to an ASR 1k with a speaker connection from ISE, then use the ASR 1k to speak them down to the WAN edge at sites. To scale this out a bit better, you could create a three tier topology, ISE speaker > ASR speaker > ISR 4k listener. Then within the LAN you leverage cts manual to perform inline tagging throughout the site. The ISR 4k utilizing the SXP IP-SGT mappings it learns, will retag any traffic egressing it's LAN interfaces. The core/distribution/access would be configured for inline tagging and sgacl enforcement. It doesn't have to be an ISR 4k either, they are just a reasonably scalable platform when it comes to mappings, this would work with legacy ISR G2 platforms.

This assumes you have inline tagging capable hardware from the remote site WAN edge down to the access layer.