CTS Border Enforcement Inline Tagging/SXP

[Benjamin-A]

Hi All,

I have a Question on how the CTS Enforcement on the in depth works.

Szenario:

Border is getting IP-SGT mappings from SXP.

Following are the SGT Mappings:

SGT 30 "Intranet": 10.0.0.0/8 (Border will learn this via ISE SXP)

SGT 20 "Server":  10.2.2.0/24 (Border will learn this via ISE SXP)

SGT 10 "Client" will be dynamic applied by ISE to Clients in IP_Pool_10.1.1.0/24 via Authorization Rules

If I enter the command "show ip cef internal" for Server1 (10.2.2.2) I will see the outgoing SGT 20 and the Uplink as outgoing interface (what I expect). I also see Client3 (10.3.3.3) with SGT 30 on the uplink (what I walso expected).

But if I enter the same command for Client1 (10.1.1.1) it will show me SGT 30 "Intranet" instead of SGT 10 "Client" (Result: encapsulation to VXLAN, Lisp Instance ID, RLOC and outgoint Interface are correct).

Per Policy communication between SGT 20 and SGT 30 is not permitted. SGT 10 to SGT 20 is permited and the communication just works fine.

The only thing what I feel strange about is that CEF says for Client1 that it is SGT 30. And that the cts enforcement is not kicking in when forwarding the packet to the edge, because the border would have Source SGT and Destination SGT. But instead it encapsulates the Packet in VXLAN and forwards it to the Edge which then correctly permits it. 

Based on BRKSEc 2203 I understand that IP_ARP, LOCAL, and INTERNAL  IP-SGT Bindings have a higher priority as SXP Bindings. Am I right that there is a IP_ARP Binding, on the Border? Or will there no Enforcement on the Border happening for Server to Client1 even if the Border know both bindings (as one would be wrong because it is SGT 30 and not 10 as I would expect).

[faylee]

Hi Benjamin,

Yes, the behavior that you’re observing is correct behavior for a SDA environment.

The precedence information that you’re referring from Cisco Live to applies to non-fabric.

HTH,

Fay-Ann

[Benjamin-A]

Hi Fay-Ann,

thanks for your reply. I realy want to understand what's happening so may I ask again.

What is in depth the procedure that is happening here? Or may be you know a reference where I can have a look?

Why is the border not enforcing in this szenario? In one other Cisco Live talk the rule was "when ever the Source and Destination SGT is known there will be a enforcement". But rather the Border is taking the packet encapsulating it in VXLAN with Source of 20 and forwarding it to the edge whre the edge will have a SGACL lookup. Am I right that the border is not storing the Mapping between SGT 10 and 10.1.1.1? Shouldn't be the CEF table reliable?

Thanks for clarification (:.

[Aninda Chatterjee]

---

@Benjamin-A wrote:

Hi Fay-Ann,

thanks for your reply. I realy want to understand what's happening so may I ask again.

What is in depth the procedure that is happening here? Or may be you know a reference where I can have a look?

Why is the border not enforcing in this szenario? In one other Cisco Live talk the rule was "when ever the Source and Destination SGT is known there will be a enforcement". But rather the Border is taking the packet encapsulating it in VXLAN with Source of 20 and forwarding it to the edge whre the edge will have a SGACL lookup. Am I right that the border is not storing the Mapping between SGT 10 and 10.1.1.1? Shouldn't be the CEF table reliable?

Thanks for clarification (:.

---

The Border will not be aware of SGT 10 in this case - since it is dynamically pushed with a host authentication, this SGT will be pushed to the Edge which is acting as a NAD. Additionally, enforcement is always on the destination, so any policy which includes SGT 10 as a destination SGT will in turn be downloaded to this Edge.

I've documented the general principles and some "behind the scenes" of this works here - https://www.theasciiconstruct.com/ specifically in the following blogs:

https://www.theasciiconstruct.com/post/cisco-sda-and-security-part-ii-micro-segmentation-in-sda <= intro to SGTs and micro-segmentation

https://www.theasciiconstruct.com/post/cisco-sda-and-security-part-iii-micro-segmentation-in-sda-continued <= how SGACLs are pushed to a device and data flow

https://www.theasciiconstruct.com/post/cisco-sda-and-security-part-iv-micro-segmentation-in-sda-continued-some-more <= SXP tunnels

[Benjamin-A]

Hi Aninda Chatterjee,
nice work on your blog.
Well I know how cts for general is working. I know that it is working. I can verify it in our Customers SD-Access Deployments that the border is not enforcing - but the edge will. And I already got it, that the enforcement is outgoing and that the border won't do it for general because there will be no mapping as I stated in my quetions and previous posts.
My Problem of understanding is:
The border is having a mapping via SXP for 10.1.1.1 - even it is not correct - as I can verify via the ip cef table and the IP-SGT mapping table. CTS Enforcement is enabled. And CTS enforcement is also enabled on the interfaces. But still the border is not enforcing and is sending the traffic to the FE which will have the correct mapping and then can enforce correctly.
I want to know, why is the border not enforcing the wrong mapping which it has via SXP. Is there a priority? Is there a rule which says sth like "I will do enforcement for Ethernet Traffic outgoing traffic but I will not do enforcment - even if I have a SRC SGT and DST SGT - for traffic which I will encapsulate in VXLAN"?

Well I think I will call out to our Cisco representative if we have on in our office the next time and ask him. Maybe my question is a bit odd and/or I missing a key fact (:

If I got a satisfactory reply I will post it as awsner. 

Thanks guys.

[prasannakumarpalakollu78931]

cisco trust sec security architecture builds secure networks by establishing domains of trusted network devices.

CTS uses the device and user credentials acquired during authentication for classifying the packets by security groups as they enter the network.

[jeaves@cisco.com]

Hi,

answering this assuming you have separate Borders and Edges (not Fabric in a Box) as it wasn't explicit.

When ISE assigns SGT 10 to the client, the mapping is assigned on the Edge, not on the Border. So, the Border see's the route but doesn't know there's an SGT mapping for it. SGT mappings are not carried in control-plane routing updates, they are only carried in specific data-plane mechanisms or SXP.

So, what you see is correct, the Border forwards the packet to the Edge and will propagate the source SGT in the VXLAN header if present, and enforcement will occur on the Edge.

[Benjamin-A]

Ok, thanks. Yes Border and Edge are two different devices.
But what I really don't get - sorry for that ^^' - is the following:
The Border "knows" a mapping via SXP as Intranet is 10.0.0.0/8 SGT 30. Sure it is not the correct mapping because the right mapping would be 10.1.1.1 > SGT 10 as the Edge knows it via ISE Authroization Profile.
The border even shows the "wrong" mapping in its CEF table:
Client1 10.1.1.1 > SGT 30 > Interface facing to edge. And it still does not enforce.
So the border "knows" the destination" - not the correct one - but it knows a destination SGT via SXP and still does not enforce ^^ (cts role-based enforcement is configured on the downlink facing to the edge).

So lets say we go in the other direction: Client 10.1.1.1 sends a packet to Client3 10.3.3.3. Regard to its cef table it will have a destination SGT of 30 for Client3 which is hosted behind a fusion router through IP Transit. And in this case it will do an enforcement as it knows both SRC and DST SGT.

I know that what you say is the correct behavior and I see is in production enviroments happening in the correct way. I just want to understand its technically, why in one direction is an enforcement and in the other it is not. Even commands like show ip cef internal will show me such mappings.

[jeaves@cisco.com · Accepted solution]

Hi,

The Border "knows" the right mapping as 10.0.0.0/8 SGT 30 as you're sending it via SXP.

So, it correctly shows Client1 10.1.1.1 > SGT 30 > Interface facing to edge.

If you're trying to enforce Southbound on the Border, it will not, ever. That function is disabled when traffic is destined to the VXLAN interface. That holds true for Border or Edge.

So, Southbound, traffic will be enforced on the Edge. Northbound, traffic will be enforced on the Border (if the Border is configured to enforce and it understands the destination group mapping).

Hopefully that answers your questions.

[jeaves@cisco.com]

I'll preempt a follow up question that may be posted in response to my post.

How come we recommend disabling enforcement on the Edge uplinks then if Default-deny is deployed?

Well, that is a recommendation due to client broadcast traffic (DHCP for example) being subjected to default policy as it will not be resolved to a specific SGT/DGT pair. So, with a default-deny, you would need to explicitly allow the DHCP traffic within an SGACL. The best practice though is to disable enforcement on all the uplinks instead, and it's actually best practice to disable enforcement on switch to switch links anyway.

[Benjamin-A]

Hi,
So I will conclude from this discussion:
If there is Packet destined southbound the border will have a look at its FIB. If there is a encapsulation of VXLAN (or some special other cases) it will basically ignore the listed DST SGT as the internal CTS process is defined to not have a look at the policy table.
So when troubleshooting the Fabric I also have to ignore such entrys in show commands and follow the concept idea.

And thanks for your preemting reply. Because mainly we use such whitelisting concepts.

i thought maby there is a documentation simmilar to the Firepower or IPsec dokumentation for the internal packet flow and internal desicion making inside if using CTS (like whats happening when...).
But thanks, I think I can live with this confirmation ;)