Contracts and TCAM utilization

[jgesualdi]

I'm trying some different configurations in my lab with contracts and how to efficiently use TCAM. In my lab I'm using VZany and preferred groups as much as possible. Initially my policy is allow all. My setup is as follows

Tenant PROD

VRF PROD

60  BD'S and EPG's in this VRF.

all epg's are preferred group

Tenant QA

VRF QA

35  BD'S and EPG's in this VRF.

all epg's are preferred group

With this setup all epg's in their respective vrf can communicate but not between VRF's.

I modified my config to allow inter vrf communications between all epg's. I created a contract called "inter-vrf-allow-all, this contract has 3 entries. All my production EPG's are providing this contract  and on my QA vrf I'm consuming it via VZany. This works and uses  aprox 1300 TCAM entries.

I changed my config again by removing from the qa vrf the "inter-vrf-allow-all" contract consumed by Vzany.  Now I also configured every qa epg to consume this "inter-vrf-allow-all" contract. Things still work, everyone is communicating but my TCAM usage shoots up dramatically. Goes from 1300 to 6000.

My questions is why? I was expecting the TCAM usage to be the same since it is essentially the same policy. What's the difference if I'm consuming "inter-vrf-allow-all" contract via vzany or individually via each qa epg?

Thanks

[EduardR · Accepted solution]

Hello,

We got some issue with the TCAM calculations in our fabric aswell. The reason (at least what I understood from our vendor support) is that those numbers depends in the quantity of objects that need to be created in each approach, i.e:

When you have 60 EPG with contract with a VzAny, it creates one objetc for each relationship, it means (in a very simplistic way) that we got 60 objects... But when you use the contract with each EPG, it creates one object for each relationship between the EPG... so, if you have all the EPG consuming and providing the contract, you got 60*60 objects.

Now, this is just an example, we do not got the details of the calculations, but them told us that it includes the number of subjects and the direction of the contract. In our fabric we got like 32 thousand TCAM entries, and with the VzAny it dropped only to a pair of thousand.

[gmonroy · Accepted solution]

jgesualdi, 

    vzAny is a special classifier underneath the VRF that allows for classification of all EPGs within that VRF. What this means, is that it can greatly cut down on rules that would otherwise cause a "spider web of programming" if you were to use the same Contracts across multiple EPGs.

At a high level, each EPG has its own Pctag. If you put contracts from EPG to EPG, the rule is programmed form source pctag to dest pctag. If you multiple that by 60x60 EPGs all using the same contract (cons and prov), it is exponential.

If you then bring vzAny into the equation as a replacement, vzAny has a special classifier that basically translates to "all EPGs/L3EPGs within this VRF". This means that if you had some contract that you provided on vzAny, but then consumed on an individual EPG, you would see a rule entry along the lines of:

dest vzAny, src EPG pctag.

So if you then multiplied the number of consumers by 60, you would then only end up with 60 of those above entries, as opposed to the 60x60 spider web that would have been generated by doing all cons/prov contract definitions as EPG to EPG.

Supporting documentation on vzANy:

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/kb/b_KB_Use_vzAny_to_AutomaticallyApplyCommunicationRules_toEPGs.html

Supporting documentation to verify contract rule programming:

https://www.cisco.com/c/en/us/support/docs/cloud-systems-management/application-policy-infrastructure-controller-apic/119023-technote-apic-00.html

-Gabriel