What Tcam number mean on Nexus9000?

[mhiyoshi]

Dear all,
I have been investigating the following output. There are several types of Tcam however I think Tcam number means just the slice number right?

https://www.cisco.com/c/en/us/support/docs/switches/nexus-9000-series-switches/119032-nexus9k-tcam-00.html 

Understand how to Carve TCAM Space - Nexus 9000

Slice 1 (512):    IPv4 PACL
   Slice 2 (512):    VACL
   Slice 3 (512):    RACL
   Slice 4 (512):    Redirect
   Slice 5 (256):    Port QOS
   Slice 6 (256):    Port QOS
   Slice 7 (256):    SPAN
   Slice 8 (256):    VPC Convergence
   Slice 9 (256):    Ingress CoPP
   Slice 10 (256):  Ingress CoPP
   Slice 11 (256):  Ingress System
   Slice 12 (256):  Ingress System

N9K-C9336C-FX2# sh hardware capacity | section label
L4 op labels, Tcam 0                    0       30      0.00
L4 op labels, Tcam 1                    0       62      0.00
L4 op labels, Tcam 7                    0       1023    0.00
L4 op labels, Tcam 8                    0       254     0.00
L4 op labels, Tcam 11                   0       1023    0.00
L4 op labels, Tcam 12                   0       1023    0.00
L4 op labels, Tcam 13                   0       1023    0.00
L4 op labels, Tcam 14                   0       1023    0.00
L4 op labels, Tcam 16                   0       1023    0.00
L4 op labels, Tcam 17                   0       1023    0.00
L4 op labels, Tcam 0                    0       30      0.00
L4 op labels, Tcam 1                    0       62      0.00
L4 op labels, Tcam 7                    0       1023    0.00
L4 op labels, Tcam 8                    0       254     0.00
L4 op labels, Tcam 11                   0       1023    0.00
L4 op labels, Tcam 12                   0       1023    0.00
L4 op labels, Tcam 13                   0       1023    0.00
L4 op labels, Tcam 14                   0       1023    0.00
L4 op labels, Tcam 16                   0       1023    0.00
L4 op labels, Tcam 17                   0       1023    0.00

I appreciate if you can let me know any related information.

Best Regards,

Masanobu Hiyoshi

[anpetit]

Hello @mhiyoshi , 

Hope you are doing well. 

If I'm understanding correctly, you mean slice 1 = TCAM 1 ?

TCAM number is not linked to slices. We have 12 slices but we can observe TCAM 17 for instance. 

L4 op labels, Tcam 17

Slice - A unit of memory allocation. Slices can be of size 256 or of size 512, measured in bytes.
TCAM - Ternary Content Addressable Memory. This is the space in hardware where access-lists (ACLs) are stored. This is a specialized piece of memory that stores complex tabular data and supports very rapid parallel lookups.

In TCAM you will allocate label depending on you configuration, ACL, QoS on interface for example. 

We see that slice 5 and 6 are for QoS but on this N9K-C9336C-FX2 platform, when you will allocate QoS on a port, you will use 1 label from "L4 op Labels, TCAM 0". 

Hope it answers you request. 

Have a nice day.
Antoine

[mhiyoshi]

Hi @anpetit 

Thank you for your information. Please let me make sure below.

>If I'm understanding correctly, you mean slice 1 = TCAM 1 ?
>TCAM number is not linked to slices. We have 12 slices but we can observe TCAM 17 for instance. 

TCAM number is not linked to slice number right? For example if I check below then Ingress RACL TID is 3 however L4 op labels is Tcam 1. 

N9K-C9336C-FX2# show system internal access-list globals
<omit>
                    Region          TID     Base     Size     Width
--------------------------------------------------------------------------------
                Ingress PACL          1        0        0         1
                Ingress RACL          3        0     2304         1

N9K-C9336C-FX2# show hardware access-list resource utilization | i "L4 op labels, Tcam 1 "
L4 op labels, Tcam 1                    0       62      0.00
L4 op labels, Tcam 1                    0       62      0.00

I just want to know this number (TID and Tcam num#) is indivisual output (no relationship). that is why this is not Linked.

Best Regard,

Masanobu Hiyoshi

[anpetit · Accepted solution]

Hi @mhiyoshi , 

Yes, you are completely right. TCAM number is not linked to slice number as highlighted in my first reply. 

Please mark this post as resolved if your question is resolved. 

Have a nice day. 

Kind Regards,

Antoine

[mhiyoshi]

Hi @anpetit ,

Thank you very much! I have accepted this as solution.

[mlobue]

Hello @mhiyoshi,

One of my teammates recently created a walk-through video on Nexus 9000 TCAM carving. I would encourage you to watch it to see if it helps fill any knowledge gaps regarding TCAM carving. You can find the link here: https://www.youtube.com/watch?v=ISJ4C8ISKCw

[mhiyoshi]

Hi @mlobue ,

Thank you very much. this is good VoD! I have just quickly checked it but when I have a problem then later I'll watch again.

Best Regards,

Masanobu Hiyoshi

[Darian O'Dirling - TCE]

Hi @mhiyoshi,

By default TCAM space is fully allocated by default. It can definitely be challenging to find where to get the space from.

There is different sizes single slices of 256 or double wide slices of 512. Hopefully this image can better visualizes where that memory is from.

[mhiyoshi]

Hi @Darian O'Dirling - TCE ,

Thank you for your information and png. By modifying TCAM carving as double wide slice of 512. I think unique RACL might be increased however the following URL is that Only 62 unique ACLs can be configured per slice of asic. 

Cisco Nexus 9000 Series NX-OS Verified Scalability Guide, Release 9.3(3)

As for this info, only 62 unique RACLs can be supported. is that possible to expand by TCAM carving like this?

Best Regards,

Masanobu Hiyoshi