Nexus9000 - Maximum Label Sizes Supported for ACL Types

[mhiyoshi]

Dear all, 

I have been investigating the relationship between ACL and unique label.
According to the following URL, only 62 unique ACL can be configured however it can be configured more than 100 name ACLs.
How can I understand this situation?

https://www.cisco.com/c/ja_jp/td/docs/switches/datacenter/nexus9000/sw/93x/security/configuration/guide/b-cisco-nexus-9000-nx-os-security-configuration-guide-93x/b-cisco-nexus-9000-nx-os-security-configuration-guide-93x_chapter_01010.html#id_72594

Maximum Label Sizes Supported for ACL Types
Only 62 unique ACLs (RACL/PBR/VACL/ L3-VLAN QoS/L3-VLAN SPAN ACL) can be configured. Each ACL takes one label. If the same ACL is configured on multiple interfaces, the same label is shared.
If each ACL has unique entries, the ACL labels are not shared, and the label limit is 62.

N9K-C9336C-FX2# sh hardware capacity | section label
L4 op labels, Tcam 0                    0       30      0.00
L4 op labels, Tcam 1                    0       62      0.00

N9K-C9336C-FX2# sh ip access-lists TEST1 summary

IPV4 ACL TEST1
        Total ACEs Configured: 1
        Configured on interfaces:
                Vlan1 - ingress (Router ACL)
        Active on interfaces:
<omit>
N9K-C9336C-FX2# sh ip access-lists TEST100 summary

IPV4 ACL TEST100
        Total ACEs Configured: 1
        Configured on interfaces:
                Vlan100 - ingress (Router ACL)
        Active on interfaces:
N9K-C9336C-FX2#

Regards,
Masanobu Hiyoshi

[anpetit]

Hello @mhiyoshi , 

Hope you are doing well. 

Regarding your question on ACL label allocation and label sharing.
Label sharing is possible if IPv4 and IPv6 ACLs are applied on one or more interfaces but these ACLs need to be used in combination. 
Please find below an example:

We are applying testv4 and testv6 on interface 1/1 and 1/2 - 1 label is consumed 

interface Ethernet1/1
 ip access-group testv4 in
 ipv6 traffic-filter testv6 in
 no shutdown

interface Ethernet1/2
 ip access-group testv4 in
 ipv6 traffic-filter testv6 in
 no shutdown

N9K# show hardware access-list resource utilization | i "L4 op labels, Tcam 1 "
L4 op labels, Tcam 1                    0       62      0.00
L4 op labels, Tcam 1                    1       61      1.61    >>>> 1 label is consumed.

Now if either testv4 or testv6 are used individually on any other interface, another label will be consumed. 

In the example below, we are configuring interface 1/1 with testv4 and testv6 and we are configuring interface 1/2 with only testv4. We observe that 2 labels are consumed. 

interface Ethernet1/1
 ip access-group testv4 in
 ipv6 traffic-filter testv6 in
 no shutdown

interface Ethernet1/2
 ip access-group testv4 in
 no shutdown

N9K#   show hardware access-list resource utilization | i "L4 op labels, Tcam 1 "
L4 op labels, Tcam 1                    0       62      0.00
L4 op labels, Tcam 1                    2       60      3.22   >>>> 2 labels are consumed.

Hope it answers your request. 

Kind Regards,

Antoine

[mhiyoshi]

Hi @anpetit ,

Thank you for your reply and verification output. I think this label sharing is basically physical interface basis right?
Because if I configure below, then the utilization ratio does not increase.

interface Ethernet1/5
  switchport
  switchport mode trunk
  no shutdown
!
interface Vlan2
  no shutdown
  ip access-group test4 in
!
interface Vlan3
  no shutdown
  ip access-group test4 in
!
N9K-C9336C-FX2# show hardware access-list resource utilization | i "L4 op labels, Tcam 1 "
L4 op labels, Tcam 1                    0       62      0.00
L4 op labels, Tcam 1                    0       62      0.00

Actually how many named ACL can be configurable with interface vlan?
is that possible with maximum vlan range and what's happend if the label is using more than 62 ?

N9K-C9336C-FX2(config)# vlan ?
  <1-3967>       VLAN ID 1-4094 or range(s): 1-5, 10 or 2-5,7-19

[anpetit]

Hi @mhiyoshi , 

Yes, you will consume label only when you will attached ACL to physical interface. It will work the same way with QoS policy. 

Note: TCAM resources are not shared when a VACL is applied to multiple VLANs.

When you will use more than 62 labels, a syslog error message will be display in "show logging logfile" about TCAM memory size is full.
Example of syslog when you reach scale limit: 

%ACLQOS-SLOT1-2-ACLQOS_OOTR: Tcam resource exhausted:

Therefore, reaching when reaching the scale limit, you will not be able to apply the policy on the interface. 

Hope it answers your request. Please note this post as resolved if all your doubt are cleared. 
Have a nice day. 

Kind Regards,

Antoine

[mhiyoshi]

Hi @anpetit ,

Thank you for your information. Now I  can accept it as solution.

The following is just information. label for INGR-RACL is 62 by default however if I confgure below it can be possible to expand the maximum value. *config save & reload is necessary. 

N9K-C9336C-FX2# show hardware access-list resource utilization | i "L4 op labels, Tcam 1 "
L4 op labels, Tcam 1                    0       62      0.00
L4 op labels, Tcam 1                    0       62      0.00

N9K-C9336C-FX2(config)# hardware access-list tcam label ing-racl ?
  9  Size in bits for BD-Labels allocated to Ingress-RACL region

N9K-C9336C-FX2(config)# hardware access-list tcam label ing-racl 9
Warning: This knob increases the INGR-RACL BD-Label size to 512. Please save config and reload the system for the configuration to take effect
N9K-C9336C-FX2(config)#

Besides, if there is any TCAM carving recommendation, it will be very helpful to me.

Thank you very much. and best regards,

Masanobu Hiyoshi

[anpetit]

Hello @mhiyoshi , 

Regarding TCAM carving recommendation, there is no specific recommendation. It will always need a configuration save and reload of the switch to apply the carving.

Depending on the role of the switch, TCAM carving might be needed for specific use case. However, most of the time, the default carving is used. 
You can find more information in the link below if you want. 

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/93x/security/configuration/guide/b-cisco-nexus-9000-nx-os-security-configuration-guide-93x/b-cisco-nexus-9000-nx-os-security-configuration-guide-93x_chapter_01010.html#id_72594

Hope I've answered all your questions.

Please mark this post as resolved if all your doubt are cleared. 

Have a nice day. 

Kind Regards,

Antoine