New to TCAM reallocation for Nexus n3k

[Jeffrey_233]

Hi All

I've never done a TCAM reallocation.
I'm a bit stuck on what I should be reallocating to where I need it allocated to.
And trying to add more acl rules, I'm getting the tcam is full.

I've got a pair of n3k in a VPC.
I use a few object-groups, copp-system and primarily ip access-lists.
Internal network configuration only.
They are not external facing I have a firewall for all the NATing.

I understand I need to take from somewhere and apply to where I need to.
I would think that i would need to allocate more vacl but according to my utilization report I'm not even using it and it's the ingress SUP that's low.
Could someone help and explain, what I can take from so I can have more ip access-lists?

# show hardware profile tcam region 
         sup size =   48 
        vacl size =  640 
       ifacl size =  400 
         qos size =  192 
        span size =    0 
        racl size = 1536 
      e-racl size =  256 
      e-vacl size =  640 
      qoslbl size =    0 
        ipsg size =    0 
      arpacl size =    0 
   ipv6-racl size =    0 
 ipv6-e-racl size =    0 
    ipv6-sup size =    0 
    ipv6-qos size =    0 
       e-qos size =   64 
         pbr size =    0 
    ipv6-pbr size =    0 
  e-ipv6-qos size =    0 
   e-mac-qos size =    0 
  e-qos-lite size =    0 
 mcast-bidir size =    0 
   ipv6-span size =    0 
ipv6-span-l2 size =    0 
         nat size =  256 
       rbacl size =    0 
        copp size =   64 
         fhs size =    0 
     Unknown size =    0 
arp-storm-acl size =    0 
         svi size =    0 

# show hardware access-list resource utilization 

slot  1
=======



INSTANCE 0x0
-------------


         ACL Hardware Resource Utilization (Mod 1)
         ----------------------------------------------------------
                                        Used    Free    Percent 
                                                        Utilization
------------------------------------------------------------------- 
Ingress IFACL                           0       400     0.00   
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00
Ingress RACL                            522     1014    33.98  
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00
Ingress VACL                            0       640     0.00   
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00
Ingress SUP                             44      4       91.66  
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00
Ingress QOS IPV4                        2       190     1.04   
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00
Egress Racl                             0       256     0.00   
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00
Egress QoS                              49      15      76.56  
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00
Ingress COPP - Egr SUP                  49      15      76.56  
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00
Egress VACL                             0       640     0.00   
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00
Ingress NAT                             0       256     0.00   
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00
                                        0               0.00

LOU                                     2       29      6.45   
Both LOU Operands                       2       
Single LOU Operands                     0       
LOU L4 src port:                        0
LOU L4 dst port:                        2
LOU L3 packet len:                      0
LOU IP tos:                             0
LOU IP dscp:                            0
LOU ip precedence:                      0
LOU ip TTL:                             0
TCP Flags                               0       16      0.00   
L4 op labels, Tcam 0                    0       63      0.00   
L4 op labels, Tcam 2                    8       55      12.69  
L4 op labels, Tcam 6                    1       62      1.58   

Ingress Dest info table                 0       512     0.00   
Egress Dest info table                  0       512     0.00   

[Jeffrey_233]

Basically what I'm trying to do is vlan out some workstations from an existing vlan interface.
But I'm now stuck where I try apply the new vlan interface with the new ip access-group.
I get "ERROR: Sufficient free ACL stats entries are not available in TCAM"
And I need to create one more vlan interface with an ip access-group.

[MHM Cisco World]

Friend as I mention before 

You dont have free room

You need to select one of process (as I see you can use NAT or racl) and reduce it room' this will give you some free room to add new ifacl.

MHM

[Jeffrey_233]

Hi @MHM Cisco World 
I understand i need to reallocate.
I just don't know where I need to allocate to.

Are you saying that if i allocate to ifacl this will give me more room to apply ip access-groups to vlan interfaces,
and then assign acl to ip access-lists?

[MHM Cisco World]

https://www.cisco.com/c/en/us/support/docs/switches/nexus-9000-series-switches/119032-nexus9k-tcam-00.html

This guide for tcan carve.

Now what you see is many types of acl, this guide give some details about each type of acl appear in tcam. 

There is port acl there router acl and there is vacl.

So what type of acl you need to give it some room?

Also I see in your tcam there is room for NAT, do you use NAT?

If no then reduce the NAT tcam and increase the racl/ifacl/vacl.

MHM

[Jeffrey_233]

I'm still a bit confused here thought.
Cause my print out shows that my Ingress SUP is being utilised and not my racl/ifacl/vacl
But i need to apply more vacl to be able to add a ip access-groups to a new vlan interface?

[MHM Cisco World]

If you add acl under SVI of vlan that not meaning it vacl it is racl ( i know the name confuse but what can we do with cisco terminology)

I see the racl is aroubd 33% so I think it not issue of acl room' and as you mention it issue of SUP room

So decrease NAT to zero and increase SUP by more 256 room and check.

MHM

[MHM Cisco World]

Just wait dont modify tcam' let me check one point here.

Thanks 

MHM

[MHM Cisco World]

Share exactly log message appear when you add racl.

It can be bug

MHM

[Jeffrey_233]

What logs do you want or just the error message?

[Jeffrey_233]

@MHM Cisco World 
"ERROR: Sufficient free ACL stats entries are not available in TCAM"

[MHM Cisco World]

Hi friend 

Please share exact the log you receive about tcam is full. 

MHM

[Jeffrey_233]

Just trying to figure out how to get you that log entry.
currently the only way i can see is if i push it to a syslog server.

[MHM Cisco World]

https://bst.cisco.com/quickview/bug/CSCvo89681

I afraid your issue is same as above bug' and even if we add more room to tcam rcal the issue will not solve' and you need to upgrade.

Check bug' check your N3K and your NS-XO ver. 

Good luck freind 

MHM

[Jeffrey_233]

Thanks for that.
On version 9.3.8
Will drop in a support case to cisco.