cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2112
Views
25
Helpful
9
Replies

N3K-3064PQ ACLQOS-SLOT1-4-ACLQOS_OVER_THRESHOLD error

blackmetal
Level 1
Level 1

Hello,

i have a N3k-c3064pq-10gx and sometimes i see the following error on my logs :

2020 Oct 23 16:20:51 HOSTNAME %ACLQOS-SLOT1-4-ACLQOS_OVER_THRESHOLD: Inst 0 Tcam 8 Bank 1's (NEW MAX TCAM COMPAT REGION) usage has
reached its threshold
2020 Oct 23 16:20:51 HOSTNAME %ACLQOS-SLOT1-4-ACLQOS_OVER_THRESHOLD: Inst 0 Tcam 8 Bank 1's ((null)) usage has reached its threshol
d
2020 Oct 23 16:20:51 HOSTNAME %ACLQOS-SLOT1-4-ACLQOS_OVER_THRESHOLD: Inst 0 Tcam 8 Bank 1's (NEW MAX TCAM COMPAT REGION) usage has
reached its threshold
2020 Oct 23 16:20:51 HOSTNAME %ACLQOS-SLOT1-4-ACLQOS_OVER_THRESHOLD: Inst 0 Tcam 8 Bank 1's ((null)) usage has reached its threshol
d

and here is my  show system internal access-list resource utilization out put :

slot  1
=======



INSTANCE 0x0
-------------


         ACL Hardware Resource Utilization (Mod 1)
         ----------------------------------------------------------
                                        Used    Free    Percent
                                                        Utilization
-------------------------------------------------------------------
Ingress IPv4 PACL                       16      368     4.16
Ingress IPv4 VACL                       2       510     0.39
N3k Ingress IPv4 RACL                   35      221     13.67
Ingress IPv4 PBR                        24      232     9.37
Egress IPv4 VACL                        3       509     0.58
Egress IPv4 RACL                        3       509     0.58
N3K SUP                                 125     3       97.65
N3K IPV6 SUP                            9       247     3.51
Ingress IPv4 QoS                        4       252     1.56
N3K SPAN                                14      114     10.93
N3K SPAN                                7       121     5.46

LOU                                     2       22      8.33
Both LOU Operands                       2
Single LOU Operands                     0
LOU L4 src port                         1
LOU L4 dst port                         1
LOU L3 packet len                       0
LOU IP tos                              0
LOU IP dscp                             0
LOU IP precedence                       0
LOU ip TTL                              0
TCP Flags                               0       16      0.00
L4 op labels, Tcam 0                    1       62      1.58
L4 op labels, Tcam 2                    2       61      3.17
L4 op labels, Tcam 6                    0       2047    0.00

Ingress Dest info table                 1       511     0.19
Egress Dest info table                  0       512     0.00

so my questions are :

1) what is that error in my logs and how can i solve it?

2) why does N3K SUP is high ? currently i have ~200x SVI on my nexus and i want to add another 250-280x SVI to this switch, if i do this , do i face any problem or does it impact on N3K SUP utilization? and what is usages of N3K SUP exactly?

Thank you.

2 Accepted Solutions

Accepted Solutions

balaji.bandi
Hall of Fame
Hall of Fame

Look at the guide lines :  (Guidelines and Limitations for ACLs)

 

N3K SUP                                  : 62 valid entries   3 free entries

Only 62 unique ACLs can be configured. Each ACL takes one label. If same ACL is configured on multiple interfaces, the same label is shared; but if each ACL has unique entries, the ACL labels are not shared and that label limit is 62.

 

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus3000/sw/security/6x/b_Cisco_n3k_Security_Config_6x/b_Cisco_n3k_Security_Config_6x_chapter_0111.html

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

View solution in original post

Hello!

This is non-default CoPP configuration - it looks like an additional class-map named "COPP-Permit-Trusted" was added to your CoPP policy. If this was an intentional change, then depending on the size of the ACLs associated with this class-map, it's most likely expected behavior that SUP TCAM utilization has increased and is close to being filled up.

If you do not plan on making any additions to your CoPP configuration, then this syslog can be ignored. It's simply informing you that the SUP TCAM region is extremely close to full. You will not see an operational impact on the device's performance as a direct result of this syslog unless more additions are made to the device's CoPP policy.

Thank you!

-Christopher

View solution in original post

9 Replies 9

balaji.bandi
Hall of Fame
Hall of Fame

Can you post below output also :

 

show version
show run | i feature sh hardware access-list tcam region show hardware access-list resource utilization
show processes cpu sort | ex 0.00

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

here you are :

show version
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (C) 2002-2020, Cisco and/or its affiliates.
All rights reserved.
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under their own
licenses, such as open source.  This software is provided "as is," and unless
otherwise stated, there is no warranty, express or implied, including but not
limited to warranties of merchantability and fitness for a particular purpose.
Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or
GNU General Public License (GPL) version 3.0  or the GNU
Lesser General Public License (LGPL) Version 2.1 or
Lesser General Public License (LGPL) Version 2.0.
A copy of each such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://opensource.org/licenses/gpl-3.0.html and
http://www.opensource.org/licenses/lgpl-2.1.php and
http://www.gnu.org/licenses/old-licenses/library.txt.

Software
  BIOS: version 4.5.0
  NXOS: version 7.0(3)I7(8)
  BIOS compile time:  11/09/2017
  NXOS image file is: bootflash:///nxos.7.0.3.I7.8.bin
  NXOS compile time:  3/3/2020 20:00:00 [03/04/2020 08:19:49]


Hardware
  cisco Nexus3000 C3064PQ Chassis
  Intel(R) Celeron(R) CPU        P4505  @ 1.87GHz with 3902972 kB of memory.
  Processor Board ID X

  Device name: HOSTNAME
  bootflash:    1638000 kB
  usb1:               0 kB (expansion flash)


Last reset at 781025 usecs after Wed Oct 21 00:57:55 2020
  Reason: Disruptive upgrade
  System version: 6.0(2)U6(7)
  Service:

plugin
  Core Plugin, Ethernet Plugin

Active Package(s):




show run | i feature
feature bgp
feature pbr
feature interface-vlan
feature lacp
feature vpc
feature lldp




sh hardware access-list resource entries

slot  1
=======



INSTANCE 0x0
-------------

Ingress IPv4 PACL                        : 13 valid entries   368 free entries
Ingress IPv4 VACL                        : 0 valid entries   510 free entries
N3k Ingress IPv4 RACL                    : 15 valid entries   221 free entries
Ingress IPv4 PBR                         : 23 valid entries   232 free entries
Egress IPv4 VACL                         : 0 valid entries   509 free entries
Egress IPv4 RACL                         : 0 valid entries   509 free entries
N3K SUP                                  : 62 valid entries   3 free entries
N3K IPV6 SUP                             : 7 valid entries   247 free entries
Ingress IPv4 QoS                         : 4 valid entries   252 free entries
N3K SPAN                                 : 0 valid entries   114 free entries
N3K SPAN                                 : 7 used entries    121 free entries
NAT Rewrite Table                        : 0 valid entries   2048 free entries





show hardware access-list resource utilization

slot  1
=======



INSTANCE 0x0
-------------


         ACL Hardware Resource Utilization (Mod 1)
         ----------------------------------------------------------
                                        Used    Free    Percent
                                                        Utilization
-------------------------------------------------------------------
Ingress IPv4 PACL                       16      368     4.16
Ingress IPv4 VACL                       2       510     0.39
N3k Ingress IPv4 RACL                   35      221     13.67
Ingress IPv4 PBR                        24      232     9.37
Egress IPv4 VACL                        3       509     0.58
Egress IPv4 RACL                        3       509     0.58
N3K SUP                                 125     3       97.65
N3K IPV6 SUP                            9       247     3.51
Ingress IPv4 QoS                        4       252     1.56
N3K SPAN                                14      114     10.93
N3K SPAN                                7       121     5.46

LOU                                     2       22      8.33
Both LOU Operands                       2
Single LOU Operands                     0
LOU L4 src port                         1
LOU L4 dst port                         1
LOU L3 packet len                       0
LOU IP tos                              0
LOU IP dscp                             0
LOU IP precedence                       0
LOU ip TTL                              0
TCP Flags                               0       16      0.00
L4 op labels, Tcam 0                    1       62      1.58
L4 op labels, Tcam 2                    2       61      3.17
L4 op labels, Tcam 6                    0       2047    0.00

Ingress Dest info table                 1       511     0.19
Egress Dest info table                  0       512     0.00




show processes cpu sort | ex 0.00


PID    Runtime(ms)  Invoked   uSecs  1Sec    Process
-----  -----------  --------  -----  ------  -----------
14691      4479119   2936161   1525   6.00%  t2usd
12836          152       198    770   3.00%  pktmgr
12819          148        94   1579   1.00%  arp
13767      3056726  29328899    104   1.00%  stp
CPU util  :    5.00% user,    9.50% kernel,   85.50% idle
Please note that only processes from the requested vdc are shown above

balaji.bandi
Hall of Fame
Hall of Fame

Look at the guide lines :  (Guidelines and Limitations for ACLs)

 

N3K SUP                                  : 62 valid entries   3 free entries

Only 62 unique ACLs can be configured. Each ACL takes one label. If same ACL is configured on multiple interfaces, the same label is shared; but if each ACL has unique entries, the ACL labels are not shared and that label limit is 62.

 

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus3000/sw/security/6x/b_Cisco_n3k_Security_Config_6x/b_Cisco_n3k_Security_Config_6x_chapter_0111.html

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

@balaji.bandi 

can i carve from other region and add to n3k sup ? fo rexample carve 256 from Ingress IPv4 VACL and add to n3k sup, it will solve my isue? am i right?

May be, but  i need to think on that how that is shared.  but if you looking to go ahead, slow steps and observ the impact before moving to high level.

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

Do you suggest me to reduce Ingress IPv4 VACL or or Ingress IPv4 PACL  and add it to n3k sup ?

because i can not find any solution for that error, i am not sure if ACLQOS-SLOT1-4-ACLQOS_OVER_THRESHOLDis related to n3k sup tcam

can you guide me more?

Christopher Hart
Cisco Employee
Cisco Employee

Hello!

Can you share the output of show running-config copp from this switch?

It is typically expected for the N3K SUP TCAM region to be heavily utilized. This is where CoPP ACLs are programmed into hardware. The default CoPP policy usually takes up the majority of this TCAM region with a small amount of room kept free for customization.

If you have modified the CoPP policy applied to the switch (particularly, if you've added additional ACLs or class-maps to the CoPP policy), then the N3K SUP TCAM region may be approaching full utilization and causing these syslogs..

@balaji.bandi suggested that you recarve TCAM regions (such that part of an unused region is reallocated to the SUP TCAM region) to work around this issue. Unfortunately, Nexus 3000 series switches do not support allocating additional space to the SUP TCAM region. You will observe an error message as shown below:

N3K(config)# hardware profile tcam region sup 256
ERROR: Can not change the size of SUP TCAM region size.

Thank you!

-Christopher

blackmetal
Level 1
Level 1

Hello @Christopher Hart 

here is the output :

class-map type control-plane match-any COPP-Permit-Trusted
  match access-group name switch-input
class-map type control-plane match-any copp-icmp
  match access-group name copp-system-acl-icmp
class-map type control-plane match-any copp-ntp
  match access-group name copp-system-acl-ntp
class-map type control-plane match-any copp-s-arp
class-map type control-plane match-any copp-s-bfd
class-map type control-plane match-any copp-s-bpdu
class-map type control-plane match-any copp-s-dai
class-map type control-plane match-any copp-s-default
class-map type control-plane match-any copp-s-dhcpreq
class-map type control-plane match-any copp-s-dhcpresp
  match access-group name copp-system-dhcp-relay
class-map type control-plane match-any copp-s-dpss
class-map type control-plane match-any copp-s-eigrp
  match access-group name copp-system-acl-eigrp
  match access-group name copp-system-acl-eigrp6
class-map type control-plane match-any copp-s-glean
class-map type control-plane match-any copp-s-igmp
  match access-group name copp-system-acl-igmp
class-map type control-plane match-any copp-s-ipmcmiss
class-map type control-plane match-any copp-s-l2switched
class-map type control-plane match-any copp-s-l3destmiss
class-map type control-plane match-any copp-s-l3mtufail
class-map type control-plane match-any copp-s-l3slowpath
class-map type control-plane match-any copp-s-mpls
class-map type control-plane match-any copp-s-pimautorp
class-map type control-plane match-any copp-s-pimreg
  match access-group name copp-system-acl-pimreg
class-map type control-plane match-any copp-s-ping
  match access-group name copp-system-acl-ping
class-map type control-plane match-any copp-s-ptp
class-map type control-plane match-any copp-s-routingProto1
  match access-group name copp-system-acl-routingproto1
  match access-group name copp-system-acl-v6routingproto1
class-map type control-plane match-any copp-s-routingProto2
  match access-group name copp-system-acl-routingproto2
class-map type control-plane match-any copp-s-selfIp
class-map type control-plane match-any copp-s-ttl1
class-map type control-plane match-any copp-s-v6routingProto2
  match access-group name copp-system-acl-v6routingProto2
class-map type control-plane match-any copp-s-vxlan
class-map type control-plane match-any copp-snmp
  match access-group name copp-system-acl-snmp
class-map type control-plane match-any copp-ssh
  match access-group name copp-system-acl-ssh
class-map type control-plane match-any copp-stftp
  match access-group name copp-system-acl-stftp
class-map type control-plane match-any copp-tacacsradius
  match access-group name copp-system-acl-tacacsradius
class-map type control-plane match-any copp-telnet
  match access-group name copp-system-acl-telnet
policy-map type control-plane copp-system-policy
  class copp-s-default
    police pps 400
  class copp-s-l2switched
    police pps 200
  class copp-s-ping
    police pps 100
  class copp-s-l3destmiss
    police pps 100
  class copp-s-glean
    police pps 500
  class copp-s-selfIp
    police pps 500
  class copp-s-l3mtufail
    police pps 100
  class copp-s-ttl1
    police pps 100
  class copp-s-ipmcmiss
    police pps 400
  class copp-s-l3slowpath
    police pps 100
  class copp-s-dhcpreq
    police pps 300
  class copp-s-dhcpresp
    police pps 300
  class copp-s-dai
    police pps 300
  class copp-s-igmp
    police pps 400
  class copp-s-eigrp
    police pps 200
  class copp-s-pimreg
    police pps 200
  class copp-s-pimautorp
    police pps 200
  class copp-s-routingProto2
    police pps 1300
  class copp-s-v6routingProto2
    police pps 1300
  class copp-s-routingProto1
    police pps 1000
  class copp-s-arp
    police pps 200
  class copp-s-ptp
    police pps 1000
  class copp-s-vxlan
    police pps 1000
  class copp-s-bfd
    police pps 350
  class copp-s-bpdu
    police pps 12000
  class copp-s-dpss
    police pps 1000
  class copp-s-mpls
    police pps 100
  class copp-icmp
    police pps 200
  class copp-telnet
    police pps 500
  class copp-ssh
    police pps 500
  class copp-snmp
    police pps 500
  class copp-ntp
    police pps 100
  class copp-tacacsradius
    police pps 400
  class copp-stftp
    police pps 400
  class COPP-Permit-Trusted
    police pps 0
control-plane
  service-policy input copp-system-policy

Hello!

This is non-default CoPP configuration - it looks like an additional class-map named "COPP-Permit-Trusted" was added to your CoPP policy. If this was an intentional change, then depending on the size of the ACLs associated with this class-map, it's most likely expected behavior that SUP TCAM utilization has increased and is close to being filled up.

If you do not plan on making any additions to your CoPP configuration, then this syslog can be ignored. It's simply informing you that the SUP TCAM region is extremely close to full. You will not see an operational impact on the device's performance as a direct result of this syslog unless more additions are made to the device's CoPP policy.

Thank you!

-Christopher

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Review Cisco Networking products for a $25 gift card