10-23-2020 07:09 AM
Hello,
i have a N3k-c3064pq-10gx and sometimes i see the following error on my logs :
2020 Oct 23 16:20:51 HOSTNAME %ACLQOS-SLOT1-4-ACLQOS_OVER_THRESHOLD: Inst 0 Tcam 8 Bank 1's (NEW MAX TCAM COMPAT REGION) usage has reached its threshold 2020 Oct 23 16:20:51 HOSTNAME %ACLQOS-SLOT1-4-ACLQOS_OVER_THRESHOLD: Inst 0 Tcam 8 Bank 1's ((null)) usage has reached its threshol d 2020 Oct 23 16:20:51 HOSTNAME %ACLQOS-SLOT1-4-ACLQOS_OVER_THRESHOLD: Inst 0 Tcam 8 Bank 1's (NEW MAX TCAM COMPAT REGION) usage has reached its threshold 2020 Oct 23 16:20:51 HOSTNAME %ACLQOS-SLOT1-4-ACLQOS_OVER_THRESHOLD: Inst 0 Tcam 8 Bank 1's ((null)) usage has reached its threshol d
and here is my show system internal access-list resource utilization out put :
slot 1 ======= INSTANCE 0x0 ------------- ACL Hardware Resource Utilization (Mod 1) ---------------------------------------------------------- Used Free Percent Utilization ------------------------------------------------------------------- Ingress IPv4 PACL 16 368 4.16 Ingress IPv4 VACL 2 510 0.39 N3k Ingress IPv4 RACL 35 221 13.67 Ingress IPv4 PBR 24 232 9.37 Egress IPv4 VACL 3 509 0.58 Egress IPv4 RACL 3 509 0.58 N3K SUP 125 3 97.65 N3K IPV6 SUP 9 247 3.51 Ingress IPv4 QoS 4 252 1.56 N3K SPAN 14 114 10.93 N3K SPAN 7 121 5.46 LOU 2 22 8.33 Both LOU Operands 2 Single LOU Operands 0 LOU L4 src port 1 LOU L4 dst port 1 LOU L3 packet len 0 LOU IP tos 0 LOU IP dscp 0 LOU IP precedence 0 LOU ip TTL 0 TCP Flags 0 16 0.00 L4 op labels, Tcam 0 1 62 1.58 L4 op labels, Tcam 2 2 61 3.17 L4 op labels, Tcam 6 0 2047 0.00 Ingress Dest info table 1 511 0.19 Egress Dest info table 0 512 0.00
so my questions are :
1) what is that error in my logs and how can i solve it?
2) why does N3K SUP is high ? currently i have ~200x SVI on my nexus and i want to add another 250-280x SVI to this switch, if i do this , do i face any problem or does it impact on N3K SUP utilization? and what is usages of N3K SUP exactly?
Thank you.
Solved! Go to Solution.
10-23-2020 07:39 AM
Look at the guide lines : (Guidelines and Limitations for ACLs)
N3K SUP : 62 valid entries 3 free entries
Only 62 unique ACLs can be configured. Each ACL takes one label. If same ACL is configured on multiple interfaces, the same label is shared; but if each ACL has unique entries, the ACL labels are not shared and that label limit is 62.
10-24-2020 05:17 AM
Hello!
This is non-default CoPP configuration - it looks like an additional class-map named "COPP-Permit-Trusted" was added to your CoPP policy. If this was an intentional change, then depending on the size of the ACLs associated with this class-map, it's most likely expected behavior that SUP TCAM utilization has increased and is close to being filled up.
If you do not plan on making any additions to your CoPP configuration, then this syslog can be ignored. It's simply informing you that the SUP TCAM region is extremely close to full. You will not see an operational impact on the device's performance as a direct result of this syslog unless more additions are made to the device's CoPP policy.
Thank you!
-Christopher
10-23-2020 07:23 AM
Can you post below output also :
show version
show run | i feature sh hardware access-list tcam region show hardware access-list resource utilization
show processes cpu sort | ex 0.00
10-23-2020 07:28 AM
here you are :
show version Cisco Nexus Operating System (NX-OS) Software TAC support: http://www.cisco.com/tac Copyright (C) 2002-2020, Cisco and/or its affiliates. All rights reserved. The copyrights to certain works contained in this software are owned by other third parties and used and distributed under their own licenses, such as open source. This software is provided "as is," and unless otherwise stated, there is no warranty, express or implied, including but not limited to warranties of merchantability and fitness for a particular purpose. Certain components of this software are licensed under the GNU General Public License (GPL) version 2.0 or GNU General Public License (GPL) version 3.0 or the GNU Lesser General Public License (LGPL) Version 2.1 or Lesser General Public License (LGPL) Version 2.0. A copy of each such license is available at http://www.opensource.org/licenses/gpl-2.0.php and http://opensource.org/licenses/gpl-3.0.html and http://www.opensource.org/licenses/lgpl-2.1.php and http://www.gnu.org/licenses/old-licenses/library.txt. Software BIOS: version 4.5.0 NXOS: version 7.0(3)I7(8) BIOS compile time: 11/09/2017 NXOS image file is: bootflash:///nxos.7.0.3.I7.8.bin NXOS compile time: 3/3/2020 20:00:00 [03/04/2020 08:19:49] Hardware cisco Nexus3000 C3064PQ Chassis Intel(R) Celeron(R) CPU P4505 @ 1.87GHz with 3902972 kB of memory. Processor Board ID X Device name: HOSTNAME bootflash: 1638000 kB usb1: 0 kB (expansion flash) Last reset at 781025 usecs after Wed Oct 21 00:57:55 2020 Reason: Disruptive upgrade System version: 6.0(2)U6(7) Service: plugin Core Plugin, Ethernet Plugin Active Package(s): show run | i feature feature bgp feature pbr feature interface-vlan feature lacp feature vpc feature lldp sh hardware access-list resource entries slot 1 ======= INSTANCE 0x0 ------------- Ingress IPv4 PACL : 13 valid entries 368 free entries Ingress IPv4 VACL : 0 valid entries 510 free entries N3k Ingress IPv4 RACL : 15 valid entries 221 free entries Ingress IPv4 PBR : 23 valid entries 232 free entries Egress IPv4 VACL : 0 valid entries 509 free entries Egress IPv4 RACL : 0 valid entries 509 free entries N3K SUP : 62 valid entries 3 free entries N3K IPV6 SUP : 7 valid entries 247 free entries Ingress IPv4 QoS : 4 valid entries 252 free entries N3K SPAN : 0 valid entries 114 free entries N3K SPAN : 7 used entries 121 free entries NAT Rewrite Table : 0 valid entries 2048 free entries show hardware access-list resource utilization slot 1 ======= INSTANCE 0x0 ------------- ACL Hardware Resource Utilization (Mod 1) ---------------------------------------------------------- Used Free Percent Utilization ------------------------------------------------------------------- Ingress IPv4 PACL 16 368 4.16 Ingress IPv4 VACL 2 510 0.39 N3k Ingress IPv4 RACL 35 221 13.67 Ingress IPv4 PBR 24 232 9.37 Egress IPv4 VACL 3 509 0.58 Egress IPv4 RACL 3 509 0.58 N3K SUP 125 3 97.65 N3K IPV6 SUP 9 247 3.51 Ingress IPv4 QoS 4 252 1.56 N3K SPAN 14 114 10.93 N3K SPAN 7 121 5.46 LOU 2 22 8.33 Both LOU Operands 2 Single LOU Operands 0 LOU L4 src port 1 LOU L4 dst port 1 LOU L3 packet len 0 LOU IP tos 0 LOU IP dscp 0 LOU IP precedence 0 LOU ip TTL 0 TCP Flags 0 16 0.00 L4 op labels, Tcam 0 1 62 1.58 L4 op labels, Tcam 2 2 61 3.17 L4 op labels, Tcam 6 0 2047 0.00 Ingress Dest info table 1 511 0.19 Egress Dest info table 0 512 0.00 show processes cpu sort | ex 0.00 PID Runtime(ms) Invoked uSecs 1Sec Process ----- ----------- -------- ----- ------ ----------- 14691 4479119 2936161 1525 6.00% t2usd 12836 152 198 770 3.00% pktmgr 12819 148 94 1579 1.00% arp 13767 3056726 29328899 104 1.00% stp CPU util : 5.00% user, 9.50% kernel, 85.50% idle Please note that only processes from the requested vdc are shown above
10-23-2020 07:39 AM
Look at the guide lines : (Guidelines and Limitations for ACLs)
N3K SUP : 62 valid entries 3 free entries
Only 62 unique ACLs can be configured. Each ACL takes one label. If same ACL is configured on multiple interfaces, the same label is shared; but if each ACL has unique entries, the ACL labels are not shared and that label limit is 62.
10-23-2020 07:49 AM
can i carve from other region and add to n3k sup ? fo rexample carve 256 from Ingress IPv4 VACL and add to n3k sup, it will solve my isue? am i right?
10-23-2020 07:54 AM
May be, but i need to think on that how that is shared. but if you looking to go ahead, slow steps and observ the impact before moving to high level.
10-23-2020 08:09 AM
Do you suggest me to reduce Ingress IPv4 VACL or or Ingress IPv4 PACL and add it to n3k sup ?
because i can not find any solution for that error, i am not sure if ACLQOS-SLOT1-4-ACLQOS_OVER_THRESHOLDis related to n3k sup tcam
can you guide me more?
10-23-2020 09:29 AM
Hello!
Can you share the output of show running-config copp from this switch?
It is typically expected for the N3K SUP TCAM region to be heavily utilized. This is where CoPP ACLs are programmed into hardware. The default CoPP policy usually takes up the majority of this TCAM region with a small amount of room kept free for customization.
If you have modified the CoPP policy applied to the switch (particularly, if you've added additional ACLs or class-maps to the CoPP policy), then the N3K SUP TCAM region may be approaching full utilization and causing these syslogs..
@balaji.bandi suggested that you recarve TCAM regions (such that part of an unused region is reallocated to the SUP TCAM region) to work around this issue. Unfortunately, Nexus 3000 series switches do not support allocating additional space to the SUP TCAM region. You will observe an error message as shown below:
N3K(config)# hardware profile tcam region sup 256 ERROR: Can not change the size of SUP TCAM region size.
Thank you!
-Christopher
10-23-2020 11:14 PM
Hello @Christopher Hart
here is the output :
class-map type control-plane match-any COPP-Permit-Trusted match access-group name switch-input class-map type control-plane match-any copp-icmp match access-group name copp-system-acl-icmp class-map type control-plane match-any copp-ntp match access-group name copp-system-acl-ntp class-map type control-plane match-any copp-s-arp class-map type control-plane match-any copp-s-bfd class-map type control-plane match-any copp-s-bpdu class-map type control-plane match-any copp-s-dai class-map type control-plane match-any copp-s-default class-map type control-plane match-any copp-s-dhcpreq class-map type control-plane match-any copp-s-dhcpresp match access-group name copp-system-dhcp-relay class-map type control-plane match-any copp-s-dpss class-map type control-plane match-any copp-s-eigrp match access-group name copp-system-acl-eigrp match access-group name copp-system-acl-eigrp6 class-map type control-plane match-any copp-s-glean class-map type control-plane match-any copp-s-igmp match access-group name copp-system-acl-igmp class-map type control-plane match-any copp-s-ipmcmiss class-map type control-plane match-any copp-s-l2switched class-map type control-plane match-any copp-s-l3destmiss class-map type control-plane match-any copp-s-l3mtufail class-map type control-plane match-any copp-s-l3slowpath class-map type control-plane match-any copp-s-mpls class-map type control-plane match-any copp-s-pimautorp class-map type control-plane match-any copp-s-pimreg match access-group name copp-system-acl-pimreg class-map type control-plane match-any copp-s-ping match access-group name copp-system-acl-ping class-map type control-plane match-any copp-s-ptp class-map type control-plane match-any copp-s-routingProto1 match access-group name copp-system-acl-routingproto1 match access-group name copp-system-acl-v6routingproto1 class-map type control-plane match-any copp-s-routingProto2 match access-group name copp-system-acl-routingproto2 class-map type control-plane match-any copp-s-selfIp class-map type control-plane match-any copp-s-ttl1 class-map type control-plane match-any copp-s-v6routingProto2 match access-group name copp-system-acl-v6routingProto2 class-map type control-plane match-any copp-s-vxlan class-map type control-plane match-any copp-snmp match access-group name copp-system-acl-snmp class-map type control-plane match-any copp-ssh match access-group name copp-system-acl-ssh class-map type control-plane match-any copp-stftp match access-group name copp-system-acl-stftp class-map type control-plane match-any copp-tacacsradius match access-group name copp-system-acl-tacacsradius class-map type control-plane match-any copp-telnet match access-group name copp-system-acl-telnet policy-map type control-plane copp-system-policy class copp-s-default police pps 400 class copp-s-l2switched police pps 200 class copp-s-ping police pps 100 class copp-s-l3destmiss police pps 100 class copp-s-glean police pps 500 class copp-s-selfIp police pps 500 class copp-s-l3mtufail police pps 100 class copp-s-ttl1 police pps 100 class copp-s-ipmcmiss police pps 400 class copp-s-l3slowpath police pps 100 class copp-s-dhcpreq police pps 300 class copp-s-dhcpresp police pps 300 class copp-s-dai police pps 300 class copp-s-igmp police pps 400 class copp-s-eigrp police pps 200 class copp-s-pimreg police pps 200 class copp-s-pimautorp police pps 200 class copp-s-routingProto2 police pps 1300 class copp-s-v6routingProto2 police pps 1300 class copp-s-routingProto1 police pps 1000 class copp-s-arp police pps 200 class copp-s-ptp police pps 1000 class copp-s-vxlan police pps 1000 class copp-s-bfd police pps 350 class copp-s-bpdu police pps 12000 class copp-s-dpss police pps 1000 class copp-s-mpls police pps 100 class copp-icmp police pps 200 class copp-telnet police pps 500 class copp-ssh police pps 500 class copp-snmp police pps 500 class copp-ntp police pps 100 class copp-tacacsradius police pps 400 class copp-stftp police pps 400 class COPP-Permit-Trusted police pps 0 control-plane service-policy input copp-system-policy
10-24-2020 05:17 AM
Hello!
This is non-default CoPP configuration - it looks like an additional class-map named "COPP-Permit-Trusted" was added to your CoPP policy. If this was an intentional change, then depending on the size of the ACLs associated with this class-map, it's most likely expected behavior that SUP TCAM utilization has increased and is close to being filled up.
If you do not plan on making any additions to your CoPP configuration, then this syslog can be ignored. It's simply informing you that the SUP TCAM region is extremely close to full. You will not see an operational impact on the device's performance as a direct result of this syslog unless more additions are made to the device's CoPP policy.
Thank you!
-Christopher
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: