ACL memory on the ACE module is used for programming traffic processing policy in the data plane. ACLs allow users to control network connection setups rather than processing each packet. The ACE supports a maximum of 8,192 unique ACLs and 64,000 ACL entries. Some ACLs use more memory than others, such as an ACL that uses large port number ranges or overlapping networks. Even using object groups in ACL entries, you enter fewer actual ACL entries, but the same number of expanded ACL entries as you did when you entered entries without object groups. Expanded ACL entries count toward the system limit. Complicated ACL, NAT and load balancing configurations can exponentially increase it's usage to the point that new configuration changes won't be able to fit into the available memory.
ACL memory overview and resource allocation
ACL memory consists of different type of nodes - Compressed, Uncompressed, Leaf Head, Leaf Parameter and Policy action nodes. In a system we have fixed number of those various nodes. For example, ACE has 262143 compressed nodes, 19999 uncompressed nodes, 204799 Leaf Head Nodes, 409600 Leaf Parameter Nodes and 204800 Policy Action nodes. “Show np 1 access-list resource” command from *Admin* context will show the max-limit for each type of nodes except the Policy Action nodes. When users configure ACL resources such as min 10% and max *equal to min*, ACE guarantees 10% of each nodes, except action nodes, to that resource class. What that means is ACE configurations for that resource class need to be accommodated with 10% of each node. If any of the nodes consumption goes beyond the 10% limit, ACL resource allocation would fail even though the other nodes usages are well below 10%.
Recommendation for ACL resource allocation
ACL nodes usages depend on the configurations. It doesn’t depend on the number of lines of configurations and also it doesn’t increase linearly with the number of lines of configurations. Nodes allocation for a given configurations is handled by ACL compiler using a complex data structure and it is extremely difficult to calculate the node usage for a large configurations before applying the configurations. So the recommendation would be
Apply the configurations on the system.
Find the maximum used node, in terms of percentage, for the applied configuration from the “show np 1 access-list resource” command.
While configuring ACL resource, make sure the max resource percentage is above the percentage calculated in step 2.
Reducing ACL memory usage
One common cause for high ACL memory usage is per server NAT ACLs, where a single line of ACL is written for each server that must be processed by a policy. As this policy gets applied to multiple interfaces, across multiple policies, it begins to grow exponentially and uses up a lot of ACL memory. One method to reduce the usage is to summarize ACL entries, so instead of having an ACL for server 10.87.102.100, 10.87.102.101, 10.87.102.102 and 10.87.102.103, these can be summarized into 10.87.102.100/30, reducing 4 entries to 1.
Another method to reduce the exponential growth of ACL memory usage is to break up large single context configurations into smaller multiple contexts. Following the simplified math, if a single context has 8*8 entries, that may expand to 64 total used, whereas if that configuration is broken down into two contexts, both containing 4*4, the total usage is 16+16=32. This math is greatly simplified and it's much more difficult to calculate the actual ACL memory usage, however is meant to demonstrate the logic behind exponential vs smaller linear growth.
You can instruct the ACE to check the health of servers and server farms by configuring health probes (sometimes referred to as keepalives). After you create a probe, you assign it to a real server or a server farm. A probe can be one of many types, including TCP, ICMP, Telnet, HTTP, and so on. The ACE sends out probes periodically to determine the status of a server, verifies the server response, and checks for other network problems that may prevent a client from reaching a server. Based on the server response, the ACE can place the server in or out of service, and, based on the status of the servers in the server farm, can make reliable load-balancing decisions.
Use "show resource usage" command, in Admin context, to display resource utilization in your ACE and check acl-memory usage.
ACE_module/Admin# show resource usage
Resource Current Peak Min Max Denied
conc-connections 10 18 0 8000000 0
mgmt-connections 2 10 0 100000 0
---------- snip ------------
acl-memory 33448 33448 7858944 70749384 0
regexp 0 0 0 1048576 0
---------- snip -------------
Use "limit-resource acl-memory" command to allocate ACL resources to all member contexts of a resource class.
ACE_module/Admin# limit-resource acl-memory minimum <percentage> maximum <percentage>
Hello dear community members, We have two switches Nexus 3064-X and we would like to build a new iSCSI SAN. Expected iSCSI storage performance is 30 Gig iSCSI-A + 30 Gig iSCSI-BMy questions:1) can Nexus 3064-X handle such workload?2) in general, what...
Hi there, I'm trying to get it four Nexus 3524-X to login authenticate with windows 2019 NPS via radius protocol, but so far no luck.I see in the NPS logs there 2 of the switches are sending radius requests, but somehow NPS doesn't like them, i have ...
I'm trying to configure l3out between the ACI fabric and firewall with static routeconfigured all thing that i need form the ACI and firewall.and configure a contract between external EPG that for l3out and my EPG's, but not workingit work when unen...
Hi everyone, I have a pair of N9300 with the latest NX-OS software. The following is the topology I'm looking to install: Topology: The green links are two vPCs which connect the two N9300 switches to different VM Hypervisors. The...