Regex limits on ACE

jeremy.stitt · ‎03-17-2011

Hello!

We are moving from CSS to ACE and I'm concerned about the number of regular expression rules we will have to implement.

I've read that using double wildcard expressions (.*) in too many class-maps can cause memory problems for the entire ACE. For example .*website.*

If I consolidate my regular expression rules into something like the following:

(www\.)*website(1|2)*\.(mobi|net|org).*

should I still be concerned about using too much memory? Does a statement like (www)* constitue a wildcard expression?

We have more than 100 domains that we need rules like this for.

sachinga.hcl · ‎03-17-2011

Hi Jeremy,

-------------------------------

I ran into an issue someday where I had exhausted all of the regexp memory in my ACE from using too many wildcards in my URL matching. The symptoms were class-maps not matching regex despite being the correct syntax.

I opened a ticket with Cisco, but they could not reproduce the issue in their labs.

We were stumped until the engineer examined my “show tech-support” output and saw that the regexp memory usage was very high (i.e. maxed out!):

ACE/Staging# show resource usage
Allocation
Resource Current Peak Min Max Denied
-------------------------------------------------------------------------------
Context: Staging
conc-connections 756 2902 0 7999900 0
mgmt-connections 0 17 0 99900 0
proxy-connections 0 267 0 1048574 0
xlates 0 0 0 1048574 0
bandwidth 2104660 124535514 0 622500016 0
throughput 2104406 124522546 0 498750016 0
mgmt-traffic rate 254 12968 0 123750000 0
connections rate 3 389 0 324900 0
ssl-connections rate 0 21 0 1000 0
mac-miss rate 0 5 0 2000 0
inspect-conn rate 0 3 0 6000 0
acl-memory 37688 102608 0 70844416 0
sticky 0 4 41942 0 0
regexp 915387 1047557 0 1048576 63
syslog buffer 0 0 0 4194304 0
syslog rate 0 59 0 100000 0

Wow, that looks ugly in HTML.

Anyway, the line you’re interested in is the regexp line with 915387 bytes currently used, 1047557 bytes used at peak, 1048576 bytes max, and 63 denied requests to access the regexp memory.

Note that the regexp memory usage was consuming 900+ KB of the 1MB for regexp.

Also note the denies in the right-hand column.

I knew something was up because when I added my regex class-maps, my policy-map said that Regex dnld status: FAILED.

ACE/Staging# sh service-policy int530 class-map Secure-Stage-Map

Status : ACTIVE
-----------------------------------------
Interface: vlan 520 530
service-policy: int530
class: Secure-Stage-Map
ssl-proxy server: secure-stage
loadbalance:
L7 loadbalance policy: Secure-Stage-Policy
Regex dnld status : FAILED
VIP Route Metric : 77
VIP Route Advertise : ENABLED-WHEN-ACTIVE
VIP ICMP Reply : ENABLED-WHEN-ACTIVE
VIP State: INSERVICE
curr conns : 0 , hit count : 0
dropped conns : 0
client pkt count : 0 , client byte count: 0
server pkt count : 0 , server byte count: 0
conn-rate-limit : 0 , drop-count : 0
bandwidth-rate-limit : 0 , drop-count : 0
Parameter-map(s):
HTTP_MAP

The Cisco TAC engineer said that they recommend no more than 10 double wildcard regex statements per ACE.

A double wildcard regex is one where you have a wildcard (.*) before and after a string:

class-map type http loadbalance match-any Website-Class
5 match http url .*Website.*

I had probably 20-something of those.

For whatever reason, double wildcard statements use MUCH more memory than other single wildcard statements.

So I locked my regex down to use only one wildcard per match:

class-map type http loadbalance match-any Website-Class
5 match http url /Website/.*

Once I removed the double wildcards, the memory usage stayed roughly the same.

Hhmm… Another call to my Cisco TAC engineer reveals that if the memory is not immediately freed, I need to remove the class-maps and policy-maps using the regex.

Basically, when I created the VIP class-map and then created the regex class-map to be tied to the policy-map, the VIP requested the regexp memory for the regex class-map. So in order to free that memory, I had to remove the VIP class-maps.

Actually, I had to remove the VIP class-maps, regex class-maps, AND relevant policy-maps.

Good thing I had a backup to paste it all in once I typed “no class-map” a couple hundred times.

About a minute after removing and then re-adding the class-maps, the regexp memory usage was at a much more desirable level:

ACE/Staging# show resource usage
Allocation
Resource Current Peak Min Max Denied
-------------------------------------------------------------------------------
Context: Staging
conc-connections 1008 2902 0 7999900 0
mgmt-connections 0 17 0 99900 0
proxy-connections 0 267 0 1048574 0
xlates 0 0 0 1048574 0
bandwidth 1004219 124535514 0 622500016 0
throughput 1004219 124522546 0 498750016 0
mgmt-traffic rate 0 12968 0 123750000 0
connections rate 4 389 0 324900 0
ssl-connections rate 0 21 0 1000 0
mac-miss rate 0 5 0 2000 0
inspect-conn rate 0 3 0 6000 0
acl-memory 37688 102608 0 70844416 0
sticky 0 4 41942 0 0
regexp 2198 1047557 0 1048576 63
syslog buffer 0 0 0 4194304 0
syslog rate 0 59 0 100000 0

Note that the regexp line is now consuming only 2198 bytes of the 1MB of regexp memory. Success!

Sure enough, once I checked to make sure my regex was matching URLs in my class-maps, it was working like a champ.

Would’ve rather spent the day implementing more ACE config than finding this problem, but meh.

------------------------------------------------------------------------------------------------------------------------
Hope you get your answer in this scenario.

And when I am seeing your examle you are using triple wildcard expression.

(www\.)*website(1|2)*\.(mobi|net|org).*

I don't know how much memory it would use to process your triple wildcard regular expressions , but all the time always with your configuration you can estimate how much memory is getting consumed by your regular expressions , using the above mentioned resource usage commands and then modify your regular expressions accordingly so as to best otimized usage of your memory for inculcating more regular expressions using wildcards but probably no more then 10 double wildcard regular expresion or take consultation from your Cisco TAC engineer according to the model of your ace device/card.

Also remember that header-map also consumes regexp.

As I said the limit is 10 regular expression.

Each header-map consume 1 regexp.

Each sticky cookie consume also 1 regexp.

So, check your whole configuration for counting the total number of regular expression this way may be if you have hit the limit.

For that sake try to combine header-maps.

------------------------------------------------------------------------------------------------------

In regard to ACE,

You can allocate all resources or individual resources to all member contexts of a resource class. For example, you can allocate only concurrent connections or sticky table memory or management traffic. To allocate system resources to all members (contexts) of a resource class, use the limit-resource command in resource-class configuration mode.

The syntax of this command is as follows:

limit-resource {acl-memory | all | buffer {syslog} | conc-connections | mgmt-connections | proxy-connections | rate {bandwidth | connections | inspect-conn | mac-miss | mgmt-traffic | ssl-bandwidth | syslog} | regexp | sticky | xlates} {minimum number} {maximum {equal-to-min | unlimited}}

Note: The limit that you set for individual resources when you use the limit-resource command overrides the limit that you set for all resources when you use the limit-resource all command.

If you lower the limits for one context (context A) in order to increase the limits of another context (context B), you may experience a delay in the configuration change because the ACE will not lower the limits of context A until the resources are no longer being used by the context.

Following is the limit for 4 GBps bandwidth ACE device

Regular Expression Memory 1,048,576 bytes( approx 1MB)

You can upgrade the ACE maximum bandwidth to 8 Gbps or 16 Gbps by purchasing a separate license from Cisco.

Displaying the ACE Resource Allocation and Usage

To view the current resource allocation in your ACE, enter the following command:

ACE_mdule5/Admin# show resource allocation
---------------------------------------------------------------------------
Parameter Min Max Class
---------------------------------------------------------------------------

acl-memory 0.00% 100.00% default

syslog buffer 0.00% 100.00% default

conc-connections 0.00% 100.00% default

mgmt-connections 0.00% 100.00% default

proxy-connections 0.00% 100.00% default

bandwidth 0.00% 100.00% default

connection rate 0.00% 100.00% default

inspect-conn rate 0.00% 100.00% default

syslog rate 0.00% 100.00% default

regexp 0.00% 100.00% default

sticky 0.00% 100.00% default

xlates 0.00% 100.00% default

ssl-connections rate 0.00% 100.00% default

mgmt-traffic rate 0.00% 100.00% default

mac-miss rate 0.00% 100.00% default

throughput 0.00% 100.00% default

To view the current resource usage, enter the following command:

ACE_mdule5/Admin# show resource usage
Allocation
Resource         Current       Peak        Min        Max       Denied
-------------------------------------------------------------------------------
Context: Admin
conc-connections              0          0          0    8000000          0
mgmt-connections              2          8          0     100000          0
proxy-connections             0          0          0    1048574          0
xlates                        0          0          0    1048574          0
bandwidth                  1094      80192          0 625000000          0
throughput                938      75902          0 500000000          0
mgmt-traffic rate         156       4290          0 125000000          0
connection rate               1         28          0    1000000          0
mac-miss rate                 0          0          0       2000          0
inspect-conn rate             0          0          0       6000          0
acl-memory                23776      28616          0   78610432          0
sticky                        0          0          0          0          0
regexp                        0          0          0    1048576          0
syslog buffer                 0          0          0    4194304          0
syslog rate                   0          0          0     100000          0     Note:      All bandwidth values are in bytes per second. To convert to bits per second (bps), multiply the values by eight. The ACE guarantees 1 Gbps of bandwidth for management traffic. So, the total bandwidth for a 4-Gbps ACE license is actually 5 Gbps. Throughput is still 4 Gbps.

To display the data plane resource allocation and usage and to cross-check the output of the above two commands, enter the following command:
ACE_module5/Admin# show np 1 me-stats -L0
Resource limts for context : 0
Rate                  Configured     Counters
Policer Name          Min      Max   min-toks max-toks   peak-toks    deny
bandwidth:       0   ee6b280        0 ee6b0fa     d8a4        0
throughput:       0   ee6b280        0 ee6b280     d8a4        0
mgmt-traffic rate:      0   3b9aca0        0 3b9aca0      a0e        0
connection rate:       0     7a120        0    7a120       11        0
ssl-connections rate:   0       9c4        0      9c4        0        0
mac-miss rate:       0       3e8        0      3e8        0        0
inspect-conn rate:      0       bb8        0      bb8        0        0

Resource              Configured        Counters
Policer Name          Min      Max      Min      Max     peak     deny
conc-connections:       0   3d0900        0        0        0        0
mgmt-connections:       0     c350        0        0        4        0
proxy-connections:      0    7ffff        0        0        0        0
ip-reassemble buffer:   0        0        0        0        0        0
tcp-ooo buffer:       0        0        0        0        0        0
regexp:       0        0        0        0        0        0
xlates:       0    7ffff        0        0        0        0

ACE/Admin# show resource usage virtual partition eng
Allocation
Resource Current Peak Min Max Denied
---------------------------------------------------------------------
Virtual partition: eng
conc-connections 1076261 2852238 4000000 4000000 0
mgmt-connections 8 24 2500 2500 0
proxy-connections 136152 201376 524288 524288 0
xlates 170226 293546 524288 524288 0
bandwidth 62946459 156087362 250000000 250000000 0
connection rate 117652 283887 500000 500000 0
ssl-connections rate 124 397 500 500 0
mgmt-traffic rate 16754661 313562819 62500000 62500000 0
mac-miss rate 0 0 1000 1000 0
inspect-conn rate 824 1782 3000 3000 0
acl-memory 9806253 20248853 39305216 39305216 0
regexp 148878 524288 524288 524288 3
syslog buffer 577166 1199438 2097152 2097152 0
syslog rate 234 786 1500 1500 0

The Admin context has a context ID of 0. To display the resource allocation and and usage statistics for another context, change the "0" in the "-L" parameter to the context ID of another context.

Use the below mentione dsite to get few more similar details for the same:

http://www.cisco.com/en/US/prod/collateral/modules/ps2706/ps6906/White_Paper_Cisco_Application_Control_Engine_A_Technical_Overview_of_Virtual_Partitioning_ps7027_Products_White_Paper.html

For more details on limit of different resources of ACE kindly refer the following URL:

Cisco Application Control Engine (ACE) Troubleshooting Guide -- ACE Resource Limits

http://docwiki.cisco.com/wiki/Cisco_Application_Control_Engine_(ACE)_Troubleshooting_Guide_--_ACE_Resource_Limits

Cisco Application Control Engine (ACE) Troubleshooting Guide -- Managing Resources

http://docwiki.cisco.com/wiki/Cisco_Application_Control_Engine_(ACE)_Module_Troubleshooting_Guide,_Release_A2(x)_--_Managing_Resources

Also use the following site to know more about using regular expressions and their impact:

http://www.cisco.com/web/about/security/intelligence/preventing_activex_ace.html

HTH

Please rate if you find the above mentioned information useful to you.

Sachin