ACE 4710 resource allocation issue

asepiacci · ‎01-22-2014

HI,

I cannot understand the resource allocation of the ACE I have configured, in particular why the MAX allocation is lower than the MIN allocation, or even 0.

Probably, it's related to the RC0 class, where the maximum is set as unlimited, but I expected every context to have the maximum allocation equal to the class it belongs to. The only exception should be the Context D, the one associated to RC0 class, that could have a maximum of 54% = 30%(the minimum) + 24% (100% - 76% that is the sum of the minimum of all other contexts) .

This is the resource configuration:

ACE4710-O1-PRI/Admin# show run

....

resource-class RC0

limit-resource all minimum 30.00 maximum unlimited

resource-class RC1

limit-resource all minimum 10.00 maximum equal-to-min

limit-resource sticky minimum 10.00 maximum equal-to-min

resource-class RC2

limit-resource all minimum 20.00 maximum equal-to-min

limit-resource sticky minimum 20.00 maximum equal-to-min

resource-class RC3

limit-resource all minimum 5.00 maximum equal-to-min

limit-resource sticky minimum 5.00 maximum equal-to-min

resource-class RC4

limit-resource all minimum 1.00 maximum equal-to-min

limit-resource sticky minimum 1.00 maximum equal-to-min

.....

context A

allocate-interface vlan 27

allocate-interface vlan 324

allocate-interface vlan 605

allocate-interface vlan 607

member RC2

context B

allocate-interface vlan 603

allocate-interface vlan 606

member RC1

context C

allocate-interface vlan 192-193

allocate-interface vlan 251

member RC3

context D

allocate-interface vlan 388-389

member RC0

context E

allocate-interface vlan 608-609

member RC4

context F

member RC1

This is the actual resource usage and allocation:

ACE4710-O1-PRI/Admin# show resource usage

Allocation

Resource Current Peak Min Max Denied

-------------------------------------------------------------------------------

.......

Context: A

conc-connections 11439 12244 400000 0 0

mgmt-connections 44 46 20000 0 0

proxy-connections 11439 12244 52429 0 0

xlates 0 0 13107 0 0

acc-connections 0 0 2000 0 0

bandwidth 3413889 21806480 26843400 125000000 0

throughput 3413213 5408647 26843400 0 0

mgmt-traffic rate 676 676 0 125000000 0

connection rate 59 85 200000 0 0

ssl-connections rate 50 79 1000 0 5238386

mac-miss rate 0 0 400 0 0

inspect-conn rate 0 0 8000 0 0

http-comp rate 2075434 4143487 13107200 0 21746221

acl-memory 16712 16712 7508336 0 0

sticky 10 10 163840 0 0

regexp 10138 10138 209715 0 0

syslog buffer 208896 208896 208896 0 0

syslog rate 1 3 20000 0 0

Context: B

conc-connections 164 5571 200000 0 0

mgmt-connections 0 32 10000 0 0

proxy-connections 164 5571 26214 0 0

xlates 0 0 6554 0 0

acc-connections 0 0 1000 0 0

bandwidth 56475 4014023 13421700 125000000 648

throughput 53655 4001175 13421700 0 648

mgmt-traffic rate 2820 12848 0 125000000 0

connection rate 9 5847 100000 0 0

ssl-connections rate 7 201 500 0 0

mac-miss rate 0 1 200 0 0

inspect-conn rate 0 0 4000 0 0

http-comp rate 0 0 6553600 0 0

acl-memory 12896 12912 3753136 0 0

sticky 15 15 81920 0 0

regexp 1314 1314 104858 0 0

syslog buffer 105472 208896 104448 0 0

syslog rate 0 53 10000 0 0

Context: C

conc-connections 6 436 100000 0 0

mgmt-connections 0 30 5000 0 0

proxy-connections 4 41 13107 0 0

xlates 0 0 3277 0 0

acc-connections 0 0 500 0 0

bandwidth 176 857478 6710850 125000000 0

throughput 90 841826 6710850 0 0

mgmt-traffic rate 86 15652 0 125000000 0

connection rate 0 6298 50000 0 0

ssl-connections rate 0 19 250 0 0

mac-miss rate 0 200 100 0 979

inspect-conn rate 0 0 2000 0 0

http-comp rate 0 0 3276800 0 0

acl-memory 16576 16592 1875504 0 0

sticky 1 3 40960 0 0

regexp 1062 1062 52429 0 0

syslog buffer 0 0 52224 0 0

syslog rate 0 0 5000 0 0

Context: D

conc-connections 7124 18425 600000 480000 0

mgmt-connections 8 70 30000 24000 0

proxy-connections 7124 18421 78643 62914 0

xlates 0 0 19661 15728 0

acc-connections 0 0 3000 2400 0

bandwidth 4971240 16084230 40265100 157212080 0

throughput 4971140 16079538 40265100 32212080 0

mgmt-traffic rate 100 4692 0 125000000 0

connection rate 126 844 300000 240000 0

ssl-connections rate 117 855 1500 1200 0

mac-miss rate 0 109 600 480 0

inspect-conn rate 0 0 12000 9600 0

http-comp rate 0 0 19660800 15728640 0

acl-memory

24400 24416 11263536 9012511 0

sticky 3399 6713 245760 0 0

regexp 2438 2438 314573 251658 0

syslog buffer 0 0 314368 253952 0

syslog rate 0 0 30000 24000 0

Context: E

conc-connections 0 16 20000 0 0

mgmt-connections 2 44 1000 0 0

proxy-connections 0 4 2621 0 0

xlates 0 0 655 0 0

acc-connections 0 0 100 0 0

bandwidth 262 79518 1342170 125000000 0

throughput 90 72846 1342170 0 0

mgmt-traffic rate 172 6672 0 125000000 0

connection rate 2 10 10000 0 0

ssl-connections rate 0 0 50 0 0

mac-miss rate 0 0 20 0 0

inspect-conn rate 0 0 400 0 0

http-comp rate 0 0 655360 0 0

acl-memory 24272 24272 373424 0 0

sticky 0 7 8192 0 0

regexp 914 914 10486 0 0

syslog buffer 0 0 10240 0 0

syslog rate 0 0 1000 0 0

Context: F

conc-connections 0 0 200000 0 0

mgmt-connections 0 0 10000 0 0

proxy-connections 0 0 26214 0 0

xlates 0 0 6554 0 0

acc-connections 0 0 1000 0 0

bandwidth 0 0 13421700 125000000 0

throughput 0 0 13421700 0 0

mgmt-traffic rate 0 0 0 125000000 0

connection rate 0 0 100000 0 0

ssl-connections rate 0 0 500 0 0

mac-miss rate 0 0 200 0 0

inspect-conn rate 0 0 4000 0 0

http-comp rate 0 0 6553600 0 0

acl-memory 6976 7040 3753136 0 0

sticky 0 0 81920 0 0

regexp 914 914 104858 0 0

syslog buffer 0 0 104448 0 0

syslog rate 0 0 10000 0 0

This is the show ver:

ACE4710-O1-PRI/Admin# show ver

Cisco Application Control Software (ACSW)

TAC support: http://www.cisco.com/tac

The copyrights to certain works contained herein are owned by

other third parties and are used and distributed under license.

Some parts of this software are covered under the GNU Public

License. A copy of the license is available at

http://www.gnu.org/licenses/gpl.html.

Software

loader: Version 0.95

system: Version A3(2.3) [build 3.0(0)A3(2.3) adbuild_16:41:21-2009/07/23_/auto/adbu-rel2/rel_a3_2_3_throttle/REL_3_0_0_A

3_2_3]

system image file: (hd0,1)/c4710ace-mz.A3_2_3.bin

Device Manager version 1.2 (0) 20090702:2137

installed license: ACE-AP-VIRT-020 ACE-AP-C-500-LIC ACE-AP-OPT-LIC-K9 ACE-AP-SSL-05K-K9

Hardware

cpu info:

Motherboard:

number of cpu(s): 2

Daughtercard:

number of cpu(s): 16

memory info:

total: 6226388 kB, free: 4129876 kB

shared: 0 kB, buffers: 23172 kB, cached 0 kB

cf info:

filesystem: /dev/hdb2

total: 861668 kB, used: 729204 kB, available: 88692 kB

last boot reason: Unknown

configuration register: 0x1

ACE4710-O1-PRI kernel uptime is 435 days 15 hours 36 minute(s) 40 second(s)

Thanks.

Alessandro

Kanwaljeet Singh · ‎01-22-2014

Hi Alessandro,

I came across this document. Good read to understand resource allocation.

     ===============================================================
          Understanding Throughput Allocation on the ACE
     ===============================================================


`show resource usage all`
                                                     Allocation
        Resource         Current       Peak        Min        Max       Denied
-------------------------------------------------------------------------------

Context: Admin
     :
  bandwidth                   794      62282   75000000  200000000          0
    throughput                177       3183   75000000   75000000          0
    mgmt-traffic rate         617      59099          0  125000000          0


Context: CTX1
     :
  bandwidth             357573782  425023811   50000000  550000000 3343868110
    throughput          357569156  425000000   50000000  425000000 3343868110
    mgmt-traffic rate        4626      23811          0  125000000          0


The above was taken from an ACE licensed for 4 Gbps of throughput.  There are
only two contexts configured on the ACE and they are each a member of a 
unique resource-class.


resource-class RC-Admin
  limit-resource all minimum 15.00 maximum equal-to-min
resource-class RC-CTX1
  limit-resource all minimum 10.00 maximum unlimited


In order to see how the Min and Max allocations above correlate to 4 Gps, we
have to keep in mind that the numbers above are in bytes, not bits.

We can see from the resource-class configurations above that 25% of the bandwidth
is reserved for the two contexts (15% for the Admin context and 10% for CTX1).

     Admin Context:

     Admin context min alloc.             75,000,000     bytes
     Multiply by 8 to convert to bits            x 8     bits-per-byte
                              ----------------------
     Admin context min Total bps            600,000,000   bps (or .6 Gbps)


     =========================================================================

     CXT1 Context:

     CTX1 context min alloc.                   50,000,000     bytes
     Multiply by 8 to convert to bits            x 8     bits-per-byte
                              ----------------------
     Admin context min Total bps              400,000,000   bps (or .4 Gbps)


     =========================================================================


     Admin context min alloc.            600,000,000   bps (or .6 Gbps)
     CTX1 context min alloc.                 +  400,000,000   bps (or .4 Gbps)
                              ----------------------
     Total min allocated bps               1,000,000,000   bps (or 1 Gbps)


Notice that 1 Gbps is 25% of 4 Gbps, which is the aggregate of the minimum 
throughput allocations for both contexts.

Now we move on to see how the remaining 3 Gbps is carved out for the two contexts
to use.  

We can see from the resource-class configurations that the Admin context's 
maximum throughput allocation is equal-to-min.  Therefore, it cannot use anymore
than its 15%, or .6 Gbps (or 600 Mbps).  The CXT1 context has a maximum throughput 
allocation of unlimited, therefore it has access to all the remaining bandwidth on 
the ACE, if necessary.

Since its maximum is shown as 425,000,000 bytes, we can see how this works out to
the remaining 3 Gbps as shown below:


     CTX1 context max alloc.                 425,000,000     bytes
     Subtract the min alloc.                   - 50,000,000     bytes
                              ----------------------
     Total remaining bytes available            375,000,000     bytes
     Multiply by 8 to convert to bits            x 8     bits-per-byte
                              ----------------------

     Total bps remaining in shared pool     3,000,000,000   bps (or 3 Gbps)


Now you can see how we've accounted for the 4 Gbps:


     Total min allocated bps               1,000,000,000   bps (or 1 Gbps)
     Total bps remaining in shared pool     3,000,000,000   bps (or 3 Gbps)
                              ---------------
                              4,000,000,000     bps (or 4 Gbps)


In the output at the top of this document, you can see the throughput denies
on CTX1.  Since there is a minimum of .4 Gbps allocated to this context, plus 
another 3 Gbps in the shared pool available if needed, CTX1 has access to a total
of 3.4 Gbps of throughput.  A chart sent by the customer indicated the throughput 
on the ACE was peaking at roughly 3.8 Gbps during peak hours, thereby causing the 
bandwidth denies.


Solution:

     In this case there are two options.  

     1. There is currently 15% of resources allocated to the Admin context.
        Since the Admin context is only using a small fraction of this, it
        could be lowered to 2%, which would add another 510 Mbps of throughput
        to the shared pool.  This would bring the throughput that CXT1 has 
        access to from 3.4 Gbps to 3.91 Gbps.  Since your peak is roughly
        3.8 Gbps, this would give you approximately 110 Mbps extra.

          NOTE:  Resources freed from one context, so another context can use
            them, can only be used by the new context if the first context
            is not using them.  Resources must be free in order for them
            to be transfered.

     2. At this point, a better long-term solution would be to upgrade the 
        4 Gbps license to a 8 Gbps license.

 
 Regards,
 Kanwal

asepiacci · ‎01-23-2014

Thanks Fnu for your kind reply, but if you have a look at the "show resource usage" I posted, you can see some values I can't understand. Focusing on ssl-connections:

Context A has a MIN = 1000, while MAX is 1000

Context B has MIN = 500 and MAX = 0

Context C has MIN = 250, and MAX = 0

Context D MIN = 1500 > MAX = 1200 and so on:

how is it possible?

And it's the same for other variables: the MAX column is 0 while MIN is "something" (or MIN > MAX).

I really can't understand.