6500 QOS_TCAM issue on 12.2(33)SXI2a

Martin Picard · ‎10-31-2013

Hi,

I need to implement a single QoS marking policy on +/- 100 VLANs. The MQC-policy contains 8 classes (including the class-default).

The 8 class-maps refer to 8 named extended ACL for a total of +/- 1000 lines. I don't need marking stats.

So I did the "qos vlan-based" on my interfaces and "no mls qos marking stats", but the 6500 still populates the QOS_TCAM with every class for every VLAN on every modules... no surprise that I get into TCAM_Mask capacity exceeded.

When I get this message, does it only means that ACL treatment will not be done in hardware but only in software, or does it means no ACL treatment will be done ?

In any case, how can I have the QoS_TCAM to hold a single copy of my policy ACLs for all the VLANs and modules on that 6500 ???

Here's the "show version"

Cisco IOS Software, s72033_rp Software (s72033_rp-ADVENTERPRISEK9_WAN-M), Version 12.2(33)SXI2a, RELEASE SOFTWARE (fc2)

Technical Support: http://www.cisco.com/techsupport

Compiled Wed 02-Sep-09 01:00 by prod_rel_team

ROM: System Bootstrap, Version 12.2(17r)SX5, RELEASE SOFTWARE (fc1)

ai-lab40_279 uptime is 9 weeks, 6 days, 21 hours, 25 minutes

Uptime for this control processor is 9 weeks, 6 days, 21 hours, 22 minutes

Time since ai-lab40_279 switched to active is 9 weeks, 6 days, 21 hours, 20 minutes

System returned to ROM by power cycle at 14:58:13 EST Mon Jan 17 2011 (SP by power on)

System restarted at 20:42:50 EDT Thu Aug 22 2013

System image file is "sup-bootflash:s72033-adventerprisek9_wan-mz.122-33.SXI2a.bin"

Last reload reason: Reload Command

This product contains cryptographic features and is subject to United

States and local country laws governing import, export, transfer and

use. Delivery of Cisco cryptographic products does not imply

third-party authority to import, export, distribute or use encryption.

Importers, exporters, distributors and users are responsible for

compliance with U.S. and local country laws. By using this product you

agree to comply with applicable laws and regulations. If you are unable

to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to

export@cisco.com.

cisco WS-C6509-E (R7000) processor (revision 1.6) with 983008K/65536K bytes of memory.

Processor board ID SMC1716002C

SR71000 CPU at 600Mhz, Implementation 0x504, Rev 1.2, 512KB L2 Cache

Last reset from s/w reset

84 Virtual Ethernet interfaces

115 Gigabit Ethernet interfaces

22 Ten Gigabit Ethernet interfaces

1917K bytes of non-volatile configuration memory.

8192K bytes of packet buffer memory.

65536K bytes of Flash internal SIMM (Sector size 512K).

Configuration register is 0x2102

and the "show modules"

Mod Ports Card Type Model Serial No.

--- ----- -------------------------------------- ------------------ -----------

1 16 16 port 1000mb GBIC ethernet WS-X6416-GBIC SAD05030A4P

2 4 CEF720 4 port 10-Gigabit Ethernet WS-X6704-10GE SAD105103JM

3 48 SFM-capable 48 port 10/100/1000mb RJ45 WS-X6548-GE-TX SAD0830098E

4 48 SFM-capable 48 port 10/100/1000mb RJ45 WS-X6548-GE-TX SAD074003L3

6 5 Supervisor Engine 720 10GE (Active) VS-S720-10G SAL1336XZBU

8 16 CEF720 16 port 10GE WS-X6716-10GE SAL13463XBR

Mod MAC addresses Hw Fw Sw Status

--- ---------------------------------- ------ ------------ ------------ -------

1 0001.c9b0.8e4c to 0001.c9b0.8e5b 1.2 5.4(2) 12.2(33)SXI2 Ok

2 001a.6d65.e640 to 001a.6d65.e643 2.4 12.2(18r)S1 12.2(33)SXI2 Ok

3 0011.bb34.9bf8 to 0011.bb34.9c27 10.1 7.2(1) 12.2(33)SXI2 Ok

4 000d.29f6.c6a0 to 000d.29f6.c6cf 5.0 7.2(1) 12.2(33)SXI2 Ok

6 0023.33ab.93f0 to 0023.33ab.93f7 3.1 8.5(2) 12.2(33)SXI2 Ok

8 001f.6ca4.2278 to 001f.6ca4.2287 1.1 12.2(18r)S1 12.2(33)SXI2 Ok

Mod Sub-Module Model Serial Hw Status

---- --------------------------- ------------------ ----------- ------- -------

2 Distributed Forwarding Card WS-F6700-DFC3C SAL1317NXEU 1.4 Ok

6 Policy Feature Card 3 VS-F6K-PFC3C SAL1337YKSF 1.1 Ok

6 MSFC3 Daughterboard VS-F6K-MSFC3 SAL1231YZ6Y 1.0 Ok

8 Distributed Forwarding Card WS-F6700-DFC3C SAL13484Y7C 1.4 Ok

Mod Online Diag Status

---- -------------------

1 Pass

2 Pass

3 Pass

4 Pass

6 Pass

8 Pass

Tx

Martin

Jose Solano · ‎11-05-2013

Hi Martin,

See this:http://www.cisco.com/en/US/products/hw/switches/ps708/products_tech_note09186a008074d6b1.shtml#tcam-limits

It explains a bit better the limitation for QOS TCAM on the 6500 platforms. About the first question I really don´t think that once you reached the TCAM limit this will be perform at the sofware level since if that was the case it may cause high cpu issues and that is why you see the error message warning that the limit was reached. The second question not sure if it is possible because when you configure an ACL, map the ACL to the QoS and when you apply the QoS policy on the interface, the switch programs the TCAM with that information.

Hope this helps...

Martin Picard · ‎11-05-2013

Hi Leo,

I agree that if ACL processing is done in software it might cause high CPU usage but since I saw some documented high-cpu issues related to TCAM i thought maybe when HW capacity was exceeded it would revert back to software processing. ACL software processing is not all that bad since regular routers do it all the time. Not talking about the same speed, I know but... In some cases, higher CPU usage is better than no ACL treatment at all. Having an option to fall back, or not, to software processing would be even better.

On the second point, there might be a need for a separate TCAM area for each interface for policy-map counters or netflow stats etc, but there should be a way to share the QOS_TCAM entries when the same QOS_ACL is applied to multiple interfaces and no fancy features are needed.. How do people provision large ACLs on multiple VLANs? In the most efficient usage, the QOS_TCAM contains 4000masks and 32000entries. If these entries have to be split/dedicated to specific interfaces and not shared between interfaces, then if the same ACL is applied to 10 interfaces, the ACL has to be less than 400masks/3200entries. On 100 interfaces, it needs to be less than 40masks/320entries. It would make more sense if these entries could be shared instead of split if I don't need fancy features. And on these machines, 100interfaces is not rare.

Thanks for your time.
Very much appreciated.

Sent from Cisco Technical Support iPad App