04-22-2014 01:26 AM
Hi cisco community team,
Recently we started getting a lot of next message:
Line 388168: Apr 22 11:09:50: %CPP_FM-3-CPP_FM_TCAM_WARNING: F1: cpp_sp: TCAM limit exceeded: HW TCAM cannot hold Class group ess-tc-cg:3311009747. Use SW TCAM instead.
Line 388169: Apr 22 11:09:51: %CPP_FM-3-CPP_FM_TCAM_WARNING: F1: cpp_sp: TCAM limit exceeded: HW TCAM cannot hold Class group ess-tc-cg:4120738916. Use SW TCAM instead.
Line 388170: Apr 22 11:09:59: %CPP_FM-3-CPP_FM_TCAM_WARNING: F1: cpp_sp: TCAM limit exceeded: HW TCAM cannot hold Class group ess-tc-cg:3311009747. Use SW TCAM instead.
Line 388171: Apr 22 11:10:04: %CPP_FM-3-CPP_FM_TCAM_WARNING: F1: cpp_sp: TCAM limit exceeded: HW TCAM cannot hold Class group ess-tc-cg:3311009747. Use SW TCAM instead.
etc.
ASR1006-fvpn#show platform hardware qfp active tcam resource-manager usage
QFP TCAM Usage Information
80 Bit Region Information
--------------------------
Name : Leaf Region #0
Number of cells per entry : 1
Current 80 bit entries used : 0
Current used cell entries : 0
Current free cell entries : 0
160 Bit Region Information
--------------------------
Name : Leaf Region #1
Number of cells per entry : 2
Current 160 bits entries used : 3919
Current used cell entries : 7838
Current free cell entries : 4450
320 Bit Region Information
--------------------------
Name : Leaf Region #2
Number of cells per entry : 4
Current 320 bits entries used : 0
Current used cell entries : 0
Current free cell entries : 0
Total TCAM Cell Usage Information
----------------------------------
Name : TCAM #0 on CPP #0
Total number of regions : 3
Total tcam used cell entries : 7838
Total tcam free cell entries : 516450
Threshold status : below critical limit
ASR1006-fvpn#$rm hardware qfp active tcam resource-manager error parity
QFP TCAM Errors
Total parity errors: 0
As I could see there is a lot of free cell entries.
Hardware: ASR1000-RP2; ASR1000-ESP40
IOS XE 154-1.S
Please, help me understand what is happening.
Regards,
Konstantin
04-23-2014 05:34 AM
Hello Kostantin,
kindly if we can have a look at the following outputs:
show platform software status control-processor brief
show platform hardware qfp active infra exmem statistics
show platform hardware qfp active infra exmem statistics user
Once you identify the feature that holds most of the memory (from the above logs it seems TCAM), collect the output from:
show platform hardware qfp active feature tcam.
Do you have lots of deny statements in your ACL !? if so you might be hitting CSCtz33305
HTH,
Michel.
04-23-2014 11:12 PM
Hi Michel,
Here is an outputs.
#show platform software status control-processor brief Load Average Slot Status 1-Min 5-Min 15-Min
RP0 Healthy 0.01 0.04 0.00
ESP1 Healthy 0.00 0.00 0.00
SIP0 Healthy 0.01 0.00 0.00
Memory (kB)
Slot Status Total Used (Pct) Free (Pct) Committed (Pct)
RP0 Healthy 8097552 4098132 (51%) 3999420 (49%) 5509808 (68%)
ESP1 Healthy 8097532 1484240 (18%) 6613292 (82%) 1174680 (15%)
SIP0 Healthy 969052 346592 (36%) 622460 (64%) 604188 (62%)
CPU Utilization
Slot CPU User System Nice Idle IRQ SIRQ IOwait
RP0 0 6.60 1.40 0.00 91.99 0.00 0.00 0.00
1 2.00 1.20 0.00 96.60 0.00 0.20 0.00
ESP1 0 1.29 8.79 0.00 89.81 0.00 0.09 0.00
1 0.90 0.70 0.00 98.39 0.00 0.00 0.00
SIP0 0 2.19 0.99 0.00 96.70 0.00 0.09 0.00
ASR1006-fvpn#show platform hardware qfp active infra exmem statistics QFP exmem statistics
Type: Name: DRAM, QFP: 0
Total: 1073741824
InUse: 252996608
Free: 820745216
Lowest free water mark: 819333120
Type: Name: IRAM, QFP: 0
Total: 134217728
InUse: 8620032
Free: 125597696
Lowest free water mark: 125597696
Type: Name: SRAM, QFP: 0
Total: 32768
InUse: 14848
Free: 17920
Lowest free water mark: 17920
ASR1006-fvpn#show platform hardware qfp active infra exmem statistics user
Type: Name: IRAM, QFP: 0
Allocations Bytes-Alloc Bytes-Total User-Name
-------------------------------------------------------------------------------
2 230400 231424 CPP_FIA
Type: Name: GLOBAL, QFP: 0
Allocations Bytes-Alloc Bytes-Total User-Name
-------------------------------------------------------------------------------
7 20112 24576 P/I
1 16384 16384 FHS
1 512 1024 FME
1 4384 5120 EPC
1 4384 5120 SBC
1 4 1024 CFT
1 4 1024 CVLA
10 279152 284672 CEF
1 512 1024 B2B HA
1 1138256 1138688 QM RM
1 65536 65536 Qm 16
3 15761408 15761408 ING_EGR_UIDB
1 4194304 4194304 TCAM
41 42319872 42319872 ING EGR INPUT CHUNK_Config_0
41 671744 671744 ING EGR INPUT CHUNK_Sm_Name_0
1 32768 32768 ING EGR INPUT CHUNK_Lg_Name_0
41 37617664 37617664 ING EGR OUTPUT CHUNK_Config_0
41 671744 671744 ING EGR OUTPUT CHUNK_Sm_Name_0
1 32768 32768 ING EGR OUTPUT CHUNK_Lg_Name_0
1 16384 16384 ING EGR OUTPUT CHUNK_Queue_0
41 671744 671744 ING-EGR_IfMap_0
15 116754 128000 CACE
3481 22459776 24951808 GIC
1 1048576 1048576 PLU Mgr_CEF_0_0
10 16816620 16823296 FNF
2 2097152 2097152 PLU Mgr_CEF_0_3
2 3145728 3145728 PLU Mgr_CEF_0_8
3 3145728 3145728 PLU Mgr_CEF_0_9
1 1048576 1048576 PLU Mgr_PLU_GLOBAL_0_0
1 1048576 1048576 PLU Mgr_PLU_GLOBAL_0_1
1 786432 786432 PLU Mgr_PLU_GLOBAL_0_2
1 1048576 1048576 PLU Mgr_PLU_GLOBAL_0_3
Type: Name: GLOBAL, QFP: 0
Allocations Bytes-Alloc Bytes-Total User-Name
-------------------------------------------------------------------------------
1 1310720 1310720 PLU Mgr_PLU_GLOBAL_0_4
1 786432 786432 PLU Mgr_PLU_GLOBAL_0_5
1 917504 917504 PLU Mgr_PLU_GLOBAL_0_6
1 1048576 1048576 PLU Mgr_PLU_GLOBAL_0_7
1 1572864 1572864 PLU Mgr_PLU_GLOBAL_0_8
1 1048576 1048576 PLU Mgr_PLU_GLOBAL_0_9
1 1310720 1310720 PLU Mgr_PLU_GLOBAL_0_10
1 1572864 1572864 PLU Mgr_PLU_GLOBAL_0_11
1 1835008 1835008 PLU Mgr_PLU_GLOBAL_0_12
1 1048576 1048576 PLU Mgr_PLU_GLOBAL_0_13
1 1310720 1310720 PLU Mgr_PLU_GLOBAL_0_14
1 1572864 1572864 PLU Mgr_PLU_GLOBAL_0_15
1 917504 917504 PLU Mgr_PLU_GLOBAL_0_16
1 1048576 1048576 PLU Mgr_PLU_GLOBAL_0_17
1 1179648 1179648 PLU Mgr_PLU_GLOBAL_0_18
11 180224 180224 Unprotected CEF Exmem
4 34772 37888 SSLVPN
1 16 1024 cpp_epc_sbs_client
9 1467384 1474560 BFD
3 4400 7168 LI
1 64 1024 cpp_li_sbs_client
3 53248 53248 SC
1 16384 16384 CPP SC VRF TABLE CHUNK
1 16384 16384 CPP SC VRFNAME TABLE CHUNK
1 4096 4096 SMI
1 40 1024 cpp_smi_sbs_client
1 6528 7168 cpp_pbr_sbs_client
2 8000 8192 TFC
4 1024000 1024000 TUNNEL
1 4384 5120 ERSPAN
1 112 1024 cpp_erspan_sbs_client
12 3756352 3759104 ESS
Type: Name: GLOBAL, QFP: 0
Allocations Bytes-Alloc Bytes-Total User-Name
-------------------------------------------------------------------------------
11 360448 360448 Subscriber switch encap info
11 540672 540672 Subscriber switch encaps chunk
3 393216 393216 Subscriber drop config block
2 32 2048 ICMP
42 1344000 1376256 cpp_icmp_sb_chunk
1 32000 32768 cpp_icmp6_sb_chunk
1 4096 4096 SPAMARMOT
2 65536 65536 ethernet
2 181504 182272 PALCI CLIENT
205 1327312 1459200 cpp_punt_sbs_client
1 320 1024 punt path chunk 0
42 1344000 1376256 punt subblock chunk
32 12288 32768 punt policer chunk
22 719476 736256 PKTLOG
1 512 1024 queue info chunk 0
1 16 1024 CPP IPHC
7 1286432 1288192 IPFRAG
11 176000 180224 cpp_ipfrag_sb_chunk
10 26048 34816 cpp_ipfrag_sbs_client
11 528000 529408 cpp_ipreass_sb_chunk
11 176000 180224 cpp_ipreass_cur_dgram_cnt_chunk
1 96000 96256 cpp_ipv6reass_sb_chunk
26 169728 186368 sbs_cef
1629 7686528 8369152 cpp_ess_sbs_client
3 393216 393216 ESF TC SB
3 1572864 1572864 ESF TC ISD ENTRY
1 960 1024 CPP ACL SBS Client
2 524288 524288 L2TP HASH ELEM
2 262144 262144 L2TP CXT SB
2 262144 262144 L2TP TX SEQ ELEM
2 1310720 1310720 Subscriber LTERM ctx info
4 524288 524288 ESF Subscriber CXT SB
Type: Name: GLOBAL, QFP: 0
Allocations Bytes-Alloc Bytes-Total User-Name
-------------------------------------------------------------------------------
8 1048576 1048576 ESF Policing HW WORD
4 524288 524288 ESF Policing CXT SB
1 163840 163840 ESF L4R CXT SB
1 32768 32768 ESF L4R TRANS MGR SB
1 262144 262144 ESF L4R 32-ENTRY HASH SB CHUNK
1 768 1024 ESF L4R RULE SB
1 1048992 1049600 ESF L4R TRANS POOL
1 1573280 1573888 ESF L4R IP6 TRANS POOL
1 131072 131072 IPSUB DOWNSTREAM SESSION
1 393216 393216 Subscriber LITE VSI CTX ARRAY
There aren't a lot of deny statement.
Regards,
Konstantin
03-02-2017 03:11 AM
Please do let me know if you have any solution for this
03-02-2017 08:15 AM
We're working on a customer facing document explaining how TCAM is being used in ASR1000.
Meanwhile, let me address this concern here.
On ASR1000 TCAM is used as a resource for all classification clients, including ACLs, QoS, etc...
Since XE3.10, when the system runs out of hardware TCAM, it will automatically switch to software TCAM.
The HW TCAM can be monitored using the below CLIs:
#show platform resources
#show platform hardware qfp active tcam resource-manager usage
The DRAM memory used by SW TCAM is shown under CACE component.
# show platform hardware qfp active infrastructure exmem statistics user | inc CACE
In IOS-XE 3.10 onwards, when a large policy/ACL needs to be programmed and it does not fit into HW TCAM then the system will automatically move that policy/ACL to SW TCAM instead of rejecting the policy.
The below warning message will be generated:
CPP_FM-3-CPP_FM_TCAM_WARNING F0: cpp_sp: TCAM limit exceeded: HW TCAM cannot hold Class group <xxxxxxxx>. Use SW TCAM instead.
There should be no service disruption, aside from the fact that HW TCAM will not be used (a potential performance impact).
HTH,
Michal
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide