cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
3629
Views
5
Helpful
4
Replies

ASR1006 TCAM limit exceeded.

kzryumov1
Level 1
Level 1

Hi cisco community team,

Recently we started getting a lot of next message:

Line 388168: Apr 22 11:09:50: %CPP_FM-3-CPP_FM_TCAM_WARNING: F1: cpp_sp:  TCAM limit exceeded: HW TCAM cannot hold Class group ess-tc-cg:3311009747.  Use SW TCAM instead.
    Line 388169: Apr 22 11:09:51: %CPP_FM-3-CPP_FM_TCAM_WARNING: F1: cpp_sp:  TCAM limit exceeded: HW TCAM cannot hold Class group ess-tc-cg:4120738916.  Use SW TCAM instead.
    Line 388170: Apr 22 11:09:59: %CPP_FM-3-CPP_FM_TCAM_WARNING: F1: cpp_sp:  TCAM limit exceeded: HW TCAM cannot hold Class group ess-tc-cg:3311009747.  Use SW TCAM instead.
    Line 388171: Apr 22 11:10:04: %CPP_FM-3-CPP_FM_TCAM_WARNING: F1: cpp_sp:  TCAM limit exceeded: HW TCAM cannot hold Class group ess-tc-cg:3311009747.  Use SW TCAM instead.
 

etc.

 

ASR1006-fvpn#show platform hardware qfp active tcam resource-manager usage
QFP TCAM Usage Information

80 Bit Region Information
--------------------------
Name                                : Leaf Region #0
Number of cells per entry           : 1
Current 80 bit entries used         : 0
Current used cell entries           : 0
Current free cell entries           : 0

160 Bit Region Information
--------------------------
Name                                : Leaf Region #1
Number of cells per entry           : 2
Current 160 bits entries used       : 3919
Current used cell entries           : 7838
Current free cell entries           : 4450

320 Bit Region Information
--------------------------
Name                                : Leaf Region #2
Number of cells per entry           : 4
Current 320 bits entries used       : 0
Current used cell entries           : 0
Current free cell entries           : 0


Total TCAM Cell Usage Information
----------------------------------
Name                                : TCAM #0 on CPP #0
Total number of regions             : 3
Total tcam used cell entries        : 7838
Total tcam free cell entries        : 516450
Threshold status                    : below critical limit

 

ASR1006-fvpn#$rm hardware qfp active tcam resource-manager error parity

QFP TCAM Errors


  Total parity errors: 0

 

As I could see there is a lot of free cell entries.

Hardware: ASR1000-RP2; ASR1000-ESP40
IOS XE 154-1.S

 

Please, help me understand what is happening.

 

Regards,

Konstantin

 

4 Replies 4

 

Hello Kostantin,

 

kindly if we can have a look at the following outputs:

show platform software status control-processor brief
show platform hardware qfp active infra exmem statistics
show platform hardware qfp active infra exmem statistics user

 

Once you identify the feature that holds most of the memory (from the above logs it seems TCAM), collect the output from:

show platform hardware qfp active feature tcam.

 

Do you have lots of deny statements in your ACL !? if so you might be hitting CSCtz33305

 

HTH,

 

Michel.

 

Hi Michel,

Here is an outputs.

#show platform software status control-processor brief Load Average  Slot  Status  1-Min  5-Min 15-Min

  RP0 Healthy   0.01   0.04   0.00

 ESP1 Healthy   0.00   0.00   0.00

 SIP0 Healthy   0.01   0.00   0.00

 

Memory (kB)

 Slot  Status    Total     Used (Pct)     Free (Pct) Committed (Pct)

  RP0 Healthy  8097552  4098132 (51%)  3999420 (49%)   5509808 (68%)

 ESP1 Healthy  8097532  1484240 (18%)  6613292 (82%)   1174680 (15%)

 SIP0 Healthy   969052   346592 (36%)   622460 (64%)    604188 (62%)

 

CPU Utilization

 Slot  CPU   User System   Nice   Idle    IRQ   SIRQ IOwait

  RP0    0   6.60   1.40   0.00  91.99   0.00   0.00   0.00

         1   2.00   1.20   0.00  96.60   0.00   0.20   0.00

 ESP1    0   1.29   8.79   0.00  89.81   0.00   0.09   0.00

         1   0.90   0.70   0.00  98.39   0.00   0.00   0.00

 SIP0    0   2.19   0.99   0.00  96.70   0.00   0.09   0.00

 

ASR1006-fvpn#show platform hardware qfp active infra exmem statistics QFP exmem statistics

 

Type: Name: DRAM, QFP: 0

  Total: 1073741824

  InUse: 252996608

  Free: 820745216

  Lowest free water mark: 819333120

Type: Name: IRAM, QFP: 0

  Total: 134217728

  InUse: 8620032

  Free: 125597696

  Lowest free water mark: 125597696

Type: Name: SRAM, QFP: 0

  Total: 32768

  InUse: 14848

  Free: 17920

  Lowest free water mark: 17920

 

ASR1006-fvpn#show platform hardware qfp active infra exmem statistics user

Type: Name: IRAM, QFP: 0

  Allocations  Bytes-Alloc  Bytes-Total  User-Name                              

  -------------------------------------------------------------------------------

  2            230400       231424       CPP_FIA                               

Type: Name: GLOBAL, QFP: 0

  Allocations  Bytes-Alloc  Bytes-Total  User-Name                              

  -------------------------------------------------------------------------------

  7            20112        24576        P/I                                   

  1            16384        16384        FHS                                    

  1            512          1024         FME                                   

  1            4384         5120         EPC                                   

  1            4384         5120         SBC                                   

  1            4            1024         CFT                                   

  1            4            1024         CVLA                                  

  10           279152       284672       CEF                                   

  1            512          1024         B2B HA                                

  1            1138256      1138688      QM RM                                 

  1            65536        65536        Qm 16                                 

  3            15761408     15761408     ING_EGR_UIDB                          

  1            4194304      4194304      TCAM                                  

  41           42319872     42319872     ING EGR INPUT CHUNK_Config_0          

  41           671744       671744       ING EGR INPUT CHUNK_Sm_Name_0         

  1            32768        32768        ING EGR INPUT CHUNK_Lg_Name_0         

  41           37617664     37617664     ING EGR OUTPUT CHUNK_Config_0         

  41           671744       671744       ING EGR OUTPUT CHUNK_Sm_Name_0        

  1            32768        32768        ING EGR OUTPUT CHUNK_Lg_Name_0        

  1            16384        16384        ING EGR OUTPUT CHUNK_Queue_0          

  41           671744       671744       ING-EGR_IfMap_0                        

  15           116754       128000       CACE                                  

  3481         22459776     24951808     GIC                                   

  1            1048576      1048576      PLU Mgr_CEF_0_0                        

  10           16816620     16823296     FNF                                   

  2            2097152      2097152      PLU Mgr_CEF_0_3                       

  2            3145728      3145728      PLU Mgr_CEF_0_8                       

  3            3145728      3145728      PLU Mgr_CEF_0_9                       

  1            1048576      1048576      PLU Mgr_PLU_GLOBAL_0_0                

  1            1048576      1048576      PLU Mgr_PLU_GLOBAL_0_1                

  1            786432       786432       PLU Mgr_PLU_GLOBAL_0_2                

  1            1048576      1048576      PLU Mgr_PLU_GLOBAL_0_3                

Type: Name: GLOBAL, QFP: 0

  Allocations  Bytes-Alloc  Bytes-Total  User-Name                             

  -------------------------------------------------------------------------------

  1            1310720      1310720      PLU Mgr_PLU_GLOBAL_0_4                

  1            786432       786432       PLU Mgr_PLU_GLOBAL_0_5                

  1            917504       917504       PLU Mgr_PLU_GLOBAL_0_6                

  1            1048576      1048576      PLU Mgr_PLU_GLOBAL_0_7                

  1            1572864      1572864      PLU Mgr_PLU_GLOBAL_0_8                

  1            1048576      1048576      PLU Mgr_PLU_GLOBAL_0_9                

  1            1310720      1310720      PLU Mgr_PLU_GLOBAL_0_10               

  1            1572864      1572864      PLU Mgr_PLU_GLOBAL_0_11               

  1            1835008      1835008      PLU Mgr_PLU_GLOBAL_0_12               

  1            1048576      1048576      PLU Mgr_PLU_GLOBAL_0_13               

  1            1310720      1310720      PLU Mgr_PLU_GLOBAL_0_14               

  1            1572864      1572864      PLU Mgr_PLU_GLOBAL_0_15               

  1            917504       917504       PLU Mgr_PLU_GLOBAL_0_16               

  1            1048576      1048576      PLU Mgr_PLU_GLOBAL_0_17               

  1            1179648      1179648      PLU Mgr_PLU_GLOBAL_0_18               

  11           180224       180224       Unprotected CEF Exmem                 

  4            34772        37888        SSLVPN                                

  1            16           1024         cpp_epc_sbs_client                     

  9            1467384      1474560      BFD                                   

  3            4400         7168         LI                                    

  1            64           1024         cpp_li_sbs_client                     

  3            53248        53248        SC                                    

  1            16384        16384        CPP SC VRF TABLE CHUNK                

  1            16384        16384        CPP SC VRFNAME TABLE CHUNK            

  1            4096         4096         SMI                                   

  1            40           1024         cpp_smi_sbs_client                    

  1            6528         7168         cpp_pbr_sbs_client                    

  2            8000         8192         TFC                                   

  4            1024000      1024000      TUNNEL                                

  1            4384         5120         ERSPAN                                

  1            112          1024         cpp_erspan_sbs_client                 

  12           3756352      3759104      ESS                                   

Type: Name: GLOBAL, QFP: 0

  Allocations  Bytes-Alloc  Bytes-Total  User-Name                             

  -------------------------------------------------------------------------------

  11           360448       360448       Subscriber switch encap info          

  11           540672       540672       Subscriber switch encaps chunk        

  3            393216       393216       Subscriber drop config block          

  2            32           2048         ICMP                                  

  42           1344000      1376256      cpp_icmp_sb_chunk                     

  1            32000        32768        cpp_icmp6_sb_chunk                    

  1            4096         4096         SPAMARMOT                             

  2            65536        65536        ethernet                              

  2            181504       182272       PALCI CLIENT                          

  205          1327312      1459200      cpp_punt_sbs_client                   

  1            320          1024         punt path chunk 0                     

  42           1344000      1376256      punt subblock chunk                   

  32           12288        32768        punt policer chunk                    

  22           719476       736256       PKTLOG                                

  1            512          1024         queue info chunk 0                     

  1            16           1024         CPP IPHC                              

  7            1286432      1288192      IPFRAG                                

  11           176000       180224       cpp_ipfrag_sb_chunk                    

  10           26048        34816        cpp_ipfrag_sbs_client                 

  11           528000       529408       cpp_ipreass_sb_chunk                  

  11           176000       180224       cpp_ipreass_cur_dgram_cnt_chunk       

  1            96000        96256        cpp_ipv6reass_sb_chunk                

  26           169728       186368       sbs_cef                               

  1629         7686528      8369152      cpp_ess_sbs_client                    

  3            393216       393216       ESF TC SB                             

  3            1572864      1572864      ESF TC ISD ENTRY                      

  1            960          1024         CPP ACL SBS Client                    

  2            524288       524288       L2TP HASH ELEM                        

  2            262144       262144       L2TP CXT SB                           

  2            262144       262144       L2TP TX SEQ ELEM                      

  2            1310720      1310720      Subscriber LTERM ctx info             

  4            524288       524288       ESF Subscriber CXT SB                 

Type: Name: GLOBAL, QFP: 0

  Allocations  Bytes-Alloc  Bytes-Total  User-Name                             

  -------------------------------------------------------------------------------

  8            1048576      1048576      ESF Policing HW WORD                  

  4            524288       524288       ESF Policing CXT SB                   

  1            163840       163840       ESF L4R CXT SB                        

  1            32768        32768        ESF L4R TRANS MGR SB                  

  1            262144       262144       ESF L4R 32-ENTRY HASH SB CHUNK        

  1            768          1024         ESF L4R RULE SB                       

  1            1048992      1049600      ESF L4R TRANS POOL                    

  1            1573280      1573888      ESF L4R IP6 TRANS POOL                

  1            131072       131072       IPSUB DOWNSTREAM SESSION              

  1            393216       393216       Subscriber LITE VSI CTX ARRAY         

 

There aren't a lot of deny statement.

 

Regards,

Konstantin

 

Please do let me know if you have any solution for this

We're working on a customer facing document explaining how TCAM is being used in ASR1000.

Meanwhile, let me address this concern here.

On ASR1000 TCAM is used as a resource for all classification clients, including ACLs, QoS, etc...

Since XE3.10, when the system runs out of hardware TCAM, it will automatically switch to software TCAM.

The HW TCAM can be monitored using the below CLIs:

#show platform resources

#show platform hardware qfp active tcam resource-manager usage

The DRAM memory used by SW TCAM is shown under CACE component. 

# show platform hardware qfp active infrastructure exmem statistics user | inc  CACE

 

In IOS-XE 3.10 onwards, when a large policy/ACL needs to be programmed and it does not fit into HW TCAM then the system will automatically move that policy/ACL to SW TCAM instead of rejecting the policy.

 

The below warning message will be generated:

CPP_FM-3-CPP_FM_TCAM_WARNING F0: cpp_sp:  TCAM limit exceeded: HW TCAM cannot hold Class group <xxxxxxxx>. Use SW TCAM instead.

There should be no service disruption, aside from the fact that HW TCAM will not be used (a potential performance impact).

HTH,

Michal

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: