'prm_server' detected the 'warning' condition 'TCAM error encountered.

[anas.abdullkarim]

hello , i use ASR9006 with Ios xr 6.6.3

#show platform 
Tue Jun  6 10:51:30.385 AST
Node            Type                      State            Config State
-----------------------------------------------------------------------------
0/RSP0/CPU0     A9K-RSP440-SE(Active)     IOS XR RUN       PWR,NSHUT,MON
0/RSP1/CPU0     A9K-RSP440-SE(Standby)    IOS XR RUN       PWR,NSHUT,MON
0/3/CPU0        A9K-36x10GE-TR            IOS XR RUN       PWR,NSHUT,MON

i have many ACL that applied to interfaces and some use a as PBR , when i try to add some seq to my currnet ACL , i got this error , i remove all unused ACL and make reload to my router and the issues is still

ipv4 access-list ACL-PBR-CORE-BNG-COSTUMERS
 1 permit ipv4 any 10.0.0.0/8
 2 permit ipv4 any host 10.250.250.250
 9990 permit tcp any any eq www nexthop1 ipv4 10.57.80.86
 10001 permit ipv4 any any

ipv4 access-list ACL-PBR-CORE-BNG-COSTUMERS
 701 permit ipv4 any host 1.2.3.4
!!% TCAM error encountered.: Following processes are out of resources
!!% process : pkg/bin/pfilter_ea pid : 532655 node : node0_3_CPU0 rc :'prm_server' detected the 'warning' condition 'TCAM error encountered.'
!
end

[tkarnani · Accepted solution]

Hi,

we will need some more information about this TCAM error.

can you please check "show controller np drvlog location 0/3/cpu0"

Thanks

[anas.abdullkarim]

hi , all log messages like this 

#show controller np drvlog location 0/3/cpu0
p-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 13:28:34.717 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 13:28:34.767 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 13:28:34.810 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 13:28:34.855 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 13:28:34.900 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 13:28:34.947 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 13:28:34.988 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 13:28:35.032 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 13:28:35.074 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 13:28:35.121 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 13:28:35.164 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 13:28:35.211 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 13:28:35.254 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 13:28:35.313 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 13:28:35.355 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 13:28:35.404 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 13:28:35.446 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688

Jun 6 13:33:53.850 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 13:33:53.927 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 13:33:54.010 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 13:33:54.083 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 13:33:54.138 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 13:33:54.186 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 13:33:54.233 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 13:33:54.276 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 13:33:54.319 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 13:33:54.361 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
prmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:10.403 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:10.482 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:10.567 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:10.632 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:10.690 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:10.738 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:10.786 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:10.827 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:10.874 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:10.916 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:10.961 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:11.003 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:11.048 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:11.091 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:11.136 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:11.179 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:11.226 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:11.271 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:11.327 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:11.372 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:11.421 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:11.463 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:11.511 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:11.555 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:11.600 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:11.644 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:11.689 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:11.733 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:11.790 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:11.845 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:11.895 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:11.936 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:11.984 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:12.026 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:12.073 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:12.117 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:12.165 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:12.206 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:12.255 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:12.302 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:17.443 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:17.487 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:17.536 ERROR! 0x80001750 NPprmCAMC_ExtTCAMCommand: command fails due to time out error in file 'drivers/chips/np/npchip-4c/src/host/driver/src/prm/chn/NPprmCAMC.c' line 1325
Jun 6 14:09:17.578 ERROR! 0x80001750 NPapiPrm_ExtTCAMCommand: NPprmCAMC_ExtTCAMCommand failed, channel Id 5. in file 'drivers/chips/np/npchip-4c/src/host/driver/src/api/NPapiPrm.c' line 688
Jun 6 14:09:17.621

[tkarnani · Accepted solution]

This appears to be a hardware fault.

can we re-seat the line card, if this error persists can we replace it with a spare?

thanks

[anas.abdullkarim]

Hello, what linecard? RSP or Internet cardline ? 

[tkarnani]

Line card 3. 0/3/cpu0

[anas.abdullkarim]

This cpu inside RAP or LC?

[anas.abdullkarim]

thanks for supporting , the issues is solved after re-seat cardline 

[MHM Cisco World]

Sorry it ASR 9k not 1k.

[smilstea]

This is not helpful, we are talking about asr9k not asr1k, different hw and sw.

[anas.abdullkarim]

hello, so what do you suggest? reset LineCard of router 

[smilstea]

Tarun already shared the next steps "can we re-seat the line card, if this error persists can we replace it with a spare?". I agree with his recommendation, TCAM lives on the NP of the LC so its a LC problem not an RSP problem. Also you are running unsupported hw/sw combination, the last supported release for RSP440 is 6.4.2, beyond that is not supported or tested.

Sam

[MHM Cisco World]

show prm server tcam summary all all np0 location 0/x/cpu0 <<- check the TCAM free allocation 

[anas.abdullkarim]

RP/0/RSP0/CPU0:ASR9006-TAM-XR#show prm server tcam summary all all np0 location 0/x/cpu0
                                     ^
% Invalid input detected at '^' marker.
RP/0/RSP0/CPU0:ASR9006-TAM-XR#show pfilter-ea fea summary loc 0/3/cpu0
Wed Jun  7 14:42:24.839 AST

******** NP Resource Usage Summary ************

Chan #  160-bit TCAM Entries  640-bit TCAM Entries  Stats   SS Hash Entries
========================================================================
  0           118                    0           114            0
  1           118                    0           114            0
  2           118                    0           114            0
  3           118                    0           114            0
  4           118                    0           114            0
  5           118                    0           114            0