01-07-2020 12:02 AM
how to verify TCAM ASR9K IOS-XR ?
please help me
01-07-2020 10:43 AM
What do you want to verify?
How many TCAM entries an ACE/ACL uses?
How many used / free TCAM entries there are?
Due to bit-boundaries and what ACL options are used a particular ACE may use multiple TCAM entries
1 ACE without any ranges (TCP ports, UDP ports, ttl) will typicall map to 1 TCAM entry.
ACEs with ranges occupy multiple TCAM entries
Note: 1 ACL cannot have more than 64K TCAM entries
Using bit boundaries helps to minimize the number of TCAM entries needed:
For example matching ports ranged 0 - 1023 would take a single TCAM entry
However, if you needed to match 0-1024, that would be two rules:
As well interfaces on the same NP having the same ACL applied in the same direction share TCAM entries, this helps minimize the TCAM space that is used.
What affects the ACL scale for a LC?
show pfilter-ea fea ipv4-acl <ACL> loc <loc>
shows you how many ACEs, how many TCAM entries, and TCAM entries per ACE (must be applied to see)
ex:
ACE List for This Region:
seq_num 94, tcam_entries 2, stats_ace_id 0x535880 (0x63c400) new 0
seq_num 95, tcam_entries 2, stats_ace_id 0x535881 (0x63c408) new 0
seq_num 96, tcam_entries 1, stats_ace_id 0x535882 (0x63c410) new 0
seq_num 97, tcam_entries 1, stats_ace_id 0x535883 (0x63c418) new 0
seq_num 98, tcam_entries 1, stats_ace_id 0x535884 (0x63c420) new 0
seq_num 99, tcam_entries 1, stats_ace_id 0x535885 (0x63c428) new 0
seq_num 100, tcam_entries 7, stats_ace_id 0x535886 (0x63c430) new 0
seq_num 101, tcam_entries 2, stats_ace_id 0x535887 (0x63c438) new 0
seq_num 102, tcam_entries 2, stats_ace_id 0x535888 (0x63c440) new 0
seq_num 103, tcam_entries 2, stats_ace_id 0x535889 (0x63c448) new 0
seq_num 104, tcam_entries 2, stats_ace_id 0x53588a (0x63c450) new 0
seq_num 105, tcam_entries 2, stats_ace_id 0x53588b (0x63c458) new 0
seq_num 106, tcam_entries 1, stats_ace_id 0x53588c (0x63c460) new 0
seq_num 107, tcam_entries 2, stats_ace_id 0x53588d (0x63c468) new 0
seq_num 108, tcam_entries 16, stats_ace_id 0x53588e (0x63c470) new 0
how pfilter-ea fea summary loc {location}
shows how many total ACEs/TCAM entries/stats counters are used on the linecard (per NP, where NP=“chan#”)
RP/0/RSP0/CPU0:ASR9006-E#show pfilter-ea fea summary loc 0/1/cpu0
Tue Jun 24 20:17:22.041 UTC
******** NP Resource Usage Summary ************
Chan # 144-bit TCAM Entries 576-bit TCAM Entries Stats SS Hash Entries
========================================================================
0 36 20 56 0
1 0 0 0 0
2 0 0 0 0
3 0 0 0 0
4 139 0 70 0
5 0 0 0 0
6 0 0 0 0
7 0 0 0 0
Here we have the number of TCAM entries used (139 out of 24k) and 70 ACE entries for NP 4
show access-lists ipv4 <acl> hardware [ingress | egress] resource-usage loc {location}
shows compiled ACL hardware stats (TCAM, compression, etc)
RP/0/RSP0/CPU0:ASR9006-E#show access-lists ipv4 test-acl hardware egress resource-usage loc 0/1/cpu0
Tue Jun 24 20:18:30.946 UTC
NP : 4
Rules (ACE) : 70
ACL compression level : 0
Fields compressed : None
TCAM Entries used : 139 ( 24k total)
TCAM Key Width : 160 ( 0 total for compressed fields)
show prm server tcam summary all [ACL | AFMON | IFIB | LI | PBR | QOS | all] [all | np0 | npx] location {location}
shows TCAM utilization for the application(s) specified
RP/0/RSP0/CPU0:ASR9006-E#show prm server tcam summary all all np4 loc 0/1/cpu0
Wed Jun 25 16:07:18.618 UTC
Node: 0/1/CPU0:
----------------------------------------------------------------
TCAM summary for NP4:
TCAM Logical Table: TCAM_LT_L2 (1)
Partition ID: 0, priority: 2, valid entries: 3, free entries: 317
Partition ID: 1, priority: 2, valid entries: 0, free entries: 320
Partition ID: 2, priority: 1, valid entries: 0, free entries: 320
Partition ID: 3, priority: 1, valid entries: 0, free entries: 11840
Partition ID: 4, priority: 0, valid entries: 76, free entries: 11700
TCAM Logical Table: TCAM_LT_ODS2 (2), free entries: 16304, resvd 128
ACL Common Region: 0 entries allocated. 0 entries free
Application ID: NP_APP_ID_IFIB_IPV4 (0)
Total: 1 vmr_ids, 8005 active entries, 8005 allocated entries.
Application ID: NP_APP_ID_QOS (1)
Total: 0 vmr_ids, 0 active entries, 0 allocated entries.
Application ID: NP_APP_ID_IPV4_ACL (2)
Total: 1 vmr_ids, 139 active entries, 139 allocated entries.
Application ID: NP_APP_ID_AFMON (3)
Total: 0 vmr_ids, 0 active entries, 0 allocated entries.
Application ID: (null) (4)
Total: 0 vmr_ids, 0 active entries, 0 allocated entries.
Application ID: NP_APP_ID_PBR (5)
Total: 0 vmr_ids, 0 active entries, 0 allocated entries.
TCAM Logical Table: TCAM_LT_ODS8 (3), free entries: 1929, resvd 64
ACL Common Region: 0 entries allocated. 0 entries free
Application ID: NP_APP_ID_IFIB_IPV6 (0)
Total: 1 vmr_ids, 2103 active entries, 2103 allocated entries.
Application ID: NP_APP_ID_QOS_IPV6 (1)
Total: 0 vmr_ids, 0 active entries, 0 allocated entries.
Application ID: NP_APP_ID_ACL_IPV6 (2)
Total: 0 vmr_ids, 0 active entries, 0 allocated entries.
Application ID: (null) (3)
Total: 0 vmr_ids, 0 active entries, 0 allocated entries.
Application ID: (null) (4)
Total: 0 vmr_ids, 0 active entries, 0 allocated entries.
Application ID: NP_APP_ID_PBR_IPV6 (5)
Total: 0 vmr_ids, 0 active entries, 0 allocated entries.
As we have seen before the NP is using 139 TCAM entries for ACLs
We also have 8005 IFIB TCAM entries
We are using 8144 entries on the local table TCAM_LT_ODS2 and have 16304 entries free (24K for -TR cards)
We also get the number of vmr_ids which is essentially how many unique ACLs are applied in the TCAM on the NP.
Sam
03-08-2023 02:47 AM
Hi Sam,
Thank you for the insights, I know this is an old thread, but I have some addition to add for future reference and also some questions someone hopefully can answer.
You can view the actual TCAM entries that were generated for your ACL, for example with this ACL:
ipv4 access-list tcam-test
10 permit ipv4 any host 10.0.0.254
20 permit tcp 10.56.108.0 0.0.0.255 host 10.0.230.210 eq cmd
30 permit tcp 10.56.108.0 0.0.0.255 host 10.0.230.210 range 1514 1515
40 permit ipv4 10.56.108.0 0.0.0.255 host 10.165.130.2
50 deny ipv4 any 10.0.0.0 0.255.255.255
60 deny ipv4 any 192.168.0.0 0.0.255.255
70 permit ipv4 any any nexthop1 vrf tcamtestvrf ipv4 10.56.108.1
100 deny ipv4 any any
You can get the vmr-id of the ACL via the pfilter-ea fea command which is missing from the previous excerpt:
RP/0/RSP0/CPU0:ASR9001-TCAM#show pfilter-ea fea ipv4-acl tcam-test location 0/0/CPU0
Rgn tcam-test, lkup v4, Dir In, Chan 0, acl_id 3, vmr_id 3, num_aces 8, num_tcam_entries 11, refcnt 1.
...
You can view how many active entries the specifies vmr_id takes up in tcam:
RP/0/RSP0/CPU0:ASR9001-TCAM#show prm server tcam summary all all detail np0 location 0/0/CPU0
Node: 0/0/CPU0:
----------------------------------------------------------------
TCAM summary for NP0:
TCAM Logical Table: TCAM_LT_L2 (1)
Partition ID: 0, priority: 2, valid entries: 7, free entries: 2041
Partition ID: 1, priority: 2, valid entries: 0, free entries: 2048
Partition ID: 2, priority: 0, valid entries: 0, free entries: 2048
Partition ID: 3, priority: 0, valid entries: 6, free entries: 24570
Partition ID: 4, priority: 0, valid entries: 3, free entries: 67581
TCAM Logical Table: TCAM_LT_ODS2 (2), free entries: 89239, resvd 128
ACL Common Region: 448 entries allocated. 448 entries free
Application ID: NP_APP_ID_IFIB (0)
VMR ID: 1, active entries: 8005, total entries: 8005
Total: 1 vmr_ids, 8005 active entries, 8005 allocated entries.
Application ID: NP_APP_ID_QOS (1)
VMR ID: 21, active entries: 21, total entries: 21
Total: 1 vmr_ids, 21 active entries, 21 allocated entries.
Application ID: NP_APP_ID_ACL (2)
VMR ID: 3, active entries: 11, total entries: 11
VMR ID: 4, active entries: 7, total entries: 7
VMR ID: 6, active entries: 23, total entries: 23
VMR ID: 8, active entries: 23, total entries: 23
VMR ID: 9, active entries: 210, total entries: 210
VMR ID: 16, active entries: 94, total entries: 94
VMR ID: 19, active entries: 94, total entries: 94
Total: 7 vmr_ids, 463 active entries, 463 allocated entries.
Application ID: NP_APP_ID_AFMON (3)
Total: 0 vmr_ids, 0 active entries, 0 allocated entries.
Application ID: NP_APP_ID_LI (4)
Total: 0 vmr_ids, 0 active entries, 0 allocated entries.
Application ID: NP_APP_ID_PBR (5)
Total: 0 vmr_ids, 0 active entries, 0 allocated entries.
TCAM Logical Table: TCAM_LT_ODS8 (3), free entries: 15113, resvd 123
ACL Common Region: 448 entries allocated. 448 entries free
Application ID: NP_APP_ID_IFIB (0)
VMR ID: 1, active entries: 603, total entries: 603
Total: 1 vmr_ids, 603 active entries, 603 allocated entries.
Application ID: NP_APP_ID_QOS (1)
Total: 0 vmr_ids, 0 active entries, 0 allocated entries.
Application ID: NP_APP_ID_ACL (2)
VMR ID: 2, active entries: 22, total entries: 22
VMR ID: 5, active entries: 34, total entries: 34
VMR ID: 6, active entries: 34, total entries: 34
Total: 3 vmr_ids, 90 active entries, 90 allocated entries.
Application ID: NP_APP_ID_LI (4)
Total: 0 vmr_ids, 0 active entries, 0 allocated entries.
Application ID: NP_APP_ID_PBR (5)
VMR ID: 7, active entries: 6, total entries: 6
VMR ID: 8, active entries: 1, total entries: 1
And after that you can view the specific ACL in TCAM by the vmr_id:
RP/0/RSP0/CPU0:ASR9001-TCAM#show prm server tcam entries 144-LT vmr-id 3 100 np0 location 0/0/CPU0
Node: 0/0/CPU0:
----------------------------------------------------------------
ODS NP: 0, LT: 2, AppId: 2, VmrId 3, Offset 0, Entries 12, Shadow 0
5df02 V: 400c 0040 0000 0000 0000 0000 0000 0000 fe00 000a 11
M: 0003 ffbf ffff ffff ffff ffff ffff ffff 0000 0000
R: 110a0000 00000003 28126100 00000000 00000000 00000000 00000000 00000000
5df04 V: 400c 0041 0000 0000 0000 0202 006c 380a d2e6 000a 11
M: 0003 ffbc ffff ffff ffff 0000 ff00 0000 0000 0000
R: 110a0000 00000003 b05d6300 00000000 00000000 00000000 00000000 00000000
5df4a V: 400c 0043 0000 0000 0000 0000 006c 380a d2e6 000a 11
M: 0003 ffbc ffff ffff ffff ffff ff00 0000 0000 0000
R: 110a0000 00000003 b05d6300 00000000 00000000 00000000 00000000 00000000
5df4c V: 400c 0041 0000 0000 0000 05ea 006c 380a d2e6 000a 11
M: 0003 ffbc ffff ffff ffff 0001 ff00 0000 0000 0000
R: 110a0000 00000003 c0156e00 00000000 00000000 00000000 00000000 00000000
5df90 V: 400c 0043 0000 0000 0000 0000 006c 380a d2e6 000a 11
M: 0003 ffbc ffff ffff ffff ffff ff00 0000 0000 0000
R: 110a0000 00000003 c0156e00 00000000 00000000 00000000 00000000 00000000
5e302 V: 400c 0040 0000 0000 0000 0000 006c 380a 0282 a50a 11
M: 0003 ffbf ffff ffff ffff ffff ff00 0000 0000 0000
R: 110a0000 00000003 30126100 00000000 00000000 00000000 00000000 00000000
5eb02 V: 400c 0040 0000 0000 0000 0000 0000 0000 0000 000a 11
M: 0003 ffbf ffff ffff ffff ffff ffff ffff ffff ff00
R: 111a0000 00000003 40126100 00000000 00000000 00000000 00000000 00000000
5ef02 V: 400c 0040 0000 0000 0000 0000 0000 0000 0000 a8c0 11
M: 0003 ffbf ffff ffff ffff ffff ffff ffff ffff 0000
R: 111a0000 00000003 48126100 00000000 00000000 00000000 00000000 00000000
5f302 V: 400c 0040 0000 0000 0000 0000 0000 0000 0000 0000 11
M: 0003 ffbf ffff ffff ffff ffff ffff ffff ffff ffff
R: 112a0100 11000003 50126100 016c380a 00000000 00000000 00000000 00000000
5f304 V: 400c 0040 0000 0000 0000 0000 0000 0000 0000 0000 11
M: 0003 ffbf ffff ffff ffff ffff ffff ffff ffff ffff
R: 111a0000 00000003 58126100 00000000 00000000 00000000 00000000 00000000
5fb02 V: 400c 0040 0000 0000 0000 0000 0000 0000 0000 0000 11
M: 0003 ffbf ffff ffff ffff ffff ffff ffff ffff ffff
R: 111a0000 00000003 60126100 00000000 00000000 00000000 00000000 00000000
Lets see this line for example:
5df4c V: 400c 0041 0000 0000 0000 05ea 006c 380a d2e6 000a 11
M: 0003 ffbc ffff ffff ffff 0001 ff00 0000 0000 0000
R: 110a0000 00000003 c0156e00 00000000 00000000 00000000 00000000 00000000
Not sure about the other fields but this is enough to identify rules.
Now for the weird thing and my question... as you can see for the TCP permit rules, it actually takes up 2 TCAM entry spaces... one with the port information and one without, see here:
5df4c V: 400c 0041 0000 0000 0000 05ea 006c 380a d2e6 000a 11
M: 0003 ffbc ffff ffff ffff 0001 ff00 0000 0000 0000
R: 110a0000 00000003 c0156e00 00000000 00000000 00000000 00000000 00000000
5df90 V: 400c 0043 0000 0000 0000 0000 006c 380a d2e6 000a 11
M: 0003 ffbc ffff ffff ffff ffff ff00 0000 0000 0000
R: 110a0000 00000003 c0156e00 00000000 00000000 00000000 00000000 00000000
this is the same for UDP permit rules as well... regards to the doubling of permit rules
I did some testing and for deny rules the case as you would suppose, only one entry with the ports assigned correctly...
So here is my question, why would permit rules with a single port or ports which can be matched by a single rule use two TCAM slots? Model is an ASR9001.
03-08-2023 09:22 AM
We have two entries for the sequence numbers that specify a source and destination instead of any, for the other permit statements they take up 1 entry such as sequence 10. You also don't necessarily need a deny any any at the end as there is an implicit deny any any applied.
RP/0/RSP0/CPU0:ASR-9001-B#show pfilter-ea fea ipv4-acl tcam-test loc 0/0/cPU0
Wed Mar 8 17:18:54.876 UTC
Rgn tcam-test, lkup v4, Dir In, Chan 1, acl_id 2, vmr_id 3, num_aces 9, num_tcam_entries 11, refcnt 1.
ACE List for This Region:
seq_num 10,tcam_entries 1,stats_ace_id 0x531760 (0x61bb00) new 0
seq_num 20,tcam_entries 2,stats_ace_id 0x531761 (0x61bb08) new 0
seq_num 30,tcam_entries 2,stats_ace_id 0x531762 (0x61bb10) new 0
seq_num 40,tcam_entries 1,stats_ace_id 0x531763 (0x61bb18) new 0
seq_num 50,tcam_entries 1,stats_ace_id 0x531764 (0x61bb20) new 0
seq_num 60,tcam_entries 1,stats_ace_id 0x531765 (0x61bb28) new 0
seq_num 70,tcam_entries 1,stats_ace_id 0x531766 (0x61bb30) new 0
seq_num 100,tcam_entries 1,stats_ace_id 0x531767 (0x61bb38) new 0
seq_num 2147483647,tcam_entries 1,stats_ace_id 0x531768 (0x61bb40) new 0
Intf List for This Region:
Te0/0/2/3, hw_count 0.
Sam
03-08-2023 09:40 AM - edited 03-08-2023 09:41 AM
Hi Sam,
thank you for the reply, it is true, that the deny any any at the end is not needed. I am just wondering why the double entry is needed for seq 20 and 30.... seq 40 also specifies a source and a destination and still uses only one entry.
Seems to me that specifying a port would be the trigger to this behavior.
Two rules are created for seq 30, which in the tcam list are referenced as 5df4c and 5df90. To me looks like 5df4c wholly covers 5df90, meaning that 5df90 seems to be redundant... i am wondering if that is the case or if not, what is the reason behind creating a second rule without the port definition in 5df90?
g
03-08-2023 10:25 AM
I see what you mean, I changed the permit entries to deny and their tcam count went down to 1. I need to do some investigation internally to see why there is this difference.
Sam
03-22-2023 08:29 AM
I got a response from development:
The behaviour seen is expected.
When we filter TCP/UDP protocol with port fields, two TCAM entries will be created. First entry is to match non-fragmented packet which has complete L3 and L4 header and the second entry is to match fragmented packets which has only L3 header and does not have any L4 header. Thats why in second TCAM entry there is no port field programmed. The purpose of this is to permit all the fragmented packets as well.
But incase of deny ACE there is no need to create another tcam entry to match fragmented packets because the initial packets which has L3/L4 header will anyways be dropped by first tcam entry and reassembly of fragmented packets anyhow will not happen and gets dropped. Thats the reason we don't create two tcam entries for deny ACE.
Sam
03-23-2023 02:42 AM
Right, fragmentation...
Does the ASR reassemble fragmented packets before forwarding? Or will the fragments (which come after the original packet with L4 information was dropped) be forwarded for example if you have an acl like this?
10 deny udp any any
20 permit ipv4 any any
So my question is will the asr drop the fragmented packets or will the router pass the remaining fragments down to the destination and it is the destinations responsibility to drop the invalid packets?
03-23-2023 09:01 AM
Reassembly only happens on for-us packets as that is cpu intensive.
For regular data traffic we forward the packets as is, otherwise the next hop would just have to fragment the packet, then the next hop reassemble, etc etc.
Fragments can be completely valid, but its not upto the transit node (asr9k) to determine that for transit packets.
Sam
03-24-2023 02:27 AM
HI Sam,
was my intuition as well, just wanted to make sure, thanks.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide