nexus 3k TCAM carving region cannot be configured

[Chris_78]

Hi 

I'm trying to configure PBR and i'm getting pushed back when I try to apply

switch(config-if)# ip policy route-map PBR
under SVI  with an error 0x410400c5(TCAM region is not configured. Please configure TCAM region and retry the command) so i'm trying to allocate some space in TCAM  by running:

switch (config-int)# hardware profile tcam region pbr 128

 ERROR: Aggregate TCAM region configurationIFACL+VACL+RACL+PBR+IPv6_RACL*2+QOS+IPv6_QOS*2+IPv6_PBR*2.exceeded the available size of 2048. Please re-configure.

I'm researching further - apparently we cannot apply settings to the default TCAM template, there should be a template - however on 3k switch is not allowing me to create templates... at least by running: hardware profile tcam resource template (nothing comes in resource) i'm running version NXOS: version 7.0(3)I4(7)

Any info is greatly appreciated!

Thanks!

Chris

[Kevin SAS]

Hello,

I am not an expert on this subject but have you tried to reduce the others memory allocation :

show hardware profile tcam region 

And reduce what is not needed for you:

hardware profile tcam region {arpacl | {ipv6-e-racl | e-racl} | ifacl | ipsg | {ipv6-qos | qos} |qoslbl | {ipv6-racl | racl} | vacl } tcam_size 

Something like 

hardware profile tcam region ipv6-e-racl 0

according to your version and this guide :

security guide n3k v7

According to the configuration guide in PBR section:

routing config guide v7

in prerequisites :

Because the default size of IPv4 and IPv6 TCAM regions is 0, you must carve the PBR TCAMs and reload the switch before applying PBR policy on an interface 

So you should reduce the other memory alloc in order to gain memory for PBR.

Regards

HTH

[Chris_78]

Thanks Kevin 

I guess this is in the right direction - however I already carved from other regions and i'm not allowed to allocate at certain registers

switch(config)# show hardware capacity module

Supervisor Redundancy HW State(Dual-SUPs): Enabled
Redundancy state: Active with no standby

Switching Resources:
-------------------
Module Model Number Part Number Serial Number
---------------------------------------------------------
1 N3K-C3048TP-1GE 73-14147-03 FOC16231R2R

Flash/NVRAM Resources:
------------------------------------------------------------
Usage: Module Device Total(KB) Free(KB) %Used
------------------------------------------------------------
1 bootflash 1609984 360116 77
1 logflash 1609984 360116 77

`show hardware capacity power`

Power Resources Summary:
------------------------
Power Supply redundancy mode(administratively): PS-Redundant
Power Supply redundancy mode(operationally): Non-Redundant
Total Power Capacity 3000.00 W
Power reserved for SUP,Fabric,and Fan Module(s) 159.84 W ( 5.33 % )
Power currently used by Modules 0.00 W ( 0.00 % )
Total Power Available 2840.16 W ( 94.67 % )
Total Power Output (actual draw) 7920.00 W
Total Power Input (actual draw) 46291.00 W

`show hardware capacity eobc`

`show system resources module all `
CPU Resources:
-----------------------------------------------------------
CPU utilization: Module 5 seconds 1 minute 5 minutes
-----------------------------------------------------------
1 10 6 6

-----------------------------------------------------------
Processor memory: Module Total(KB) Free(KB) % Used
-----------------------------------------------------------
1 3903332 1449984 62
`show vdc resource`

vlan 12 used 4 unused 4082 free 4078 avail 4094 total

vrf 3 used 0 unused 4093 free 4093 avail 4096 total

port-channel 0 used 0 unused 104 free 104 avail 104 total

u4route-mem 1 used 127 unused 1535 free 1408 avail 1536 total

u6route-mem 1 used 95 unused 639 free 544 avail 640 total

m4route-mem 1 used 57 unused 199 free 142 avail 200 total

m6route-mem 1 used 7 unused 79 free 72 avail 80 total

L2 table utilization on Module = 1

Asic Max Count Used Count
-----+---------+---------
0 131072 19

INSTANCE 0x0
-------------

ACL Hardware Resource Utilization (Mod 1)
----------------------------------------------------------
Used Free Percent
Utilization
-------------------------------------------------------------------
Ingress IPv4 PACL 3 381 0.78
Ingress IPv4 VACL 2 382 0.52
N3k Ingress IPv4 RACL 16 496 3.12
Egress IPv4 VACL 3 509 0.59
Egress IPv4 RACL 3 509 0.59
N3K SUP 106 22 82.81
N3K IPV6 SUP 8 248 3.12
Ingress IPv4 QoS 4 252 1.56
N3K SPAN 14 242 5.47

LOU 2 22 8.33
Both LOU Operands 2
Single LOU Operands 0
LOU L4 src port: 1
LOU L4 dst port: 1
LOU L3 packet len: 0
LOU IP tos: 0
LOU IP dscp: 0
LOU ip precedence: 0
LOU ip TTL: 0
TCP Flags 0 16 0.00
L4 op labels, Tcam 0 0 63 0.00
L4 op labels, Tcam 2 0 63 0.00
L4 op labels, Tcam 6 0 2047 0.00

Ingress Dest info table 0 512 0.00
Egress Dest info table 0 512 0.00
CLI not supported on this platform

QoS Resource Utilization
------------------------

Resource Module Total Used Free
-------- ------ ----- ---- ----
Aggregate policers: 1 4096 34 4062
Distributed policers: 1 1024 0 1024
Policer Profiles: 1 4096 34 4062

switch(config)#  sh hardware profile tcam region
sup size = 128
vacl size = 384
ifacl size = 384
qos size = 128
span size = 128
racl size = 512
e-racl size = 512
e-vacl size = 512
qoslbl size = 256
ipsg size = 256
arpacl size = 0
ipv6-racl size = 0
ipv6-e-racl size = 0
ipv6-sup size = 256
ipv6-qos size = 0
e-qos size = 0
pbr size = 0
ipv6-pbr size = 0
e-ipv6-qos size = 0
e-mac-qos size = 0
e-qos-lite size = 0

[Chris_78]

Because the default size of IPv4 and IPv6 TCAM regions is 0, you must carve the PBR TCAMs and reload the switch before applying PBR policy on an interface 

- They mean I should load some there and then I should be able to add to PBR?

[Kevin SAS · Accepted solution]

Hi, 

Try to reduce the following :

racl size from 512 to 256

ipv6 sup cant be changed

ipsg too

but you can also reduce egress acl and egress vacl (minimum 0)

Maybe you should reload before to increase memory for pbr.

And for this one

Because the default size of IPv4 and IPv6 TCAM regions is 0, you must carve the PBR TCAMs and reload the switch before applying PBR policy on an interface 

I don't understand, where did you find it ?

HTH

[Chris_78]

Hi Kevin 

That was awesome - after carving from the regions you mentioned i'm able to allocate space for PBR. 

I'm not sure if that matters but I had "no feature pbr" prior doing the changes. 

Side note 

"And for this one

Because the default size of IPv4 and IPv6 TCAM regions is 0, you must carve the PBR TCAMs and reload the switch before applying PBR policy on an interface" 

that comes from your first msg :) 

Thanks again!

Chris