Virtual Nexus N9K-C9300v - TCAM configuration

TONY SMITH · ‎03-30-2020

Hi,

I'm running a couple of Nexus 9300 virtual switches to simulate a network prior to receiving the actual hardware. It's NX-OS 9.3.3. I keep getting warning/error messages saying that TCAM isn't assigned for one thing or another, and in fact that's correct if I have a look with "show system internal access-list globals" then it shows zero for everything. Furthermore I tried to configure for one of the functions it was complaining about, reloaded as directed, but after reload the assignments are still all zero.

My simulations are running under KVM withing GNS3, but I note it's the same for the ESXi version, TCAM regions are all zero.

Is there any special way to setup TCAM on the virtual appliance, compared to the real thing? Or is this just cosmetic and should be ignored?

Thanks, Tony S

Sergiu.Daniluk · ‎03-30-2020

Hi,

According to nexus 9000v config guides, TCAM still needs to be carved for specific features to be enabled.

When you configure the supported Cisco Nexus 9000 features on Cisco Nexus 9000v, it is necessary that you configure the TCAM carving. For example, when configuring ARP suppression with BGP-EVPN, use the hardware access-list tcam region arp-ether size double-wide command to accommodate ARP in this region. (You must decrease the size of an existing TCAM region before using this command.)

Reference: https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/93x/nx-osv/configuration/guide/b-cisco-nexus-9000v-guide-93x/m_cisco_nexus_9000v_9x.html#reference_81F2BB5B76B0484EB488FDCA1960E3D2

Do you see the feature which was complaining about tcam, working correctly? If yes, this could be just a cosmetic thing.

Cheers,

Sergiu

TONY SMITH · ‎03-30-2020

@Sergiu.Daniluk wrote:
Hi,

According to nexus 9000v config guides, TCAM still needs to be carved for specific features to be enabled.
When you configure the supported Cisco Nexus 9000 features on Cisco Nexus 9000v, it is necessary that you configure the TCAM carving. For example, when configuring ARP suppression with BGP-EVPN, use the hardware access-list tcam region arp-ether size double-wide command to accommodate ARP in this region. (You must decrease the size of an existing TCAM region before using this command.)

Thanks for the reply. The bit I've highlighted is contradictory. "You must decrease the size of an existing TCAM region before using this command". I can't see how that can be done given that all regions are currently zero.

Sergiu.Daniluk · ‎03-31-2020

Hi Tony,

Definitely value 0 for all regions does not look good. So it might be just a cosmetic thing. Either that, or TCAM carving is not supported on Nexus 9000v - my guess would be that is not supported.

Regards,

Sergiu

TONY SMITH · ‎03-31-2020

So the attached file shows the whole output, showing TCAM zero all the way.

I have been getting the following log messages that drew my attention to this issue ..

2020 Mar 30 14:44:34 N9K-A %ACLQOS-SLOT1-2-ACLQOS_FAILED: ACLQOS failure: TCAM region is not configured for feature BFD class IPv4 direction ingress. Please configure TCAM region Redirect [redirect] and retry the command.

This corresponded to an inability to get EIGRP peering over L3 interface (although OK at L2). Looking at "sh ip eigrp nei det" at both ends suggested problems with BFD. I can't remember and didn't save the actual message, but the far end (IOS) showed the neighbour as awaiting BFD initialisation.

Disabling BFD on EIGRP everywhere has resolved the neighbour issue.

Sergiu.Daniluk · ‎03-31-2020

So it seems that it is supported and even required to carve the TCAM for features to work.

So it is a cosmetic issue then. Though not sure how you can then modify the TCAM.Have you tried loading NXOSv 9.2?

Regards,

Sergiu

TONY SMITH · ‎03-31-2020

I haven't tried different NX-OS versions on these appliances, but running the 9.2.1 appliance N9K-9000v shows the same. All regions are zero on boot up. And even before any configuration is added the following was logged ...

2020 Mar 31 14:16:54 switch %ACLQOS-SLOT1-2-ACLQOS_FAILED: ACLQOS failure: TCAM region is not configured for feature QoS class IPv4 direction ingress. Please configure TCAM region Ingress COPP [copp] and retry the command.

And it's not prepared to even let me try to configure it ..

switch(config)# hardware access-list tcam region qos 256
ERROR: Aggregate TCAM region configuration exceeded the available Ingress TCAM slices. Please re-configure.
switch(config)#

It let's me configure for redirect, but it's not clear that the configuration has done anything.

switch(config)# hardware access-list tcam region redirect 256
Warning: Please save config and reload the system for the configuration to take effect

After reboot the regions are all still zero.

So the same as 9.3.3

ccie.22939 · ‎02-11-2021

hardware access-list tcam region racl 512
hardware access-list tcam region arp-ether 256 double-wide