Fabric nodes in moderate-to-high memory usage without user traffic?

tuanquangnguyen · ‎12-26-2019

Hi community,

While configuring a new ACI fabric, I've taken notice that all nodes (leaves, spines and APICs) seem to operate with a rather high memory usage, consistently exceeding 50% over a week of monitoring (detailed amount below). The fabric has yet been integrated into the ongoing system, thus there aren't any user traffic or spikes. Should this be expected at all?

Thanks in advance.

The fabric consists of:

1. 3x APIC-SERVER-M3 on 4.2(2f) code - continuously > 68% since I've started monitoring

2. 2x N9K-C93180YC-EX Leaves - > 57%

3. 4x N9K-C9348GC-FXP Leaves - > 59%

4. 2x N9K-C9332C Fixed Spines - > 78%.

All leaves and spines are running on NX-OS 14.2(2f)

What I've been configuring:

1. Network Centric deployment, but with gateway migrated onto ACI and traffic PBR'ed to an external firewall cluster

2. Barely over 1000 static ports - On demand deployment immediacy

3. About 150 EPG contracts (1 subject for each, matching IP traffic). All subjects are associated with a PBR SG.

4. One L3Out, with only one L3Out EPGs but a few Logical Node Profiles, SVI Logical Interface Profiles.

Jayesh Singh · ‎06-30-2021

Hi,

This is an expected condition.

Below is the snip from official document:

Switch memory utilization of 70-75% is normal, even for leaf and spine switches in a new deployment where no configuration has been pushed. Switch memory utilization in ACI mode should not be compared to utilization in standalone NX-OS mode due to the additional ACI-specific processes, which consume additional memory.

You should be concerned only if memory utilization exceeds 90%. Open a TAC case for proactive troubleshooting if memory utilization exceeds 85%.

Document for your reference:

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/Operating_ACI/guide/b_Cisco_Operating_ACI/b_Cisco_Operating_ACI_chapter_01011.html#concept_393DF77C7B2A47E194F3F0D8059971DC

It's an old post but I think the information will be useful for future queries.

Regards,

Jayesh