Hi All,
I'm noticing these TAHUSD error logs on N9K
Software
BIOS: version 07.69
NXOS: version 9.3(13)
BIOS compile time: 04/07/2021
NXOS image file is: bootflash:///nxos.9.3.13.bin
NXOS compile time: 1/31/2024 12:00:00 [12/13/2023 06:06:50]
Hardware
cisco Nexus9000 C9236C chassis
Intel(R) Xeon(R) CPU @ 1.80GHz with 16399028 kB of memory.
2025 Jun 19 12:54:43 N9K %TAHUSD-SLOT1-4-BUFFER_THRESHOLD_EXCEEDED: Module 1 Instance 4 Pool-group buffer 90 percent threshold is exceeded! (message repeated 1 time)
2025 Jun 19 12:57:08 N9K %TAHUSD-SLOT1-4-BUFFER_THRESHOLD_EXCEEDED: Module 1 Instance 4 Pool-group buffer 90 percent threshold is exceeded! (message repeated 1 time)
2025 Jun 19 12:59:28 N9K %TAHUSD-SLOT1-4-BUFFER_THRESHOLD_EXCEEDED: Module 1 Instance 4 Pool-group buffer 90 percent threshold is exceeded! (message repeated 1 time)
2025 Jun 19 13:09:23 N9K %TAHUSD-SLOT1-4-BUFFER_THRESHOLD_EXCEEDED: Module 1 Instance 4 Pool-group buffer 90 percent threshold is exceeded! (message repeated 2 times)
2025 Jun 19 13:43:24 N9K %TAHUSD-SLOT1-4-BUFFER_THRESHOLD_EXCEEDED: Module 1 Instance 4 Pool-group buffer 90 percent threshold is exceeded! (message repeated 1 time)
2025 Jun 19 13:58:44 N9K %TAHUSD-SLOT1-4-BUFFER_THRESHOLD_EXCEEDED: Module 1 Instance 4 Pool-group buffer 90 percent threshold is exceeded! (message repeated 1 time)
2025 Jun 19 14:01:09 N9K %TAHUSD-SLOT1-4-BUFFER_THRESHOLD_EXCEEDED: Module 1 Instance 4 Pool-group buffer 90 percent threshold is exceeded! (message repeated 1 time)
2025 Jun 19 14:08:49 N9K %TAHUSD-SLOT1-4-BUFFER_THRESHOLD_EXCEEDED: Module 1 Instance 4 Pool-group buffer 90 percent threshold is exceeded! (message repeated 1 time)
2025 Jun 19 14:11:29 N9K %TAHUSD-SLOT1-4-BUFFER_THRESHOLD_EXCEEDED: Module 1 Instance 4 Pool-group buffer 90 percent threshold is exceeded! (message repeated 1 time)
2025 Jun 19 14:18:59 N9K %TAHUSD-SLOT1-4-BUFFER_THRESHOLD_EXCEEDED: Module 1 Instance 4 Pool-group buffer 90 percent threshold is exceeded! (message repeated 1 time)
2025 Jun 19 14:20:59 N9K %TAHUSD-SLOT1-4-BUFFER_THRESHOLD_EXCEEDED: Module 1 Instance 4 Pool-group buffer 90 percent threshold is exceeded! (message repeated 1 time)
Unit: 0 Slice: 4
====================
|------------------------------------------------------------------------------------------------------------------|
| Output Pool-Group drops |
| Drop-PG No-drop CPU--PG LCPU-PG RCPU-PG SPAN-PG |
|------------------------------------------------------------------------------------------------------------------|
Occupancy drops 621672561 0 0 0 0 0 |
AQM drops 0 0 N/A N/A N/A N/A |
|--------------------------------------------------------------------------------------------------------------------|
| Output UC Pool counters |
| Pool 0 Pool 1 Pool 2 Pool 3 Pool 4 Pool 5 Pool 6 Pool 7 |
|--------------------------------------------------------------------------------------------------------------------|
Dynamic Threshold (cells) 6784 6784 6784 6500 6784 6784 6784 6784 |
Occupancy drops 621672284 0 0 0 0 0 0 0 |
AQM drops 0 0 0 0 0 0 0 0 |
|--------------------------------------------------------------------------------------------------------------------|
| Output MC Pool counters |
| Pool 0 Pool 1 Pool 2 Pool 3 Pool 4 Pool 5 Pool 6 Pool 7 |
|--------------------------------------------------------------------------------------------------------------------|
Dynamic Threshold (cells) 6784 6784 6784 6784 6784 6784 6784 6784 |
Dynamic Threshold (desc) 7984 7984 7984 7984 7984 7984 7984 7984 |
Dynamic Threshold (inq thr) 6784 6784 6784 6784 6784 6784 6784 6784 |
Occupancy drops 277 0 0 0 0 0 0 0 |
|--------------+---------+---------+---------+---------+---------+--------+---------+---------+|
| Additional counters |
|--------------+---------+---------+---------+---------+---------+--------+---------+---------+|
MEM cell drop reason : 0
MEM descriptor drop reason : 0
OPG cell drop reason : 132
OPG descriptor drop reason : 0
OPG CPU cell drop reason : 0
OPG CPU descriptor drop reason : 0
OPG SPAN cell drop reason : 0
OPG SPAN descriptor drop reason : 0
OPOOL cell drop reason : 38704003
OPOOL descriptor drop reason : 0
UC OQUEUE cell drop reason : 621665268
MC OQUEUE cell drop reason : 1022
OQUEUE descriptor drop reason : 0
MC OPOOL cell drop reason : 412
FWD DROP : 581888
SOD : 0
BMM BP : 0
No Drop : 0
Packets received : 1434426025112
TRUNC MTU : 0
TRUNK BMM BP : 0
VOQFC messages sent : 0
SOD messages sent : 0
SPAN desciptor drop : 0
I've come across this document
https://bst.cisco.com/bugsearch/bug/CSCwe13666?rfs=qvlogin
Looks to be not resolved and the buffer threshold errors continued even after the device was isolated due to buffer accounting values stuck at unexpectedly high values without any traffic...
Any input will be appreciate.
Thanks,
Nishan
