Cisco 9800-40 Max used disk capacity of 90% for /bootflash exceeded

[HVO-cisco]

Hello,
I was trying to do a Radioactive Trace and got this error:
Error: Not enough disk space avaliable.

The trace output was like this:
executing cmd on chassis 2 ...
sending cmd to chassis 1 ...
executing cmd on chassis 1 ...
Files being merged in the background, please check [/bootflash/debugTrace_6e16.7640.6cd9.txt] output file
Collecting files from chassis 1.
Not enough available disk space in /bootflash to run this command. Maximum used disk capacity of 90% for /bootflash exceeded. Aborting ...

Any idea how I can fix this ?

Regards
Johann Folkestad

[HVO-cisco]

Hello, thank you for the fast reply.

I understand the problem, and it seems that in Administration-Management-File manager I can delete files and folders.

But how do I know what files or foldere I can delete ?

[Scott Fella]

It's really hard to see what files there are when you are looking at the gui.  You can always delete any .txt, .logs or .crash files, but if you have been upgrading, there is also an option when you upgrade to remove any unused images.  You are better off also posting the directory/files from the cli as later version have the option to show as a list.

Here is my dir from a 9800-CL I have with no configuration:

9800-cl-01#dir

Directory of bootflash:/
655361  drwx             4096  Mar 19 2025 13:39:08 +00:00  certs
262147  drwx             4096  Mar 19 2025 13:39:04 +00:00  license_evlog
1572869  drwx             4096  Mar 19 2025 13:39:00 +00:00  .product_analytics
19      -rw-             2087  Mar 19 2025 13:38:59 +00:00  cvac.log
22      -rw-              157  Mar 19 2025 13:38:58 +00:00  csrlxc-cfg.log
23      -rw-              736  Mar 19 2025 13:38:57 +00:00  vlan.dat
18      -rw-               30  Mar 19 2025 13:38:38 +00:00  throughput_monitor_params
17      -rw-           142769  Mar 19 2025 13:37:58 +00:00  memleak.tcl
786433  drwx             4096  Mar 19 2025 13:37:44 +00:00  dc_profile_dir
1048577  drwx            36864  Mar 19 2025 13:37:42 +00:00  tracelogs
131073  drwx             4096  Mar 19 2025 13:37:09 +00:00  .installer
1048703  drwx             4096  Jan 29 2025 20:51:48 +00:00  webui_helper
1572867  drwx             4096  Jan 29 2025 18:59:06 +00:00  .cloudm
917506  dr-x             4096  Jan 29 2025 18:29:55 +00:00  .wl_persistent_db
262151  drwx             4096  Jan 29 2025 17:39:32 +00:00  pnp-tech
524290  drwx             4096  Jan 29 2025 17:26:50 +00:00  .dbpersist
20      -rw-                1  Jan 29 2025 16:43:21 +00:00  .cvac_version
21      -rw-               16  Jan 29 2025 16:43:21 +00:00  ovf-env.xml.md5
1048582  drwx             4096  Jan 29 2025 16:43:17 +00:00  pnp-info
1179652  drwx             4096  Jan 29 2025 16:43:17 +00:00  onep
786439  drwx             4096  Jan 29 2025 16:43:15 +00:00  .geo
917507  drwx             4096  Jan 29 2025 16:43:15 +00:00  .cloudfs
655363  drwx             4096  Jan 29 2025 16:42:52 +00:00  stats
1441795  drwx             4096  Jan 29 2025 16:42:40 +00:00  .IOT_IMAGES
655362  drwx             4096  Jan 29 2025 16:42:27 +00:00  regulatory
1441794  drwx             4096  Jan 29 2025 16:42:19 +00:00  virtual-instance
1179650  drwx             4096  Jan 29 2025 16:42:16 +00:00  tech_support
262145  drwx             4096  Jan 29 2025 16:42:11 +00:00  core
393217  drwx             4096  Jan 29 2025 16:42:07 +00:00  bootlog_history
1441793  drwx             4096  Jan 29 2025 16:41:57 +00:00  ap_spectral_recording
14      -rw-         56629552  Jan 29 2025 16:40:33 +00:00  C9800-CL-rpboot.17.16.01.SPA.pkg
15      -rw-             4345  Jan 29 2025 16:40:33 +00:00  packages.conf
13      -rw-       1642099764  Jan 29 2025 16:40:32 +00:00  C9800-CL-mono-universalk9.17.16.01.SPA.pkg
917505  drwx             4096  Jan 29 2025 16:39:18 +00:00  .rollback_timer
1572865  drwx             4096  Jan 29 2025 16:38:51 +00:00  pcap
524289  drwx             4096  Jan 29 2025 16:38:51 +00:00  SHARED-IOX
12      -rwx                0  Jan 29 2025 16:38:51 +00:00  mode_event_log
1310721  drwx             4096  Jan 29 2025 16:38:51 +00:00  .prst_sync
11      drwx            16384  Jan 29 2025 16:38:40 +00:00  lost+found 

[Scott Fella]

SSH to your controller and run the following:

delete /recursive /force flash:tracelogs/
delete /recursive /force flash:*.crash
delete /recursive /force flash:*.txt

[Rich R]

@james783allen that looks suspiciously like a chatbot response which simply restates the problem and does not say anything about how to fix it which really is not helpful at all.

Scott's reply explains what can be deleted.  I would also add that you can do:
install remove inactive
to delete any software files which are no longer needed.  You don't mention what version of code you're running so just a warning that there is a bug in 17.15.x that it will sometimes offer to delete the running software in which case you need to answer NO to the prompt and delete the correct files manually yourself.  I reported this issue in beta but nothing has been done to fix it so it is still there in the released code (I've seen it on 17.15.2 and 17.15.2b).  If you delete the running software then the WLC will not recover after a reload so check very carefully before saying YES to deletion of the files.

If there still isn't enough space after cleaning up the bootflash then your only option is to redirect the RA trace to disk (which has more space) instead of bootflash but that can't be done on the GUI, it can only be done from the CLI.

[eglinsky2012]

Regarding "install remove inactive" - Isn't there an issue where if SMUs/APSPs are installed, running that command could prevent future software upgrades? I thought it was advised to perform a "clear install state" prior to "install remove inactive" - please clarify if I'm wrong or if the issue has been resolved in a newer version.

[Rich R]

It's not install remove inactive which causes that - it's upgrading without deactivating and removing each SMU and APSP individually before the upgrade, because they remain in the (now corrupted) install database even though they are not in use anymore.  At that point install remove inactive is just removing the files from bootflash: and makes no difference either way to the corrupted install database.  So to avoid having to remove each SMU and APSP before upgrade (to prevent the database corruption) clear install state removes them all in one quick shot (with a reload) so that you can then do a clean upgrade. This is supposed to be resolved in 17.15.x and later but remains to be confirmed ...

[Scott Fella]

For the bootflash you should be safe deleting these files within these directories.  Don't go too crazy, but just make sure you delete any trace files you perform when you are done.

show bootflash:
delete /recursive /force bootflash:debugBundleCollector/
delete /recursive /force bootflash:bootlog_history/
delete /recursive /force bootflash:license_evlog/ 

[HVO-cisco]

Thank you for your replays. This was very helpful.
The code is 17.9.5, and output of the files in the directory are attached.

I will do this first:

delete /recursive /force flash:tracelogs/
delete /recursive /force flash:*.crash
delete /recursive /force flash:*.txt

Then this:

delete /recursive /force bootflash:debugBundleCollector/
delete /recursive /force bootflash:bootlog_history/
delete /recursive /force bootflash:license_evlog/ 

and then this command:
install remove inactive

Regards
Johann Folkestad

[Leo Laohoo]

Please share the output to the following command: 

sh platform software mount | include tmpfs.*tmp

[HVO-cisco]

Here is the output:

JF

[Leo Laohoo]

That output is fine. 

TMPFS should be <11%.

If the TMPFS >50%, raise a TAC Case because TAC needs to get into shell to delete the offending bloatware. 

Or reboot the switch.