topic Re: ISE root disk filesystem in Network Access Control

ISE root disk filesystem

registry — Wed, 29 May 2019 08:11:39 GMT

We are using the Cisco ISE and one of the appliance servers is not calculating used disk space correctly...it states 75%, but it should be more like 70%. We have the threshold snmp trap set to 25 ( =75% used ) and the server started to send alerts reaching that level. After removing some patches and corefile dumps - "show disk" still says 75% used. Now, if I poll the server with snmpwalk, the used disk is 70%. But it is still trapping...

show disk:

Internal filesystems:
/ : 75% used ( 10532540 of 14987616)
/dev : 0% used ( 0 of 32842248)
/dev/shm : 0% used ( 0 of 32851936)
/run : 1% used ( 1092 of 32851936)
/sys/fs/cgroup : 0% used ( 0 of 32851936)
/storedconfig : 2% used ( 1588 of 95054)
/boot : 21% used ( 94882 of 487634)
/boot/efi : 4% used ( 8792 of 276312)
/tmp : 1% used ( 7000 of 1983056)
/opt : 26% used ( 276781400 of 1125348968)
/run/user/440 : 0% used ( 0 of 6570388)
/run/user/301 : 0% used ( 0 of 6570388)
/run/user/321 : 0% used ( 0 of 6570388)
/run/user/0 : 0% used ( 0 of 6570388)
/run/user/304 : 0% used ( 0 of 6570388)
/run/user/303 : 0% used ( 0 of 6570388)
/run/user/322 : 0% used ( 0 of 6570388)
/run/user/308 : 0% used ( 0 of 6570388)
all internal filesystems have sufficient free space

System info:

Cisco Application Deployment Engine OS Release: 3.0
ADE-OS Build Version: 3.0.3.030
ADE-OS System Architecture: x86_64

The second question I have is: What is taking up all space? There are no big files in the root system, but still 70% is used. I have tried to reload the system.

Regards / Fred

Re: ISE root disk filesystem

Jason Kunst — Wed, 29 May 2019 15:12:52 GMT

I would checking out the info under http://cs.co/ise-help how to get help in the community, we are not the TAC, you should open a case with them to debug

Re: ISE root disk filesystem

hslai — Wed, 29 May 2019 18:56:08 GMT

As Jason said, the root file system can't be cleaned up without TAC assistance. What we are able to clean usually is only the /localdisk, which uses the file system mounted in /opt. The "show disks" gives the outputs from Linux command "df" and is more accurate than the data obtained from a SNMP query, which is also unclear from which OID you received the data.

Re: ISE root disk filesystem

Arne Bier — Wed, 29 May 2019 21:25:37 GMT

Maybe it's time to create a cron job for ISE systems that does a daily disk cleanup (around midnight or whatever) - these used to be part of every unix system ever since I can remember. I am sure the TAC have gathered enough best practice experience of problems that occur most frequently that this can be automated. This should not require human intervention.

Re: ISE root disk filesystem

packetplumber9 — Mon, 03 Jun 2019 13:22:57 GMT

I've seen the "show disk" output take some time to update after deleting files to free up space. This could at least temporarily explain the difference between the command output and SNMP polling results

Re: ISE root disk filesystem

hslai — Tue, 04 Jun 2019 08:47:02 GMT

ISE does have such cron jobs in cleaning files. The original post did not provide enough data and would be best investigated by TAC.