free up space cisco ISE

[Kashish_Patel]

In Cisco ISE, I see that disk space is 82% full... How can I find out what is filling up space so much? How can I free up space?

ise# sh disks

disk repository: 3% used (370780 of 14877092)

Internal filesystems:

/ : 82% used ( 137597412 of 177740076)

/storedconfig : 7% used ( 5691 of 93327)

/tmp : 3% used ( 47076 of 1976268)

/boot : 9% used ( 40452 of 489956)

/dev/shm : 0% used ( 0 of 2008188)

  all internal filesystems have sufficient free space

[Marcin Latosiewicz]

Kashish,

What kind of a node is it? Typically purge operations should take care of most problems with disk usage on / , however on older ISE releases we saw a few different problems.

M.

[Jatin Katyal]

Kashish,

What is the version of ISE? Are you running it on Vmware or appliance?

If its VM and MnTISE vm space should be capped at 600gb

http://www.cisco.com/en/US/docs/security/ise/1.1.1/installation_guide/ise_vmware.html#wp1110217

If it's MnT than I'd say this is one the most common problems with ISE MNT nodes is the inability to purge the database. This leads to the tablespace running out of the allocated space and the eventual rise in CPU usage due to failure to write to the DB.The current workaround is to manually purge the database. The /opt size has been increased to 40% in ISE 1.1.3 patch 1.  What should also be considered is the reason for the DB to fill up and the purge to fail. Here are the few common ones:

- load balancer are configured to probe radius to check the availability of the server. Since there is no collaction filter feature like in ACS so we can't filter them out.

- reauthentication also plays an important role. Re-auth timer in the authorization profile should have appropriate value.

- mis-configured end-points and repeated failures should be tracked down and fixed.

- prevent to use any script in the network for radius probing.

Jatin Katyal

- Do rate helpful posts -

[askhuran]

Hello Kashish,

You may consider a short purge interval, regular archive of old data to a  repository, in addition to disabling un-needed probes. Review your  profiling and authorizing conditions for more appropriate probes and  attributes, so they can be optimized.

Are you running ISE on VMWare or ISE appliance? You may need to regularly backup / clear the logs from the MnT node

I agree with Jatin Katyal. He has given good suggestions that you may consider.

On any node that has the Monitoring persona enabled, 30% of the VM disk  space is allocated for log storage. For a Monitoring node with 600 GB VM  disk space, 180 GB is allocated for log storage. A deployment with  100,000 user endpoints generates 2 GB of logs approximately per day. In  this case, you can store 30 days of logs in the Monitoring node, after  which you must transfer the old data to a repository and purge it from  the Monitoring database. For extra log storage, you can increase the VM  disk space. For every 100 GB of disk space that you add, you get 30 GB  more for log storage. Depending on your requirements, you can increase  the VM disk size up to a maximum of 600 GB or 180 GB log storage. The  30% disk space allotment is applicable only for fresh installations. If  you upgrade to 1.1.x, a maximum of 150 GB is allocated for the MnT node  irrespective of the VM disk size.

The following may be more helpful:

http://www.cisco.com/en/US/docs/security/ise/1.1.1/installation_guide/ise_vmware.html

Important Note: Only use Firefox Browser for ISE administration and configuration

[harvisin]

Hello,

According to me  this is  common problem with ISE  MNT nodes is the inability to purge the database which leads to the  tablespace running out of the allocated space and the eventual rise in  CPU usage due to failure to write to the DB.The current workaround is to  manually purge the database. The /opt size has been increased to 40% in  ISE 1.1.3 patch 1.  The reason for the DB to fill up and the purge to  fail can be:-

  reauthentication also plays an important role.  Re-auth timer in the authorization profile should have appropriate  value.

- mis-configured end-points and repeated failures  should be tracked down and fixed.

[MDBee]

If the /opt directory usage is too high then configuration backups will fail. In my case the /opt directory was at 76% usage. To reduce the size of it I had a TAC case with Cisco. They attempted to reduce space with root access with no success. The fix when working with cisco was the below (this does not require root access).

1. application configure ise
2. Type "1" to select the "Reset M&T Database".

This took 10-15 minutes to complete but after it was complete the /opt directory was then down to 24% usage. 

Note: It doesn't work on the primary node as the database is being used by the active session. You need to promote the secondary node to the primary and ran the "Reset M&T Database" on the ISE node that is now the secondary one. If this still doesn't help further Cisco TAC help may be required.