Cisco ISE 2.X Available Disk after Upgrade

[jj2048]

Overview:

I was assigned to plan out the upgrade the version of Cisco ISE for a certain customer.

I did some simulations by creating a staging environment where it consists of a Cisco ISE, Switch (Authenticator), AD, and an Endpoint for 802.1X authentications, in order to test out CLI upgrade guides and create a document for method of procedure.

After trying out upgrading, by patching first the old version to the latest one, then upgrading it to a newer version, I noticed that the available disk space changed. So I then patched the 2.6 to the latest version still the disk is still small.

I've tried searching into communities all concerning disk free up needs TAC, but I haven't found yet a topic similar about this available disk space.

And I thought as well that I have another customer with a physical appliance (Multi-node Deployment), I noticed that it's disk space was 30GB as well, and was wondering before I tried to upgrade on a staging environment. I saw hints of upgrade since there was an upgrade bundle on their disk. There will be upcoming upgrade as well for this customer, so I need to be sure to involve TAC with this in order to get back the original available disk space.

Question:

Is this behavior is expected and will require assistance from TAC?

System: (Standalone)

Cisco ISE 2.1.X VM, patch 8 200GB storage 16GB ram

Objective:

Upgrade Version to 2.6.X patch 6, (CLI Upgrade)

Problem:

Available Disk Space from approx 200GB to 30GB after upgrade.

Outputs:

Before Upgrade

Cisco ISE 2.1.0.474, Patch 8

Usage for disk: filesystem
26179149824 bytes total used
199386726400 bytes free
237662035968 bytes available (200GB approx)

After Upgrade

Cisco ISE 2.6.0.156, Patch 6

Usage for disk: filesystem
2783711232 bytes total used
26455691264 bytes free
30829043712 bytes available (30GB approx)

Thank you in advance.

[Colby LeMaire · Accepted solution]

At the CLI, run the "show disks" command.  If the local filesystem is full, you can clean up the files.  If the other partitions (i.e. /dev, /opt, /boot, etc.) are full, then you will need TAC assistance.  If it is the local filesystem, run the "dir recursive" command and it will show you all of the files and their sizes.  Most common files that take up space are coredump files and those can be deleted.  The other common files that take up space are your patch bundles and installation packages.  If you don't need them anymore after they are installed, just delete them.

[jj2048]

Hi, Colby.

Thank you for the response.

"Show disks", gave me the following output: (This is from staging environment)

disk repository: 10% used (2718468 of 30106488)

Internal filesystems:
/ : 14% used ( 1863036 of 14987616)
/dev : 0% used ( 0 of 8121724)
/dev/shm : 0% used ( 0 of 8132660)
/run : 1% used ( 1384 of 8132660)
/sys/fs/cgroup : 0% used ( 0 of 8132660)
/tmp : 1% used ( 6868 of 1983056)
/storedconfig : 2% used ( 1583 of 95054)
/opt : 22% used ( 47245800 of 232091832)
/boot : 25% used ( 112574 of 487634)
/run/user/440 : 0% used ( 0 of 1626536)
/opt/docker/runtime/overlay/000052231dec967c60b3931d75486852d3259d6e1d5a51f63332855d59f2484b/merged : 22% used ( 47245800 of 232091832)
/opt/docker/runtime/containers/d4525affd1481509a213d1d43de8154ee1775d763f3ab047640c5cc1aeff011b/shm : 0% used ( 0 of 65536)
/run/user/301 : 0% used ( 0 of 1626536)
/run/user/321 : 0% used ( 0 of 1626536)
/run/user/0 : 0% used ( 0 of 1626536)
/run/user/304 : 0% used ( 0 of 1626536)
/run/user/322 : 0% used ( 0 of 1626536)
all internal filesystems have sufficient free space

I'm not sure what the unit is given here, though assuming it's bytes, it is approximately 500MB in total.

I was wondering where all the the other 170GB gone to.

Question:

In the current scenario, where the "dir disk:/" shows 30GB available space, and the free space will be around 25-28GB.

If another upgrade will commence, will this suffice for another upgrade and the next? I encountered an error last time the disk space if <21GB upgrade will not continue.
I might try upgrading twice, from 2.1 to 2.4 to 2.6 just to check if the available space will still decrease in size.

There might be a special case that we can't do re-imaging due site is inaccessible for physical access due to our situation we are experiencing. 

So the 170GB missing, is what I'm looking for, or how could I get it back to disk available space.

[Damien Miller · Accepted solution]

To be fair here, dir disk: should show only 30 GB of space. This is because disk: is mapped to /localdisk which is a preallocated 30 GB mount found within /opt.

[jj2048]

Hi, Damien.

Thank you for the clarification.

To my understanding of your statement, that this is an expected behavior.

I'm not into depth with the file systems of Cisco ISE, and was new to the experience of what happened after the upgrade.

Once I'm done simulating the scenario where 2 jumps of upgrade is done I'll post results.

[Damien Miller]

Yes, it's expected that "dir disk:" will show the 30 GB provisioned for /localdisk in 2.4+. The command Colby shared, "show disks" will provide more granular allocations from the underlying linux subsystem.

[jj2048]

So i got the results from upgrading 2.1 to 2.4 to 2.6.

Sequence

Patch 2.1 to the latest version 8

Upgrade 2.1 to 2.4

Patch 2.4 to the latest version 12

Upgrade 2.4 to 2.6

Patch 2.6 to the latest version 6

Outputs:

Cisco ISE 2.1.0.474 , Patch 8
Usage for disk: filesystem
26079166464 bytes total used
199486709760 bytes free
237662035968 bytes available

Cisco ISE 2.4.0.357, No patch
Usage for disk: filesystem
46170112 bytes total used
29193232384 bytes free
30829043712 bytes available (Disk size changed from 200GB to 30GB)

Cisco ISE 2.4.0.357, Patch 12
Usage for disk: filesystem
46979248128 bytes total used
178586628096 bytes free
237662035968 bytes available (Disk size changed from 30GB to 200GB)

Cisco ISE 2.6, No Patch
Usage for disk: filesystem
46166016 bytes total used
29193236480 bytes free
30829043712 bytes available (Disk size changed from 200GB to 30GB)

Cisco ISE 2.6, Patch 6
Usage for disk: filesystem
46170112 bytes total used
29193232384 bytes free
30829043712 bytes available

So my questions were answered.

Is the change of disk size is expected? - Seems like so

Do I require assistance from TAC in case disk is insufficient? - Yes

What I found out.

Cisco ISE patch can get back the missing 170GB on the dir disk:/, refer to the Patch 12 on 2.4.

Thank you to Mr. Colby LeMaire and Damien Miller for their helpful answers.

Hope this discussion help someone out.