UC560 CUE disk issue

[rick.mancinelli]

All-

So we have a client with a UC560 that is reporting that it is running out of disk space:

"Feb 24 18:06:14 localhost platform_config: Critical amount of disk space used, please backup log files using the copy log command"

Below is the result of a sh sysdb /sw/info/filesys:

dev/root/usage/mountPoint               /

dev/root/usage/oneKblocks               2048124

dev/root/usage/usedBlocks               1960264

dev/root/usage/availBlocks              87860

dev/root/usage/usedPercent              96

I took the system offline and attempted a "database compact" and, after a minute or two, received the following:

INFO:  vacuuming "pg_catalog.pg_largeobject"

ERROR:  invalid page header in block 22545 of relation "pg_largeobject"

We just recently deleted and recreated all of the voice mail boxes, so we're not using space there.  The log files are all just a few hundred K, so nothing noteworthy there, either.

Without a decent disk interface, it is hard to tell WHAT is gobbling the space...?   Also, what is this invalid page header issue?

Any assistance would be appreciated.

Thanks,

Rick

[victorycf]

No, voice mail is only at around 30% usage.   It's basically CSCtr07804 that allegedly is fixed in 8.6.1

se-10-1-10-1# show sysdb /sw/info/filesys

dev/root/usage/mountPoint               /

dev/root/usage/oneKblocks               2048124

dev/root/usage/usedBlocks               2048124

dev/root/usage/availBlocks              0

dev/root/usage/usedPercent              100

Backups won't complete. I can back up the config, but not data....

se-10-1-10-1(offline)# backup category configuration

Backup progress: 6930782 bytes

Backup Complete.

Check Backup history for detailed information.

se-10-1-10-1(offline)# backup category data

Backup progress: 1979 bytes

Backup Failed.

Check Backup history for detailed information.

se-10-1-10-1(offline)#

Cisco support said I had to upgrade to 8.6.1 and warned me since I have 8.2 I will lose voice mail... Will tackle the upgrade shortly, but I'm worried that when the backup fails, the upgrade process won't continue.   Hopefully I can bypass it.   I've already told my boss voice mails are almost certainly going to be lost. 

[Brett Walters]

Ack - it is 100%.  Pretty sure they are right, then...without any previous backup, all your voice mail is history.   You must have an environment that goes through a bunch of voice mails (and properly deletes them) like mine did.  I have other UC560's with minimal issue in this area because their VM usage is minimal.  We upgrade them anyway so not to lose track of them, but they did not melt down.

[victorycf]

I resolved this fairly painlessly (well, except for losing all voicemails).   My main concern was not being able to do a backup during the upgrade procedure so that procedure would restore the system after it was done.  I fixed it doing the following.

The first step was to go into the CUE web management page and delete a few AA prompt messages that I didn't really need. This freed up enough space on the CUE flash so it started working properly temporarily.

I then exectued the following in offline mode

• backup category configuration  (I was unable to backup data without it failing)
• restore factory default

After it rebooted I then went into offline mode and did:

• backup server url ..... (point to my original backup ftp site)
• show backup server (to get the latest id of the backup I did above)
• restore id n category configuration (where n is the backup id)
• reload

This dropped my flash usage down to about 30% and got the system working again.  At this point  I could have waited to upgrade to a later time, but went ahead and did the upgrade now that it could complete a backup.

The upgrade went fairly painlessly. I did have a problem with pilot numbers for the CUE not working (vm, AAs) but doing a reload on the CUE fixed that.    I also lost my remote phones. Hopefully I don't have to drag them back in to reconfigure. I'll worry about that tomorrow.

Hopefully this helps someone who finds this article the same way I did, by doing a search on the symptons.

[Alexander Maroukian]

Hello Ken,

Thanks for sharing! Just to add if the remote phones are SPA525 then they are using SSL VPN.

After upgrade you need to recreate the certificate for the SSL. You may try to do so in CCA Configure> Security> SSL VPN-> Generate Certificate.

HTH,

Alex

[victorycf]

Thank you. That fixed that issue, as well as an issue of CCA not connecting over SSL locally.

[Brian Rapier]

I got bit by this a month ago.  Was not aware of any bug reports out on the issue.  We lost the entire voicemail/auto attendant setup.

It started the night before with a call that someone "could not use the voicemail menus" while listening to the messages.

Turned out that the message they were listening to was actually recording the the menus themselves (you know the press 1 to, press 2 to, press 3 to etc.) , so when they tried to push a button nothing happened as it wasn't really the menu, but a recording of the menu.  I chalked it up to user error and thought nothing of it.  Well 16 hours later the whole thing died.

Started with a user telling me they could not get into thier voicemail, once I looked at the logs I started seeing the disk space errors.  I also tried the disk cleanup utility to end up with a corrupt db.

Of course the first thing we did with TAC was reload and update the software from 8.6 to 8.6.5 and then realized none of my backups would work.  8.5 hours later the system was rebuilt and running.

So far things have been stable since we moved to 8.6.5.