So we have a client with a UC560 that is reporting that it is running out of disk space:
"Feb 24 18:06:14 localhost platform_config: Critical amount of disk space used, please backup log files using the copy log command"
Below is the result of a sh sysdb /sw/info/filesys:
I took the system offline and attempted a "database compact" and, after a minute or two, received the following:
INFO: vacuuming "pg_catalog.pg_largeobject"
ERROR: invalid page header in block 22545 of relation "pg_largeobject"
We just recently deleted and recreated all of the voice mail boxes, so we're not using space there. The log files are all just a few hundred K, so nothing noteworthy there, either.
Without a decent disk interface, it is hard to tell WHAT is gobbling the space...? Also, what is this invalid page header issue?
Any assistance would be appreciated.
Rick, you MUST call support and open a trouble ticket. I have gone through this issue 3 times. 1 time the whole unit had to be RMA'd. 1 time, the client lost everything and I had to rebuild the whole UC from scratch. The 3rd time, i raised enough hell and it got escalated to the moon.
For almost a year, i have an engineer from cisco, california assigned specifically to my case who installed special scripts and the cue is running diagnistic logging 24/7 onto an FTP and syslog server. he checks on the cue status every few weeks.
At this state, if you are lucky, you MAY be able to save the cue configuration.
Call support right away and tell them this issue has been tracked, specifically for uc560 for a couple of years.
Below is my email address. If they give you a run around, email me directly and I can give you my case number so that they can get an idea what is going on.
There is a good chance that the Compact Flash card has been hit with some bad blocks, this is not uncommon as CFC's don't have a very high means failure rate, and on busy systems with plenty of CUE action they can experience failures prematurely.
I would log a support case imediatly on this machine, in the mean time I would use CCA to create a full backup of the system just incase the CUE fails completly.
Please do not leave anything to chance, also dont run a check disk on it as this can kill it (Has happened to me more then 3 times) especially if the CFC is on its last legs.
Anyone have an update from Cisco regarding this issue/bug? I have a UC system and a case open with this exact issue and they are telling me to reload the module to factory defaults and start from sratch which IS NOT AN OPTION at all.
Myself and the customer are pretty livid and will be pushing and escalating this case ALL the way up until someone can address this issue within Cisco.
Am going through this right now. Had no idea this 'bug' even existed until the CUE was full. Have an escalation in place now, but really, wipe out the CUE and start over is the answer? I cannot believe there is not method to moev a file out of the system to free up space, or delete a mailbox or two to do so and get it back online to upgrade and allegedly fix it.
Has anyone seen 8.6 in action to actually *resolve* this, or does the issue still exist beyond that?
Per Cisco - 8.6 helps alleviate the issue, but does not resolve it, as you mention. They are hopeful that the version they are currently working on will actually resolve it. I am not really sure how you can code a database to grow until *all* disk space is used - and not provide a method to watch/verify it automatically (and not a manual investigation), but at least when you get to the escalated support, Cisco is there to work hard to resolve. Brandon, if you are out there watching these threads - thank you for all your hard work and attempts to fix this!
If your database gets to 100%, as mine did, you are unlikely to be able to recover anything, as we were unable to. Unfortunately, another symptom that the database is full and/or corrupted is that automated backups of CUE fail - and if no one is watching them...restores are ugly in terms of recovered data (though the CUE reinstall seemed pretty simple and straightforward).
Just FYI for anyone else out there. Set up automated CUE backups (and have it email you if you can get that to work) and make sure *someone* is watching that, and disk usage. Otherwise, you end up here.
Update - scheduled backups in this instance *are* sending emails to the local admin at my client's site - and oddly enough, they are all saying, every day, that backups were successful and completed, so this may not be a harbginer of failure. The darn thing said it was backing up fine and hadn't for months...ugh.
Message was edited by: Brett Walters
Has anybody heard anything new about this bug? I spent 11 hours last Tueday night with a UC560 getting it back up and running with the newest software pack and the support tech sent me this:
Right now it appears as though the database is not corrupt, but only the engineer build will fix the issue. If you are happy with the way it is now, then make sure a regularly scheduled backup is done, but if you would like to move forward with the engineer build, just give us a call. The information you sent me is now attached to the case.
I am just wanting to know if this bug creeps up on you as in the database fills up at a slow rate or if it just one day goes from 21% to 100%. This unit that I had has been in production since November 2010 and Monday night was when the database showed this bug. Right now, I am sitting at 21% usage and on the latest software pack 8.6.0 and everything seems to be running smoothly.
Also, does anybody have any kind of scripts that I can run to get the information on the database used percentage automatically? Having to log in everyday and run the command is not ideal.
Thomas - talk to higher level support within the Small Business Support group. There is a temporary fix (new CUE installation version) that will alleviate some or most of the growth causing this, but you will need their assistance in obtaining it and installing it, even if you are a CUE expert and CLI guru. I am told there will be a new version released in the near future (no promised dates, but hopefully September) that includes this version.
In my experience with the issue, it is gradual, but it will depend heavily on voice mail usage on the system. If there is a ton of voice messages left, then deleted, this will grow faster than if VM is only nominally used.
I do not have any scripts - I logged in manually, and understand your frustration with it. Same ones I have with not being able to automatically backup CCA (yes, I know I can backup CUE regularly, and do).
PS: 8.6.0 does not fix it - the engineer build is better suited to alleviate more of the issue. Been there, done that.
When we did my latest one, with some work, we were able to - but your engineer will explain it. I don't want to make anything public they may not wany public - but yes, you will be able to backup your config, upgrade, and restore. At least we were in my case - I was not on 8.6.0 for mine.
I just got hit with this bug and got the bad news from Cisco support that I have to upgrade and lose all of my voice mails (I'm on 8.2 now)
I'm amazed that there wasn't some sort of communication sent out warning people to upgrade to fix a critical bug. Or is there such a list and I'm not on it for some reason?
Well, about to call my boss and tell him the bad news.