cancel
Showing results for 
Search instead for 
Did you mean: 
cancel

Notice: UC560 Voice Mail / AA issue being investigated

We are currently investigating several cases where UC560s running on  the latest SWP 8.0.4 are experiencing Audo Chop and distortion with  Voice Mail Greeting & Recorded Voice Mail Messages.

We  believe we understand the issue now is with CUE 8.0.2 and are close to implementing a  solution

Please watch CSCti85760  in your Bug Toolkit for status.

In the mean time if  you do experience this issue, use CCA to reload CUE (we have seen this  help when the problem occurs and found very agressive proactive action  of every 2-3 days will prevent this).

If you have a factory UC560 ready to install,  please only upgrade the IOS to 15.0(1)XA3A.  Do not upgrade CUE (Stay with CUE 7.1.3)

We removed the UC560 SWP from CCO until we have a solution. 

*If you need 15.0(1)XA3a IOS image, you can get it from 8.0.4 software pack for uc540 platform which is still posted and has no problem.

NEW INFO:

Downgrading is NOT an option for CUE.  DO NOT attempt to downgrade CUE to 7.1.3 if you are running CUE 8.0.2 as it is not supported.

30 REPLIES 30

Steven Holl
Cisco Employee
Cisco Employee

I've got someone seeing this on an AIM running 7.2.2, as well, and am currently working with CUE developers on it.  May be realted, or may not.

I see a high jitter value (~800ms) on a single pack from CUE to CME.

I'm working with development in regards to this, and currently pending some coredump collections and rtp pacer debugs.  If it ends up being a new bug, I'll try to remember to report back to this thread since it could be related.  My issue may be related to CSCtf70144 and just never ported to the AIM platform, but I know CSCtf70144 was fixed for the UC5xx platforms.

finalconnect
Participant
Participant

Steve,

As a Cisco partner, I appreciate the honesty that such a post required. Most vendors including Cisco of the past, would simply cover this up as just another "bug", but this is not just another bug. This "bug" impacts everyone using a system and the core of what a phone system is supposed to provide not just a feature.

As a partner having two customers with this problem, I appreciate the effort Cisco has put into helping me, the partner, and my customer. In the end, we need the problem resolved as quickly as possible so that it does not delay future orders we have in the pipeline.

Here is how to RELOAD ONLY CUE if you find you need to using CCA 2.2.5 (click to improve picture)

Latest Status and request for Partners who want to try workaround.

As we now have reproduced the problem and have a team of developers looing at this as I write, we also are taking the action to produce a work around.

We will be using the Event Manager to create a sort of CRON job that can kick off and reboot CUE every morning at 4AM, since we know that it takes about 2 days to get into the problem where CUE is sending packets with 40ms packetization delay (should be 20ms).

We are advising Partners that rebooting CUE (not the whole UC560), will prevent the problem from occuring, but we realize this is a manual task.

When we do have this ready, are there any Partners who would like to use it?

Send me an EMAIL sdistef@cisco.com

We are also considering the following if we cannot figure out the fix in the next couple days, but its a bit complex and time consuming.  Let me explain.

Putting CUE back on CUE 7.1.3 will eliminate this problem.  However, the data structure of CUE 8.0.2 is NOT backward compatible with CUE 7.1.3 (so it cant be saved and restored).   Also, as you all know, these systems are configured in such a way where configuration is sent to BOTH IOS/CME and CUE, so there is no way to simply restore CUE to factory defaults and load the old CUE SW and rebuilt only it.   The only way to go back to CUE 7.1.3, is to factory default the entire system, downgrade CUE (I would probably factory reset again after that) and then rebuild the system.  This obviously will take 3-4 hours if you have all the data you need.

The only value an RMA option would provide is getting you on older code, and you would upgrade IOS to 15.0(1)XA3A (but leave CUE alone on 7.1.3) and then rebuild it in a staging (offline) way.  This is very possible using text files for phones and users as I have documented in TEL#25 on this community.  Then it would be more like a physical cutover of one box to the other.  You would have to transfer licenses and make absolutely sure the HW VWICs are identical in both systems to have this work, and this also is manually intensive, so we are really not encouraging this option, but want to let you know it is being considered on the table at this point.

This is IOS CLI, entered on the UC560 with privilege level 15 in Configuration Terminal mode.   When done, save it using 'write memory' which copies running to startup-config.  While IOS will not reboot with this, it's still a good idea to save it in case the router looses power or reloads due to something else.


This is intended to be a short term work around only, until we address the root cause of the CUE packetization problem that seems to occur after a period of time ranging from a couple days to 4-5 days.   The reboot of CUE initializes it preventing the longevity issue.  UC560 CUE reboot should take about 5 minutes on a UC560 with about 50 voicemail boxes.

The 4am is a time we picked, but you can adjust if that's not a good time.  When CUE reloads, all call processing continues as normal, just voice mail and AA are reported as unavailable, please try later.

NOTE:  PLEASE REMEMBER TO TAKE THIS OUT WHEN WE FIX THE ROOT CAUSE


Since this is fairly new, the SBCS is being briefed on this as well so this is pretty much real time.  We tried this script on several systems and