Delayed audio after agent selected answers

Josh Stickney · ‎10-29-2018

Hello; we are running UCCX 10.6 and every once in a while, I get a report of an agent answering a call and there not being anyone there or the agent is unable to hear the caller. When I call in and test, I often get where it sounds like the call is ringing to the agent, stops for a second, then I hear the agent saying "Hellooo?" like maybe its the second time they were saying it after not getting a response. I look in CDR records of the reported issues and it looks like the agent has hung up on the caller. I'm starting to wonder if the agents, in a hurry to churn out calls, are just hanging up if they do not get an immediate response, and they may not be getting an immediate response because they are talking before the audio stream is fully up.

I know for calls coming in to the main script, I put a delay step to avoid them missing the beginning of the first prompt to play in the script. Is there something similar that can be done to allow the audio connection to establish before the agent picks up and starts talking?

Anthony Holloway · ‎10-29-2018

There is nothing you can do in scripting to affect when the audio cuts through after an Agent has answered.

Since the call is being disconnected from MOH, and then connected to the Agent at time of answer, you could look at processing times in CUCM SDL traces to see where the delay is.

You might find, for example, that there are quite a few SIP messages involved in that process, and so you might use the Duplex Streaming setting in CUCM to help with this. Like wise, the midcall signaling pass thru media change command on the SIP gateway could help. Or maybe it's not configuration that is the answer, and you have high CPU on CUCM.

Some things to think about.

Josh Stickney · ‎10-30-2018

Thank you for some ideas to look at. Turns out, we have Duplex Streaming and mid-call signalling already. When I look at the UCM servers; CPU use is pretty low, at least right now. Sitting at 20% and just over 50% memory usage. I will have to check during more peak hours in the afternoon and maybe if I get another report, I will pull the SDL traces.

I did make one change to some of the scripts that seemed to have helped in that I took out where we were putting calls on hold while they waited in queue to get hold music, and changed it to just play a wav file. Not only did it seem to help reduce the dropped calls; the music has better audio quality as well. Will see where that goes.

Anthony Holloway · ‎10-30-2018

Good on you for switching over. I have been an advocate of using Play Prompt as opposed to Call Hold for MOH for a while now. It's less resource intensive, as it doesn't involved CUCM and the Gateway to update signaling when switching back and forth. Just imagine how busy the network/components would be with 100 calls in queue, all cycling a 30 second hold to announcement loop. By switching over to Play Prompt however, doesn't remove the Hold feature entirely, as the system will implicitly put the call on hold while transferring to an Agent. Though, indirectly, you may be improving things, by reducing load on the system.

Josh Stickney · ‎10-30-2018

Thats a good point and I can hear the old MOH from UCM when I am going to an agent. What is interesting is before; when I had it as my hold music and made a test call, I could watch on the CUBE and see I got flipped to G.729 while I heard the music and then would flip back to G.711 when I would go back to a prompt in UCCX and then when talking to the agent. I haven't caught it showing me flip to G.729 when I get that brief MOH while ringing the agent but maybe its just too short for the timing of my commands in the CUBE. I've seen a lot less transcoder use since and I like that so long as we don't see performance drops elsewhere.

Anthony Holloway · ‎10-30-2018

Really, so your region setting from CUBE to MOH server is 8kbps? That's not great, since g729 is specifically designed to compress human speech and not music. I get it, cost savings and all, but man, that's not the way to do it. All music should be g711ulaw at least, and then if you need to save bandwidth, use multicast, and short of that, use local streaming to avoid WAN usage. Additionally, if you are doing mid-call codec changes, that's SIP signaling you cannot avoid, and therefore your mid-call signaling passthru command on CUBE is not able to save you any messages.

Since you mentioned transcoder usage, which is expensive and therefore should be carefully considered, another common mistake I see is the lack of an out of band DTMF relay on CUBE for calls to UCCX. This can cause MTP and/or Transcoder usage as well. So, if you see dtmf-relay rtp-nte on all of your dial peers, then you do not have an out of band option. You would need something like dtmf-relay sip-kpml rtp-nte.

Josh Stickney · ‎10-30-2018

Anthony; i would like to thank you immensely for just having the conversation with me. This system has been running since 2010 and just upgraded over the years, largely carrying over the settings from before with many different hands in the cookie jar and some things I just had never thought to look at. MOH Servers being one of them. Just looked and we have 3 to match our 3 UCMs, and just one of them was set in a DP to is G.729. It also happens to be the UCM in the same datacenter as our busiest CUBE. Our bandwidth should be fine and I dont think this was intended. O might flip it over and see if it sounds any better.

As for DTMF; you are spot on. We have dtmf-relay rtp-nte. Would it be worth changing if we are safely within our transcoder useage limits?

Anthony Holloway · ‎10-30-2018

Hey, if you like having conversations on these topics, we can chat in real time some time. I really like talking about this stuff with people. PM me your details if you're interested.

On the G729 for Music topic, it's all subjective, but the general recommendation is to not use G729 for music. You may not hear a difference. And recall that switch codecs mid call causes additional signaling to take place, which could cause audio cut through delays.

On the DTMF topic, you know, it's all up to you. If the resources are already paid for, and to fix the system means that money now goes to waste, then maybe keep 'em. Maybe when it comes time to replace those routers, you can mention to your boss that you know of a way to reduce costs by re-configuring things such that the new purchase doesn't need to have PVDMs.