WebDialer: usage and restrictions when working at scale

[floatingpurr]

Hello folks. I'm designing and testing a solution for the implementation of a simple click-2-dial machinery from an external application. I need to work with the CUCM, then I've found two solutions:

- WebDialer

- Remote CTI

I set up a working PoC based on the WebDialer. All runs smoothly in my lab. However I've just read that the WebDialer accepts a Maximum Concurrent Call Requests number, ranging between 1 to 8 per sec.

Just to make sure that I am on the right page: is this limitation only referred to the max number of http calls to the service <makeCallSoap>? Or is it a serverwide restriction that affects all the http requests hitting https://[cucm]:8443/webdialer/services/WebdialerSoapservice70 regardless of the requested service?

And again: is this limitation clusterwide or per each server in the same cluster? (It seems a clusterwide limitation)

AFAIU, in the very best case scenario, you'll be placing a maximum of 8 new calls per second. If you hit it such a throttling limit, you'll get back an error like "9 Service overloaded" and the call is not placed. If that constraint doesn't fit your need, there is no way to scale, unless you are working in a multi-cluster environment with more inter-cluster WebDialers. In a single cluster, each server can run its very own WebDialer but this redundancy can't be used to increase the Maximum Concurrent Call Requests restriction (i.e., this is indeed a redundancy, not a way to scale).

The Remote CTI way looks not being affected by restrictions but it is absolutely less convenient. Indeed, you need to post custom XMLs to the phone's HTTP server to control it. In this case, you need the IP address of each phone (not easy) and you are supposed to have a direct access to the phone (not always possible). To be honest, there is also the JTAPI / TAPI way, but we do not use Java or C in our stack. Moreover, although I've not explored this scenario, I'm quite afraid that it probably suffers from same problems.

It feels like I'm in front of a by-design limitation with no way out.

Any remarks, hints on how to make a scalable click-2-call service are more than welcome. Also just sharing your experience with that and your feedback would be a great help.

[Steven L]

8 CPS is a lot. How many are you wanting?

[floatingpurr]

Hi @Steven L, I'm trying to understand shortcomings arising from the CPS restrictions and how to deal with it. I try to summarize the key points of my questions. I really appreciate any help on those.

• Does the "8 calls/sec" limit refer (i) just to the calls to the service <makeCallSoap> (i.e., new calls) or (ii) to any http request to the SOAP endpoint? I think (i) but it's not crystal clear to me.
• In a single cluster environment, is such a limitation considered (i) clusterwide or (ii) per server? I think (i). This means that you can't distribute <makeCallSoap> requests among servers in a load balanced fashion to avoid the error "Service overloaded".
• What you can do if you need to provide a working click to call to users? Even with a few dozen of telephones you may hit the 8 CPS restrictions during peak hours. In that case, it's not nice to get back an error or keep retrying in background until the SOAP request starts the call.

@Steven L you are right, 8 CPS is a good threshold. However sometimes it may lead to corner cases that can be really annoying for users. I need to understand better the aforementioned points to deal with them.

[Steven L]

i think you need to understand that 8 CPS is simply 8 NEW calls being presented within 1 second.

I have a call center with 3,000 agents that peaks out at 9 CPS.

I am not sure you will ever hit that number honestly.

[floatingpurr]

Hi @Steven L, thanx for pointing that out. So, let me recap.

1. "8 calls/sec" limit refers just to the requests to the service <makeCallSoap> (i.e., new calls). Any other HTTP request to https://[cucm]:8443/webdialer/services/WebdialerSoapservice70 hitting other services does not contribute to the throttling mechanism
2. Such a limitation is clusterwide 
3. The previous point entails that there is no way to increase the 8 CPS in a cluster, if necessary. As per Steve remarks, 8 CPS might be more than enough and fit most scenarios. However, that is a by-design constraint the can't be overcome in a single cluster environment

Can anybody confirm? I'd be happy to shed light on those points and to accept her/his answer.

[dstaudt · Accepted solution]

My understanding matches your summary...a few thoughts:

- 8 CPS is a lot, and if you are getting throttle responses you can store and retry (at a sane pace) after a bit

- TBH I'm not sure what exactly happens if you exceed the configured rate limit.  Often these settings will just trigger an admin alert or a performance counter increment rather than fail.  It's likely the limits are quite conservative in light of modern hardware capabilities - you might try higher rates to see if you get actively throttled, and/or what the effect is on CUCM CPU/performance
- The authentication model of WebDialer is simple, but pretty inefficient.  Authentication (especially backed by LDAP/AD) can be surprisingly slow/expensive, and WebDialer starts this sequence anew for each incoming request (AFAIK it does not provide a cookie or cache sessions - CUCM SSO/token can help, but is not common). 

- Building a specific targeted WebDialer-like service based on TAPI/JTAPI can potentially be several of orders of magnitude more efficient, while allowing full control over the behaviour (e.g. end calls based on software, detect calls in progress, screen-pops, mid-call features like forward/conference, etc.) as the CTI app/service authenticates once at startup and there is no subsequent auth overhead for call operations

[floatingpurr]

Hi @dstaudt thanks for your thoughts. I reply with mine 

> "you can store and retry (at a sane pace) after a bit"

- yes, if something went wrong, the application can retry to place a call behind the scenes several times. This is not a technical problem per se, but It's something I'd like to avoid. In my opinion, it'd be a bad user experience for operators.

> "TBH I'm not sure what exactly happens if you exceed the configured rate limit"

- It's not completely clear from the doc what happens when you hit the rate limit, but I'm pretty sure you are going to get back some kind of error and not just an alert. The CUCM manual states that you should receive an error like Service Temporarily Unavailable. Maybe such an error is returned by the Web version of the Dialer, but it should be the same for SOAP requests. As per the devnet guide, you'll probably get back "6 - Service temporarily unavailable" or maybe "9 - Service overloaded". Those codes aren't described further there.

> "The authentication model of WebDialer is simple, but pretty inefficient"

- I've not found any documented session management mechanism. It looks like you need to provide credentials in every request (user or proxy auth doesn't matter). Luckily, in my case there aren't other components involved, such as LDAP, so the authentication process should be quite quick and smooth. I'm not sure if relying on CUCM SSO is worth the effort in my case.

>"TAPI/JTAPI can potentially be several of orders of magnitude more efficient"

- I see. Unfortunately, we do not have Java or C in our stack and we'd avoid extending our tech surface if it is not really necessary. PS: if you need it, also the WebDialer allows to end a call via software. Of course, as you pointed out, it lacks all the advanced capabilities you mentioned.

Thanks @dstaudt, I really appreciate your help.