The browser-based Finesse Agent Desktop 12.5, that ships with the Cisco Unified CCE / CCX contact center solutions, has made massive strides in the performance of its underlying infrastructure, which improves its throughput and reliability by an order of magnitude compared to Finesse 11.x and below.
This is part one of a series of posts from the engineering team that aims to describe the changes made in the 12.5 release and the resulting improvements.
Faster failover is one of the major improvements that was added in Finesse 12.5. Failover is the process by which the desktop reloads from the alternate Finesse server when it senses a problem with the currently connected server.
Finesse failover involves 2 major scenarios,
A desktop reload, requires all the clients to reload the complete desktop & its resources from the alternate server within a very short window.
This operation is akin to a complete browser relaunch for the desktop and produces the same load as a login operation for a single client.
While shift changes and consequent user logins typically happen over a period of a few minutes, a browser failover would generate the entire load at the Finesse server in a much shorter time period, comprising of static HTTP resource loads and REST API calls from all the clients involved.
This made scaling thousands of clients to simultaneously become available in the shortest possible time, an exacting challenge.
Therefore, this requirement became the source of a major list of optimisations made to the Finesse desktop.
One of the critical challenges faced by a web server is scale. In the initial days of Common Gateway Interface (CGI) web , HTTP servers used to serve a request by spawning a new process for each request, which was no doubt influenced by the Unix process spawning philosophy. Once the scale requirements grew, this was deemed to be too inefficient and thread-based request handling replaced it and has been more or less in force for a decade or two.
However, this architecture became inadequate beyond a certain scale, and c10k refers to the challenge of scaling a single web server to handle simultaneous connections from 10k clients. The act of spawning a different thread for each request becomes untenable at this scale due to the inherent in-efficiency present in context switching between different thread stacks and synchronising between them.
Asynchronous request processing architecture (which was definitely not pioneered by the web industry) was the next family of architecture that was picked by web servers to overcome this challenge.
Asynchronous requests refer to the nature of application code that aims to remove extra thread requirements by using only non-blocking API calls, which does not entail any waits. The results of the API calls are communicated over callbacks to the application code to process later.
This allows a single thread to run without being blocked and cater to multiple requests at the same time.
Finesse uses Apache Tomcat as its underlying web server. Tomcat HTTP connector with the fastest throughput (APR) uses synchronous style of request processing and improving upon this, became a requirement to meet the rate of requests pushed during a failover scenario. Though Tomcat does have a non-blocking connector, it handles only the TCP connection establishment phase in an async manner and does not seem to be able to scale the static resource processing in an async manner.
Therefore, Finesse opted to use Nginx since it has an asynchronous / event-based architecture, which was a departure from the thread-based request processing used by Apache Tomcat. Nginx was one of the pioneers of this new web server architecture and has now become one of the leading web servers in the industry.
These are the changes we introduced to improve the tomcat scale issues
Overall, this change has provided significant gains from all the above perspectives and enables new opportunities due to the flexibility it provides to the application stack.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.