06-27-2022 07:14 AM
Hello there,
We have a custom desktop application integrated with finesse (PCCE 12.5), we are facing an issue on call drop randomly, sometimes there is no issue and it worked fine, the agent is in an active call (TALKING state), and sometimes when he invokes end call action there is no response from the system, it seems like system got stuck and not responding, please advise, is it due to connection lost between our application and finesse or some other reason?
Please note that we have downgraded the security to TLSv1 as java 5 does not support TLSv1.2, can it be a reason for this issue?
06-27-2022 08:06 AM
We have had a similar issue on 12.6 from time to time.
TAC informed my colleagues that the following bug might be causing it: https://bst.cisco.com/bugsearch/bug/CSCvi53649
A possible workaround is to disable external access to the Cisco Finesse Notification Service TCP-based XMPP port (5223) if it is not used by any of your custom applications on both Finesse servers.
You can do this by running the following command:
utils finesse set_property webservices enableExternalNotificationPortAccess false
Regards,
Gerry
06-27-2022 12:37 PM
Hello Gerry,
Thanks for your reply, our application is using port 5222 to establish XMPP connection, please advise if we disable this port (5223) it will not impact other scenarios? Also, please confirm whether this workaround worked for you or not?
Regards,
Muhammad Arslan
06-28-2022 01:00 AM
Muhammad,
The Command will NOT impact on port 5222.
If you wanted (which you do not) - to disable 5222 - use the below command.
Use the following CLI commands to enable or disable the Cisco Finesse Notification Service plain XMPP port (5222). This port can be enabled only if you have third-party solutions that connect directly to the Cisco Finesse Notification Service over plain Transmission Control Protocol (TCP) connection. This port is not required for the Finesse desktop or BOSH/WebSocket based integrations. By default, the port is disabled.
To enable: utils finesse set_property webservices enableInsecureOpenfirePort true
To disable: utils finesse set_property webservices enableInsecureOpenfirePort false
Reference:
06-27-2022 05:32 PM
Hi @Gerry O'Rourke , what was the correlation, just that something was hitting the server on that port which was causing the high CPU which led to the system not responding promptly or something?
06-28-2022 01:04 AM
Bill,
No reason was given except for mentioning the bug id.
So whether this is the actually root cause (and how this becomes the root cause) if this port is disabled I don't know!
I am a bit suspect.....
Gerry
06-29-2022 09:01 PM - edited 06-29-2022 09:02 PM
It will be hard to speculate the underlying reason for the request hung scenario. The said condition could be because of several reasons such as custom desktop connection issue, flaky connection, finesse server responding with delay or even back-end PCCE server taking long time to respond. Did you get a chance to look into the custom desktop logs to check, whether custom desktop receives any response in the said scenario? I suggest you to look into the custom desktop logs to find out any error/delays while getting response from Finesse server. Also, where there any drop in XMPP connection?
From Finesse end, if could retrieve the web services logs for the said scenario, we could further drill down the cause of such an event.
Below is the reference material to collect the Finesse logs
https://developer.cisco.com/docs/finesse/#!log-collection-log-collection/log-collection.
-Brinal
07-03-2022 11:23 AM
hello @bmachado,
Thanks for your reply.
I tried to check the logs for the application and found the agent state in dialog XML as FAILED and state cause as BAD_DESTINATION, I have also attached the XML for your review. the issue is that whenever the agent logs in to finesse desktop or custom application, he finds himself always in TALKING state and he can't change his state or even drop the call. Please advise what is the reason behind this and how we can release the call for such agents and how to overcome this issue in the future?
Please note that this issue is with a few agents and happened randomly.
thanks in advance.
Regards,
Muhammad Arslan
07-04-2022 09:46 PM
Hi Muhammad,
From the logs you have shared, looks like call state in upstream system (PCCE) has been corrupted. Reason for it is hard to guess only from Dialog XML (BAD_DESTINATION field). As explained earlier, it would be helpful, if you could share the finesse web services log for the scenario, so that we could pin point the issue. Waiting for your response with the logs requested.
Thanks
Brinal
07-04-2022 11:04 PM
Hello Brinal,
Please advise can I execute the below command directly or do I need to execute the install command first? and if I mention the number of days in command, can I collect the logs for two days back?
file get activelog desktop recurs compress
file get activelog desktop recurs reltime days 2
How can I collect the logs for a specific agent?
Also, advise if there is a way to collect logs from the finesse web desktop?
Regards,
Muhammad Arslan
07-05-2022 06:57 AM
Hi Muhammad,
You can use either of the above mentioned command to download the logs files. The webservices logs will be the finesse server logs.
Sadly, there is no way to obtain the logs specific to single user or from finesse web desktop.
So I suggest you to download the entire set of logs from cli and filter only the one for the agent affected.
Once you have logs filtered, you can point the messages flowing to CTI server and responses from CTI server to pinpoint the error you are encountering.
From the whatever information you have provided till now, issue seems to be from corrupted call state in CTI backend. Once you confirm the same with logs, you can optionally contact TAC to resolve/fix the same.
Thanks,
Brinal
07-24-2022 10:47 PM
Hello @bmachado,
I have collected the finesse webservices logs as requested, please find the attached zip file for your kind review, the agent is still facing the same issue, please your support to check and advise what was the issue, you can check the logs against anyone of the below value:
Agent ID: 50015
Agent Extension: 42893
Call id: 35094667
In the below sample logs, I found one message coming from the backend CTI server (in bold) please advise if this is the cause of the issue and how we can resolve it for the time being? Waiting for your feedback, thanks.
Sample Logs (check the attached zip file for full logs):
%[REQUEST_URL=SystemInfo][agent_id=][requestId=][request_method=systemInfo.GET][request_parameters=]: Request from client to webservice api
0024592706: 10.240.26.26: %[call-id []]% %[agent-id []]% Jul 03 2022 13:46:51.477 +0300: %CCBU_http-apr-127.0.0.1-8082-exec-26-6-REQUEST_END: %[elapsed_time=0]: Request complete
0024592707: 10.240.26.26: %[call-id []]% %[agent-id []]% Jul 03 2022 13:46:51.537 +0300: %CCBU_EVENT_POOL-4-worker-22-6-MESSAGE_FROM_CTI_SERVER: %[cti_message=CTICallFailedEvent [ invokeID=null, cti_sequence_id=16618878, msgID=16, msgName=CallFailedEvent, deploymentType=CCE, callId=35094667, peripheralId=5000, connectionDeviceIdType=0, failingDeviceType=0, calledDeviceType=0, localConnectionState=1, calledDeviceId=, connectionDeviceId=42893, failingDeviceId=42893, eventCause=13]][cti_message_name=CallFailedEvent]: Message coming to Finesse from backend cti server
0024592708: 10.240.26.26: %[call-id [35094667]]% %[agent-id []]% Jul 03 2022 13:46:51.538 +0300: %CCBU_EVENT_POOL-4-worker-22-6-Call.getDialogTO - No skill group name found.: For callId 35094667, no skill group name found for skill group number 0
0024592709: 10.240.26.26: %[call-id [35094667]]% %[agent-id []]% Jul 03 2022 13:46:51.538 +0300: %CCBU_EVENT_POOL-4-worker-22-6-XMPP_PUBLISH_ASYNCHRONOUS: %[NodeId=/finesse/api/User/50015/Dialogs][Payload=<notification xmlns='http://jabber.org/protocol/pubsub'><Update><data><dialog><associatedDialogUri></associatedDialogUri><fromAddress>42893</fromAddress><id>35094667</id><mediaProperties><DNIS></DNIS><callType>AGENT_INSIDE</callType><dialedNumber></dialedNumber><outboundClassification></outboundClassification></mediaProperties><mediaType>Voice</mediaType><participants><Participant><actions><action>DROP</action></actions><mediaAddress>42893</mediaAddress><mediaAddressType>AGENT_DEVICE</mediaAddressType><startTime>2022-07-03T10:46:36.523Z</startTime><state>FAILED</state><stateCause>BAD_DESTINATION</stateCause><stateChangeTime>2022-07-03T10:46:51.537Z</stateChangeTime></Participant></participants><state>FAILED</state><uri>/finesse/api/Dialog/35094667</uri></dialog></data><event>PUT</event><requestId></requestId><source>/finesse/api/Dialog/35094667</source></Update></notification>]: Publishing XMPP Message Asynchronously
0024592710: 10.240.26.26: %[call-id [35094667]]% %[agent-id []]% Jul 03 2022 13:46:51.538 +0300: %CCBU_EVENT_POOL-4-worker-22-6-DnE-CHECKPOINT-5 - : %[Dispatch Stats=CTI_CMD_QUEUE_TIME: 0, CTI_CMD_DISPATCH_TIME: 0, CTI_CMD_EXECUTION_TIME: 1, CTI_CMD_TOTAL_EXECUTION_TIME: 1][Overall Msg Stats=CTI_MSG_RCV_TIME:Jul 03 2022 13:46:51.537,CTI_MSG_TOTAL_PROCESSING_TIME:1,CTI_MSG_DECODE_TIME:0,CTI_MSG_ACMI_NOTIFY_TIME:0,CTI_MSG_DISPATCH_TIME:0,CTI_MSG_PROCESSING_TIME:1,CTI_MSG_LATENCY:0,CTI_SOCKET_WAIT_TIME:0][Payload=CTICallFailedEvent [ invokeID=null, cti_sequence_id=16618878, msgID=16, msgName=CallFailedEvent, deploymentType=CCE, callId=35094667, peripheralId=5000, connectionDeviceIdType=0, failingDeviceType=0, calledDeviceType=0, localConnectionState=1, calledDeviceId=, connectionDeviceId=42893, failingDeviceId=42893, eventCause=13]][SEQ=[42893, 35094667]]: Completed task
0024592711: 10.240.26.26: %[call-id []]% %[agent-id []]% Jul 03 2022 13:46:51.538 +0300: %CCBU_EVENT_POOL-4-worker-22-6-MESSAGE_FROM_CTI_SERVER: %[cti_message=CTICallOriginatedEvent [ invokeID=null, cti_sequence_id=16618879, msgID=15, msgName=CallOriginatedEvent, deploymentType=CCE, callId=35094667, peripheralId=5000, connectionDeviceIdType=0, skillGroupNumber=-1, callingDeviceType=0, calledDeviceType=0, localConnectionState=1, connectionDeviceId=42893, callingDeviceId=42893, calledDeviceId=, eventCause=13]][cti_message_name=CallOriginatedEvent]: Message coming to Finesse from backend cti server
Regards,
Muhammad Arslan
07-25-2022 01:03 AM
Hi Muhammad,
Thats the good finding to track down the actual issue. As you already pointed out in logs, CTICallFailedEvent event from CTI backend is causing the error you observe in desktop. I suggest you to contact TAC to root cause the actual issue back in CCE or CallManager application. This clearly doesn't look like finesse issue. TAC will be able to guide on the CTI end.
-Brinal
07-25-2022 02:08 AM
Hello @bmachado ,
Thanks for the feedback, have you checked the below message in Logs, which is actually coming from the CTI backend, which is showing that no skill group name is found for skill group number 0, also advise what is TAC?
[cti_message_name=CallFailedEvent]: Message coming to Finesse from backend cti server
0024592708: 10.240.26.26: %[call-id [35094667]]% %[agent-id []]% Jul 03 2022 13:46:51.538 +0300: %CCBU_EVENT_POOL-4-worker-22-6-Call.getDialogTO - No skill group name found.: For callId 35094667, no skill group name found for skill group number 0
Regards,
Arslan
08-01-2022 03:59 AM
Hello @bmachado ,
Kindly, your feedback is awaited here, also please advise if we can drop such failed calls or (corrupted calls) using DROP action, as I can see that the Drop is an allowable action in case of participant state is FAILED, but I also need to confirm if call dialog state is also FAILED, can we do the DROP action and will it work as expected?
Dear @dekwan your feedback will be appreciated in this case.
Regards,
Muhammad Arslan
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide