This document was generated from CDN thread Created by: Phillip Wagoner on 03-02-2011 11:33:15 AM <!-- [DocumentBodyStart:c7c4bacf-a8e7-4717-90f2-11a860cf2399] -->Hello, I'm having a rather unusual issue with Cisco Agent Desktop that I hope somone could give me some insight on. First let me give you a little background. All users in the company, over 300, were on Windows XP SP3 using CAD 4.5.7.4. It was then decided to upgrade everyone to Windows 7. We knew CAD 4.5.7.4 would not be compatible with Windows 7. So intead of upgrading to most current version of CAD it was decided to try and get CAD 4.5.7.4 to work with Windows 7. The only way we were able to do this was install Windows XP Mode to every machine that needed CAD and use it from within the virtual machine. This for the most part has worked great. Except some users, around 30-40, are getting a "Critical Error" message randomly with CAD that will log them out. When I look at the CAD logs I'm finding the same error, "Heartbeat Failure, CallManager Offline." I then went to the JTAPI logs and found some more information, see below. Can anyone give me some insight as to why this is happening and what I can do to fix it? Thank you!! Phillip 429: Feb 03 09:39:30.531 CST %JTAPI-PROTOCOL-7-UNK
P1-10.130.0.12) [ProviderRetryThread] sending: com.cisco.cti.protocol.GetMonitorDNCloseRequest { sequenceNumber = 32 enumHandle = 1 }430: Feb 03 09:39:30.561 CST %JTAPI-PROTOCOL-7-UNK
P1-10.130.0.12) received Response: com.cisco.cti.protocol.GetMonitorDNCloseResponse { sequenceNumber = 32 monitorDNenumerationHandle = 0 }431: Feb 03 09:39:30.561 CST %JTAPI-CTI-7-UNK
P1-10.130.0.12) reopening device (P1-2803) SEP001192CC20BB(0,0)432: Feb 03 09:39:30.621 CST %JTAPI-JTAPI-7-UNK:[8447]Request: removeObserver(com.spanlink.jtapiclient.splkJtapiObj@21d1f4)433: Feb 03 09:39:30.631 CST %JTAPI-JTAPI-7-UNK:AddrObservationEndedEv event to com.spanlink.jtapiclient.splkJtapiObj@21d1f4434: Feb 03 09:39:30.631 CST %JTAPI-JTAPI-7-UNK
P1-2803) [8447] AddrObservationEndedEv [#6]435: Feb 03 09:39:30.641 CST %JTAPI-JTAPIIMPL-7-UNK:[com.spanlink.jtapiclient.splkJtapiObj@21d1f4]ObserverProxy.queueEvents: dispatching synchronously436: Feb 03 09:39:30.641 CST %JTAPI-JTAPIIMPL-7-UNK:[com.spanlink.jtapiclient.splkJtapiObj@21d1f4]ObserverProxy.deliverEvents()437: Feb 03 09:39:30.641 CST %JTAPI-JTAPIIMPL-7-UNK:[com.spanlink.jtapiclient.splkJtapiObj@21d1f4]delivering to addressChangedEvent438: Feb 03 09:39:30.641 CST %JTAPI-JTAPIIMPL-7-UNK:[com.spanlink.jtapiclient.splkJtapiObj@21d1f4]ObserverProxy.deliverEvents() completed439: Feb 03 09:39:30.641 CST %JTAPI-JTAPI-7-UNK:[8447]Request: removeCallObserver(com.spanlink.jtapiclient.splkJtapiObj@21d1f4)440: Feb 03 09:39:30.641 CST %JTAPI-JTAPI-7-UNK:CallObservationEndedEv event to com.spanlink.jtapiclient.splkJtapiObj@21d1f4441: Feb 03 09:39:30.641 CST %JTAPI-JTAPI-7-UNK:8447 CallObservationEndedEv [#7]442: Feb 03 09:39:30.641 CST %JTAPI-JTAPIIMPL-7-UNK:[com.spanlink.jtapiclient.splkJtapiObj@21d1f4]ObserverProxy.queueEvents: dispatching synchronously443: Feb 03 09:39:30.641 CST %JTAPI-JTAPIIMPL-7-UNK:[com.spanlink.jtapiclient.splkJtapiObj@21d1f4]ObserverProxy.deliverEvents()444: Feb 03 09:39:30.641 CST %JTAPI-JTAPIIMPL-7-UNK:[com.spanlink.jtapiclient.splkJtapiObj@21d1f4]delivering to callChangedEvent445: Feb 03 09:39:30.641 CST %JTAPI-JTAPIIMPL-7-UNK:[com.spanlink.jtapiclient.splkJtapiObj@21d1f4]ObserverProxy.deliverEvents() completed446: Feb 03 09:39:30.641 CST %JTAPI-JTAPI-7-UNK
P1-2803){Line:SEP001192CC20BB:8447(0,0)} close () Removed Line into Vector size = 0447: Feb 03 09:39:32.669 CST %JTAPI-JTAPI-7-UNK
P1-2803)[(P1-sa-ccm1) EventThread](P1-2803) Request: removeObserver448: Feb 03 09:39:32.790 CST %JTAPI-JTAPI-7-UNK
rovObservationEndedEv event to com.spanlink.jtapiclient.splkJtapiObj@21d1f4449: Feb 03 09:39:32.820 CST %JTAPI-JTAPI-7-UNK
rovObservationEndedEv [#8]450: Feb 03 09:39:32.840 CST %JTAPI-JTAPIIMPL-7-UNK:[com.spanlink.jtapiclient.splkJtapiObj@21d1f4]ObserverProxy.queueEvents: dispatching synchronously451: Feb 03 09:39:32.850 CST %JTAPI-JTAPIIMPL-7-UNK:[com.spanlink.jtapiclient.splkJtapiObj@21d1f4]ObserverProxy.deliverEvents()452: Feb 03 09:39:32.860 CST %JTAPI-JTAPIIMPL-7-UNK:[com.spanlink.jtapiclient.splkJtapiObj@21d1f4]delivering to providerChangedEvent453: Feb 03 09:39:32.870 CST %JTAPI-JTAPIIMPL-7-UNK:[com.spanlink.jtapiclient.splkJtapiObj@21d1f4]ObserverProxy.deliverEvents() completed454: Feb 03 09:39:32.940 CST %JTAPI-JTAPI-7-UNK
P1-2803)[(P1-sa-ccm1) EventThread](P1-2803) Request: shutdown455: Feb 03 09:39:32.970 CST %JTAPI-MISC-7-UNK
rovider "(P1-2803)" changing state to SHUTDOWN456: Feb 03 09:39:32.980 CST %JTAPI-JTAPI-7-UNK
P1-2803)[(P1-sa-ccm1) EventThread](P1-2803) Request: getObservers457: Feb 03 09:39:55.376 CST %JTAPI-PROTOCOL-7-UNK
P1-10.130.0.12) received server Heartbeat: com.cisco.cti.protocol.Heartbeat { }458: Feb 03 09:40:04.301 CST %JTAPI-PROTOCOL-7-UNK
P1-10.130.0.12) received server Heartbeat: com.cisco.cti.protocol.Heartbeat { }459: Feb 03 09:40:43.068 CST %JTAPI-PROTOCOL-7-UNK
P1-10.130.0.12) received server Heartbeat: com.cisco.cti.protocol.Heartbeat { }460: Feb 03 09:40:43.249 CST %JTAPI-PROTOCOL-7-UNK
P1-10.130.0.12) received Event: com.cisco.cti.protocol.ProviderClosedEvent { eventSequence = 9 reason = 1 }461: Feb 03 09:40:43.259 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) EventThread: queuing com.cisco.cti.protocol.ProviderClosedEvent462: Feb 03 09:40:43.279 CST %JTAPI-CTIIMPL-7-UNK
P1-10.130.0.12) ReceiveThread: caught com.cisco.cti.protocol.DeserializationException: EOF while reading message headerAssociated Bytes:463: Feb 03 09:40:43.560 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) ReceiveThread: exiting464: Feb 03 09:40:43.570 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) ReceiveThread: notifying handler of demise465: Feb 03 09:40:43.580 CST %JTAPI-PROTOCOL-7-UNK
P1-10.130.0.12) received Event: com.cisco.cti.protocol.ProviderOutOfServiceEvent { eventSequence = 10 PROVIDER_OUT_OF_SERVICE_EVENT = 200 }466: Feb 03 09:40:43.580 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) EventThread: queuing com.cisco.cti.protocol.ProviderOutOfServiceEvent467: Feb 03 09:40:43.600 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) ReceiveThread: shutting down468: Feb 03 09:40:43.610 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) ReceiveThread: finished shutting down469: Feb 03 09:40:43.610 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) EventThread: shutting down470: Feb 03 09:40:43.640 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) EventThread: waiting for exit471: Feb 03 09:40:43.289 CST %JTAPI-CTIIMPL-7-UNK
P1-10.130.0.12) EventThread handling event com.cisco.cti.protocol.ProviderClosedEvent[9]472: Feb 03 09:40:43.650 CST %JTAPI-PROTOCOL-7-UNK
P1-10.130.0.12) Received ProviderClosedEvent473: Feb 03 09:40:43.660 CST %JTAPI-CTIIMPL-7-UNK
P1-10.130.0.12) EventThread handling event com.cisco.cti.protocol.ProviderOutOfServiceEvent[10]474: Feb 03 09:40:43.690 CST %JTAPI-CTIIMPL-7-UNK
P1-10.130.0.12) deviceMap is empty? -false475: Feb 03 09:40:43.700 CST %JTAPI-CTIIMPL-7-UNK
P1-10.130.0.12) lineMap is empty? -true476: Feb 03 09:40:43.700 CST %JTAPI-CTIIMPL-7-UNK
P1-10.130.0.12) callMap is empty? -true477: Feb 03 09:40:43.710 CST %JTAPI-CTIIMPL-7-UNK
P1-10.130.0.12) clearing park DN map478: Feb 03 09:40:43.720 CST %JTAPI-JTAPI-7-UNK
P1-2803)[(P1-10.130.0.12) EventThread](P1-2803) Request: getCalls479: Feb 03 09:40:43.730 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) EventThread: exiting480: Feb 03 09:40:43.831 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) EventThread: finished shutting down481: Feb 03 09:40:43.831 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) HeartbeatSendThread shutting down482: Feb 03 09:40:43.851 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) HeartbeatSendThread exiting483: Feb 03 09:40:43.871 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) HeartbeatSendThread waiting for exit484: Feb 03 09:40:43.881 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) HeartbeatSendThread finished shutting down485: Feb 03 09:40:43.901 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) HeartbeatReceiveThread shutting down486: Feb 03 09:40:43.921 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) HeartbeatReceiveThread server heartbeat or message not received for 60600 mili-second487: Feb 03 09:40:43.931 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) HeartbeatReceiveThread exiting488: Feb 03 09:40:43.951 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) HeartbeatReceiveThread waiting for exit489: Feb 03 09:40:43.951 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) HeartbeatReceiveThread finished shutting down490: Feb 03 09:40:43.961 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) DeviceLineUpdateThread: shutting down491: Feb 03 09:40:43.981 CST %JTAPI-MISC-7-UNK:[com.cisco.cti.util.BlockingQueue@9fd93]BlockingQueue:take() Threadname=(P1-10.130.0.12) DeviceLineUpdateThread Thread interrupted492: Feb 03 09:40:43.992 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) DeviceLineUpdateThread: exiting493: Feb 03 09:40:44.012 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) DeviceLineUpdateThread: waiting for exit494: Feb 03 09:40:44.012 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) DeviceLineUpdateThread: finished shutting down495: Feb 03 09:40:44.062 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) ProviderRetryThreadwaiting for 2180 msecs496: Feb 03 09:40:46.260 CST %JTAPI-MISC-7-UNK
P1-10.130.0.12) ProviderRetryThread starting retriesSubject: RE: JTAPI ProviderClosedEvent Replied by: Joseph Moskie on 03-02-2011 01:36:27 PMPhillip,There is clearly a request to shut down the JTAPI Provider coming in from the app here:1454: Feb 03 09:39:32.940 CST %JTAPI-JTAPI-7-UNKP1-2803)[(P1-sa-ccm1) EventThread](P1-2803) Request: shutdown
2455: Feb 03 09:39:32.970 CST %JTAPI-MISC-7-UNKrovider "(P1-2803)" changing state to SHUTDOWN
This would shut down JTAPI, but would not be responsible for the error you are seeing."Heartbeat Failure, CallManager Offline" means exactly what it says. Every few seconds, JTAPI sends a heartbeat to the CUCM server. If it does not respond, we increment a counter, and if it doesn't respond several times in a row, we know we have a connectivity problem to the CUCM server, and we throw the exception you are seeing.There is nothing in the log you posted that shows a Heartbeat Failure.However, I do see another potential issue:1461: Feb 03 09:40:43.259 CST %JTAPI-MISC-7-UNKP1-10.130.0.12) EventThread: queuing com.cisco.cti.protocol.ProviderClosedEvent
2462: Feb 03 09:40:43.279 CST %JTAPI-CTIIMPL-7-UNKP1-10.130.0.12) ReceiveThread: caught com.cisco.cti.protocol.DeserializationException: EOF while reading message header
DeserializationExceptions generally happens when the JTAPI version is higher than the CUCM version.Did you upgrade CAD (and JTAPI) without upgrading the your CUCM servers?What version of the CUCM is running, and what version of JTAPI is being run (generally visible as the first line in the JTAPI log)?Subject: RE: JTAPI ProviderClosedEvent Replied by: Phillip Wagoner on 03-02-2011 02:28:55 PMI'm told our CCM version is 4.1. From the information on one of end user's machine our CAD version isCAD/CSD Product Suite Version 4.5.7.4JTAPI: Cisco JTAPI version 2.1(3.22)ReleaseSubject: RE: JTAPI ProviderClosedEvent Replied by: Phillip Wagoner on 07-02-2011 11:54:41 AMIs this a problem with the product versions we are using? If it was, wouldn't be seeing this with all users and not just a select few? We have three users in particular that this keeps happening with. I see the same exception being thrown on all three computers. Is there anything else that might cause this?Subject: RE: JTAPI ProviderClosedEvent Replied by: Joseph Moskie on 16-02-2011 03:58:07 PMI'm told our CCM version is 4.1. From the information on one of end user's machine our CAD version is
CAD/CSD Product Suite Version 4.5.7.4
JTAPI: Cisco JTAPI version 2.1(3.22)Release
Those versions should be fine, I'm not entirely sure why you'd be getting a DeserializationException. CAD might package a different version that what is recommended for the CCM itself, but I don't know about that. CCM 4.1 has been marked End of Life a while ago, and we no longer support it.