Re: PerfMonPort "Error found in Adding counters"

Murat Demirel · ‎06-03-2015

Hi team,

After upgrading CUCM from 8.6 to 9.1, we started to receive the following error from PerfMonPort. We cannot get the number of registered hard phones. Do you have any idea why we get this error?

AxisFault

faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.generalException

faultSubcode:

faultString: Error found in Adding counters: Error=9 ErrorMsg=\\10.73.243.10\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.11\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.12\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.13\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.14\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.15\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.16\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.17\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.11\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.12\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.13\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.14\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.15\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.16\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.17\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.18\Cisco CallManager\RegisteredHardwarePhones;

faultActor:

faultNode:

faultDetail:

{http://xml.apache.org/axis/}stackTrace:Error found in Adding counters: Error=9 ErrorMsg=\\10.73.243.10\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.11\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.12\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.13\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.14\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.15\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.16\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.17\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.11\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.12\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.13\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.14\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.15\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.16\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.17\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.18\Cisco CallManager\RegisteredHardwarePhones;

at com.cisco.ccm.serviceability.soap.perfport.PerfmonBindingImpl.perfmonAddCounter(Unknown Source)

at com.cisco.ccm.serviceability.soap.perfport.PerfmonBindingSkeleton.perfmonAddCounter(Unknown Source)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at org.apache.axis.providers.java.RPCProvider.invokeMethod(RPCProvider.java:397)

at org.apache.axis.providers.java.RPCProvider.processMessage(RPCProvider.java:186)

at org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:323)

at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)

at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)

at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)

at org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:454)

at org.apache.axis.server.AxisServer.invoke(AxisServer.java:281)

at org.apache.axis.transport.http.AxisServlet.doPost(AxisServlet.java:699)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)

at org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:277)

at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:274)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAsPrivileged(Subject.java:515)

at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:309)

at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:169)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:299)

at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:57)

at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:193)

at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:189)

at java.security.AccessController.doPrivileged(Native Method)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)

at com.cisco.vos.platform.tomcat.valves.CiscoResponseHeaderFilter.doFilter(Unknown Source)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:277)

at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:274)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAsPrivileged(Subject.java:515)

at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:309)

at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:249)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)

at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:57)

at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:193)

at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:189)

at java.security.AccessController.doPrivileged(Native Method)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)

at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)

at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)

at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:611)

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)

at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)

at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)

at org.apache.catalina.authenticator.SingleSignOn.invoke(SingleSignOn.java:314)

at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)

at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)

at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)

at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)

at java.lang.Thread.run(Thread.java:682)

{http://xml.apache.org/axis/}hostname:KLUCPCCMT1

Error found in Adding counters: Error=9 ErrorMsg=\\10.73.243.10\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.11\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.12\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.13\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.14\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.15\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.16\Cisco CallManager\RegisteredHardwarePhones;\\10.73.243.17\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.11\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.12\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.13\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.14\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.15\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.16\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.17\Cisco CallManager\RegisteredHardwarePhones;\\10.86.55.18\Cisco CallManager\RegisteredHardwarePhones;

at org.apache.axis.message.SOAPFaultBuilder.createFault(SOAPFaultBuilder.java:222)

at org.apache.axis.message.SOAPFaultBuilder.endElement(SOAPFaultBuilder.java:129)

at org.apache.axis.encoding.DeserializationContext.endElement(DeserializationContext.java:1087)

at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:604)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1789)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2950)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)

at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:513)

at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:815)

at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:744)

at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:128)

at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1208)

at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:543)

at javax.xml.parsers.SAXParser.parse(SAXParser.java:395)

at org.apache.axis.encoding.DeserializationContext.parse(DeserializationContext.java:227)

at org.apache.axis.SOAPPart.getAsSOAPEnvelope(SOAPPart.java:696)

at org.apache.axis.Message.getSOAPEnvelope(Message.java:435)

at org.apache.axis.handlers.soap.MustUnderstandChecker.invoke(MustUnderstandChecker.java:62)

at org.apache.axis.client.AxisClient.invoke(AxisClient.java:206)

at org.apache.axis.client.Call.invokeEngine(Call.java:2784)

at org.apache.axis.client.Call.invoke(Call.java:2767)

at org.apache.axis.client.Call.invoke(Call.java:2443)

at org.apache.axis.client.Call.invoke(Call.java:2366)

at org.apache.axis.client.Call.invoke(Call.java:1812)

at com.cisco.schemas.ast.soap.PerfmonBindingStub.perfmonAddCounter(PerfmonBindingStub.java:359)

at PerfMonPort_Test.main(PerfMonPort_Test.java:146)

Thanks and regards,

Murat

npetrele · ‎06-04-2015

It looks like you're using Axis - can I assume you're using Axis-generated Java classes based on the WSDL? If so, did you regenerate these classes using the 9.1 WSDL?

Murat Demirel · ‎06-05-2015

Hi Nicholas,

Yes, we already regenerated the classes using 9.1 WSDL and then we started to get these errors.

Thanks and regards,

Murat

npetrele · ‎06-10-2015

Hi Murat,

I gave up on Axis a while ago because of errors that are difficult to troubleshoot, and now use wsimport. However, I'll keep digging to see if I can find an answer for you.

npetrele · ‎06-10-2015

Hi Murat,

One thing I'm not clear on: You say, "We cannot get the number of registered hard phones." But the error has to do with adding performance counters, not getting the number of registered phones.

Also, if you are looking for a list of phones, are you aware that 9.1 adds a new API called selectCMDeviceExt, which eliminates duplicates across nodes?

Murat Demirel · ‎06-12-2015

Hi Nicholas,

We are getting both the number of registered phones and CPU-memory counters of the servers. We receive same error for both.

In addition, I will check the new API selectCMDeviceExt.

Thanks and regards,

Murat