Error APIC EM Installation (Cassandra DB)

[achmad.nazmy]

Hi,

We are installing APIC EM in our VM Lab. But we are facing an error with the installation after completing the config wizard. It seems to be a problem with cassandra DB. Below is the stack trace:

2015-04-15 06:57:04,083 | DEBUG  | 32207 | ShellCmdThread-561552 | 140248664151808 | gr

apevine.common.shell.ShellCommand.ShellCommand   | ShellCommand.py:99 | Command '/opt/cis

co/grapevine-db/cassandra/bin/nodetool -h 192.168.16.1 netstats' finished (ret: 2)

=== stdout ===

Mode: STARTING

Not sending any streams.

=== stderr ===

error: org.apache.cassandra.db:type=StorageProxy

-- StackTrace --

javax.management.InstanceNotFoundException: org.apache.cassandra.db:type=StorageProxy

    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanSer

verInterceptor.java:1095)

    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBea

nServerInterceptor.java:643)

    at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)

    at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.ja

va:1464)

    at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.jav

a:97)

    at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConne

ctionImpl.java:1328)

    at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnect

ionImpl.java:1420)

    at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(RMIConnectionImpl.j

ava:657)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.j

ava:43)

    at java.lang.reflect.Method.invoke(Method.java:606)

    at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)

    at sun.rmi.transport.Transport$2.run(Transport.java:202)

    at sun.rmi.transport.Transport$2.run(Transport.java:199)

    at java.security.AccessController.doPrivileged(Native Method)

    at sun.rmi.transport.Transport.serviceCall(Transport.java:198)

    at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:567)

    at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:82

8)

    at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(TCPTransport.j

ava:619)

    at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:6

84)

    at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(TCPTransport.java:6

81)

    at java.security.AccessController.doPrivileged(Native Method)

    at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:681

)

    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145

)

    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615

)

    at java.lang.Thread.run(Thread.java:745)

    at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCal

l.java:275)

    at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)

    at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)

    at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)

    at javax.management.remote.rmi.RMIConnectionImpl_Stub.getAttribute(Unknown Source

)

    at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getAttrib

ute(RMIConnector.java:906)

    at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHand

ler.java:267)

    at com.sun.proxy.$Proxy13.getReadRepairAttempted(Unknown Source)

    at org.apache.cassandra.tools.NodeProbe.getReadRepairAttempted(NodeProbe.java:944

)

    at org.apache.cassandra.tools.NodeTool$NetStats.execute(NodeTool.java:612)

    at org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:247)

    at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:161)

|

Can please advise what the root cause is? Do we did something wrong with the configuration?

Regards,

Nazmy

[yawming]

Hi Nazmy,

Did you see this on console after finishing the last step of config wizard ?

Thanks,

Yawming

[yawming]

In order to look into more detail can you please collect a RCA bundle of the cluster for further analysis?

How to create a RCA bundle ?

• ssh into the root VM
• Run the "rca" command
• Send us the resulting RCA tarball that is created

If file is too big please put it in dropbox or box and share the link.

Thanks,

Yawming

[achmad.nazmy]

Hi Yawchen,

Thanks for your response.

The error is not displayed in the console. I get the error message from grapevine_config_wizard.log.

Please download the RCA tarbal from the URL below:

Dropbox - grapevine-root-192.168.168.97-rca-2015-04-15_08-01-30-AM_UTC+0000.tar.gz

Thanks,

Nazmy

[yawming · Accepted solution]

Hi Nazmy,

Thanks for the files. Our dev team took a look and found the same IP address is configured on both of the root VM interfaces. Please run the config to correct that.

eth0	Link encap:Ethernet  HWaddr 00:50:56:a7:b4:bc
	inet addr:192.168.168.97 Bcast:192.168.168.255  Mask:255.255.255.0
	UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
	RX packets:6233 errors:0 dropped:0 overruns:0 frame:0
	TX packets:235 errors:0 dropped:0 overruns:0 carrier:0
	collisions:0 txqueuelen:1000
	RX bytes:513013 (513.0 KB)  TX bytes:20534 (20.5 KB)

eth1	Link encap:Ethernet  HWaddr 00:50:56:a7:e7:bc
	inet addr:192.168.168.97  Bcast:192.168.168.255  Mask:255.255.255.0
	UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
	RX packets:6847 errors:0 dropped:68 overruns:0 frame:0
	TX packets:716 errors:0 dropped:0 overruns:0 carrier:0
	collisions:0 txqueuelen:1000
	RX bytes:566677 (566.6 KB)  TX bytes:388742 (388.7 KB)

eth1 interface should have an IP from public network - different than that from the private network IP assigned to eth0.