Workaround for Using JConsole with WLS 10gR3 JMX (James Bayer's Blog)

Workaround for Using JConsole with WLS 10gR3 JMX

By james.bayer on March 21, 2009 11:39 AM

4-24-2009 Update – Please see the comments of this entry for a solution that only requires setting the classpath and does not require a custom RMI/IIOP proxy. Thanks to Kai and Mic for commenting with an easier solution. James

I came across a surprising interoperability issue the other day using JConsole to browse WLS 10gR3 JMX MBeans. Read on to see how to I got around it. One of my customers wanted to monitor some WLS services running in a cluster recently, so in doing a little research I decided to pull up JConsole to see what the JMX Beans looked like. Of course WLST works well for this, but I didn’t quite know where the MBean was located so I wanted something visual that I could navigate in a tree fashion without typing commands.

In order to bring up JConsole, you need to follow some basic steps in the documentation to enable RMI on the server you want to connect to: http://edocs.bea.com/wls/docs103/jmxinst/accesscust.html#wp1107229 Be sure to set the default RMI user and password and you will also need to restart the servers after you perform this configuration so it takes effect.

So normally you could just launch JConsole from the JDK’s bin directory and plug-in your connection info. In my case I am using JDK 6 from the WLS Example server which is running on port 7001. This is the URI I put into JConsole:

service:jmx:iiop:///jndi/iiop://localhost:7001/weblogic.management.mbeanservers.runtime

My RMI default user and password is “weblogic”

Well, to my surprise I got a JConsole Output window that kept scrolling with errors. Here is the beginning of the stack:

com.sun.corba.se.impl.encoding.CDRInputStream_1_0_read_value
WARNING: "IOP00810257: (MARSHAL) Could not find class"
 at com.sun.corba.se.imple.logging.ORBUtilSystemException.couldNotFindClass(ORBUtilSystemException:7756)
 ....

I looked around a bit on Google and found that since JDK 5 there has been a bug in the IIOP RMI stack in the JDK. Thankfully there is a work-around that involves creating a simple poor-man’s proxy. So I compiled and ran the RMIIIOPProxy.java file from that link (I removed the package name for brevity) and tried again to connect again while the proxy was running. This time it worked.


C:\temp>java RMIIIOPProxy 7011 weblogic.management.mbeanservers.runtime localhost 7001 weblogic.management.mbeanservers.runtime
Connection established with south server at: service:jmx:iiop:///jndi/iiop://localhost:7001/weblogic.management.mbeanservers.runtime
Proxy server listening at: service:jmx:rmi:///jndi/rmi://localhost:7011/weblogic.management.mbeanservers.runtime
Strike <Enter> to exit

So now there is a simple proxy that listens on port 7011 which then connects to port 7001 on my behalf and gets around the bug. I simply take the JMX URI from the proxy console output and plug that back in to JConsole, this time with a successful connection. Notice that this URI does not include IIOP. Now I get browse the MBeans, see the attributes, and execute operations like I expect. I also tried this with JDK 5’s JConsole and the same issue was present and solved by the work-around.

Permalink

Comments (8)

luke:

Hi James,

I've same problem, so I've tried to use your way, but this it seems doesn't working

this Exception stack trace:

run:

Connection established with south server at: service:jmx:iiop:///jndi/iiop://localhost:7001/weblogic.management.mbeanservers.domainruntime

Proxy server listening at: service:jmx:rmi:///jndi/rmi://localhost:7011/weblogic.management.mbeanservers.domainruntime

Strike to exit

23-apr-2009 14.39.56 com.sun.corba.se.impl.encoding.CDRInputStream_1_0 read_value

AVVERTENZA: "IOP00810257: (MARSHAL) Could not find class"

org.omg.CORBA.MARSHAL: vmcid: SUN minor code: 257 completed: Maybe

at com.sun.corba.se.impl.logging.ORBUtilSystemException.couldNotFindClass(ORBUtilSystemException.java:7756)

at com.sun.corba.se.impl.encoding.CDRInputStream_1_0.read_value(CDRInputStream_1_0.java:1013)

at com.sun.corba.se.impl.encoding.CDRInputStream.read_value(CDRInputStream.java:253)

at com.sun.jmx.remote.internal.PInputStream.read_value(Unknown Source)

at org.omg.stub.javax.management.remote.rmi._RMIConnection_Stub.getMBeanInfo(Unknown Source)

at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getMBeanInfo(RMIConnector.java:1037)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at jmxjmrpproxy.RMIIIOPProxy$MBeanServerProxyHandler.invoke(RMIIIOPProxy.java:139)

at $Proxy0.getMBeanInfo(Unknown Source)

at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1420)

at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)

at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1264)

at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1359)

at javax.management.remote.rmi.RMIConnectionImpl.getMBeanInfo(RMIConnectionImpl.java:864)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)

at sun.rmi.transport.Transport$1.run(Transport.java:159)

at java.security.AccessController.doPrivileged(Native Method)

at sun.rmi.transport.Transport.serviceCall(Transport.java:155)

at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)

at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)

at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:619)

Caused by: java.lang.ClassNotFoundException

thanks in advance

Posted by luke | April 23, 2009 6:16 AM

Posted on April 23, 2009 06:16

James Bayer:

Luke, I'm not sure what to tell you. I don't have that problem on my machine with WLS 10.3 using the proxy. Are you using the same JVM for the Proxy and JConsole that you're using on WLS? You might also want to open a support ticket to get additional help or check the WLS General forum on OTN.

Thanks, James

Posted by James Bayer | April 23, 2009 6:50 AM

Posted on April 23, 2009 06:50

Mic:

Hello,

We found a workaround that doesn't need to compile and run a proxy.

Just set JAVA_HOME and BEA_HOME and run:

RedHat
jconsole -J-Djava.class.path=$JAVA_HOME/lib/jconsole.jar:/usr/bea40/wlserver_10.3/server/lib/wljmxclient.jar -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote

Windows
jconsole -J-Djava.class.path=%JAVA_HOME%\lib\jconsole.jar;%BEA_HOME%\wlserver_10.3\server\lib\wljmxclient.jar -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote

Connection string:
service:jmx:rmi:///jndi/iiop://localhost:7001/weblogic.management.mbeanservers.domainruntime

Cheers, Mic

Posted by Mic | April 24, 2009 5:04 AM

Posted on April 24, 2009 05:04

Kai Ellinger:

1. After having the same issue, it turned out that it is just a CLASSPATH issue when using jConsole from the latest SUN JVM:

a) Working for connect string starting with 'service:jmx:rmi:///jndi/iiop://...' only.
set JAVA_HOME=D:\SUN_jdk1.6.0_13
set PATH=%JAVA_HOME%\bin;%PATH%
jconsole -J-Djava.class.path=%JAVA_HOME%\lib\jconsole.jar;%bea_home%\wlserver_10.3\server\lib\wljmxclient.jar

b) Working for connect string starting with 'service:jmx:rmi:///jndi/iiop://...' and 'service:jmx:iiop:///jndi/iiop://...' .

set JAVA_HOME=D:\SUN_jdk1.6.0_13
set PATH=%JAVA_HOME%\bin;%PATH%
jconsole -J-Djava.class.path=%JAVA_HOME%\lib\jconsole.jar;%bea_home%\wlserver_10.3\server\lib\weblogic.jar

NOTE: The RMI or IIOP in the connect string!

c) I tried the following URLs that worked fine:

* service:jmx:rmi:///jndi/iiop://localhost:7001/weblogic.management.mbeanservers.runtime

* service:jmx:iiop:///jndi/iiop://localhost:7001/weblogic.management.mbeanservers.runtime

* service:jmx:rmi:///jndi/iiop://localhost:7001/weblogic.management.mbeanservers.domainruntime

* service:jmx:iiop:///jndi/iiop://localhost:7001/weblogic.management.mbeanservers.domainruntime

2. In addition, the parameter -J-Djmx.remote.proto.provider.pkgs=weblogic.management.remote can be set if the current version of the JVM has any issue:

jconsole -J-Djava.class.path=%JAVA_HOME%\lib\jconsole.jar;%bea_home%\wlserver_10.3\server\lib\wljmxclient.jar -J-Djmx.remote.proto.provider.pkgs=weblogic.management.remote

I hope this helps a bit!

Posted by Kai Ellinger | April 24, 2009 7:36 AM

Posted on April 24, 2009 07:36

John Lee:

visualvm seems to have the same problem. Any idea what the fix for it would be?

Posted by John Lee | June 4, 2009 7:08 AM

Posted on June 4, 2009 07:08

Tobias Hoevel:

Thanks for all your useful hints and comments, you helped me solving this painful issue.

It seems the MBean deserialization bug (causing the interoperability problem) has currently been examined and fixed in JVM Rel. 1.6.0_11(JRockit) and 1.6.0.12 (Sun). :-)

Posted by Tobias Hoevel | July 6, 2009 12:24 AM

Posted on July 6, 2009 00:24

Dave:

hi, guys

I tried the above suggestions but it's not work. I am using weblogic92, JDK 1.5. I also changed the weblogic domain to allow anonymous JMX access but still no luck.

The exception is the permission problem I guess:

Exception in thread "JConsole.addUrl" org.omg.CORBA.NO_PERMISSION: vmcid: 0x0 minor code: 0 completed: No

Posted by Dave | July 22, 2009 11:28 AM

Posted on July 22, 2009 11:28

Kai Ellinger:

As far as I'm aware of, the described steps are only needed in a 1.6 JVM.

Always make sure sure that you have weblogic.jar in you class path. Best is to do this using $ cd domain_hopme/bin; . ./setDomainEnv.sh to have any other dependencies as well.

I would try to start

$ cd domain_hopme/bin
$ . ./setDomainEnv.sh
$ jconsole -J-Djava.class.path=$CLASSPATH -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote

And than connect to the mentioned URLs.

The org.omg.CORBA.NO_PERMISSION might come from not allowing anonymous IIOP access and setting the default IIOP user to the weblogic user under section servers/serverName/protocols/iiop of the WLS Admin Console.

Posted by Kai Ellinger | September 24, 2009 10:08 AM

Posted on September 24, 2009 10:08

James Bayer's Blog