Workaround for Using JConsole with WLS 10gR3 JMX

4-24-2009 Update – Please see the comments of this entry for a solution that only requires setting the classpath and does not require a custom RMI/IIOP proxy.  Thanks to Kai and Mic for commenting with an easier solution.  James

I came across a surprising interoperability issue the other day using JConsole to browse WLS 10gR3 JMX MBeans.  Read on to see how to I got around it.  One of my customers wanted to monitor some WLS services running in a cluster recently, so in doing a little research I decided to pull up JConsole to see what the JMX Beans looked like.  Of course WLST works well for this, but I didn’t quite know where the MBean was located so I wanted something visual that I could navigate in a tree fashion without typing commands.

In order to bring up JConsole, you need to follow some basic steps in the documentation to enable RMI on the server you want to connect to: http://edocs.bea.com/wls/docs103/jmxinst/accesscust.html#wp1107229  Be sure to set the default RMI user and password and you will also need to restart the servers after you perform this configuration so it takes effect.

So normally you could just launch JConsole from the JDK’s bin directory and plug-in your connection info.  In my case I am using JDK 6 from the WLS Example server which is running on port 7001.  This is the URI I put into JConsole:

service:jmx:iiop:///jndi/iiop://localhost:7001/weblogic.management.mbeanservers.runtime

My RMI default user and password is “weblogic”

Well, to my surprise I got a JConsole Output window that kept scrolling with errors.  Here is the beginning of the stack:

com.sun.corba.se.impl.encoding.CDRInputStream_1_0_read_value
WARNING: "IOP00810257: (MARSHAL) Could not find class"
 at com.sun.corba.se.imple.logging.ORBUtilSystemException.couldNotFindClass(ORBUtilSystemException:7756)
 ....

I looked around a bit on Google and found that since JDK 5 there has been a bug in the IIOP RMI stack in the JDK.  Thankfully there is a work-around that involves creating a simple poor-man’s proxy.  So I compiled and ran the RMIIIOPProxy.java file from that link (I removed the package name for brevity) and tried again to connect again while the proxy was running.  This time it worked.

C:\temp>java RMIIIOPProxy 7011 weblogic.management.mbeanservers.runtime localhost 7001 weblogic.management.mbeanservers.runtime
Connection established with south server at: service:jmx:iiop:///jndi/iiop://localhost:7001/weblogic.management.mbeanservers.runtime
Proxy server listening at: service:jmx:rmi:///jndi/rmi://localhost:7011/weblogic.management.mbeanservers.runtime
Strike <Enter> to exit

So now there is a simple proxy that listens on port 7011 which then connects to port 7001 on my behalf and gets around the bug.  I simply take the JMX URI from the proxy console output and plug that back in to JConsole, this time with a successful connection.  Notice that this URI does not include IIOP.  Now I get browse the MBeans, see the attributes, and execute operations like I expect.  I also tried this with JDK 5’s JConsole and the same issue was present and solved by the work-around.

cropper

Comments:

Hi James, I've same problem, so I've tried to use your way, but this it seems doesn't working this Exception stack trace: run: Connection established with south server at: service:jmx:iiop:///jndi/iiop://localhost:7001/weblogic.management.mbeanservers.domainruntime Proxy server listening at: service:jmx:rmi:///jndi/rmi://localhost:7011/weblogic.management.mbeanservers.domainruntime Strike to exit 23-apr-2009 14.39.56 com.sun.corba.se.impl.encoding.CDRInputStream_1_0 read_value AVVERTENZA: "IOP00810257: (MARSHAL) Could not find class" org.omg.CORBA.MARSHAL: vmcid: SUN minor code: 257 completed: Maybe at com.sun.corba.se.impl.logging.ORBUtilSystemException.couldNotFindClass(ORBUtilSystemException.java:7756) at com.sun.corba.se.impl.encoding.CDRInputStream_1_0.read_value(CDRInputStream_1_0.java:1013) at com.sun.corba.se.impl.encoding.CDRInputStream.read_value(CDRInputStream.java:253) at com.sun.jmx.remote.internal.PInputStream.read_value(Unknown Source) at org.omg.stub.javax.management.remote.rmi._RMIConnection_Stub.getMBeanInfo(Unknown Source) at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getMBeanInfo(RMIConnector.java:1037) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at jmxjmrpproxy.RMIIIOPProxy$MBeanServerProxyHandler.invoke(RMIIIOPProxy.java:139) at $Proxy0.getMBeanInfo(Unknown Source) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1420) at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1264) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1359) at javax.management.remote.rmi.RMIConnectionImpl.getMBeanInfo(RMIConnectionImpl.java:864) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305) at sun.rmi.transport.Transport$1.run(Transport.java:159) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:155) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) Caused by: java.lang.ClassNotFoundException thanks in advance

Posted by luke on April 22, 2009 at 09:16 PM PDT #

Luke, I'm not sure what to tell you. I don't have that problem on my machine with WLS 10.3 using the proxy. Are you using the same JVM for the Proxy and JConsole that you're using on WLS? You might also want to open a support ticket to get additional help or check the WLS General forum on OTN. Thanks, James

Posted by James Bayer on April 22, 2009 at 09:50 PM PDT #

Hello, We found a workaround that doesn't need to compile and run a proxy. Just set JAVA_HOME and BEA_HOME and run: RedHat jconsole -J-Djava.class.path=$JAVA_HOME/lib/jconsole.jar:/usr/bea40/wlserver_10.3/server/lib/wljmxclient.jar -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote Windows jconsole -J-Djava.class.path=%JAVA_HOME%\lib\jconsole.jar;%BEA_HOME%\wlserver_10.3\server\lib\wljmxclient.jar -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote Connection string: service:jmx:rmi:///jndi/iiop://localhost:7001/weblogic.management.mbeanservers.domainruntime Cheers, Mic

Posted by Mic on April 23, 2009 at 08:04 PM PDT #

1. After having the same issue, it turned out that it is just a CLASSPATH issue when using jConsole from the latest SUN JVM: a) Working for connect string starting with 'service:jmx:rmi:///jndi/iiop://...' only. set JAVA_HOME=D:\SUN_jdk1.6.0_13 set PATH=%JAVA_HOME%\bin;%PATH% jconsole -J-Djava.class.path=%JAVA_HOME%\lib\jconsole.jar;%bea_home%\wlserver_10.3\server\lib\wljmxclient.jar b) Working for connect string starting with 'service:jmx:rmi:///jndi/iiop://...' and 'service:jmx:iiop:///jndi/iiop://...' . set JAVA_HOME=D:\SUN_jdk1.6.0_13 set PATH=%JAVA_HOME%\bin;%PATH% jconsole -J-Djava.class.path=%JAVA_HOME%\lib\jconsole.jar;%bea_home%\wlserver_10.3\server\lib\weblogic.jar NOTE: The RMI or IIOP in the connect string! c) I tried the following URLs that worked fine: * service:jmx:rmi:///jndi/iiop://localhost:7001/weblogic.management.mbeanservers.runtime * service:jmx:iiop:///jndi/iiop://localhost:7001/weblogic.management.mbeanservers.runtime * service:jmx:rmi:///jndi/iiop://localhost:7001/weblogic.management.mbeanservers.domainruntime * service:jmx:iiop:///jndi/iiop://localhost:7001/weblogic.management.mbeanservers.domainruntime 2. In addition, the parameter -J-Djmx.remote.proto.provider.pkgs=weblogic.management.remote can be set if the current version of the JVM has any issue: jconsole -J-Djava.class.path=%JAVA_HOME%\lib\jconsole.jar;%bea_home%\wlserver_10.3\server\lib\wljmxclient.jar -J-Djmx.remote.proto.provider.pkgs=weblogic.management.remote I hope this helps a bit!

Posted by Kai Ellinger on April 23, 2009 at 10:36 PM PDT #

visualvm seems to have the same problem. Any idea what the fix for it would be?

Posted by John Lee on June 03, 2009 at 10:08 PM PDT #

Thanks for all your useful hints and comments, you helped me solving this painful issue. It seems the MBean deserialization bug (causing the interoperability problem) has currently been examined and fixed in JVM Rel. 1.6.0_11(JRockit) and 1.6.0.12 (Sun). :-)

Posted by Tobias Hoevel on July 05, 2009 at 03:24 PM PDT #

hi, guys I tried the above suggestions but it's not work. I am using weblogic92, JDK 1.5. I also changed the weblogic domain to allow anonymous JMX access but still no luck. The exception is the permission problem I guess: Exception in thread "JConsole.addUrl" org.omg.CORBA.NO_PERMISSION: vmcid: 0x0 minor code: 0 completed: No

Posted by Dave on July 22, 2009 at 02:28 AM PDT #

As far as I'm aware of, the described steps are only needed in a 1.6 JVM. Always make sure sure that you have weblogic.jar in you class path. Best is to do this using $ cd domain_hopme/bin; . ./setDomainEnv.sh to have any other dependencies as well. I would try to start $ cd domain_hopme/bin $ . ./setDomainEnv.sh $ jconsole -J-Djava.class.path=$CLASSPATH -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote And than connect to the mentioned URLs. The org.omg.CORBA.NO_PERMISSION might come from not allowing anonymous IIOP access and setting the default IIOP user to the weblogic user under section servers/serverName/protocols/iiop of the WLS Admin Console.

Posted by Kai Ellinger on September 24, 2009 at 01:08 AM PDT #

James, I am able to make a remote JConsole connection to a Linux-WebLogic system from Windows using T3 - but when the JConsole window comes up, the Summary, Threads, Memory, Overview tabs are all disabled - only the MBeans tab is enabled, and in the contents I cannot find java.lang. What do I need to do to access the other tabs?

Posted by Calvin Arnason on June 07, 2011 at 06:49 AM PDT #

Calvin, To get the JConsole tabs to work and show the platform MBeans, then you need to use the JVM options like this that I put at the top of DOMAIN_HOME/bin/setDomainEnv.sh export JAVA_OPTIONS="${JAVA_OPTIONS} -Dcom.sun.management.jmxremote.port=7091 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false" Then I start jconsole on the remote machine like this: jconsole hostWhereWLSIsRunning:7091

Posted by james.bayer on June 07, 2011 at 07:34 AM PDT #

Post a Comment:
Comments are closed for this entry.
About

James Bayer Image
I was formerly a Product Manager on the WebLogic Server team based out of Oracle HQ. You can find my new blog at http://iamjambay.com.
Follow Me on Twitter
Oracle WebLogic

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today