Tuesday May 22, 2007

Troubleshooting BOF (JavaOne 2007) slides

We received few comments/emails asking for the slides of troubleshooting tools BOF (JavaOne 2007). Alan Bateman has uploaded the slides (PDF) of the troubleshooting tools BOF.

Wednesday May 16, 2007

Programmatically dumping heap from Java applications

In the troubleshooting BOF, we demonstrated how to use the jmap -dump option to dump heap dump of a running application and showed how to browse/analyze the resulting binary heap dump using the jhat tool.

One of the questions was how to programmatically dump the heap from applications. For example, you may want to dump multiple heap snapshots from your application at various points in time and analyze those off line using jhat. Yes, you can dump heap from your application -- but you have to do a bit of programming as shown below:


import javax.management.MBeanServer;
import java.lang.management.ManagementFactory;
import com.sun.management.HotSpotDiagnosticMXBean;

public class HeapDumper {
    // This is the name of the HotSpot Diagnostic MBean
    private static final String HOTSPOT_BEAN_NAME =
         "com.sun.management:type=HotSpotDiagnostic";

    // field to store the hotspot diagnostic MBean 
    private static volatile HotSpotDiagnosticMXBean hotspotMBean;

    /\*\*
     \* Call this method from your application whenever you 
     \* want to dump the heap snapshot into a file.
     \*
     \* @param fileName name of the heap dump file
     \* @param live flag that tells whether to dump
     \*             only the live objects
     \*/
    static void dumpHeap(String fileName, boolean live) {
        // initialize hotspot diagnostic MBean
        initHotspotMBean();
        try {
            hotspotMBean.dumpHeap(fileName, live);
        } catch (RuntimeException re) {
            throw re;
        } catch (Exception exp) {
            throw new RuntimeException(exp);
        }
    }

    // initialize the hotspot diagnostic MBean field
    private static void initHotspotMBean() {
        if (hotspotMBean == null) {
            synchronized (HeapDumper.class) {
                if (hotspotMBean == null) {
                    hotspotMBean = getHotspotMBean();
                }
            }
        }
    }

    // get the hotspot diagnostic MBean from the
    // platform MBean server
    private static HotSpotDiagnosticMXBean getHotspotMBean() {
        try {
            MBeanServer server = ManagementFactory.getPlatformMBeanServer();
            HotSpotDiagnosticMXBean bean = 
                ManagementFactory.newPlatformMXBeanProxy(server,
                HOTSPOT_BEAN_NAME, HotSpotDiagnosticMXBean.class);
            return bean;
        } catch (RuntimeException re) {
            throw re;
        } catch (Exception exp) {
            throw new RuntimeException(exp);
        }
    }

    public static void main(String[] args) {
        // default heap dump file name
        String fileName = "heap.bin";
        // by default dump only the live objects
        boolean live = true;

        // simple command line options
        switch (args.length) {
            case 2:
                live = args[1].equals("true");
            case 1:
                fileName = args[0];
        }

        // dump the heap
        dumpHeap(fileName, live);
    }
}

By including the above class in your application, you can call HeapDumper.dumpHeap whenever you want to dump the heap as shown in the sample main.

Important note: while you can dump multiple heap snapshots from your application, you can not correlate the objects from multiple dumps. jmap tool uses object addresses as object identifiers - which vary between garbage collections [recall that GC may move objects which changes the object addresses]. But, you can correlate by aggregate stat such as histogram etc.

Tuesday May 15, 2007

JavaOne'07 -- my summay.

After 19 hours of flight, I finally reached Chennai from SFO (on Sunday morning). I'm still going through the timezone change. I wish for a "Star Trek" like transport :-)

This JavaOne turned to be a "device" JavaOne for me -- I attended talks on SunSPOT, Java ME, Blu-ray, OCAP etc. In some sense, it is device JavaOne anyway -- after all one of the big announcements is about JavaFX mobile. I wanted to attend, but missed the talk on "x86 (JPC?) Java emulator".

Random bits from this year's JavaOne:

  • Our troubleshooting BOF went well. As usual there were many questions to the point that they would hardly let me finish jhat demo :-).
  • I was at the performance and diagnostics POD for some time - mostly demonstrating jconsole, jhat to few folks.
  • I felt that there was more crowd this time. JavaOne is getting bigger and bigger. Long queues everywhere ...
  • On Friday, I had a chat with Fat Cat Air. The chat was mostly about the flight that he is building and not about "tiered compilation" :-)
  • I had to travel to L.A for a day. I stayed with my friend Gurumoorthy over the week end. Now, I've a new friend "Akhilan" (Gurumoorthy's son).
  • I had to explain quite a few folks how I work at Chennai when there is no Sun office at Chennai. Yes, I work from home. If you are curious, you may want to check out OpenWork.
That's all....

Friday May 11, 2007

JavaOne'07 Thur (May 10)

I attended only two sessions:
  • TV sessions kick off -- introducing to many Java TV related technologies. One problem with the TV domain is that there are loo...ots of acronyms! One feels lost in the words ... For example, GEM stands for "Globally Executable MHP" and "MHP" stands for Multimedia Home Platform and so on. Add to that the telecommunication/MPEG related jargons. You may want to check out a few sites that I find interesting. Sumit Malik (of Sony) showed very interesting demos on Blu-Ray.
  • I attended the talk on OCAP. Too many APIs :-)
One important aspect of JavaOne is about meeting people as much as attending sessions/BOFs. I met and had nice chat with Matthias Ernst. And of course many of the Sun folks as well. I got the CD for blu-ray development - there is a competition to be won (52 inch LCD HDTV!) :-) :-)

Thursday May 10, 2007

JavaOne'07 Wed (May 9)

I attended the following sessions/BOFs (in no specific order):
  • Building PhoneME project and applications on it. The speakers (Stuart Marks and Hinkmond Wong) demonstrated how to SVN checkout, build the platform - both Phone ME feature (CLDC) and Phone ME advanced (CDC). A sample protocol handler (called "upcase" protocol) was added to the platform and the platform was built. Also, the speakers showed how to build CLDC/CDC apps using NetBeans IDE. Stuart Marks made the IDE to use the Phone ME feature build rather than the bundled wireless toolkit version of CLDC [He didn't tell how to do so. You may want to watch out for Terrence Barr's blog.
  • Vincent Hardy's talk on using JSR 226, 287 and 290 (SVG/XHTML/CSS/ECMAScript) on phones. It was interesting talk. I knew a bit about SVG format - but I learned about how to use it on Java ME platform.
  • JavaFX talk by Chris Oliver. Seeing is believing -- you got to see the stuff - you may want to download and try it out!.
  • I attended the session on developing flashy graphics for Java ME platform -- it was mostly the "common sense" stuff ((like don't call System.gc() explicitly, use double-buffering etc). But, sometimes it is worthwhile reminding the "obvious".
  • I attended the Java SE performance BOF - it was an interactive Q & A session. The "performance gurus" of Sun answered the questions. Most questions were about GC - as one would expect in such sessions :-)

Wednesday May 09, 2007

JavaOne'07 Tuesday (May 8)

Here is what I did on the first day of JavaOne 2007.
  • I attended the keynote session - main messages are:
    • "JavaFX script" language - a dynamic, statically typed language for GUI/Java2D/Swing type applications (can be used for general purpose as well). It was formerly called F3. In future, there will be an authoring tool that emits JavaFX script.
    • JavaFX/Mobile - complete software platform for mobible - sort of like "Java OS" for mobile phones. You have Java SE, yes - you read it right, Java SE for mobile phones! And JavaFX script wil run on this platform too.
    • OpenJDK
    • NetBeans 6.0
    I attended the following talks:
    • Language oriented programming - this was very nice presentation on DSLs (Domain Specific Languages). Very nice examples for internal DSLs (where you stay within the general purpose language) in Groovy, Java and JRuby - then followed by brief description of external DSLs (do it in the hard way - create your own lexer, parser) and tools for the same. Interesting talk!
    • Then, I attended the talk on SunSPOT. The talk was around constructing virtual reality/game/3D stuff using SunSPOT devices and desktop. SunSPOT runs Squawk - VM that runs on the bare ARM-9 hardware and provides low-level operating system support, as well as application isolation (isolates). Over the week end, I played on Wii console [which my friend bought recently]. After attending this session on SunSPOT, I appreciate how difficult it is to create such a system! Cool stuff!
    At 8.00 PM, Alan, Mandy and I presented the usual JDK troubleshooting tools BOF. There were lots of questions. We wanted it to be interactive - it was so interactive that I could barely complete the demo on jhat tool :-) If you are keen on improving heap dumps, you may want to join and contribute http://heap-snapshot.dev.java.net project. One important suggestion was to create a web page of tools and products that import hprof heap dump format files. After the talk, we discussed on importing the heap dump to a database and allowing SQL queries -- so that we can handle very large heap dumps. We will have to revisit this. But, now that we have Open JDK, you can contribute with new ideas and implementations/improvements!

Thursday May 03, 2007

Meet you at BOF - 2816 JavaOne 2007

As usual, there is troubleshooting BOF this year as well (2816 - Tuesday 8.00 PM). Alan Bateman, Mandy Chung and yours truely will present. Meet you all there!

Wednesday Apr 18, 2007

Desktop scripting applications with NetBeans 5.5

How about developing desktop Java applications using scripting, XML and database with NetBeans? This afternoon I played with scripting using NetBeans 5.5. Before proceeding further, I will describe the devepment environment:

In addition, I've CVS checked out the following java.net projects:

I started with a simple Java program project. Added a script file with the name "hello.js" to the project:

I added the following code using the JavaScript editor:

JavaScript editor supports syntax highlighting etc. The "importPackage" built-in function imports a Java package to script."importClass" imports a specific Java class. Everything above is straight forward -- except may be the listener for the button. JavaScript engine takes care of wrapping the supplied JavaScript function to ActionListener interface. Also, JavaScript engines supports JavaBean convention so that we can write f.visible = true instead of f.setVisible(true).

I added the following code to the main class:


package helloscript;

import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import javax.script.ScriptEngine;
import javax.script.ScriptEngineManager;

public class Main {
    public static void main(String[] args) throws Exception {
        // create manager
        ScriptEngineManager m = new ScriptEngineManager();
        // create javascript script engine
        ScriptEngine js = m.getEngineByName("javascript");
        // evaluate "hello.js"
        InputStream strm = Main.class.getResourceAsStream("/hello.js");
        Reader r = new InputStreamReader(strm);
        js.eval(r);
    }   
}

Note that NetBeans copies "hello.js" to build/classes directory and packages it into dist/HelloScript.jar as well. So, the above getResourceAsStream works as expected. When running the project, I got this:

Now, how about throwing some database access and XML? I'm too lazy to use JDBC API directly. I wanted to try out the persistence API. So, I followed the steps below to generate entity classes from the sample database [shipped with NetBeans enterprise pack? I've no idea, as long as it works... :-) ]. You probably know this already: You do not need to use Java EE to use the persistence API. You can use it within Java SE

Password is also "app". Don't forget to choose "Remember password during this session" option!

I added all tables from the sample db and I just selected package name to be "helloscript".

Created "persistence unit" as suggested!

Now, I need to add Java code to fetch all "Customer" instances from DB. So, I added the following to Main.java:



package helloscript;

import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.util.List;
import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;
import javax.persistence.Persistence;
import javax.persistence.Query;
import javax.script.ScriptEngine;
import javax.script.ScriptEngineManager;

public class Main {
    public static void main(String[] args) throws Exception {
        // create JavaScript engine
        ScriptEngineManager m = new ScriptEngineManager();
        ScriptEngine js = m.getEngineByName("javascript");
        
        // Create Entity manager factory - "HelloScriptPU"
        // is the name given when creating persistence unit
        EntityManagerFactory emf = Persistence.createEntityManagerFactory("HelloScriptPU");
        // create Entity manager
        EntityManager em = emf.createEntityManager();

        // select all Customer instances
        Query q = em.createQuery("SELECT c FROM Customer c");

        // get the result list (of Customers)
        List l = q.getResultList();

        // expose the Customer list to script
        // as a global variable by the name "customers"
        js.put("customers", l);

        // now evaluate script -- the script may use
        // "customers" variable        
        InputStream strm = Main.class.getResourceAsStream("/hello.js");
        Reader r = new InputStreamReader(strm);
        js.eval(r);
    }   
}

Now, I added logic to display the customer list in the JavaScript code:

Again, I've used JavaBean convention support to access Customer name and email -- so that I wrote c.name and c.email instead of calling c.getName() and c.getEmail() on the entity instances. I used JEditorPane to display the generated HTML table inside the JFrame. When I ran the above code. I got the following error:


Exception in thread "main" Local Exception Stack: 
Exception [TOPLINK-4003] (Oracle TopLink Essentials - 2006.8 (Build 060830)): oracle.toplink.essentials.exceptions.DatabaseException
Exception Description: Configuration error.  Class [org.apache.derby.jdbc.ClientDriver] not found.
        at oracle.toplink.essentials.exceptions.DatabaseException.configurationErrorClassNotFound(DatabaseException.java:86)
        at oracle.toplink.essentials.sessions.DefaultConnector.loadDriver(DefaultConnector.java:168)
        at oracle.toplink.essentials.sessions.DefaultConnector.connect(DefaultConnector.java:83)
        at oracle.toplink.essentials.sessions.DatasourceLogin.connectToDatasource(DatasourceLogin.java:170)
        at oracle.toplink.essentials.internal.sessions.DatabaseSessionImpl.loginAndDetectDatasource(DatabaseSessionImpl.java:537)
        at oracle.toplink.essentials.ejb.cmp3.EntityManagerFactoryProvider.login(EntityManagerFactoryProvider.java:180)
        at oracle.toplink.essentials.internal.ejb.cmp3.EntityManagerSetupImpl.deploy(EntityManagerSetupImpl.java:230)
        at oracle.toplink.essentials.internal.ejb.cmp3.base.EntityManagerFactoryImpl.getServerSession(EntityManagerFactoryImpl.java:78)
        at oracle.toplink.essentials.internal.ejb.cmp3.base.EntityManagerFactoryImpl.createEntityManagerImpl(EntityManagerFactoryImpl.java:113)
        at oracle.toplink.essentials.internal.ejb.cmp3.base.EntityManagerFactoryImpl.createEntityManagerImpl(EntityManagerFactoryImpl.java:107)
        at oracle.toplink.essentials.internal.ejb.cmp3.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:76)
        at helloscript.Main.main(Main.java:20)
Java Result: 1

Seems like persistence runtime did not find the driver classes. I added derbyclient.jar from $JDK/db/lib directory to the project's "Libraries".

After that, it ran as expected!

But, I mentioned XML. Where is it? Well, I generated (X)HTML for table :-) How about using E4X? (easier interface to work with XML). Although Mozilla Rhino JavaScript supports E4X, the bundled version in JDK 6 does not support E4X. Rhino's E4X implementation depends on Apache XML Beans. We do have a jsr-223 script engine for "full" Rhino with E4X support in the http://scripting.dev.java.net project. I added js-engine.jar from this project, js.jar [Rhino jar - either download it from Mozilla site or from CVS workspace of scripting project] and xmlbeans.jar [which I got from the Phobos project $PHOBOS/phobos/dist/lib/xmlbeans.jar] to the project:

Then, I modified the "hello.js" script to use E4X:

Note that the HTML table generation does not involve strings. I use E4X to generate it -- I've expressions inside { } for the 'dynamic' portions of the (X)HTML. Also, in the Java code, I made a small change:


        ScriptEngine js = m.getEngineByName("rhino-nonjdk");

Note that I used "rhino-jdk" as the engine name instead of "javascript" so that the manager would choose the jsr-223 engine that is bundled with JDK 6. Without this, it would choose the not bundled jsr-223 engine in rt.jar. With the change above, E4X works! I learned a bit about NetBeans 5.5 JavaScript support, persistence API and E4X. Hmm..., some day I need to try out the bleeding edge stuff in NetBeans 6...

Tuesday Apr 10, 2007

SML subset for TEDI?

I came across "Do we need a new kind of schema language?" from Tim Bray's blog. After reading it, I can't resist this: why not use a subset of Standard ML for this purpose?

  • Basic types - SML has int, real, bool, char, string
  • Tuples - SML has t1\*t2\*t3... (for example, int\*bool, int\*string\*real and so on)
  • Lists - 'a list - int list, string list and so on
  • Records - {name1:t1, name2:t2 } (for example, {name:string, age:int})
  • type for type abbreviations - useful when defining "bigger" schema.
  • datatype for algebraic data types
  • For larger schemas, we could use parameterized data types.
  • For larger schemas, we could also use SML structures to group related types.

In the above, we have every type expression except for function types! (IMO, it is not a big list!). Annotations for bindings may be specified in SML comments [this needs more thought].



The example in James Clark's page

 { url: String, width: Integer?, height: Integer?, title: String }
 
would become the following in SML:

 { url: string, width: Maybe int, height: Maybe int, title: string }

where Maybe is

  datatype 'a Maybe = Just of 'a | Nothing

Pros:
  • We can have parametric types and module system - may be useful, for defining larger, generic schema.
  • Tuples, Lists, Records could be mapped to parametrized classes (like in Scala, Java etc.) or native data types in scripting languages.
  • sum-of-product types can be translated as classes (like case classes in Scala)
  • Type expressions are proper set of one particular language - at least few people would feel at home :-) JSON is proper subset of JavaScript object literals, functional folks could have their turn :-). BTW, I am okay with Haskell as well. Personally, I've played with SML little bit more :-)
  • If we want to have schema specified default values of various elements of data, we can include proper subset of value definitions in SML.

Monday Apr 02, 2007

Retrieving .class files from a Java core dump

In my last blog entry, I explained how to retrieve .class files from a running Java application. How about retrieving .class files from a Java core dump? You may have got a core dump from JVM due to
  • a bug in some JNI code [you may want to run your application with the -Xcheck:jni flag during development]
  • a bug in JVM. You got core dump and hs_err_pid<NNN> file [which you can decode online!]
  • or you may have forced core dump by a tool such as gcore

When you debug native core dumps, you may have asked your customer for the matching shared objects [or DLLs]. This is because text (a.k.a "code") pages are not dumped into core dump in most operating systems [although you could configure any page to be dumped into core on Solaris 10+]

In JVM, the equivalent of "text" section is the JVM internal data structures created by parsing the loaded .class files. Such data structures are stored in an area of the Java heap called "permanent generation". Because the Java heap is included into the core dumps, it should be possible to access these from Java core dumps. If so, how about reconstructing .class files from core dumps? As it turns out, there is already a tool to do exactly that!

HotSpot Serviceability Agent (SA) is a postmortem/snapshot debugger for HotSpot Java Virtual Machine. SA has a number of tools that are not bundled with JDK/JRE. But, now that HotSpot JVM is a open source, you can look/modify/build HotSpot SA. HotSpot SA lives under the $HOTSPOT/agent directory. There are tools under $HOTSPOT/agent/make directory. To use the SA tools, you need to build SA. There is a gnumake file for SA under $HOTSPOT/agent/make directory. HotSpot SA uses Mozilla Rhino JavaScript engine to implement certain features:

After building HotSpot SA, you can run any of the tools under $HOTSPOT/agent/make. There are shell scripts the SA tools - jcoreproc.sh is the tool to retrieve .class files from core dumps. There is another tool called clhsdbproc.sh that supports dbx/gdb like command line interface. This tool supports dumpclass command to retrieve .class files from core dump. There is some documentation of SA tools under $HOTSPOT/agent/doc directory. Happy core dump debugging!

Friday Mar 30, 2007

Retrieving .class files from a running app

Many Java applications generate .class files on-the-fly at runtime. Some applications modify loaded Java classes either at class load time or even later using hotswap. Few examples:
  • Scripting language implementations that generate .class files - like BeanShell, Groovy
  • OR mapping tools such as Toplink Essentials
  • Compilation of XSL stylesheets to .classes (assuming compilation is done at runtime rather than at build).
  • Profilers that use byte code instrumentation technique - like NetBeans profiler.

It becomes difficult to debug such applications - for example, you may get verifier error because of a bug in the .class generator somewhere. So, sometimes it is better to dump .class files of generated/modified classes for off-line debugging - for example, we may want to view such classes using tools like jclasslib. Instead of modifying each and every component that generates/modifies .class files, we would want to have a generic solution. The solution below uses attach-on-demand facility in JDK 6.


File: ClassDumperAgent.java

import java.lang.instrument.\*;
import java.io.File;
import java.io.FileOutputStream;
import java.security.ProtectionDomain;
import java.util.List;
import java.util.ArrayList;
import java.util.regex.Pattern;

/\*\*
 \* This is a java.lang.instrument agent to dump .class files
 \* from a running Java application.
 \*/
public class ClassDumperAgent implements ClassFileTransformer {

  // directory where we would write .class files
  private static String dumpDir;
  // classes with name matching this pattern
  // will be dumped
  private static Pattern classes;

  public static void premain(String agentArgs, Instrumentation inst) {
    agentmain(agentArgs, inst);
  }

  public static void agentmain(String agentArgs, Instrumentation inst) {    
    parseArgs(agentArgs);
    inst.addTransformer(new ClassDumperAgent(), true);

    // by the time we are attached, the classes to be
    // dumped may have been loaded already. So, check
    // for candidates in the loaded classes.
    Class[] classes = inst.getAllLoadedClasses();
    List<Class> candidates = new ArrayList<Class>();
    for (Class c : classes) {
      if (isCandidate(c.getName())) {
        candidates.add(c);
      }
    }
    try {
      // if we have matching candidates, then
      // retransform those classes so that we
      // will get callback to transform.
      if (! candidates.isEmpty()) {
        inst.retransformClasses(candidates.toArray(new Class[0]));
      }
    } catch (UnmodifiableClassException uce) {
    }
  }

  public byte[] transform(ClassLoader loader, String className,
    Class redefinedClass, ProtectionDomain protDomain,
    byte[] classBytes) {
    // check and dump .class file
    if (isCandidate(className)) {
      dumpClass(className, classBytes);
    }

    // we don't mess with .class file, just 
    // return null
    return null;
  }

  private static boolean isCandidate(String className) {
    // ignore array classes
    if (className.charAt(0) == '[') {
      return false;
    }

    // convert the class name to external name
    className = className.replace('/', '.');
    // check for name pattern match
    return classes.matcher(className).matches();
  }

  private static void dumpClass(String className, byte[] classBuf) {
    try {
      // create package directories if needed
      className = className.replace("/", File.separator);
      StringBuilder buf = new StringBuilder();
      buf.append(dumpDir);
      buf.append(File.separatorChar);
      int index = className.lastIndexOf(File.separatorChar);      
      if (index != -1) {
         buf.append(className.substring(0, index));
      }
      String dir = buf.toString();
      new File(dir).mkdirs();

      // write .class file
      String fileName = dumpDir + 
         File.separator + className + ".class";
      FileOutputStream fos = new FileOutputStream(fileName);
      fos.write(classBuf);
      fos.close();
    } catch (Exception exp) {
      exp.printStackTrace();
    }
  }

  // parse agent args of the form arg1=value1,arg2=value2
  private static void parseArgs(String agentArgs) {
    if (agentArgs != null) {
      String[] args = agentArgs.split(",");
      for (String arg: args) {
        String[] tmp = arg.split("=");
        if (tmp.length == 2) {
          String name = tmp[0];
          String value = tmp[1];
          if (name.equals("dumpDir")) {
            dumpDir = value;
          } else if (name.equals("classes")) {
            classes = Pattern.compile(value);            
          }
        }
      }
    }

    if (dumpDir == null) {
      dumpDir = ".";
    }

    if (classes == null) {
      classes = Pattern.compile(".\*");
    }
  }
}

File: manifest.mf

Premain-Class: ClassDumperAgent
Agent-Class: ClassDumperAgent
Can-Redefine-Classes: true
Can-Retransform-Classes: true

File: Attacher.java

import com.sun.tools.attach.\*;

/\*\*
 \* Simple attach-on-demand client tool that
 \* loads the given agent into the given Java process.
 \*/
public class Attacher {
  public static void main(String[] args) throws Exception {
    if (args.length < 2) {
      System.out.println("usage: java Attach <pid> <agent-jar-full-path> [<agent-args>]");
      System.exit(1);
    }

    // JVM is identified by process id (pid).
    VirtualMachine vm = VirtualMachine.attach(args[0]);

    String agentArgs = (args.length > 2)? args[2] : null;
    // load a specified agent onto the JVM
    vm.loadAgent(args[1], agentArgs);
  }
}

Steps to build class dumper:
  • javac ClassDumperAgent.java
  • jar cvfm classdumper.jar manifest.mf ClassDumperAgent.class
  • javac -cp $JAVA_HOME/lib/tools.jar Attacher.java
Steps to run class dumper:
  • start your target process -- I used java2d demo application in JDK
  • find the process id of your process using "jps" tool
  • java -cp $JAVA_HOME/lib/tools.jar:. Attacher <pid> <full-path-of-classdumper.jar> dumpDir=<dir>,classes=<name-pattern>

The above command will dump all classes matching the given name (regex) pattern into the given directory. The default dump directory is the current working directory (of the target application!). The default pattern is ".\*" i.e., match all classes loaded/will be loaded in the target application.

Now, how about a nice GUI that

  • shows all the Java processes
  • lets you can choose dump directory
  • Optionally, shows all the classes of a selected application
As usual, that is left as an exercise to the reader :-)

Saturday Mar 24, 2007

Script Beans - part 2

This is continuation of my earlier post on script beans. On rethinking, I've made few (minor) changes to ScriptObject and Callable. The changes are:
  • Using varargs instead of Object[] arguments in few places.
  • Added invoke method in ScriptObject -- this is used to directly invoke a method on a script object instead of getting a method/function valued property using get and then invoking Callable.call method. Why? Many dynamically typed languages have Smalltalk's doesNotUnderstand equivalent. i.e., if a method is not found, then a method missing handler is invoked. - like JRuby's method_missing and Groovy's invokeMethod. ScriptObject.invoke may be implemented by calling ScriptObject.get and Callable.call on result and possibly incorporating language specific method missing method as well.
  • ScriptObject.put returns boolean to tell whether the put was successful or not (may be read-only property was assigned or property addition is not possible and so on).


ScriptObject

import javax.script.ScriptException;

/\*\*
 \* Any Java object supporting this interface can be
 \* accessed from scripts with "simpler" access pattern.
 \* For example, a script engine may support natural
 \* property/field access syntax for the properties exposed
 \* via this interface. We use this interface so that we
 \* can dynamically add/delete/modify fields exposed to
 \* scripts. Also, script engines may expose this interface for
 \* it's own objects.
 \*/
public interface ScriptObject {
  // special value to denote no-result -- so that
  // null could be used as proper result value
  public static final Object UNDEFINED = new Object();
  // empty object array
  public static final Object[] EMPTY_ARRAY = new Object[0];

  /\*
   \* Returns all property names supported by this object.
   \* Property "name" is either a String or an Integer".
   \*/
  public Object[] getIds();

  /\*\*
   \* Get the value of the named property.
   \*/
  public Object get(String name);

  /\*\*
   \* Get the value of the "indexed" property. 
   \* Returns UNDEFINED if the property does not exist.
   \*/
  public Object get(int index);

  /\*\*
   \* Set the value of the named property. Returns
   \* whether the put was successful or not.
   \*/
  public boolean put(String name, Object value);

  /\*\*
   \* Set the value of the indexed property. Returns
   \* whether the put was successful or not.
   \*/
  public boolean put(int index, Object value);

  /\*\*
   \* Returns whether the named property exists or not.
   \*/
  public boolean has(String name);

  /\*\*
   \* Returns whether the indexed property exists or not.
   \*/
  public boolean has(int index);

  /\*\*
   \* Deletes the named property. Returns true on success.
   \*/
  public boolean delete(String name);

  /\*\*
   \* Deletes the indexed property. Returns true on success.
   \*/
  public boolean delete(int index);

  /\*\*
   \* Call the named method on this script object.
   \*/
  public Object invoke(String name, Object... arguments)
      throws NoSuchMethodException, ScriptException;
}



Callable.java

import javax.script.ScriptException;

/\*\*
 \* This interface is used to represent "function/method" valued
 \* properties in ScriptObjects.
 \*/
public interface Callable {
  /\*\*
   \* Call the underlying function passing the given
   \* arguments and return the result.
   \*/
  public Object call(Object... args) throws ScriptException;
}



AbstractScriptObject.java

import javax.script.ScriptException;

/\*\*
 \* Simple dummy implementation of ScriptObject.
 \*/
public abstract class AbstractScriptObject 
  implements ScriptObject {

  public Object[] getIds() {
    return EMPTY_ARRAY;
  }    

  public Object get(String name) {
    return UNDEFINED;
  }

  public Object get(int index) {
    return UNDEFINED;
  }

  public boolean put(String name, Object value) {
    return false;
  }

  public boolean put(int index, Object value) {
    return false;
  }

  public boolean has(String name) {
    return false;
  }

  public boolean has(int index) {
    return false;
  }

  public boolean delete(String name) {
    return false;
  }

  public boolean delete(int index) {
    return false;
  }

  public Object invoke(String name, Object... arguments) 
    throws NoSuchMethodException, ScriptException {
    Object value =  get(name);
    if (value instanceof Callable) {
      return ((Callable)value).call(arguments);
    } else {
      throw new NoSuchMethodException(name);
    }
  }
}

Wednesday Mar 21, 2007

Script Beans?

When we expose Java objects (or other language objects!) to a scripting language, we may want flexibility. Most scripting languages on the Java platform support JavaBean conventions. For example, JavaScript allows property style access for "getXXX" methods. Some language engines (for example, Groovy) treat java.util.Map's specially to provide map.key_name style access. But, we may want more flexiblity in addition to bean conventions. For example, we may want java.sql.ResultSet to be accessed with the natual obj.column_name syntax. But, instead of such ad-hoc special cases, we may want to have a generic way.

Also, while jsr-223 has API support to call a specific script function or method, there is no way to "reflect" on script objects. For example, you can't find out the all methods and properties supported by a specific script object. There is no engine independent way to reflect script objects - you have to use Scriptable interface for Rhino , GroovyObject interface for Groovy and IRubyObject interface for JRuby and so on.

HotSpot Serviceability Agent (SA) is a core dump/hung process debugger for HotSpot JVM. It supports JavaScript based command line interface - much like dbx/gdb's shell-like scripting interface. I had used Mozilla Rhino API directly to implement that -- because it was done before the advent of jsr-223. I am porting this to use jsr-223 API. Previously, I had used Scriptable, ScriptableObject and Function interfaces from Mozilla Rhino. I had changed this to use the following interfaces:



ScriptObject

/\*\*
 \* Any Java object supporting this interface can be
 \* accessed from scripts with "simpler" access pattern.
 \* For example, a script engine may support natural
 \* property/field access syntax for the properties exposed
 \* via this interface. We use this interface so that we
 \* can dynamically add/delete/modify fields exposed to
 \* scripts. Also, script engines may expose this interface for
 \* it's own objects.
 \*/
public interface ScriptObject {
  // special value to denote no-result -- so that
  // null could be used as proper result value
  public static final Object UNDEFINED = new Object();
  // empty object array
  public static final Object[] EMPTY_ARRAY = new Object[0];

  /\*
   \* Returns all property names supported by this object.
   \* Property "name" is either a String or an Integer".
   \*/
  public Object[] getIds();

  /\*\*
   \* Get the value of the named property.
   \*/
  public Object get(String name);

  /\*\*
   \* Get the value of the "indexed" property. 
   \* Returns UNDEFINED if the property does not exist.
   \*/
  public Object get(int index);

  /\*\*
   \* Set the value of the named property. 
   \*/
  public void put(String name, Object value);

  /\*\*
   \* Set the value of the indexed property. 
   \*/
  public void put(int index, Object value);

  /\*\*
   \* Returns whether the named property exists or not.
   \*/
  public boolean has(String name);

  /\*\*
   \* Returns whether the indexed property exists or not.
   \*/
  public boolean has(int index);

  /\*\*
   \* Deletes the named property. Returns true on success.
   \*/
  public boolean delete(String name);

  /\*\*
   \* Deletes the indexed property. Returns true on success.
   \*/
  public boolean delete(int index);
}



Callable.java

/\*\*
 \* This interface is used to represent "function/method" valued
 \* properties in ScriptObjects.
 \*/
public interface Callable {
  /\*\*
   \* Call the underlying function passing the given
   \* arguments and return the result.
   \*/
  public Object call(Object[] args) throws ScriptException;
}



Just to be sure: please note that I am thinking of ScriptObject and Callable as two way interfaces. Script engines would expose it's objects (all or some of it's objects) as ScriptObjects. Also, from Java code we can expose ScriptObject objects to script engines and the engines will support special obj.field and obj.method(...) syntax in the respective language.

Why can't we just use BeanUtils and avoid reinventing the wheel? For example, we can probably make use of BeanUtils API to expose objects to scripts and requires jsr-223 script engine implementers to treat DynaBeans specially to provide easier syntax. And optionally expose (some or all of) their own objects with DynaBean interface.

  • beanutils has the notion of DynaClass. Every DynaBean is queried to return it's DynaClass and from the DynaClass we get all properties supported. I think we can (and should?) avoid this when we have to deal with multiple languages. We would like to make minimal assumptions about objects -- allow prototype/delegation languages like JavaScript (where there is no notion of class) and allow for dynamically changing metaclass (like in Groovy, JRuby). So, in the above ScriptObject interface, we query the ScriptObject itself to return it's properties by calling getIds method. That returns the current snapshot of the properties of the object.
  • Also, beanutils API seems to concentrate only on "properties" only. There is nothing mentioned about "methods". (Did I miss anything in BeanUtils API that deals methods?). We have "Callable" - which is wrapper for any script method/function. ScriptObject.get(String name) would return a Callable object for method/function valued properties of the script object.
  • beanutils brings much more than what we need - we want to add just enough abstractions.

The ScriptObject and Callable interfaces could help in inter scripting language communication as well -- there is no need to specify a strongly typed Java interface to communicate b/w two dynamically typed languages (using say, Invocable.getInterface() ). For example, you can access Groovy object from a JRuby script and vice versa and both sides could use natural syntax to access ScriptObjects.Please let me know your comments. We can probably try this out with script engines @ scripting.dev.java.net

Monday Mar 19, 2007

JVM Talks...

Recently, I gave a series 4 talks (consecutive saturdays about 2 to 2.5 hours) at CS department of Anna University on Java Virtual Machine. The aim was to give understanding JVM at specification level and not much on any specific implementation of it. I did not create slides on my own. Also, I didn't stick only to slides - I did mostly marker-pen-and-board as well.


Initial reading to introduce the concept of virtual machines: Because we want to study a machine, it is useful to look at disassembled code. I used the following to show disassembled .class files:
  • javap - comes with JDK. Run with -p (private as well) and -c (code) options.
  • jclasslib Bytecode Viewer - very nice "visual" bytecode viewer. Allows us to nativagte various elements in a .class file by hyperlinks. There is also a NetBeans module available for jclasslib.
I used/referred the teaching materials from these sites/courses as well:

Writing code using a Java assembler gives very good understanding. I used jasmin assembler to show various JVM instructions. Also, we can see JVM's bytecode verifier in action:


.class public Adder
.super java/lang/Object

; standard constructor
.method public <init>()V
   aload_0 
   invokespecial java/lang/Object/()V
   return
.end method

.method public static main([Ljava/lang/String;)V
   .limit stack 2
   .limit locals 2
   
   ; nothing on stack, try using "iadd"
   iadd

   return
.end method

You can assmeble the above code by the command

  java -jar jasmin.jar Adder.j

When you run the "Adder" class, you get the following verifier error:


Exception in thread "main" java.lang.VerifyError: (class: Adder, method: main 
signature: ([Ljava/lang/String;)V) Unable to pop operand off an empty stack


There are also Java libraries to work with .class files:

Both the libraries can be used to read/analyze existing .class files and generate/modify .class files.

BCELifer class can be supplied with a .class file. BCELifier generates a Java program to generate the given .class file. By looking at the generated program, we can learn how to use BCEL API.

Command to generate a Java source program using BCELifier for a given "HelloWorld.class" (The generated Java program can be compiled and run to (re-)create HelloWorld.class).


java -cp bcel-5.1.jar org.apache.bcel.util.BCELifier HelloWorld.class

ASM library also comes with similar ASMifierClassVisitor class to generate Java source.


java -cp $ASM_HOME/lib/asm-util-3.0.jar:$ASM_HOME/lib/asm-3.0.jar org.objectweb.asm.util.ASMifierClassVisitor  HelloWorld.class

Tutorials on BCEL and ASM:

Friday Mar 16, 2007

Update Velocity jsr-223 engine to use version 1.5

Came to know from Erik's Linkblog about the new release of Velocity. Updated jsr-223 script engine for Velocity @ scripting.dev.java.net to use Velocity 1.5.
About

sundararajan

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
Bookmarks
Links

No bookmarks in folder

Blogroll

No bookmarks in folder

News

No bookmarks in folder