Monday May 18, 2009

Java for the FIRST Robotics Competition!

On April 16th we announced that Java would be available for the FIRST Robotics Competition. This is joint work between Sun and WPI to port Squawk (an open-source Java virtual machine) to the compactRIO robot control system, as well as the WPILib robotics library from C++ to Java.

We expect this to be a great addition for FRC teams that have some Java programming experience, and will bring not only the combination of power and safety inherent in Java to robotics programming, but also the ecosystem of Java tools, libraries and learning resources.

The exact flavor of Java that will be available is Java ME (Micro Edition), configured with the CLDC and IMP libraries. This is essentially the same flavor of Java that is installed in millions of cell phones (but without the graphic user interface libraries - LCDUI).

In addition the system is based on the libraries and SDK of the Sun SPOT project. It includes over-the-air debugger support as part of integration with the NetBeans and Eclipse IDEs and also includes ant-based command line tools.

The Java system will be in closed alpha and beta releases over the Summer and Fall, and will be ship in the "kit of parts" for the 2010 season (if not earlier).  We have also developed a software-only robot emulator that allows you to learn how to develop robotic software in Java using the Sun SPOT APIs (but doesn't yet include the WPILib APIs).

I'll be blogging more about this, including technical details, the Atlanta Championship, and Java + FIRST @ Java One, but here are some pointers to more information:

Friday Feb 06, 2009

GC debugging + surfing + nutrition?

      A year ago I was tracking down a garbage collector bug in Squawk over many days. Like most debugging, you start with a theory of what might be going wrong, figure out what data would prove or disprove the theory, and try to reproduce the bug while getting this critical data.
recycle logo
But now life starts getting interesting. We can't use a debugger on device when the system is this broken. And the system is built on an abstraction of objects (almost everything in Squawk is written in Java), but when memory overwrites can blow away metadata like object lengths and class pointers, you realize that you can't trust everything you see.

This isn't the profound, but ultimately simple, betrayal of the ground during an earthquake. Garbage collection can effect every object in the heap, so garbage collection bugs are more like betrayal in a bad Kung Foo movie:


I had spent a few days and several dives down into the system, only to rise to the surface with theories disproved or becoming deeply disoriented. So one morning, I started with a new theory and tools, took a deep breath, and dove into the system again. Suddenly I was struck with a flashback of body surfing as a kid at Hendry's Beach and getting slammed by waves, tumbling around until I didn't know which way was up. cc

After reminiscing for a few minutes, I realized how amazing it was that my metaphoric dives and confusion could invoke such as strong memory. I could feel the oily tar residue of the ocean in my mouth. I could taste the seaweed that always reminded me of ice cream and of the kelp harvesters on the horizon.

I started thinking about how high-level metaphors could trigger such low-level sensory memories, when a sudden small burp reminded me of the fish oil pills and smelly vitamins that I had washed down that morning with a cup of hot coffee. Lesson learned :-)

ps. There was still a GC bug to fix. It turned out that we were removing the stack object (yes, it's an object) for a dying thread from the set of GC roots while the thread was executing the last few steps of thread shutdown. Yikes!

Monday Dec 15, 2008

Class Literals in Java ME


Last week I learned that there was more to Java class literals than I expected - and not in a good way. If you're working in Java SE you can skip reading now, but in Java ME this might be of interest...

Class literals were added to the Java language in 1.1 days - they allow you to get a reference to a Class object as if any class had a static field called "class". So you can get the Class object for the String class by using the expression String.class instead of calling Class.forName and dealing with exceptions.

This is a great convenience - not for avoiding some try-catches (you could use a helper method for that) -  because the class name is checked at compile-time instead of run-time. If you change the name of a class everywhere but in the string passed to Class.forName, javac won't be any help.


In Java 1.5 the JVM Spec was updated to include direct support for class literals (extending the ldc bytecode). But how did class literals worked in Java 1.1 through 1.4? And how do they work in Java ME CLDC now?

It turns out that javac just keeps generating code until it works :-) There are a couple of variations in what it does depending on the Java target version used, but I'll describe what I've seen for -target 1.2 or 1.3 (which includes most (all?) Java ME CLDC applications).

Consider this trivial class using a Class Literal:

public class ClassTest {
    static Class c2 = String.class;
Javac will generate code roughly like this:
public class ClassTest {
    static Class c2;

    static Class class$java$lang$String; // cache (would make sense with a different example)

    static {
        if (class$java$lang$String == null) {
            class$java$lang$String = class$("java.lang.String");
        c2 = class$java$lang$String;

    static Class class$(String name) {
        try {
            return Class.forName(name);
        } catch (ClassNotFoundException ex) {
            Exception newex = new NoClassDefFoundError(ex.getMessage());

Bigger Issue

OK, so this is getting a little messy - the simple expression String.class is generating about 33 bytes of bytecode plus overhead for an anonymous field and method. But what happens when the class literal is referenced from an interface instead of a class? Note that interfaces can't have static methods, so javac generates an anonymous class to hold the anonymous method!
public interface InterfaceTest {
    static Class c2 = String.class;
Javac will generate code roughly like this:
public interface InterfaceTest {
    static Class c2;

    static {
        if (InterfaceTest$1.class$java$lang$String == null) {
            (InterfaceTest$1.class$java$lang$String = class$("java.lang.String");
        c2 = (InterfaceTest$1.class$java$lang$String;

public class InterfaceTest$1 {
    static Class class$java$lang$String;

    static Class class$(String name) {
        try {
            return Class.forName(name);
        } catch (ClassNotFoundException ex) {
            Exception newex = new NoClassDefFoundError(ex.getMessage());
This results in the same amount of bytecode overhead, but now we have extra class overhead. The class file for the anonymous class InterfaceTest$1 is about 587 bytes. Now multiply this by every interface that references a class literal.

Java Native Access

So why would anyone have interfaces that references a class literals? Check out the Java Native Access (JNA) project. JNA is an interesting system that lets you declare native functions you'd like to call from Java, and JNA takes care of generating a callout, without generating any JNI code. I'll be blogging about JNA in the future.

The typical JNA usage pattern is to declare an interface that contains the functions you want to call (as interface methods), and JNA creates an implementation that will actually do the call. To match the interface and implementation to a dynamic library containing the native code, the interface should declare a static variable such as:

    public interface CLibrary extends Library {
        CLibrary INSTANCE = (CLibrary)
            Native.loadLibrary((Platform.isWindows() ? "msvcrt" : "c"),
        void printf(String format, Object... args);

So What? And Where's the Squawk Connection?

OK, to recap, in Java 1.5 and later, this doesn't happen because the JVM can handle class literals. But in a Java ME system with tight memory constraints, such as what Squawk is designed for, this could add up.

But the Squawk VM handles class literals just fine, it's just that javac doesn't know that. In the process of converting normal Java bytecodes into Squawk bytecodes, we can do a lot of cleanup. The simplest trick that comes to mind is to pattern match in the static initializer "clinit", looking for the initialization of the anonymous field with a call to the anonymous method.

If we simply replace the call to class$("java.lang.String") with a load of the class literal, then the anonymous method will not be called, and the dead method elimination (DME) phase will delete the method.

Getting rid of the anonymous class is trickier than I first thought, because the cache variable ("class$java$lang$String") gets declared in the anonymous class. In theory we can handle this by constant propagation - the cache field is final, and is really initialized by a constant value, so we can replace references to the cache with the load of the constant class litereal. If we implemented dead field elimination there would be no references to any fields or methods of the anonymous class and dead class elimination (DCE) could do it's job!

I've got to get dead class elimination checked in one of these days, but in the meantime I now know where all of these anonymous classes are coming from.

ps. I'd like to thank for having the sources for everything, in particular

Thursday Dec 04, 2008

About Me

I'm Derek White. I work for Oracle Labs in Burlington, Massachusetts. The views expressed on this blog are my own and do not necessarily reflect the views of Oracle.

Currently I'm working on the Squawk project - an open source virtual machine for Java written almost entirely in Java. Squawk is a Java ME (CLDC + IMP) system designed for small embedded devices. A cell phone is big for us. The funny thing is that in the past I've worked on getting Java to scale up to large CMT systems, but now I'm getting Java to scale down to machines as small (or smaller) than the ones I first started programming on.

I work with a lot of good people here at Sun, and will link to those that blog off to the right somewhere ->

Other things I've worked on in the past include:

  • Java simulation and performance analysis for Niagara processors (the short answer is "NOPs are bad").
  • Garbage collection and JVM performance issues at Sun Labs (the "Exact VM").
  • A JVM for an unnamed 64-bit OS at Novell.
  • The Dylan programming language and development environment at Apple.
  • The Object Pascal compiler at Apple.

There is more information on my Oracle Labs Bio Page.

This is my first blog post - often when I read a blog for the first time I ask myself "Who is this guy?". Well now you now in this case.

In the future look for posts on Java VM design, Squawk, porting, performance analysis and tuning, debugging, Sun SPOTs, programming, robots, too many parenthetical remarks (a side-effect of working with the Dylan (lisp) hackers),  and obscure cultural references.

Now I wonder where Ruth is?....

Add to Technorati Favorites


Out of the fog... of bits, bytes, and really small Java Virtual Machines, by Derek White. The views expressed on this blog are my own and do not necessarily reflect the views of Oracle.


« July 2016