Week-end fun with the java compiler source code

I downloaded java compiler (javac) source code from the JDK 7 site. I did not download entire JDK – I just downloaded compiler-7-ea-src-b15-05_jul_2007.zip I've installed JDK 6 and NetBeans 5.0

I extracted the source zip file into c:\\javac directory. From NetBeans IDE, File->Open Project menu, I chose c:\\javac\\compiler directory. Then, I build the project – I scrolled the build output log to the end and I saw:

Building jar: C:\\javac\\compiler\\dist\\lib\\javac.jar

build-bin.javac:

Copying 1 file to C:\\javac\\compiler\\dist\\bin

build:

BUILD SUCCESSFUL (total time: 8 seconds)

So, I tried to run the newly compiled java compiler. I attempted to compile a simple “Hello World” program. I got the following error:

C:\\javac\\compiler\\dist\\lib>java -jar javac.jar Hello.java

Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/tools/javac/Main

What happened? I looked at the build log again. I missed the following lines – because I had seen only at the end!!

Copying 7 files to C:\\javac\\compiler\\build\\bootclasses

recompiling compiler with itself

javac: invalid flag: C:\\javac\\compiler\\build\\classes

Usage: javac <options> <source files>

use -help for a list of possible options

Java Result: 2

Copying 7 files to C:\\javac\\compiler\\build\\classes

Copying 1 file to C:\\javac\\compiler\\build\\classes\\com\\sun\\tools\\javac\\resources

Building jar: C:\\javac\\compiler\\dist\\lib\\javac.jar

build-bin.javac:

Copying 1 file to C:\\javac\\compiler\\dist\\bin

build:

BUILD SUCCESSFUL (total time: 8 seconds)

Looks like there is a build error. The compiler in built in two steps:

  1. The sources are built with javac in JDK 6 (on which my NetBeans IDE ran)

  2. Then, compiler sources are built again – but this time with the new compiler binary generated by step (1).

Looks we got error in the step (2) [see above: recompiling compiler with itself] . I searched the ant script used to build for “recompiling compiler with itself”. The following is the fragment after that:

<echo message="recompiling compiler with itself"/>

<pathconvert pathsep=" " property="src.javac.files">

<path>

<fileset dir="${src.classes}">

<patternset refid="src.javac"/>

</fileset>

</path>

</pathconvert>

<java fork="true" classpath="${build.bootclasses}" classname="com.sun.tools.javac.Main">

<arg value="-sourcepath"/>

<arg value=""/>

<arg value="-d"/>

<arg file="${build.classes}"/>

<arg value="-g:source,lines"/>

<arg line="${src.javac.files}"/>

</java>

The problem seems to be with “java” command above. Empty string is set as value for -sourcepath option. I changed that to the following:

<arg value="-sourcepath"/>

<arg value="${src.classes}"/>

When I re-built the compiler after the above change, there were no errors – yes, I scrolled the build output to check it :-) And newly compiled javac could compile “Hello World” program.

Now, I wanted to make some to “interesting” but simple change to the compiler source. From a “doc” page, I came to know that there is a hidden javac option called “-printflat”. It appears that with -printflat option javac prints source code after doing transformations for generic types, inner classes, enhanced for-loops, assertions etc. It would be great to visualize the kind of transformations done by javac. So, I wanted to make “hidden” option available. I searched for “printflat” in the project. I got three hits:

  1. JavaCompiler.java

  2. RecognizedOptions.java

  3. java.properties

As usual, I am impatient – wanted to enable printflat option always [regardless of what the command line is]. So, I changed the following line in JavaCompiler.java

printFlat = options.get("-printflat") != null;

to

printFlat = true; // options.get("-printflat") != null;

so that the secret option is enabled always. After rebuilding the compiler, I tried compiling my “Hello World” program. Surprise! I got the following error:

C:\\javac\\compiler\\dist\\lib>java -jar javac.jar Hello.java

Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/tools/javac/Main

When I checked “javac.jar” by “jar tvf javac.jar”, I saw only “.java” files instead of “.class” files! Remember I mentioned that javac is recompiled by itself (step (2) above)? Apparently with “-printflat” option, javac just write transformed files but does not generate .class files! Because I had hardcoded printflat to be true always, during the second bootstrap compilation javac did not generate .class files. Looks like my lazy way does not work! I need to find how to really change the code to accept printflat command line option explicitly. I cut the story shot and just summarize the changes I made:

  1. added a enum value to com.sun.tools.javac.main.OptionName – PRINTFLAT("-printflat");

  2. In com.sun.tools.javac.main.RecognizedOptions class, I added PRINTFLAT to “static Set<OptionName> javacOptions” initialization value.

  3. In public static Option[] getAll(final OptionHelper helper) method of RecognizedOptions class, I added “new HiddenOption(PRINTFLAT)” as an element in the returned Option[].

I managed to compile and run the compiler after the above changes! Now when I can pass “printflat” option!! I compiled the following simple Book.java:


class Book {
  private String name;
  public Book(String name) {
     this.name = name;
  }

  class Order {
     private int quantity;
     public Order(int quantity) {
        this.quantity = quantity;
     }
  }
}

with the following command:

c:\\javac\\compiler\\dist\\lib\\>java -jar javac.jar -printflat c:\\Book.java

Now, I can see the generated Book.java and Book$Order.java in the current directory where java compiler was run:


class Book {
    private String name;
    
    public Book(String name) {
        super();
        this.name = name;
    }
    {
    }
}

class Book$Order {
    /\*synthetic\*/ final Book this$0;
    private int quantity;
    
    public Book$Order(/\*synthetic\*/ final Book this$0, int quantity) {
        this.this$0 = this$0;
        super();
        this.quantity = quantity;
    }
}

Wow! I can see how java compiler generates a hidden synthetic parameter for the outer class object and so on. Note that the java compiler does not overwrite your original source files. You need to run the compiler in a different directory – compiler generates new files [which is good, you won't accidentally overwrite your original code with generics, inner classes and so on].

Now, you can experiment with constructs like asserts, inner class methods accessing outer's private methods/fields, anonymous/local classes, local class accessing final parameters/locals of enclosing method, generics, enhanced for-loop and so on and see how java compiler transforms those constructs to generate good-old “flat” classes without these features. Have fun!!

Comments:

I had the similar adventure Dive In KSL but used the Kitchen Sink Language project for playing around only with the compiler. I personnaly really liked the jtreg test case writing. There are of course a lot of examples in the current code base, so it helps for a start. For playing with the options I created a nice unit test class (link above) that really helped me for debugging. I enjoyed a lot the experience, and still working on it, but it does not feel the same like the first fun week...

Posted by Fred on July 18, 2007 at 07:01 PM IST #

Post a Comment:
Comments are closed for this entry.
About

sundararajan

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
Bookmarks
Links

No bookmarks in folder

Blogroll

No bookmarks in folder

News

No bookmarks in folder