Friday May 23, 2008

Command-line tools and the Java Module System

At JavaOne 2008, I gave a demonstration of using command line tools with the current JSR 277 implementation. Here I'll share that with the blogosphere's wider audience. (For the record, I also demo'd using VisualVM with JSR 277 modules, but a blog on that will have to wait until we've made that code is not yet ready for prime time.)

The examples below show use of javac, jam, java, jrepo. They're based on the current JSR 277 code base, available via the Modules Mercurial repository. You can download the source code for the examples from

Please realize that JSR 277 and its implementation are a presently a moving target, and the sample code, commands, etc. below are all subject to change.

Source files

A modules-related example is really only interesting if there are two modules, one which imports the other. The example uses driver.Main and a lib.Library classes and puts them into correspondingly-named modules. Here's lib/src/lib/, which prints hello world and some information about the code's module definition and repository:
package lib;

import java.module.\*;

public class Library {
    public String greet() {
        String info = "";
        try {
            Repository r = Repository.getSystemRepository();
            ModuleDefinition md = r.find("lib");
            info += " from " + md.getName() + "-" + md.getVersion()
                + " in repository '" + r.getName()
                + "' at " + r.getSourceLocation();
        } catch (Throwable t) {
            // should not happen but ignore; it's just a demo anyway
        return "hello, world" + info;
Here's the associated lib/src/lib/ file:
package lib;

import java.lang.ModuleInfo.\*;
import java.module.annotation.\*;

class module_info {
    exports lib$Library;
Note the use of annotations to import the core Java platform, and to designate this module as version 1.0.

The "exports lib$Library;" merits explanation. For a class to be visible from one module to another, it must be exported. Our implementation currently requires us to do this "by hand". module_info is a real Java class: so is java.lang.ModuleInfo.exports, and lib$Library is a field of type exports in class lib.module_info. Ultimately, we expect that the set of classes exported by a module will be determined via the normal Java visibility rules, in conjunction with the new "module" keyword, and module developers won't have to explicitly list the exports as shown above. More on this when we discuss the jam tool.

Here's driver/src/driver/

package driver;

import lib.Library;

public class Main {
    public static void main(String[] args) {
        Library ly = new Library();
And finally, the corresponding driver/src/driver/
package driver;

import java.module.annotation.\*;

    @ImportModule(name="lib", version="[1.0,2.0)")
class module_info {
You can readily see that the driver module depends on the lib module, in any version greater or equal to 1.0 and less than 2.0.

Compiling and creating JAM files

The rest of this discussion assumes you've built the modules sources and the resulting build is at $MJDK. With the current state of the modules source base, javac compilation is no different than what you're already used to. However, creating a JAM file is a little different than creating a JAR file:
(1) mkdir repo
(2) cd lib
(3) mkdir classes
(4) $MJDK/bin/javac -d classes src/lib/\*.java
(5) $MJDK/bin/jam cfsS ../repo/lib.jam lib classes -C classes lib
(6) cd ..
In (1), we create the directory which will serve as the source location for a LocalRepository. In (4) you can see ... nothing new. However line (5) is new, the jam tool. It is based on the jar tool, so the "cf" options are familiar. The new options are:
  • s: Specifies module name, in this case "lib".
  • S: Specifies the location of the module info file, in this case it is in the classes directory. (In case you're wondering about the choice of "s" and "S": In earlier times, we considered using the term "superpackage" instead of "module", and "s" made sense. We've not yet changed this, and as with other things in this blog entry, "s" and "S" are subject to change.)
The resulting lib.jam has this content:
Where did MODULE.METADATA come from? Its content is based on The current implementation creates it by copying Eventually jam will examine its argument files, and build lists of member and exported classes. These lists will and the information in will be put into MODULE.METADATA.

Compilation and jam'ing for the driver is similar:

(1) cd driver
(2) mkdir classes
(3) $MJDK/bin/javac -d classes -cp ../lib/classes src/driver/\*.java
(4) $MJDK/bin/jam cfsS ../repo/driver.jam driver classes -C classes driver
(5) cd ..
In (3), notice that we provide the classpath to the library's classes. In the fullness of time, we anticipate allowing something like this:
$MJDK/bin/javac -d classes -repository  ../repo src/driver/\*.java
We expect to more options for locating JAM files during compilation, similar to those allowed by the java launcher.

Running code in Java Modules

By now, you should have a directory named repo containing driver.jam and lib.jam. Lets run the driver:
$MJDK/bin/java -jam repo/driver.jam
which should print something like this:
hello, world from lib-1.0 in repository 'application' at file:/tmp/cli-demo/repo/
Or, you can specify a version constraint for the driver:
$MJDK/bin/java -module "driver:[1.0,2.0)" -repository repo
If no module matches the version constraint, you'll get an error message:
$MJDK/bin/java -module "driver:1.5;[2.0,3.0)" -repository repo
Error: Module not found: driver:1.5;[2.0,3.0)
The Java launcher has other modules-related options; see the Java Launcher section on our tools page for more information.

Using jrepo to work with repositories

The new JDK tool jrepo, described at the jrepo section of our tools page, lets you examine repository contents, as well as install and uninstall modules to/from repositories. This example's prior steps resulted in a directory repo with two JAMs, lib.jam and driver.jam. Here's how jrepo shows the repository's contents:
% $MJDK/bin/jrepo list -r repo
Repository file:/tmp/cli-demo/
Name                 Version             
lib                  1.0                 
driver               1.0  
With -v, jrepo gives more information:
% $MJDK/bin/jrepo list -v -r repo
Repository file:/tmp/cli-demo/repo/
Name                 Version              Platform  Arch    Modified          Filename
lib                  1.0                  generic   generic 5/22/08 1:52 PM   /tmp/cli-demo/repo/lib.jam
driver               1.0                  generic   generic 5/22/08 1:52 PM   /tmp/cli-demo/repo/driver.jam
Adding -p causes jrepo to show all contents of all repositories that are parents of the named repository:
% $MJDK/bin/jrepo list -v -p -r repo
Bootstrap repository
Name                 Version              Platform  Arch    Modified          Filename         1.7                  generic   generic n/a               n/a
corba                3.0                  generic   generic n/a               n/a
javax.xml            1.4                  generic   generic n/a               n/a
javax.xml.bind       2.0                  generic   generic n/a               n/a         2.0                  generic   generic n/a               n/a
javax.xml.soap       1.3                  generic   generic n/a               n/a          1.0                  generic   generic n/a               n/a
javax.annotation.processing 1.0                  generic   generic n/a               n/a
javax.annotation     1.0                  generic   generic n/a               n/a
javax.script         1.0                  generic   generic n/a               n/a              1.7                  generic   generic n/a               n/a
java.classpath       1.7                  generic   generic n/a               n/a
Repository file:/tmp/cli-demo/repo/
Name                 Version              Platform  Arch    Modified          Filename
lib                  1.0                  generic   generic 5/22/08 1:52 PM   /tmp/cli-demo/repo/lib.jam
driver               1.0                  generic   generic 5/22/08 1:52 PM   /tmp/cli-demo/repo/driver.jam
We can remove a module:
% $MJDK/bin/jrepo uninstall -v -r repo lib
Uninstalled lib                  1.0                  generic   generic 5/22/08 1:52 PM   /tmp/cli-demo/repo/lib.jam
% $MJDK/bin/jrepo list -r repo
Repository file:/tmp/cli-demo/repo/
Name                 Version             
driver               1.0                 
And, if there's another repository into which we want to install a JAM, we can do that as well:
% mkdir repo2
% $MJDK/bin/jrepo install -v -r repo2 repo/driver.jam
Installed repo/driver.jam: driver               1.0                  generic   generic 5/22/08 2:02 PM   /tmp/cli-demo/repo/driver-1.0.jam
There's more work for the JSR 277 development team to do with respect to command line tools. Here's some of what's on our plate:
  • 6673594: (javac) ModuleFileManager for recognizing modules' import dependencies during compilation
  • 6674169: (dependency) Support JSR 277 in javac, javap, javah, and javadoc
  • 6674173: (dependency) Support JSR 277 in jrunscript
  • 6674167: (dependency) Support JSR 277 in rmic
  • 6584410: (jam) generate list of all classes and all public classes in embedded JAR files
  • 6590185: (jrepo) Installing a module via repository management tool should check its dependencies
  • 6593310: (jrepo) "list" command should sort repository contents
  • 6605077: (jrepo) add "dependencies" subcommand
  • 6605083: (jrepo) add "validate" subcommand
  • 6628249: (jrepo) provide information about services and providers that are in modules
Last but not least, please send suggestions if any command line tool functionality seems lacking.

Tuesday May 08, 2007

Working on OpenJDK using NetBeans

I like the way Rich Green said it at this morning's JavaOne keynote: Today begins the next phase for Java. Sun continued the open-sourcing of Java, releasing the rest of the JDK. He also mentioned working on the JDK with NetBeans, and I want to say a little more about that.

The JDK download, available at, includes sources for libraries, tools, and, importantly for us here, several NetBeans projects, designed to help developers work on the JDK. Once you've unpacked the source code bundle, you'll find them in

That directory incluces a README file which provides an overview of what else you'll need to get started, settings to help work with the projects, and do on. Let me mention here that you'll need to get the latest preview release of NetBeans, available here. For working on the JDK you only need the Basic version.

There are about a half dozen NetBeans projects in the source bundle, including ones for Swing, building the whole JDK, the javac compiler, and more. Aside from those which build the whole JDK, each project focuses on an area of interest of the JDK. There's a project-specific README in each project's directory.

We can well imagine that you might like to work on a different area of the JDK, so the README also has a mini-tutorial on how to make your own project, by starting from one of the provided ones.

And there's a great set of tutorials on the NetBeans site, Getting Started with OpenJDK in NetBeans IDE.

Tuesday May 01, 2007

Meet the Java SE Core Libraries Engineering Team at JavaOne 2007

Many members of the Java SE Core libraries engineering team will be on hand at JavaOne 2007, and we're having a BOF Wednesday night. Please stop by to hear what we've been up to in the past year, what we're looking to do for the following year, and to ask questions of the team.

In the Core Libaries team our work focuses on java.lang, I/O and NIO, java.util, concurrency, jar/zip, networking, character encoding, regular expressions, and more. So if those areas interest you, join us at

Room 105
8:55 PM Wednesday, May 9

We'll briefly present what we do and are planning for the year ahead. There should be plenty of time for Q & A.

If you're interested in security issues, then please join the security team Thursday night, 8:55 PM, BOF-2516.

Hope to see you there!

Monday Dec 11, 2006

Notable changes in the Java SE 6 Core Libraries

This blog summarizes notable changes to the core libraries in Java SE 6. Think of Java's "core libraries" as lang, io, util, and so on: the stuff of which all Java programs are comprised. For good measure, throw in jar, zip, regular expressions, concurrency, and reflection too.

There's lots more information about Java SE 6's features and compatibility with earlier releases. The platform overview describes the entire Java platform briefly, with links to more detailed guides. And of course, you can download the Java SE 6 release itself at

Changes in I/O

Reading passwords without echoing their text

New to Java SE 6 is the ability to read text from a terminal without having it echo on the screen, through
    char[] password = System.console().readPassword("password: ");
    // use password
    // clear password
    java.util.Arrays.fill(password, '\\0');
A few things are worth noting:
  • System.console() will return null if either standard input or standard output is not a terminal.
  • readPassword() returns an array, so that you can null it out as shown.
  • readPassword() uses the character set of the terminal device, which might not be that of Charset.defaultCharset().

Display of the Critical Message Box

Chances are you've seen a dialog on Microsoft Windows presenting a critical message such as
        Abort, Retry, Ignore
In the class, any method which accesses removable media such as a floppy drive could cause a dialog with that message to appear. This could be a real problem if you're accessing the system remotely! So we've changed the functionality here, so that by default for the JVM (including native code) and all processes it launches, the dialog doesn't appear. In a case like this:
    new File("A:\\\\tmp.txt").exists();
if there's no floppy in the A: drive, exists() returns false.

Interruptible I/O

What does this do?
    public class Interrupt {
        public static void main(String[] args) {
Brownie points if you answered "it depends". On Microsoft Windows and Linux operating systems, it will print Hello?. On the Solaris Operating System, it will hang. That's because I/O is truly interruptible on Solaris only: once the interrupt flag is set, you won't get anything written to stdout nor stderr. Note that you can turn it off, by running java with -XX:-UseVMInterruptibleIO. This will be the default behavior in future releases of the JDK, for better cross-platform compatibility.

Finding out about disk space

We've added some new method to so you can find out how much disk space is available: Note that the returned values are only estimates at the time of the call. They don't take into account any background I/O activity (by other processes, etc.)

A few notes are in order
  • getUsableSpace() takes quotas into account only on Microsoft Windows.
  • Each returns the result as a long. The return values don't imply anything about access controls, i.e. the space might not be writable.
  • A return value of zero could mean that there's no space, or it could mean that the partition doesn't exist.

Changing access modes

In Java SE 6, provides methods to change access of files: Each returns true if the operation succeeded.

Microsoft Windows devices are no longer considered files

Prior to Java SE 6, File.isFile() returned true for Microsoft Windows devices such as CON, NUL, and LPT. Now it returns false.

File.toURL is now deprecated

Creating a URL from some files can result in an illegal URL. Consider a file named "%-2": really, those characters should be escaped. The correct way to do this is to use toURI() as a go-between:
    URL u = File.toURI("%-2").toURL()

Long path names on Microsoft Windows

Java SE 6 supports long path names on Microsoft Windows. Each path name element is limited to 260 characters, which is the platform limitation.

Other noteworthy changes to

  • 6348207: File.length() reports a length of 0 for special files such as pagefile.sys.
  • 6198547: File.createNewFile() on an existing directory incorrectly throws IOException. Now it returns false on all platforms.
  • 6395581: File.listFiles() is unable to read nfs-mounted directory. Now it returns the correct results.
  • 4809375: File.deleteOnExit() should be implemented with shutdown hooks. Prior to Java SE 6, each invocation of deleteOnExit() added the file to a list: in particular, a file could be added multiple times and during JVM shutdown, deletion was attempted for each File in the list. In Java SE 6, instead of a list, File objects are added to a LinkedHashSet, guaranteeing that each is deleted only once.

Improved selector scalability on Linux

Linux 2.6 kernel provides epoll(4), which is more scalable than poll(2). The JDK detects the platform on which it is running, and so for Linux 2.6 or later kernels, will use epoll(4) in its SelectorProvider. This should scale well when there are thousands of selectable channels registered with a selector.

Multiple locks on the same file

What does this code do?
    public class FileLock {
        public static void main(String[] args) throws Throwable {
            new FileOutputStream("foo").getChannel().lock();
            new FileOutputStream("foo").getChannel().lock();
Prior to Java SE 6, this code would deadlock on Microsoft Windows, and return two locks on Linux and Solaris. Now, on all of those platforms, the second lock attempt throws an OverlappingFileLockException, as should have been the case all along since the behavior of lock() is specified that way.

Changes in collections

Performance improved in java.util.HashMap

We were able to improve the performance hashing function used by java.util.HashMap by about 5%. A side effect is that the order of the results returned by iterating over values() has changed from the previous release. Of course, as per the specification of iterator(), clients should not depend on the order from an iterator!

Performance improved in copying arrays

We've added new methods to java.util.Arrays to copy arrays and sub-arrays: (The links above are to the int methods, but there are methods for each primitve type as well as type-parameterized methods for objects.) These methods efficiently resize, truncate, or copy subarrays of all types. They simplify your code a bit too. Before Java SE 6, you might write:
    int[] newarray = new int[oldArray.length];
    System.arraycopy(oldArray, 0, newArray, 0, oldArray.length);
Now you can instead write this:
    int[] newarray = Arrays.copyOf(oldArray, oldArray.length);
Not only is the code slightly simpler, but there's a performance benefit: The old way created an array and initialized its values to zero, then overwrite those zeroes with values from oldArray. By providing a means to do this in one step, we avoid the zero initialization. Finally, the new methods will truncate or pad the new array as required.

Bidirectional navigable collections

We've added some new interfaces to enhance Maps and Sets: These extend their related Sorted classes with navigation methods reporting closest matches (e.g. higher) and allow for traversal in either ascending or descending order. We've retrofitted the related implementations TreeMap, TreeSet, and added the new concurrent classes ConcurrentSkipListMap, and ConcurrentSkipListSet.

Double-Ended Queues

To the Queue interface in Java SE 5, we added the Deque and BlockingDeque interfaces. We updated LinkedList to implement Deque since it had everything needed to do so. We also added new classes ArrayDeque, which is an efficient implementation of Deque, and the concurrent implementation LinkedBlockingDeque.

Changes in jar and zip

Timestamps on files extracted by jar

Prior to Java SE 6, the date and time of files extracted by the jar command were the current time. Other de/compression tools use the time noted in the archive file itself. So for Java SE 6, we changed jar to conform with other tools

if the old behavior is needed, run the JVM with

Number of open ZIP files

We removed a limitation on the number of concurrently open ZIP files on Microsoft Windows. The maximum used to be 2036, but now it's whatever the platform will support.

Number of entries in a ZIP file

The ZIP file format has a 2-byte field to record the number of entries in the file, artificially imposing a 64k limit. Our implementation now ignores that field, and instead just counts the entries. Prior to this change, you could count the entries with ZipInputStream or ZipFile , but get differing results if there were more than 64k entries in the file.

Long Zip File Names on Microsoft Windows

We now use the same support for long file names in ZIP files as we use in the package.

Other changes

Support for plugable service providers

We've added java.util.ServiceLoader as the standard way for clients to load plugable code. It maintains a (clearable) cache of loaded services.

Array syntax checking

Final quiz: what does this do?
    public class LoadArray {
        public static void main(String[] args) throws Throwable {
            String name = (new String[0]).getClass().getName();
The key observations are that name is the name of an array class, and that we're using the single-argument overload of loadClass(). In earlier releases, this was allowed, but in Java SE 6, throws a ClassNotFoundException. It really is a bug: loadClass() only works for binary names, and the Java Language Specification, Third Edition, Chapter 13 defines binary names for classes and interfaces, not arrays.

Notable changes in Runtime.exec()

We had a number of bugs all related to attempts to predict whether an exec would succeed. In Java SE 6, we instead exec whatever is given, and communicate over a pipe with the exec'd process. If it fails for any reason, such as in these bugs, we can detect the failure:
  • 4052517: Runtime.exec won't execute programs belonging to other groups on Unix
  • 4811767: Runtime.exec should throw IOException when workdir does not exist (Unix)
  • 5033302: Can't execute Solaris NFS programs with uid>64k on Linux-amd64



« April 2014