JSR 199 meets JSR 203
By jjg on Sep 24, 2009
java.util.zip.\*. Clients of the Compiler API can either obtain and use a
StandardJavaFileManager, or they can provide their own implementation of a
JavaFileManager, enabling the compiler to access custom storage, such as the in-memory storage used by an IDE.
Now, in JDK 7, there is a new feature, More New I/O APIs for the Java Platform, JSR 203, which provides a much richer file system abstraction. How do these new APIs relate to the features in the Compiler API?
To find out, this past week I've put together an implementation of the
JavaFileManager API, that bridges to the new NIO APIs. This means that anyone who uses or provides a filesystem using the
java.nio.file.\* APIs can now use those same APIs in conjunction with the Compiler API. Since the new NIO APIs provide uniform access to a variety of file systems, such as the standard file system, zip files, and user-provided file systems, this is a very attractive way to use the Compiler API as well.
Just as the main class for accessing the standard file system is
java.io.File, its analog in the new NIO world is
So, to start working on the new file manager, I took a copy of
StandardFileManager and at least conceptually, replaced all occurrences of
Path, and called the resulting interface
Eventually, I trimmed the class down some, and added a few more methods: more on that later.
Next up, I created a simple new implementation of
FileObject that wraps a
Path object, and so naturally enough is called
Together, these provide enough to start filling in the "obvious" implementations.
For the most part, the implementations either call through to the underlying new NIO API, or use cut-n-paste code from the existing javac file manager.
(Eventually, I'll move the cut n paste code into a utility superclass that I can share between the two file managers.)
One of the abstractions in the Compiler API is a
Location, which abstractly represents search paths, such as the class path, source path and so on.
These are easily modeled as a map of a
Location to a list of
The code to initialize default values for these search paths -- most notably the platform class path, or bootclasspath -- uses the existing code in javac.
The biggest non-trivial method to implement in the new file manager was the
This takes a
Location, and parameters such as a package name, the types of files of interest, and whether to recurse into subpackages.
The method iterates over the contents of the
Location, searching each entry in turn.
Search paths may contain both directories and jar files.
Searching a directory is relatively obvious, and with new new NIO API, handling jar files is almost as easy.
If the entry on a search path is a file, the file manager attempts to open it as a
FileSystem, using FileSystems.newFileSystem This is the new standard way to open files like zip and jar files.
If the file manager successfully opens the file as a filesystem, it gets the root directory for the file system, and simply searches in that directory, using the same code as if it were a regular directory.
The other non-trivial method is
This is a method to guess the binary name for a
JavaFileObject without having to open it.
The guess is based on the name of the file object.
For an entry in a jar file, you can use the entry name as the basis for the binary name.
For an entry in a directory, you have to scan the paths for a given
Location and see if the file in question is in a directory on the search path, and use the directory-relative name as the basis for the binary name.
There may be more information available, depending on how the file object was created.
For all these reasons, the new file manager's inferBinaryName method delegates to an abstract method on
PathFileObject, and different anonymous subtypes of this class provide different and appropriate implementations of
inferBinaryName, depending on the information available at the time the file object is created.
What other methods appear on
The first is a method to get the underlying
Path for any file object returned by the
This is useful for any tools that want to use PathFileManager, and which need more operations (such as
exists(), etc) than those provided by the basic
In addition, there are methods on
PathFileManager to get and set a default
FileSystem for the file manager to use.
Currently, this is used by javac as the file system in which files are created when no specific location has been set, such as with the
The new code cannot trivially be added to javac, because of the standard bootstrap issues:
since javac is used to build JDK, it cannot itself use any features that are new to that version of JDK, including any new API.
So we need to figure out how to make file manager available.
By itself, this is not a big deal, but it would be nice to try and tie it into the existing
javax.tools.\* API as well, if possible.
Further out, there have been requests over the years to have javac write class files directly to a jar file. There are also known performance issues with the existing
Currently, the new NIO libraries do not come with "out of the box" support for zip files as
FileSystems, although there is a demo library that works well enough for read only access by
javac, provided the library is built and placed on the
It would be interesting to see if some zip file expert could provide a fast new zip library via the new NIO interfaces. This would not only benefit JDK users in general, but it would also mean that javac would immediately benefit as well, by way of this new file manager.
- Webrev and patch file
- OpenJDK NIO project>
- OpenJDK compiler group>
- The Java™ Programming Language Compiler, javac
- OpenJDK compiler group>
Thanks to Alan Bateman for answering all my questions about the new NIO API, and to Joe Darcy for feedback on this note.