The ZIP filesystem provider in JDK7

It started as a NIO2 demo project with two goals. First, to demonstrate how to use NIO2 file system provider interface to develop and deploy a custom file system, a ZIP file system. Second, to show how easy and fun to use the NIO2 file system APIs, to access a ZIP/Jar file. I happened to have some code around which was prepared for a possible future j.u.zip.ZipFile class update, so we wrapped it with the NIO2 file system provider SPI, packed it into zipfs.jar and dropped it into the <JDK>/demo/nio/zipfs directory, done! OK, it took weeks:-) to pull everything together, clean up all the corner cases here and there, testing, and it ended up with 5K+ lines of code. You can find all the source code at <JDK7>/demo/nio/zipfs/src.zip of your latest JDK7 directory, or "here" if you prefer a quick browsing. Next,  need come up with some sample code to achieve the second goal, how easy and fun to use the APIs. After writing down couple samples, wow,  we started to realize that it's truly easy now to access the ZIP file via the nio2 file system APIs, for example, to extract a ZIP entry SRC_NAME out of a ZIP file FILE_NAME, you only need to do

        try (FileSystem fs = FileSystems.newFileSystem(Paths.get(FILE_NAME), null)) {
           Files.copy(fs.getPath(SRC_NAME), Paths.get(DST_NAME);
       }


or to do a "fancy nio2 walk" on the ZIP file,

        try (FileSystem fs = FileSystems.newFileSystem(Paths.get(FILE_NAME), null)) {
            Files.walkFileTree(fs.getPath(DIR_NAME),  new SimpleFileVisitor<Path>() {
                    private int indent = 0;
                    private void perform(Path file, BasicFileAttributes attrs) {
                        if (attrs.isDirectory())
                            System.out.printf("%" + (indent==0?"":indent<<1) + "s[%s]%n",
                                              "",
                                              file.toString());
                        else
                            System.out.printf("%" + (indent==0?"":indent<<1) + "s%s%n",
                                              "",
                                              file.getFileName().toString());
                    }
                    @Override
                    public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
                        perform(file, attrs);
                        return FileVisitResult.CONTINUE;
                    }
                    @Override
                    public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)  {
                        perform(dir, attrs);
                        indent += 2;
                        return FileVisitResult.CONTINUE;
                    }
                    @Override
                    public FileVisitResult postVisitDirectory(Path dir, IOException ioe) {
                        indent -= 2;
                        return FileVisitResult.CONTINUE;
                    }
            });
        }


or, you can even do something you have not been able to do via jar or those classes at java.util.zip, for example

        Files.copy(Paths.get(SRC), fs.getPath(DST), COPY_ATTRIBUTES);

which copies a file into a ZIP file with creationTime and lastAccessTime attributes, below is a sample output that shows the "Demo.java" entry that was copied into the ZIP file with above line.


/Demo.java

    creationTime    : Fri Apr 29 22:41:46 PDT 2011
    lastAccessTime  : Wed May 25 20:58:36 PDT 2011

    lastModifiedTime: Fri Apr 29 22:41:46 PDT 2011

    isRegularFile   : true

    isDirectory     : false

    isSymbolicLink  : false

    isOther         : false

    fileKey         : null

    size            : 26680

    compressedSize  : 4941

    crc             : c5f6eb5a

    method          : 8

More interesting and cool NIO2/ZIP filesystem usages can be found in
<JDK7_HOME>/demo/nio/zipfs/Demo.java


Sure, in order to make this work, you need to add the <JDK>/demo/nio/zipfs/zipfs.jar into your classpath or manually drop it into the lib/ext directory, if it's not there already.

The more test cases and sample code we wrote the more we are convinced that it might be a good idea to simply deploy this ZIP file system provider into the system extensions directory, so the provider can be used directly (without playing with the -classpath to add the zipfs.jar into your classpath) to access a ZIP/Jar file via the NIO2 file system APIs, as an alternative to the existing java.util.zip/jar.ZipFile class. So since JDK7/b123, the zipfs.jar has been deployed into the lib/ext. You now can use the ZIP filesystem "out of the box" and access a ZIP/Jar file just like access a "normal" file system.

As of JDK7, the ZIP file system provider supports the legacy JAR URL syntax as defined by java.net.JarURLConnection. That is, entries in the zip/JAR file system will be identified by URIs of the form:
   jar:{uri}!/{entry}

In addition, A ZIP file system can be created using URIs of the form:
  jar:{uri}

The legacy JAR URL syntax will be a challenge to the platform once the latest URI RFE is adopted. We have decided to ignore this issue for now, as an alternative URI syntax would be confusing to developers and would be inconsistent with the rest of the platform.
For JDK 7, the zip/JAR file also must be located on the file system and so the URI scheme of "{uri}" in the above will be "file", eg:
  jar:file:/tmp/foo.jar
  jar:file:/tmp/foo.jar!/bar

When creating a new FileSystem properties may be used to configure the file system. These properties are specified to the FileSystems.newFileSystem methods via a java.util.Map. The key is the property name (String), and the value is the property value. The following 2 properties are now supported:

"create" : The value is of type java.lang.String with a default value of "false". If the value is "true" (case sensitive) then the zip or JAR file is created if it doesn't already exist.

"encoding" : The value is of type java.lang.String with a default value of "UTF-8". The value of the property indicates the encoding of the names of the entries in the Zip or JAR file.


You may want to give it a try if you happen to have some zips, jars around and let us know what works and what need improve.



Comments:

Are there any examples of working with many streams? For example, video or audio conference-systems where one stream splits to many streams and many streams are collected in one stream.

Posted by guest on June 15, 2011 at 04:21 PM PDT #

Are there any examples of working with many streams? For example, video or audio conference-systems where one stream splits to many streams and many streams are collected in one stream.

Posted by guest on June 15, 2011 at 04:37 PM PDT #

Applications requiring faster operations such as working with drivers (microphone, network card, video card, etc.) need direct access to memory in real-time. The practice show that the last official release of Java NIO does not yet deal with such problems. Are there any examples which implements such scenario?

Posted by guest on June 15, 2011 at 05:18 PM PDT #

will there be any support for encrypted zip files or do you still have to do this yourself?

wbe

Christian

Posted by guest on June 16, 2011 at 08:33 AM PDT #

Xueming,

I've integrated this into my application and have only found one problem so far: It fails whenever I try to open a zip file that is contained within another zip file.

The "outer" zip file works fine, but I get this exception on the inner zip file:

java.nio.file.ProviderNotFoundException: Provider not found
at java.nio.file.FileSystems.newFileSystem(FileSystems.java:403)

Since this is an extension and not an official part of the JDK, I wasn't sure where to report the issue... so you're getting it here. :)

Posted by Roger on August 03, 2011 at 09:43 PM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

xuemingshen

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today