ZIP64, The Format for > 4G Zipfile, Is Now Supported

We heard you! finally:-)

The support for ZIP64, the format for > 4G ZIP file, has finally been added into the latest OpenJDK7 build(b55). This RFE (request for enhancements) had been buried in the 200+ Jar/ZIP bug/rfe pile so deep that I was not even aware of / remembered its existence until a recent call from A custom strongly asking for it. Given everyone now has 200G+ disk space (and yes, most of my kid's video clips are now around 1G after I got that new camcorder), it is relatively easy for the Jar/ZIP user to run into this 4G ceiling these days. The RFE was quickly climbing itself to the top of my to-do list and is now in b55. From now on only the sky is the limit for your Jar/ZIP file:-)

So if you have > 4G stuff (either the total size of files zipped in > 4G or the individual files themselves are > 4G) to jar/zip, try out the OpenJDK7-b55 (and later), either via the java.util.jar/zip  APIs or the Jar tool. Let me know if you find anything not working or broken, I hope it's perfect though:-)

Here is the brief background info regarding the 4G size problem in Jar and ZIP file.

(1)Various size and position offset related fields in original ZIP format are 4 bytes, so by nature ZIP has this 4G size limitation.

(2)The field for total number of files zipped/stored in ZIP's Central directory record is 2 bytes, so it has the 65536 limit (Java's ZIP file implementation has some hacky code to workaround this issue though)

(3)ZIP64 format extensions was introduced in (in spec 4.5) to address above size limitation.

(4)JDK7 now fully supports the ZIP64(tm) format extensions
defined by the PKWARE's  ZIP specification

If you are interested in source code, here is my ZIP64 "to-do" list (a copy/paste note from the spec) and the code diffs.

Comments:

Hi,

Nice to see one more arbitrary limit removed. :)

Best,
Ismael

Posted by Ismael Juma on April 17, 2009 at 08:45 AM PDT #

Time to update the Wikipedia entry [1] - "while others, such as Java's built-in java.util.zip, still lack it"

[1] http://en.wikipedia.org/wiki/ZIP_(file_format)

Posted by Kirill Grouchnikov on April 27, 2009 at 08:25 AM PDT #

Hi,

I downloaded jdk-7-ea-bin-b59-windows-i586-14_May_2009.exe used the zip api to compress a >4 GB file. But it did not work. Do i need to make any other changes (in the code or setting) to make it work?

Jackson.

Posted by Jackson on May 20, 2009 at 02:58 PM PDT #

Jackson, can you describe a little more about what "it did not work" means, for example an exception? Is it possible to send me your code or a snapshot of the place where you use the "zip api"? which definitely will help me to trace down the issue. One of the test case I used is at http://cr.openjdk.java.net/~sherman/4681995/webrev.02/test/java/util/zip/LargeZip.java.html. Also the jar tool is built on top of the java.util.jar/zip APIs, I also used it to test.

Posted by Xueming Shen on May 26, 2009 at 04:33 AM PDT #

Does the zip library (java.util.zip) in Java 7 automatically use the Zip64 extensions when the zip file exceeds 4GB? Does existing code need to be modified to take advantage of the new Zip64 format?

Posted by Dem on July 30, 2009 at 06:08 PM PDT #

Yes, the java.util.jar/zip will pick up the Zip64 extension "automatically. User code does not need to change.

Posted by xueming.shen on August 14, 2009 at 05:45 PM PDT #

That's great to hear. Thanks!

Posted by Dem on August 18, 2009 at 03:38 AM PDT #

hi
Is the latest version of netbeans will have this new zip 64 support in it.Or I have to install it manually.I downloaded the latest JDK 7 b78 opensource jar file but don't know how I install it to netbeans.Can anybody help?

Posted by jeho on December 19, 2009 at 10:41 PM PST #

Xueming: this fix is deserately needed, so very glad to read about it.

Any chance that it will be backported to 1.6? I cannot wait for 1.7 to come out.

Posted by B on March 31, 2010 at 11:11 PM PDT #

I just downloaded and installed this:

java version "1.7.0-ea"
Java(TM) SE Runtime Environment (build 1.7.0-ea-b92)
Java HotSpot(TM) 64-Bit Server VM (build 18.0-b04, mixed mode)

But when packing it stops at about 4gig...

Is it implemented in jdk1.7?

thanks in advance

Ricky

Posted by Ricky Dagevos on May 13, 2010 at 09:15 PM PDT #

Sorry, I just encountered it was solely my own coding problem :-(

Posted by Ricky Dagevos on May 13, 2010 at 10:35 PM PDT #

xcvxcvxcvvxcv
xcv
xc
vcx
vcxv
cvc
xvcx
vc
vxcv
cxvcx
v

Posted by fgngnf on August 17, 2010 at 11:45 AM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

xuemingshen

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today