Superduper Slow Jar Command
By Christine Dorffi on May 15, 2009
It's well known that creating a Jar file can be a "little" slow. How slow? On my aged SunBlad1000, it takes about 1 minute and 40 seconds to jar the whole rt.jar in cf0M mode (no compress, no manifest) -- and it costs you a little more if done in compress mode.
But then we figured we were talking about creating jars for ten of thousands of classes with a total size of over 50M. Given the number of files and the total size, it seemed a reasonable amount of time. So, until now, we assumed it really needed that time — until someone accidentally noticed that "the CPU went to 100% busy for quite some time, perhaps a minute or more on my laptop, before starting to hit the disk to create the Jar archive."
That sounds strange, as the main job the Jar is supposed to do is to copy and compress the files (into the Jar). Thus it should hit the disk from the very beginning to the end.
So I peeked into the Jar source code (after many years), and it turned out we had a very embarassing bug in the jar code: We were doing a
O(n) look-up on a Hashtable (via the
contains() method) for each and every file we were jarring, where it really should be a
O(1) look-up operation with a HashSet. Given the number of files the command is working on, this simple mistake caused us to spend the majority of the time (that 1 min 40+ seconds) in collecting the list of files that need to jar, instead of the real "jarring" work. Sigh:-(
With that fixed (in JDK 7 build44 and later), the Jar is now much faster.
Following are the quick time-measure numbers of 10 runs of jarring/zipping the rt.jar/zip, in Store Only mode and Zip Compression mode.
- b43: the JDK 7/build43, which does not have the fix.
- b47: the JDK 7/build47, which does have the fix.
- zip: the default zip installed on my Solaris, which is zip2.3/1999
|Build 43||Build 471||Zip|
|Build 43||Build 471 ||Zip|
This page contains details of the fix.
We are making much progress on the Jar tool, and it is performing much better, though it is still slower compared to the Zip command. So we will continue our efforts going forward. I have to admit I do have some code that make Jar processing time much closer to Zip, but it will take time to make it into the product. Stay tuned!