Project Jigsaw

Vision The JDK is big—and hence it ought to be modularized. Doing so would enable significant improvements to the key performance metrics of download size, startup time, and memory footprint.

Java libraries and applications can also benefit from modularization. Truly modular Java components could leverage the performance-improvement techniques applicable to the JDK and also be easy to publish in the form of familiar native packages for many operating systems.

Finally, in order to realize the full potential of a modularized JDK and of modularized applications the Java Platform itself should also be modularized. This would allow applications to be installed with just those components of the JDK that they actually require. It would also enable the JDK to scale down to smaller devices, yet still offer conformance to specific Profiles of the Platform Specification.

Okay—so where do we start?

JDK 7 As a first step toward this brighter, modularized world, Sun’s primary goal in the upcoming JDK 7 release will be to modularize the JDK.

There will be other goals, to be sure—more on those later—but the modularization work will drive the release, which we hope to deliver early in 2010.

Tools Modularizing the JDK requires a module system capable of supporting such an effort. It requires, in particular, a module system whose core can be implemented directly within the Java virtual machine, since otherwise the central classes of the JDK could not themselves be packaged into meaningful modules.

Modularizing the JDK—or indeed any large code base—is best done with a module system that’s tightly integrated with the Java language, since otherwise the compile-time module environment can differ dramatically from the run-time module environment and thereby make the entire job far more difficult.

Now—which module system should we use?

JSR 277 The current draft of this JSR proposes the JAM module system, which has been the subject of much debate and is far from finished. This system is intended to be at least partly integrated with the Java language. Owing to some of its rich, non-declarative features, however, it would be impossible to implement its core functionality directly within the Java virtual machine.

Sun has therefore decided to halt development of the JAM module system, and to put JSR 277 on hold until after Java SE 7.

JSR 294 This JSR, Improved Modularity Support in the Java Programming Language, is chartered to extend the Java language and the Java virtual machine to support modular programming. Its Expert Group has already discussed language changes that have been well received for their simplicity as well as their utility to existing module systems such as OSGi.

Earlier this year JSR 294 was effectively folded into the JSR 277 effort. Sun intends now to revive 294 as a separate activity, with an expanded Expert Group and greater community involvement, in support of the immediate JDK 7 modularization work as well as the larger goal of modularizing the Java SE Platform itself.

OSGi If JSR 277’s JAM module system is an unsuitable foundation for modularizing the JDK, what about the OSGi Framework? This module system is reasonably mature, stable, and robust. Its core has even already been implemented within a Java virtual machine, namely that of Apache Harmony. OSGi is not at all integrated with the Java language, however, having been built atop the Java SE Platform rather than from within it.

This last problem can be fixed. Sun plans now to work directly with the OSGi Alliance so that a future version of the OSGi Framework may fully leverage the features of JSR 294 and thereby achieve tighter integration with the language.

Jigsaw In order to modularize JDK 7 in the next year or so, and in order better to inform the work of JSR 294, Sun will shortly propose to create Project Jigsaw within the OpenJDK Community.

This effort will, of necessity, create a simple, low-level module system whose design will be focused narrowly upon the goal of modularizing the JDK. This module system will be available for developers to use in their own code, and will be fully supported by Sun, but it will not be an official part of the Java SE 7 Platform Specification and might not be supported by other SE 7 implementations.

If and when a future version of the Java SE Platform includes a specific module system then Sun will provide a means to migrate Jigsaw modules up to that standard. In the meantime we’ll actively seek ways in which to interoperate with other module systems, and in particular with OSGi.

All work on Project Jigsaw will be done completely in the open, in as transparent a manner as possible. We hope you’ll join us!

My thanks to Alex Buckley for comments on drafts of this entry.
Comments:

Please do not stop at the modularization and dependency tracking step but also define rules for the naming and layout of java modules on the filesystem (that for Unix systems respect and integrate with the FHS design).

Giving integrators full freedom without any recommended conventions just results in many incompatible misdesigned choices.

Posted by Nicolas Mailhot on December 02, 2008 at 06:25 PM PST #

To me, the JDK isn't big and doesn't need to be modularized. The most urgent thing about Java for me is responsiveness, and shortening the develop - compile - deploy - test cycle.

Posted by Mikael Gueck on December 02, 2008 at 07:22 PM PST #

Mikael, the current monolithic nature of Java deployments is a huge impediment to rapid deployment cycles.

Modularity means you do not need to QA a huge binary lump every time there is a change, just the part that actually changed (a correctly designed modular system protects you from side-effects in other modules).

Posted by Nicolas Mailhot on December 02, 2008 at 07:59 PM PST #

The link to JSR294 actually points to JSR277. You meant to link to http://jcp.org/en/jsr/detail?id=294, right?

This is good news, and I hope the increased openness and development will encourage those with modularity experience to be able to contribute both ideas and code to the work.

Posted by Alex Blewitt on December 02, 2008 at 09:21 PM PST #

Sun plans to work with the OSGi Alliance. But what about OSGi Alliance's plans?

Late time I checked, OSGi Allianace refuse to use annotation because of some Java1.3/J2ME support.

Posted by Daniel Cheng on December 02, 2008 at 11:21 PM PST #

Daniel,

As my long-running discussion thread has shown: Java 1.3 is no longer being used in the wild (unless you count a 1% market share). According to my personal statistics, around 87% of Java users have already migrated to Java6. See http://forums.java.net/jive/thread.jspa?messageID=319705&tstart=0#319705 for more information.

Mark, I am a bit confused by the details of your announcement. On the one hand you say that the primary goal of Java7 will be to modularize the JDK but on the other hand you say that project Jigsaw will not be part of Java7. How will the two be related then?

For my part, I think this is a huge step in the right direction for Java simply from the point of view of scalability. Conceptual weight scalability, performance scalability and finally this is the stepping stone to finally looking at the possibility of breaking backwards compatibility when it makes sense. As I mentioned above, the vast majority of users are using the latest Java version. Backwards compatibility is very important, but certainly less vital than it was a few years ago. Hopefully Sun will finally be able to fix some long-running issues in Java that have been hampered by backwards compatibility, such as Generics.

Cheers,
Gili

Posted by Gili on December 02, 2008 at 11:35 PM PST #

Modularity not in JDK 7.0 when, how, why. Modularity is not necessary now?. We need modularity 4 years ago (Integrated with the language). Modularity in 2010 is not the right direction.

Posted by leonardoavs on December 03, 2008 at 12:48 AM PST #

Gili, I think he meant Jigsaw will not be part of Java SE 7 the Standard Platform Spec, but only in Sun's implementation of the spec, until such time when the spec does include a module system as part of the standard. That's just my guess though.

Mike.

Posted by Mike on December 03, 2008 at 12:54 AM PST #

For someone not following the modules saga, can you describe the differences between JSR 277, JSR 294, and OSGi?

I look forward to using whatever modules come out in the end -- the current situation is not so nice.

Posted by Kevin on December 03, 2008 at 03:42 AM PST #

Daniel,

It looks like there has been some movement from the OSGi side on dropping support for some of the older execution environments and on modernising the OSGi APIs generally, see this link for (a little) more info:

https://www.eclipsecon.org/submissions/2009/view_talk.php?id=638

note that the presenter is CTO of the OSGi Alliance as well, which gives it a fair wallop of credibility...

Posted by Ian Phillips on December 03, 2008 at 04:09 AM PST #

> key performance metrics of download size, startup time, and memory footprint

Download size is not a "key performance metric". For server-side and Swing apps, it doesn't come into play at all. For applets, it's not "key" - no one says "I don't like this applet because it took 30 seconds to start, whereas 15 seconds I wouldn't mind".

Startup time is not a "key performance metric" either. For server-side, not an issue at all. For client, an extra second or two at startup is not significant. If it were, client-side apps would just load themselves into memory on bootup.

Memory footprint can be a "key performance metric" for certain apps. Modularity will not help memory usage, though. Only the classes that are needed are loaded into memory today.

Mark, your tone is starting to sound like Neal's tone when he says "this is how closures will be". You may be making a slight gesture to indicate you want input from the community, but it really sounds like "this is how it's gonna be, and that's final".

It sure seems like you're looking for something interesting to work on, rather than making a real attempt to understand what the 6 million Java developers and their users really want/need. "Sun is the new Bell Labs" indeed.

Posted by Andy Tripp on December 03, 2008 at 04:24 AM PST #

Andy,

I think the key point is that making dependencies more explicit will encourage Java module authors to reduce module coupling which will in turn reduce the number of classes loaded into memory. Take Swing for example. If you load one Swing class there is a pretty good chance you just caused 85% of it (and many other core APIs) to get loaded into memory. Modular Java is like the difference between public and private JREs for 3rd-party libraries. Right now everyone is using the equivalent of a private JRE with every one of their applications. Yes it works and yes this is much less of a concern for server-side Java, but it is a major problem for client-side Java and is not sustainable in the long term. Java needs to scale better in order to continue its growth. I believe modularity is a crucial component for that scalability.

I also disagree with you on the point of applets. I, for one, will tell you applets take too long to load. Take identical Flash and Java applets, stick them on a webpage and compare. I actually think the JVM loads plenty fast but for some reason there is still a much larger lag visible with applets than with Flash (much less so since update 10!).

Posted by Gili on December 03, 2008 at 04:34 AM PST #

This may not be politically viable, but from a technical perspective, the NetBeans module system might be a better fit. It's just as capable and mature, and the usage patterns more closely mimic what is currently standard java (META-INF/services, etc).

Plus, it'd be easier to get chances/enhancements done, as there's only one other app to maintain compatibility with.

Talk to Jarda Tulach about it for a few minutes before diving too far down the OSGi road.

Posted by Rich Unger on December 03, 2008 at 05:28 AM PST #

Andy,

While I agree with your points about download size - frankly they should sort this with pre-installs and broad band is pervasive and getting ever faster, I think the Java platform could always use faster startup ( whether applet or application ) - I'm not a big fan of piling everything into OS startup - sometimes you want to start that up quickly too!

However I'd agreed that these deliverables are not the most important things on my list - there are lots of things they could improve - but here's why I do think it's important -
modularization is an enabling step to help the JDK improve - ie the main beneficiaries of modularization of the JDK are going to be JDK authors - and while that's not directly useful in the short term - it should be beneficial in the medium to longer term, allowing, in future, parts of the JDK to evolve and improve faster in the future, perhaps allowing third parties to be more active participates in providing core functionality.

Look at those parts of the JDK which were built with replaceability in mind - XML parsers, cryptography ( who doesn't use bouncy castle? ).

Andy if you were to chose something to target for JDK 7 what would it be?

Posted by Mrx on December 03, 2008 at 06:31 AM PST #

The JDK 7 release cycle is beginning to remind me of Windows Vista: no one really knows when it's going to be released (but it will be years behind previously-announced release cycle schedule) and as time goes on, features get jettisoned.

Posted by Christopher Brown on December 03, 2008 at 06:58 AM PST #

@Ian

Yea, they are droping Java 1.3 support -- in favor for Java 1.4, not even Java 5. I don't see any chance it will depends on a JDK 7/8 feature anytime soon.

Posted by Daniel Cheng on December 03, 2008 at 10:34 AM PST #

@Daniel,

When you measure the number of Java 1.3 runtimes out there, did you also count Java ME? In the embedded world, Java 1.4 is bleeding edge and there is still an awful lot of 1.3 around.

Still, of course OSGi needs to support later language features. The challenge is to support features like Generics without breaking compatibility with pre-Java-5 runtimes, because 1.4 is still important. One way to do this is to build optional extensions on top of the core APIs, and work is under way at the OSGi Alliance to do just that. Likewise, a solution will have to be found for taking advantage of new Java 7 language features (whatever they might be) without restricting OSGi to running only on Java 7.

So you're right, there's no chance OSGi will \*depend\* on Java 7/8 features soon, but there's a good chance OSGi will be able to take advantage of them when available.

Posted by Neil Bartlett on December 03, 2008 at 03:30 PM PST #

Neil,

I take issue with your statement that "1.4 is still important". How many users still use 1.4? Very few. How few? Under 4%. I'm referring to figures posted here: http://forums.java.net/jive/thread.jspa?messageID=319705

In my opinion it's about time that Generics get a face lift and pointing to old versions no longer makes sense.

J.T.,

I think you misunderstood what Mark is proposing.

1) You can rely on core APIs being there because you can instruct JKernel to install modules eagerly alongside your application. The only difference is that right now \*all\* modules are being installed eagerly, even ones your application does not depend on.

2) Loading-on-demand won't be slower if you pick the granularity properly. For example, if JKernel detects your application depends on Swing it downloads the whole thing instead of checking on a per-class basis. Performance is hardly an issue then. Personally I'd like to see per-package granularity.

I think there is a real divide in this discussion between server-side guys and client-side guys. Clearly the developers focusing on server-side technologies see absolutely no value in modularization. On the flip side, people from client-side Java can't imagine the world without it. Personally, I think it's fair to say that Java is already well established on the server and it's got a lot of ground left to make up on the client-end. I think Java6 update 10 also shows that when Sun tries it can achieve very strong community interest and support. Server-side Java won't suffer. It's already very mature.

Posted by Gili on December 03, 2008 at 04:30 PM PST #

IMHO, this issue is not urgent like it's being presented from the title. There are indeed things that are more urgent.

But I do believe it's important for the evolution of the platform. As such, it should solve a lot more than the download size and memory footprint: in fact, I see those as bonuses you get in the process.

What modularization should address should be the following:
\* Allow me to develop on Java 7 while making sure I am still library compatible with Java 5 (not just source compatible)
\* Allow for a path to significantly alter existing library to the point of retiring older versions. Imagine, for example, the ability to migrate to a Swing2, where Collections are used instead of arrays. The codebase that implements it could be the same, but expose two different specs. You choose which spec to use, and your javadocs, your autocomplete, etc. don't see all the old junk. Having the same module presenting two different specs with the same implementation is what allows you to transition while keeping backward compatibility (think about a web service with a different port for each version)
\* A repository systems from which modules are taken, a clear system on how to version modules, a way to tie in with OS installation, etc...

I hope Jigsaw is the first step toward a bigger goal: figuring out all the details to modularize the JDK is some kind of specific way, so that it can be understood how to create a good module system that does, among other things, what I mentioned above... which is a huge undertaking. And yes, we may get some short term benefit, but the real aim is further. Otherwise, I don't really understand the point.

Posted by Gabriele Carcassi on December 03, 2008 at 10:07 PM PST #

Gili,
I dont think people will change what libraries they used in order to reduce the classes loaded (that seems to be what you're saying). we'll all write "hello, world" the same way, regardless of whether it loads 1 class or 1000. Even a simple Swing app loads a lot of classes, and each and every one of them is required.

Yes, applets take longer to load than flash. More modularity won't change that.

Mrx: Windows has always loaded libc on bootup. Does anyone care that Windows that Windows takes 2 minutes to boot instead of 1:58? Not really.

Modularity of the code is a completely different matter than a modular library mechanism. The JDK could be made more modular without what's being discusses here. Or what's being discussed here could be implemented without making the JDK any more modular.

As for what I'd choose for JDK7, I don't know what the 6 million Java developers want enough to say. I think Sun should set up some real mechanism to get real feedback from Java developers and users. Java's features should be driven by customers (you and me), not JDK developers. I could give you my two cents (true 2D arrays for performance), but I doubt it would match the needs of most others.

Posted by Andy Tripp on December 04, 2008 at 01:42 AM PST #

@Daniel

This quote on the aforelinked page seems to indicate otherwise: "This session will explore a prototype of version 2 of the framework API which will exploit Java 5 language features and otherwise modernize the API."

Of course there's no mention of expected delivery dates for any of this, so whether it'll fit in with the JDK7 target date is another matter.

Posted by Ian Phillips on December 04, 2008 at 01:43 PM PST #

Andy,

Sure nobody cares about 1.58 version 2 mins but people care that it takes 2mins - how does it take 2 mins - death by a thousand cuts - loads of authors deciding an extra 2 secs won't matter - in my opinion this kind of view is why a lot of software is so bloated.

Then reversing it when it gets to a point were people think it's too slow then becomes a long hard road of lots of small improvements - anyway that's a side rant... yep - being selfish as a Java developer they could warm start.

I agree that modularity of the code doesn't depend on this and is a different from modular libraries per se - however making dependencies explicit and manageable could - I admit only could - help people understand the dependencies better and tools to manage at the library level could allow for easier migration paths between versions. It's like saying the existence of the Interface facility in Java doesn't mean your code is magically more modular - but it's certainly a useful tool in enabling that if you want to.

I totally agree that the way this has been presented "Hey we've had this great idea, let me tease you, ta da - this is the big new thing for JDK 7 - ship date X - it's \*the\* focus of JDK7" and then having the audacity to talk about community involvement blah blah does smack of a certain of lack of self-awareness and perhaps even self-indulgence - however that aside I'm not sure it's a bad idea as such - for example it might make Apple's job easier in shipping their versions etc - which has to be good

Posted by MrX on December 04, 2008 at 11:49 PM PST #

MrX,

I don't think people really care that Windows boots in 2 minutes rather than, say, the 1.5 minutes it might take if greedy apps didn't preload. Most people either leave their machine running forever, or boot it and go get coffee. Just look at all the software that installs as a service - if they had complaints, they'd stop doing it.

Yes, a module system might allow the JDK to become more modular. But I don't think the JDK's modularity is a big issue for Java developers (yes, it may be a big issue among JDK developers, but that's different). And even if it is, I don't see a compelling case that this module system will help more than just "using interfaces and making things modular" today.

I also agree with you that this seems to be another "Here's the cool thing we're going to do in JDK7" dictum, just as closures where. Sun should spend more time asking us what we need and less time telling us what we need.

Posted by Andy Tripp on December 05, 2008 at 06:10 AM PST #

As an often dissenting member of JSR-277, I have been hoping and arguing for something that works as easily as Maven/Ivy but during the compilation (javac) and runtime (jvm) as opposed to a separate tool. Unfortunately it appears that I'm in the minority and everyone really does want some huge complicated module system that doesn't solve the simple case trivially. I shouldn't need anything more than javac and java to declare a dependency on a 3rd party library, compile it and execute it without managing the repository myself. Am I alone in this wish?

Sam

Posted by Sam on December 22, 2008 at 07:17 AM PST #

Post a Comment:
Comments are closed for this entry.
About

This blog has moved to http://mreinhold.org/blog. <script>var p = window.location.pathname.split('/'); var n = p[p.length - 1].replace(/_/g,'-'); if (n != "301") window.location = "http://mreinhold.org/blog/" + n;</script>

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
News

No bookmarks in folder

Blogroll

No bookmarks in folder

Feeds
RSS Atom