Wednesday Sep 12, 2007

JXSE 2.5 : What's Cool #3 -- Programmers Guide 2.5

JXSE (JXTA for Java SE/EE 5.0) 2.5 contains quite a number of exciting changes for JXTA application developers. This little series will look at a few of the important changes in the upcoming release.

One of the most exciting new developments related to JXSE 2.5 is the new edition of the JXSE Programmers Guide that will accompany it's release. Mohamed Abdelaziz (hamada) began working on updating of the Programmers Guide last summer. Over the past year hamada has been leading the effort to add content, update and improve the existing content and correct hundreds of minor errors in the older Guide editions. Having an updated version of the Guide which matches the current JXSE APIs and best practices makes it much easier to get started with JXSE.

Throughout the entire process, and indeed for the last two editions of the Guide, Ted Kosan (tkosan) provided review and suggestions. For the new edition Jérôme Verstrynge provided a complete new tutorial as well doing a thorough review of the entire Guide. Kees Pieters and Mike Cumings (mcumings) also provided a complete reviews of the Guide.

This new edition of the Programmers Guide has been a long time in development. In part, this was because the Guide needed a format conversion and a lot of updates. We'd like to have it updated more frequently with a smaller number of changes in each new edition. The preparing the next edition should take about six months. If you have examples, content or suggestions now would be a great time to get involved. The focus of the next edition will likely be to create more complete write-ups of the new tutorial programs and hopefully new tutorial chapters for Peer Group Services and a replacement Secure Peer Group tutorial. Visit the JXTA Programmers Guide project for more information on getting involved with creating the next edition of the Programmers Guide!

The new edition is available in a number of formats. We are providing JXTA JXSE Programmers Guide 2.5 (PDF) and JXTA JXSE Programmers Guide 2.5 (ODT) as well as, for the first time as print-on-demand, JXTA JXSE Programmers Guide 2.5 (

Thursday Sep 06, 2007

JXSE 2.5 : What's Cool #2 -- META-INF/services

JXSE (JXTA for Java SE/EE 5.0) 2.5 contains quite a number of exciting changes for JXTA application developers. This little series will look at a few of the important changes in the upcoming release.

One thing that's cool about working on an open source project is that every so often a community member you've never heard of will speak up and make a brilliant suggestion that even after years of devoted work on the project everyone else somehow missed. Exactly this happened last fall a few weeks before JXSE 2.4.1 went final and unfortunately too late for it to be included in JXSE 2.4.1 but it is now available in JXSE 2.5. The old bugzilla was not preserved when we transfered the JXSE project to, so I have lost track of the name of the user who suggested the feature. If you're willing to step forward and identify yourself I'd be very happy to acknowledge your contribution.

The cool feature which is the subject of this blog is the JAR Service Provider, otherwise known as the META-INF/services system. The META-INF/services system allows for the dynamic registration of instance classes for a particular interface. We are now using it for several of the JXSE class factories and the JXSE Class Loader.

To register classes you list the class names implementing an interface in a file named for the interface in the META-INF/services folder of your JAR file. For example, to register new Advertisement types you include a file "net.jxta.document.Advertisement" in the META-INF/services folder of your JAR containing the full names of your Advertisement classes. During startup the JXSE AdvertisementFactory will read class names from that file and register classes. The process is repeated for each net.jxta.document.Advertisement file in all of the other JARs.

Use of the META-INF/services system makes it much, much easier to extend JXSE by adding your own components. Adding support for your advertisement type to JXSE now requires only including the class names in the net.jxta.document.Advertisement file of your JAR file. You can also register StructuredDocument types, ID formats, WireMessageFormats and JXTA Services and Modules. We have converted JXSE to use the META-INF/services system for all of the core modules.

Wednesday Sep 05, 2007

JXSE 2.5 : What's Cool #1 -- NIO TCP

JXSE (JXTA for Java SE/EE 5.0) 2.5 contains quite a number of exciting changes for JXTA application developers. This little series will look at a few of the important changes in the upcoming release.

JXSE provides two primary low-level message transports; TCP and HTTP. In the vast majority of cases, even for peers which are behind NAT and firewalls, the TCP message transport is used because it offers the best performance, lowest overhead and lowest latency. The HTTP message transport is used only when it not possible for a peer to open outbound TCP connections to other peers. Since the TCP message transport is so critical to JXSE we chose to focus on improving it for 2.5.

The refactoring of the TCP message transport was begun and has been spearheaded by Mohamed Abdelaziz (hamada). He started the job by replacing all of the traditional blocking I/O with a non-blocking java.nio.SocketChannel implementation. This initial work enabled us to start to look at the remaining bottlenecks, opportunities and challenges. A number of JXTA developers found hamada's contribution interesting and exciting. Even at this early point it managed to attract my attention as well as Henry Jen (slowhog), Roger Karis (malveaux) and others. slowhog contributed bug fixes and improvements based upon testing the code running on the JXTA public rendezvous servers. I contributed optimizations and a refactoring of the WireFormatMessageFactory and the binary message format to improve performance. malveaux, as usual, contributed his keen analysis and insight to help solve more than a few gnarly problems.

One of the new features and advantages of the new NIO based TCP message transport is much better use of Java Threads. We no longer require nearly as many threads per Socket instance and make extensive use of java.util.concurrent.Executor to effectively handle messages. As the new NIO based TCP message transport became more solid it became very clear that there were significant bottlenecks in how the JXSE Endpoint processed messages. The solution was to make even further use of Executors and perform all message processing on Executors. This surgery required enhancement of the HTTP message transport, the multicast message transport (which is now a separate message transport, but that's another story) and the relay. Again it was a good all around effort with contributions from hamada, slowhog, malveaux, myself and others. Mike Cumings (mcumings) contributed significant insights regarding best-practice use of Executors.

The result of all this work is that JXSE 2.5 makes considerably more efficient use of Threads while significantly improving throughput and latency. This is particularly important for infrastructure peers such as relays and rendezvous as it enables them to handle larger numbers of clients and much more load per instance. We're quite sure that JXTA network operators are going to appreciate this change.

For JXTA users and applications which are heavily dependant upon the HTTP message transport JXSE 2.5 does offer some improvement by using Executors for message processing, but we haven't yet transitioned the message transport to use NIO. Some early explorations have begin looking in to using the Grizzly HTTP server but we're still looking for contributors (especially those with Grizzly experience) to step forward and get something working. It also seems that some additional code sharing is possible between message transports by using the low-level Grizzly APIs. Contributors to investigate this opportunity are also eagerly sought!

The NIO based TCP message transport is a significant addition to JXSE 2.5 and probably provides the most dramatic change to the profile of JXTA applications of all the changes in JXSE 2.5. It is not a change though that's directly visible to applications in terms of new APIs or functionality. Future articles will look at some of the other changes in JXSE 2.5 and their benefits to JXTA application developers and users.

Tuesday Sep 04, 2007

Pavlova is Wrong for JXSE 2.5

A long time ago the name "Pavlova" was nominated as a JXSE (JXTA for Java SE/EE 5.0) release name. JXSE releases are named for celebratory foods and we have had releases named after foods from every continent and many cultures. Pavlova is a light, fluffy, airy, subtle meringue dessert from New Zealand or Australia (depending on who you ask or trust). JXSE 2.5 is, to contrast, the most significant JXSE release since JXTA 2.0 four years ago. JXSE 2.5 is the result of ten months of intense development and testing by more than a dozen contributors. It offers significant improvements to the functionality, performance, scalability, maintainability, monitorability, reliability and several more "ability"s... Beyond JXSE itself we will also be releasing a new and significantly updated edition of the JXTA Programmers Guide. This is a going to be a major event for JXTA!

When we began the 2.5 release cycle we originally imagined that it would be a routine release with the normal mix of new features, improvements and bug fixes. Instead we are now completing a much more significant release. The key factor which catapulted the 2.5 release from a normal JXSE release to a jumbo sized release has been contributions from the JXTA community. In particular, the Glassfish Shoal clustering project has been a big contributor to JXSE 2.5. Additionally, speaking of fish, the JXTA project's move to has moved us into a much bigger pond. It's been really exciting to see how many people are discovering JXTA as a result of this move.

At this point I don't think that the "Pavlova" name will be adequate JXSE 2.5 release. We would need a Guinness World Record sized Pavlova to suffice! I believe we need a name that truly reflects the scope, breadth and gut-busting hugeness of the JXSE 2.5 release. With a little searching I found Whole Stuffed Camel, but since it can't be proved to have been actually prepared nor can I find a proper name for the dish I am ruling it out. Any other suggestions? The winning name must include a web accessible recipe for inclusion in the release notes. Bonus points if you can provide photos of having cooked the dish yourself!




« July 2016