Montag Aug 31, 2009

High-performance TCP/IP programming on the JVM

I've recently created a white-paper about several options to do TCP/IP programming on the JVM. I'm giving here the introduction and the table of contents to give you an impression what's this all about. The white-paper comes with full source-code and is 27 pages long, so it's a bit too much for a blog entry. Get it here, if you want to take a look.


The purpose of this document is threefold. Firstly, it should give the reader a historical overview of the TCP/IP networking API's, programming paradigms, and various coding styles on the Java Virtual Machine. Secondly, it presents a couple of new options, programming-models, and demonstrates the same with Scala, one of the new languages available on the JVM. Lastly, a small and simple benchmark was conducted, to relate these different strategies with to each other on the metrics of performance.

The discussion is centred wholly around the Java Virtual Machine for a couple of reasons. First and foremost, one regularly deals with several programming languages and several libraries. Basing everything around the JVM provides the opportunity to eliminate issues such as the language implementation, memory management, interfaces to the host operating-system, etc. Besides this advantage, the JVM is the foundation for a large amount of Open Source libraries, tools, including excellent support from a couple of free IDEs that it became the platform of many web applications. The platform is mature, versatile and widely available.

To showcase the various API's, languages and programming models, and to give the programmer an impression as to how such source code looks like, five very trivial server programs that write a simple HTML page to any client, which tries to open a TCP/IP connection to such a server, were created. Since all five programs run on the Java Virtual Machine, this page-generating method has converged to a single utility-class, which is shared among the five programs. Therefore, risks arising from inconsistent behaviour and performance, because the programs were developed in different programming languages - Java and Scala, were eliminated.

Table of Contents


Classic Java networking IO:Sockets

One connection = one Thread?

Java New I/O: Java NIO


Non-blocking I/O

NioServer example

NIO Frameworks, enter: Grizzly

Problems with plain NIO

The Grizzly NIO framework

The Actor Model

Network API evolution

Threading API evolution

Actors: shared-nothing versus shared data


Functional and Object-Oriented programming combined

Statically typed

Running inside the Java Virtual Machine

Modern language constructs: closures, type interference, pattern matching, powerful literals etc.

Excellent Actor library nicely integrated in the language

A simple Scala server using the Actor Model

Sending messages

Receiving messages

Combining the Actor Model and NIO

The Benchmark

Benchmark setup


Page generation


Dienstag Mai 12, 2009

UNIX DaytimeServer - the Scala way

I'm playing around with Scala. No, frankly I'm in love with it. While reading through "Programming in Scala" (which is a perfect introduction into the subject), I converted the Daytime Server into a Scala version. Get the NetBeans Project.

Here's how it looks like:


 1 package daytimeserver
 3 import
 4 import java.nio.ByteBuffer
 5 import java.util.Date
 6 import java.nio.channels.SelectionKey
 8 import com.sun.grizzly.{ ProtocolFilter, Context, Controller, ProtocolChain,
 9                          DefaultProtocolChain, ProtocolChainInstanceHandler, TCPSelectorHandler }
11 import com.sun.grizzly.util.OutputWriter
13 class MyTCPSelectorHandler extends TCPSelectorHandler {
15     override def onAcceptInterest(key:SelectionKey, ctx: Context ) : Boolean = {
16         var channel = acceptWithoutRegistration(key)
18         if (channel != null) {
19             configureChannel(channel)
20             var readKey = channel.register(selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE)
21             readKey.attach(System.currentTimeMillis())
22         }
24         false
25     }
26 }
28 class MyProtocolChainInstanceHandler extends ProtocolChainInstanceHandler {
30     private var protocolChain = new DefaultProtocolChain()
32     def poll() : ProtocolChain = protocolChain
33     def offer(instance: ProtocolChain ) : Boolean = true
34 }
36 class DaytimeWriterFilter extends ProtocolFilter {
38     def execute(ctx: Context) : Boolean = {
39         if (ctx.getProtocol() == Controller.Protocol.TCP) {
40             var channel = ctx.getSelectionKey().channel()
41             var now = (new Date()).toString() + "\\n"
42             var buffer =  ByteBuffer.wrap(now.getBytes())
44             OutputWriter.flushChannel(channel, buffer)
46             channel.close()
47             buffer.clear()
48         }
49         false
50     }
52     def postExecute(ctx : Context ) : Boolean  = false
53 }
55 object DaytimeServer extends Application {
57     val PORT : Int = 1405
58     var controller = new Controller()
59     var tcpSelectorHandler = new MyTCPSelectorHandler()
60     var pciHandler = new MyProtocolChainInstanceHandler()
61     var protocolChain = pciHandler.poll()
63     tcpSelectorHandler.setPort(PORT)
64     controller.addSelectorHandler(tcpSelectorHandler)
65     controller.setProtocolChainInstanceHandler(pciHandler)
66     protocolChain.addFilter(new DaytimeWriterFilter())
68     try {
69         println("Starting Daytime server running on port " + PORT)
70         controller.start()
71     } catch {
72         case ex: IOException => println("Exception:" + ex)
73     }
74 }

Donnerstag Feb 26, 2009

Writing a UNIX daytime server (RFC 867) using Grizzly 1.9.x

This week, i started to write some kind of "Hello, World" programs for the superb Grizzly NIO framework. I used Jean-Francois Arcand's intro documentation as a foundation.

My goal was to create a simple Grizzly version of the UNIX daytime server (RFC 867). I based my tiny little program on Grizzly 1.9.7 therefore i had to remove the DefaultPipeline/setPipeline code which was needed with Grizzly 1.0.x in his examples. The tricky part was to tell Grizzly to process the ProtocolFilter on an accepted connection, rather than waiting for some data coming upstream. This is how the daytime server works - you connect to it, you get the date/time and the server closes the connection. 

Thanks to Oleksiy Stashok's reply on the grizzly users mailinglist this was fixed quite fast.  

Here's my poor man's version of a unix daytime server: 

 24 public class Main {
 26     private static int PORT = 1405;
 28     public static void main(String[] args) {
 29         Controller controller = new Controller();
 31         TCPSelectorHandler tcpSelectorHandler = new TCPSelectorHandler() {
 33             @Override
 34             public boolean onAcceptInterest(SelectionKey key,
 35                     Context ctx) throws IOException {
 36                 SelectableChannel channel = acceptWithoutRegistration(key);
 38                 if (channel != null) {
 39                     configureChannel(channel);
 40                     SelectionKey readKey =
 41                             channel.register(selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE);
 42                     readKey.attach(System.currentTimeMillis());
 43                 }
 44                 return false;
 45             }
 46         };
 48         tcpSelectorHandler.setPort(PORT);
 50         controller.addSelectorHandler(tcpSelectorHandler);
 52         ProtocolChainInstanceHandler pciHandler =
 53             new ProtocolChainInstanceHandler() {
 55                 final private ProtocolChain protocolChain = new DefaultProtocolChain();
 57                 public ProtocolChain poll() {
 58                     return protocolChain;
 59                 }
 61                 public boolean offer(ProtocolChain instance) {
 62                     return true;
 63                 }
 64             };
 66         controller.setProtocolChainInstanceHandler(pciHandler);
 68         ProtocolChain protocolChain = pciHandler.poll();
 69         protocolChain.addFilter(new DaytimeWriterFilter());
 71         try {
 72             System.out.println("Starting Daytime server running on port " + PORT);
 73             controller.start();
 74         } catch (IOException ex) {
 75             Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
 76         }
 78     }
 79 }
 81 class DaytimeWriterFilter implements ProtocolFilter {
 83     public boolean execute(Context ctx) throws IOException {
 84         if (ctx.getProtocol() == Controller.Protocol.TCP) {
 85             SelectableChannel channel = ctx.getSelectionKey().channel();
 87             String now = (new Date()).toString() + "\\n";
 89             ByteBuffer buffer =  ByteBuffer.wrap(now.getBytes());
 91             OutputWriter.flushChannel(channel, buffer);
 93             channel.close();
 94             buffer.clear();
 95         }
 97         return false;
 98     }
100     public boolean postExecute(Context ctx) throws IOException {
101         return false;   // game over
102     }
104 }

Freitag Feb 20, 2009

Running Openbravo on Glassfish v3

Last week i tried to run Openbravo 2.40 on Glassfish v3 (prelude version). Here i like to share my findings and how to get this setup running for those who follow. I neither explain how to install the necessary database nor the tomcat installation. The database was in my case PostgreSQL. I did the installation on a MacBook Pro/Intel running Mac OS X 10.5.6.

I had to get the "official" installation image for Openbravo on my platform (Mac OSX) which asks for the location of a pre-installed tomcat and login credentials for either a Oracle or PostgreSQL database. You have to run the standard installation process to get all schemas and data loaded into your preferred DB.

This can take up quite some time, but after this installation Openbravo could be found in tomcat's webapps directory: $TOMCAT_HOME/webapps/openbravo/openbravo.war. Trying to deploy this WAR-archive right away to Glassfish fails for reasons I'm going to explain below. Since there is one slight modification to Openbravo's deployment-descriptor (WEB-INF/web.xml) needed, we're unpack openbravo.war into it's own directory and deploy the application out of this directory.

The problem with the existing deployment-descriptor is, that it's written for Servlet 2.3 webcontainers like Glassfish v2 or tomcat. Glassfish v3 adheres to the Servlet 2.4 specification which is a bit picky about url-pattern. In principle this means you have to code this:


this way:


This is tracked, accepted and fixed as issue 7493 in Openbravo. Once this change makes it into Openbravo, this whole document becomes obsolete. But till then you have to apply this patch to WEB-INF/web.xml to rewrite the deployment descriptor.

After this modification, just deploy Openbravo using Glassfish's admin GUI using this trail: Applications/Web Applications/Deploy ...:

After a few seconds the new deployed application should show up in the list of your web-applications:

Now you should be able to login using Openbravo's default account Openbravo/openbravo :

which in turn take you to Openbravo's main dashboard:

Montag Mrz 31, 2008

Grails update.

I'm back from my Easter-vacation and just stunned about the feedback I've got for this blog.

Thanks so much to all of you!

I haven't figured out whether and how i could answer to individual comments, but i guess this isn't possible.

So here some answers and hints:

1. UNC path, @ChinaBoy:

This issue was once tracked by us here:

Dialog-box after creating a new Grails-project

After some testing I considered it fixed. Obviously it isn't. I'll look into it asap.

2. Netbeans 6.1 Beta or Development build, @Stefan:

Please take a look here:

There you can find the always up-to-date NBM's from our continous-integration system.

Take the 6 modules prefixed "org-netbeans-modules-groovy\*":







3. Stop App on Windows, @JS:

"stop application doesn't work"

That's right, this is an issue (only) under Windows since there is not the concept of process-groups under Windows.
Details can be found here:

Stop Application does not work anymore

I'm already working on a fix for this, so expect some relieve within the next one or two weeks.

"although the contextual menu is great, wouldn't it be great to have a small toolbar with 5 buttons (new [domain, controller, view, taglib, service]) and their keyboard shortcut?"

This is a great idea. I've created an issue to track this:

Adding Toolbar-buttons for common grails actions 

Thanks for your input!

"integration with servers inside the IDE: right-click on a grails project -> deploy to -> GFv3/JBoss/Tomcat"

This is definitely on our wish-list, but don't expect something in this area in the short-term.

4. CodeCompletion documentation, @baijianhua:

Hi - I feel really sorry to forget to mention that you have to have:

    1. JavaDoc enabled for the JDK: Tools/Java Platforms/J2SE "Javadoc"

    2. The "original" directory-layout from the groovy-docs-\* download-file unpacked right below the $GROOVY_HOME setting provided under Tools/Options/Groovy. For example if your groovy system is here:


      you need to have:


      This is hardcoded at the moment which is bad for several reasons, I know. This might be better placed in the customizer dialog, right after the $GROOVY_HOME setting itself. What about that?

Donnerstag Mrz 20, 2008

News from Grailsland

The past

Last year November Martin Adamek and me started to work on a full-featured set of modules to support the Groovy programming language and the grails application server. After being quite impressed (and a little bit depressed) about the feature-set of the Intellij Groovy/Grails plug-in, we wanted to create something similar for NetBeans.

One key responsibility of all OpenSource projects is to carefully look around what's already out in the world. Re-use of existing sourcecode and/or leveraging existing libraries is a given in this culture. Therefore we started off our work by using the existing, but inactive coyote plug-in. Doing a language plug-in. with an extensive feature-set comparable to the Intellij plug-in. goes way beyond the scope of the otherwise very helpful Schliemann framework.

This left us with the cumbersome tasks of coding everything from scratch against the NetBeans Editor API. Luckily we soon discovered Tor Norby's GSF Framework. GSF's purpose is to help creating exactly what we were about to create - A new scripting language support module. GSF really achieves this goal and has been one of the reasons, why we are able to develop so many features in such a short period of time.

Here are the modules and there main duties:

Collection of helper- and utility classes alongside the Customizers.


Editing Groovy sourcecode files. All the bells and whistles like syntax-highlighting, folding resides here.


Support for editing Groovy server pages files (GSP).


Here's all the code interfacing with an ex-process grails server. Architecture-specific execution of the "grails <target>".


Support for the new NetBeans project type "Grails-Project" together with all visual components like Wizards, Actions, Node-renderers and the like.


Wrapper module for the ones above to show-up as a single entity in NetBeans PluginManager.

The present


NetBeans 6.1

Initially we started in NetBeans old CVS trunk which soon became NetBeans 6.0. We soon realized, that we need to use some new API's, therefore the suite of modules needs NetBeans 6.1 as a minimum.


Please use the daily snapshots from the NetBeans source-repository continous integration system (Hudson), which can be found here: Builds

By using these builds, you are going to get all the pre-configured development UpdateCenter which include our modules. 


... Or work with the NetBeans sources from the mercurial repository:

and build it yourself (it is easy!, No really it is, no one forced me to write this.)

Here's a great description, how to do this:


The Modules


The Plug-ins are available from the UpdateCenter, therefore just go to:

Tools > Plug-ins > "Available Plug-ins" > "Groovy and Grails"

and install them.


The sourcecode for the Groovy/Grails modules lives in the "contrib" repository:

contrib repository webinterface 

there are six subdirectories starting with "groovy.\*".

If you build them in this order:

  2. groovy.editor
  3. groovy.gsp
  4. groovy.grails
  5. groovy.grailsproject
  6. groovy.kit

using ant:

ant -f groovy.\*/build.xml

you have them automatically up and running the next time you start NetBeans.


So what are you getting by using these modules? Pictures entering the stage:

  1. Method-completion including JavaDoc display for Groovy and Java:

  2. Code Folding of Groovy sourcfiles:

  3. Starting, stopping of Grails-Server:

  4. Importing existing Grails projects. Well arranged display of project structure:

  5. Groovy/Grails module settings integrated into NetBeans options dialog:

  6. Marking of sourcecode errors:

  7. Easy navigation of Groovy sourcecode by using a navigator view:

  8. Customizing of Grails Environment and Serverport. Auto-deploy to Glassfish Appserver:

  9. Starting common grails-tasks from context menu:

  10. Status of running grails server displayed in status-line:

  11. Syntax Highlighting:


I would love to get feedback about this project. No matter how much planning one is doing, at the end of the day the software should be used and should be usable for real live users. Therefore I'm bleeding to get your feedback. What you like, what you don't like, what you think we should implement. Ideally, you file your bugs and wishes right where they will end-up anyway and that's the NetBeans issue-tracking system Issuezilla.

We have our own Component-name which is (you guessed it): "groovy".

The Issuezilla landing page could be found here:

NetBeans Issuezilla 

The future

There are still many, many things missing from the modules. Depending on the interest, people are showing for the initial release ( we won't make it into NetBeans 6.1 ), these are candidates
for features of the next release:

  1. Debugging support. (yes - it's sad but true, that we have no debugging support at the moment)
  2. MultiView for easy navigation between corresponding Model-View-Controller files.
  3. Refactoring support.


Mittwoch Jan 31, 2007

JDK6 DTrace Probes presentation at Sun PartnerUniversity 2007 Berlin

Hi, i've got some requests for the slides on my presentation "JDK6 DTrace probes".

You can get them here. Please feel free to comment and discuss the topic here as well.

hope that helps,


PS.: By popular demand i've added the D and Java Sources as well: dtrace-jvm-probes.tar.gz




« Juli 2016