Porting the DRMAA Java Language Binding

There's a been quite a bit of talk on various aliases (and over private email) recently about what's required to port the Grid Engine DRMAA JavaTM language binding to another DRM. Since that is an interesting topic, I figured I'd assemble all of the answers here for easy reference.

First, a little background. The Grid Engine DRMAA Java language binding is built as a layer on top of the C binding. The Java language binding was implemented in this way because it was fast and easy. At the time, time to market was a big concern, and rightly so. We are now in the process of designing a more elegant way to access the qmaster from the Java platform, but that will take a little while to design and implement.

If you look at an architectural layer diagram for the Java language binding, you'd have the following layers:

  1. org.ggf.drmaa classes
  2. com.sun.grid.drmaa classes
  3. JNI adaptation layer
  4. C binding
  5. JAPI
  6. GDI & EVC libraries
  7. Commlib

On the top you have the implementation-independent classes, the API. Under that you have the implementation-specific classes, the service provider implementation. Under that is a thin layer of JNI code. (source/classes/src/com_sun_grid_drmaa_SessionImpl.c) The JNI code calls the DRMAA C binding, (source/libs/japi/drmaa.c) which calls the Job API (JAPI), (source/libs/japi/japi.c), which uses the GDI and event client (EVC) libraries to communicate with the qmaster via the commlib. Now you know.

From the perspective of porting, the interesting layers are 2 and 3. The top layer should stay the same for all implementations. Because the Grid Engine Java language binding is layered over the C binding, as long as the target DRM supports DRMAA, layer 2 also remains unchanged. (Not entirely, but I'll get to that later.) That leaves us with layer 3.

Originally, the JNI layer was very dependent on Grid Engine's internal libraries. However, in a fit of obsessive-compulsiveness, I removed all the Grid Engine dependencies early this year. There are now only two things that actually need to be changed. They are lines 40 and 41:

40: #include "japi/drmaa.h"
41: #include "japi/msg_drmaa.h"

Line 40 is simple to fix. It just needs to be changed to point to the drmaa.h header in the target DRM. Line 41 is a little more complicated. In Grid Engine, header files that start with "msg_" contain output messages which have been extracted from the code. For the JNI layer to work, it needs three messages defined. msg_drmaa.h defines them as:

#define MSG_JDRMAA_BAD_JOB_TEMPLATE    _MESSAGE(45101, _("Requested job template does not exist"))
#define MSG_JDRMAA_NULL_POINTER_S      _MESSAGE(45102, _("The "SFN" may not be null"))
#define MSG_JDRMAA_OVERFLOW_DS         _MESSAGE(45103, _("More than %d "SFN" were received"))

For our purposes here, just ignore the _MESSAGE macro. It just wraps the string with some message numbering information. Treat the SFN macro just like a %s. With that in mind, you should be able to see that these messages are just format strings for a printf, and that's exactly how they're used. (Well, vsnprintf(3C) to be exact.) The suffixes on the macro names indicate the parameters the format string should accept. No suffix means no parameters. "_S" means a single string parameter. "_DS" means an int followed by a string. All you need to do is provide definitions for these three messages, and you're good to go. You can either define them in a header and include it on line 41, or you can define the messages directly in the JNI file, or you could even hard code the messages into the code.

After that, aside from minor platform differences or variations in the DRMAA C binding implementations, you should have a functioning port. There is one other issue to keep in mind, however. As I mentioned above, you may have to make some changes to the service provider implementation. In the source/classes/com/sun/grid/drmaa/JobTemplateImpl.java file, there are a series of constants used to define the string names of the DRMAA properties in the C binding. The constants for the optional properties which the Grid Engine bindings do not support are commented out. If your DRMAA C binding implementation supports any of those attributes, you'll need to uncomment the appropriate constants and provide getters and setters for those properties.

OK. Now you know as much about porting the Java language binding as I do. It appears that the work I did to make it portable will pay dividends in the form of a Java language binding for Condor and one for Torque in the near future, just as soon as the C bindings for those DRMs are ready. Happy porting!


Post a Comment:
  • HTML Syntax: NOT allowed



« June 2016