Friday Oct 19, 2007

Calling Jython from a Java Servlet

I'm not really sure why anyone else would want to do this, but we have a big application written in Python that we have been experimenting in calling from Java using Jython. The complexity was that the application uses cpickle as part of its communications layer, so we needed to initialise Jython's copy of cpickle before we could use it. This was, in fact, no mean feat and took several hours of poking around in the Jython source to get to the answer!!

So, in the interests of saving other people jumping through a similar hoop in future, I've documented what worked for me here...

The Java magic you need to initialize Jython and (in particular) the cpickle class is:

    private static void initPython()
        PySystemState.initialize(PySystemState.getBaseProperties(), null, null);
        PySystemState sys = new PySystemState();
        Py.setFrame(new PyFrame(null, sys.__dict__, sys.__dict__, null));

I'm sorry that I can't tell you what each step does because other than the first step, which is the main initialisation for Jython, I really don't know. It's enough for me that it actually works ;)

Once you have the code set up and compiling okay, when you call java, you need to tell it where to find Jython:

java -cp my-app-dir:/local/packages/jython/jython.jar -Dinstall.root=/local/packages/jython my-app

This enables Jython to correctly find its registry, in which I'd already updated python.path to include the path to Jython's Python dir ("Lib") and our own Python code dir.

However, not content with this suffering, we decided to write a servlet to access the Python application. This uses the same initialisation as above, but this time we need to tell Jython what the python.path should look like, since we don't want to be setting global JVM variables in the webserver for one application.

So the magic to add to your servlet's init() method should look something like this:

    public void init() throws ServletException. IOException
        Properties pOverride = new Properties();
        String jythonPythonPath = getServletContext().getRealPath("lib/jython") + "/Lib";
        String pythonPath = jythonPythonPath + File.pathSeparator + "/my-app/lib/python";
        pOverride.put("python.path", pythonPath);
        PySystemState.initialize(PySystemState.getBaseProperties(), pOverride, null);
        PySystemState sys = new PySystemState();
        Py.setFrame(new PyFrame(null, sys.__dict__, sys.__dict__, null));
The second argument to PySystemState.initialize is (obviously) a Properties instance which contains those properties which you want to replace (or set, in our case) during Jython's initialisation. We only set python.path but if you look in the Jython registry, you can see what other options there are.

Note that this code could (and probably should) go into a static initialiser rather than a class method, but the servlet was really only a test harness, so it wasn't coded for style/robustness/whatever!

Finally, we set up the WAR file build so that it copies the Jython installation directory into "lib" at the root of our WAR file. This is probably overkill, since we already have jython.jar in the jars for the build and it only appears to need the "Lib" directory (which contains the Python code), but it was expedient and I haven't had time to test paring it down yet.

Wednesday May 18, 2005

When is free not free ?

To be honest, I'm not sure I know the answer to that question, but it arises from this article about the use of Java in

Now, obviously, I'm coming at this from the point of view of being an Engineer at Sun, so by definition (in the FSF's eyes), I am already tainted by the devil. However, it appears to these tainted eyes that Richard Stallman and others are stirring up trouble for the sake of the minority of folks who might regard Sun's Java implementation as being not free. I assume by this that they mean not free to be modified, enhanced, redistributed rather than costing money (which it does not).

I've tried putting myself in the shoes of a potential customer of, and to be honest, I can't see a problem with it using Sun's version of Java. After all, that is the real Java and no other [Open Source] implementations can be called that. As a customer, I could easily download both and Sun's Java implementation for use on Windows, Linux or Solaris (to name but 3 OS'es), and being an end-user, I can see no issue with this at all.

Whilst I support some of the ideals of Open Source Software, I do wonder if some people get too caught up in the ideology and don't actually stop to consider what might be the best thing for the end-users. In this particular case, I think it would be deplorable if the 2.0 release were delayed simply because of the work required to make it operate with GJC or some other "free" Java implementation. On the other hand, a fork in the development tree, as noted as a possibility in the article above, is as undesirable, and is one of the less endearing aspects of Open Source projects.

Don't get me wrong - I'm not against the activity to make work with other implementations of Java. It's that extension of functionality which makes Open Source the success that it is. The issue I have is that of the playground bullies (FSF zealots) using intimidation to force development along a particular path which only benefits a minority.

Friday Apr 08, 2005

I learnt something new about Swing yesterday

I've been using Swing since it first appeared and thought myself pretty competent at using it despite not having done much in anger with it for the past couple of years, where my focus has been more browser-based.

I had a little task to update some of a colleague's code yesterday, which involved adding a new message to be displayed in a JOptionPane. I noticed, and was puzzled by the fact that there were messages in the file which had HTML constructs in them. These messages are displayed in a JOptionPane too.

At first, I ignored the HTML constructs, assuming that the HTML was somehow filtered. However, when I queried my colleague on it, it turned out that you can indeed use limited HTML in Swing components, and it is documented in the manual. I guess I just haven't looked at the JLabel API docs for so long that I missed the change.

In the end, I didn't need the HTML, because I only wanted basic multiline presentation in the JOptionPane, which you can achieve by interspersing the string "\\n" in your text. For example:

    String msg = "This is a\\nmulti-line\\nmessage.";

My colleague was trying to do the same multi=line display, but using HTML:

    String msg = "<html>This is a<br>multi-line<br>message.</html>";

I may not have needed it, but I'm sure I will find a use for it in the future, so it's nice to know.

Wednesday Dec 15, 2004

Tuning Java applications

I am on an internal mail alias at Sun where there are regularly discussions on how customer X can get the best from their Java application.

Barely without exception, the reply from the people-who-know is to use some arcane option or combination of options to the Java VM.

Now, the thing that irritates me (and yes, this is a rant) is that whilst these options are no black magic, they are pretty close to it, and it relies on the application deployer to be able to convey their performance issue in the correct terms, and the right person-who-knows to interpret that and pass on the appropriate suggestion using -XXflag. My problem is that I don't think that this should be necessary. Why do these options exist at all, except in an experimental environment where testing is required to determine how to get the best from the VM ?

Sure, I appreciate that the JVM is a complex beast (particularly in the area of GC), and different configurations are required for a lightweight client as opposed to a heavyweight server app (this, after all, is why we have the "-server" and "-client" options.

What I think is required is a dynamic tuning mechanism which allows the JVM to monitor itself and then adjust its configuration based on a number of parameters such as throughput, number of incoming/outgoing connections, database activity, etc. This probably requires an application of AI, but given that there is a good profiling interface to the JVM, maybe it wouldn't be that difficult to do (for someone with a brain the size of a small planet).

Oh, and in case you think I'm just picking on Java, try figuring out what options to use to the C/C++ compilers to get the best performance out of your code, and you'll see a very similar problem. Gone are the days when "-O4" was all that was needed :(

I'd like to put addressing this voodoo to the top of my Christmas list so it makes it easier for me as a developer and for application deployment teams to get the best from their apps.

Or perhaps we just need faster CPUs and faster memory busses ?




Top Tags
« April 2014