Way to go, JT Harness
By jjg on Nov 14, 2006
Q: What exactly is JT Harness?
A: It's the open source version of Sun's JavaTest harness.
Q: So what exactly is JavaTest?
Like the word Java, the word JavaTest means different things to different people. To some people, "Java" means a programming language, to others, it is a programming environment, to others, it what makes games work on the cell phones, or it is what makes their web sites work well. (And don't forget the lawyers, to whom it is a trademark, and should always be used as such.)
Likewise, to some people, JavaTest is how they run the tests in the Java Compatibility Kit (JCK), to others, it is how they run tests in the JDK Test Framework, and to others, it's what they use to make sure that when you run games on your phone, they work as you expect. To those of us who have worked on JavaTest, it's a framework for building and deploying test suites. Who is right? Everyone, of course, it just depends on your perspective.
As I'm sure most everyone knows, compatibility has been an important goal ever since the beginning of Java, and from the beginning, there has been a compatibility test suite. The original test suite was a series of applets on a series of web pages, that you had to visit manually, with your browser -- this was back when "Java == Applet". That didn't cover the compiler, nor did it cover all the stuff that you couldn't do from an applet: and so we needed a test harness to run tests in all the ways and in all the places you could run Java code, and thus begat JavaTest.
To begin with, JavaTest was closely tied to the Java Compatibility Kit -- it \*was\* the harness for JCK. But as the need for regression tests arose, it became clear that the requirements for regression tests were very different from the requirements for conformance tests. Compatibility tests were of relatively few types (API tests, positive and negative verifier tests, and positive and negative compiler tests) and had to run on a wide variety of platforms, some of which didn't even exist yet. By contrast, the regression tests often tested a relatively complex sequence of events, but only had to be run by JDK, on platforms supporting JDK. Compatibility tests were categorized with a relatively small number of keywords so that they could be run appropriately by the harness on any given platform. But something different was required for JDK regression tests.
The first step was the JDK Test Framework Tag Specification, to describe what we wanted to be able to support in regression tests. Then, we had to use and extend JavaTest's plug-in architecture, so that it could recognize and execute JDK regression tests. The plug-in architecture allowed a test suite to specify a "test finder" to locate tests to be executed, and a "test script" to execute those tests. Using those mechanisms, the harness could read and execute both compatibility tests, and JDK regression tests. Over time, we've utilized the architecture to run tests on all sorts of platforms -- in some cases, the same tests are run on servers, desktops, and phones, to ensure the original promise of "Write Once, Run Anywhere".
And now, JavaTest begins an exciting new stage, being made available as the open source "JT harness". Way to go, JavaTest, er, make that JT Harness.