jtreg update, 2011
By jjg on Jul 21, 2011
There is a new update for jtreg 4.1 available. The most notable change is the addition of support for concurrent test execution: great for use on modern multi-core machines! Other changes have been driven by the goal to help cleanup and improve the JDK unit and regression test suite.
Concurrent test execution
As reported earlier,
jtreg now supports concurrent test execution in the agentvm mode and othervm modes. Agentvm mode
was introduced last year, and is "like samevm
mode, but better." To run tests currently, use the new
Start off with a value equal to the number of processors on your system, but depending on the tests you are
running, you may be able to raise the number higher.
Previously, jtreg could only run tests under a single root directory, identified by a TEST.ROOT file. Now, for various reasons, the JDK unit and regression tests are being split across multiple repositories, into separate directory trees, each with their own test root and TEST.ROOT fie. For example, the "langtools" tests, in langtools/test. are separate from the main set of "jdk" tests, in jdk/test. So, the restriction on running tests under a single root directory has been relaxed. Under the covers, jtreg automatically groups the tests according to their test root directory, runs each group of tests in turn, and then aggregates the results.
More result and report details
The hostname on which a test is run is recorded in the *.jtr file. This is useful when test runs have been distributed across a set of machines.
The test results can now also be written to XML files in a format recognized by Hudson,
which makes it easier to track the test results over time. Use the
to enable this feature. The data for each test.jtr file is written to a
corresponding XML file named test.jtr.xml in the same directory.
for this contribution.
The time taken to run a test is now reported in the .jtr file. Previously, the start and end times were available, but they had to be read separately and the elapsed time computed. The elapsed time is now available directly, in both milliseconds and hh:mm:ss.millis.
The distribution of times to run each of the tests in a test run is now reported in a new file
text/timeStats.txt written to the report directory at the end of the run.
The mean and standard deviation are given as well.
You can now configure the format of the status message written by jtreg to the console at the end of the test run. See the online help for details of the system property to set and the escape sequences that are recognized. The message is also written to the file text/stats.txt in the report directory.
There is a new option
-limittime:N that can be used to filter out tests which may
take a long time to run. jtreg examines the test's declared timeout value to determine whether the test
should be run or not.
ProblemList.txtjtreg now provides direct support for the ProblemList.txt file used to identify problematic tests in the jdk/test/ regression test suite. Previously, it was processed by test/Makefile into an exclude list; now, the fie can be given directly to the
javac exit codes
Since JDK 6, javac has used a small set of fixed exit codes to identify different exit conditions. jtreg now recognizes those codes. In particular, the @compile/fail option will now only succeed if the compilation exits normally, after generating diagnostics according to bad source files. It will not succeed if javac exits for a more serious reason, such as a javac crash.
For licensing reasons, we cannot ship a copy of JUnit with jtreg. If you want to use the jtreg support for running JUnit tests, install a copy of junit.jar in the jtreg lib/ directory or set the system property junit.jar to a place where it can be found.
Thanks to Joe and Maurizio for their feedback on this note.