jtreg 4.1

There is a new version of jtreg available, 4.1, with a number of useful new features. Most of these are to do with the way that jtreg runs tests, but one feature, limited support for JUnit tests, involves a spec change for the test descriptions, which means that this new version of jtreg will be required to run any testsuites that include such tests.

jtreg can be downloaded from the OpenJDK jtreg page.

New "agentvm" mode

Up to now, jtreg has provided two ways of running tests: in "othervm" mode and in "samevm" mode. In "othervm" mode, a new JVM is created for each action that needs it. This provides the maximum isolation between tests, at a very high cost in performance. In "samevm" mode, all Java actions are run in the same VM, which provides high performance at the risk of tests being able to affect the behavior of any subsequent tests. In some cases, a bad test can prevent all subsequent tests from being able to execute.

Problems can arise when setting a security manager such that it cannot be removed when a test completes, or failing to close any open files in the scratch directory, which on Windows means they cannot be deleted before the next test runs.

"Agentvm" mode is a new mode that is "like samevm mode, but better". JVMs created to run tests will be reused for subsequent tests provided that the jtreg harness can successfully perform a certain amount of cleanup after each test. If ever the cleanup fails, the corresponding JVMs will be terminated, and new ones will be started if necessary for any subsequent tests.

In "samevm" mode, because the same JVM is used for both test compilation and test execution, it is not possible to specify JVM options to be used just when executing a test. This restriction is not necessary in "agentvm" mode, and separate JVMs will be created if needed for any actions requiring different JVM characteristics. Assuming the cleanup after a test is successful, all the JVMs used by a test will be saved for reuse by later tests that may need JVMs with those same characteristics. However, if any of the cleanup fails, all the JVMs used by the test will be terminated, because in general it is not possible to determine which of the JVMs may have been at fault.

The new mode can be selected with the -agentvm option. For those situations where "samevm" mode is currently being used, it is also possible to select the new mode by setting the environment variable JTREG_USE_AGENTVM_FOR_SAMEVM to true. This may be convenient if you don't have easy access to the command line used to invoke jtreg.

Better links in HTML report files

When jtreg writes HTML report files, it creates links from the report to files in the work directory, and these links could easily be broken if the files in these directories were moved. jtreg now checks if the report directory and work directory are "closely related", meaning they are the same, or one is a parent of the other, or if they have a common parent. In these cases, jtreg will use relative links from the report files to the test result files in the work directory, so that if the two directories are moved together and their relative relationship is retained, the links will remain valid.

JUnit tests

Limited support has been added for running JUnit tests. Tests still require a standard jtreg test description, but you can use a new action "@run junit classname", which will invoke the test in the manner of JUnit 4. The class will be compiled if necessary, or it can be compiled explicitly in the standard way with the "@build" and "@compile" actions. (Thanks to John Rose for providing this feature.)

Separate compilation: -compilejdk:jdk

Up to now, jtreg has only been able to test versions of JDK, or more accurately, versions of Java that include the compiler javac. jtreg now has a new option to specify an alternate JDK to be used for any @compile actions in a test. This includes all implicit @compile actions as well. This means that jtreg is now able to test versions of the Java platform that do not include javac.

Notes:

  • If the -compilejdk option is not used, the same version of Java used to compile and run the tests, as now.
  • Previously, if no version of Java to test was specified on the command line, the value defaulted to the version of Java used to run jtreg. If this value was a JRE embedded in a JDK, the JDK was used instead. Now, if -compilejdk is specified, the preference to use a JDK instead of the embedded JRE is no longer required. If in doubt, always do one of the following:
    • specify -testjdk for the JDK used to compile and run the tests
    • specify -compilejdk for the JDK used to compile the tests, and -testjdk for the JDK or other version of Java to run the tests
  • Some shell tests invoke the compiler directly, using the value in the TESTJAVA environment variable. (For example, ${TESTJAVA}${FS}bin${FS}javac -d classes HelloWorld.java.) jtreg provides a new environment variable, COMPILEJAVA, which will be set to the value of -compilejdk if it was specified, and $TESTJAVA otherwise. If the tests in a testsuite might be run with -compilejdk set, any shell tests using $TESTJAVA to access the compiler should be updated to use $COMPILEJAVA instead.
Improved path handling

In various situations, jtreg creates or modifies search paths, such as a classpath or sourcepath, for use in the actions of a test. Previously, jtreg did not check whether the components it was adding to a path actually existed, and this could cause problems for some tests that checked the contents of these paths (for example, with the javac -Xlint or -Xlint:all options.) Components to be added to a path are now checked to verify they identify an existing file or directory.

Future source

The @compile action now checks the modification time of source files to be compiled, and gives a warning if the last modified time is in the future.

New option -allowSetSecurityManager[:value]

jtreg now allows tests in samevm mode and the new agentvm mode to set the security manager. (It was always possible to set the security manager in othervm mode.) If the security manager cannot be reset when the test completes, then in samevm mode, verbose messages will be printed directly to the console output, and in agentvm mode, the test's JVM(s) will be terminated. Acceptable values are yes, on or true and no, off or false. The feature is enabled by default; use -allowSetSecurityManager:off to disable it.

New suboption for -retain

The -retain option allows you to specify which files in the scratch directory should be retained when a test completes. The default (when the option is not specified) is to leave the files in the scratch directory until the beginning of the next test (if any.) This is convenient when running a single test, but is not convenient when there is a problem cleaning up any files at the beginning of the next test -- by then, the identity of the test that created the files has been lost.

A new suboption is now available for -retain: -retain:none. With this option, all files remaining in the scratch directory when a test completes will be removed immediately. Any problems that may arising in deleting these files will cause the test to be reported with an ERROR result.

-classpathappend in samevm mode

The -classpathappend option did not work in samevm mode, and has now been fixed.


Thanks to Joe Darcy for his feedback on this note.
Comments:

Post a Comment:
Comments are closed for this entry.
About

Discussions about technical life in my part of the OpenJDK world.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today