Friday Sep 12, 2008

OpenJDK 6: Some regression test results for b12

Running with jtreg flags -a -ignore:quiet, the basic regression test results on Linux for OpenJDK 6 build 12 are:

  • HotSpot, 5 tests pass.

  • Langtools, 1,346 tests pass, 2 tests fail.

  • JDK, 2,985 tests pass, 36 tests fail, 4 tests have errors.

For langtools, from the previous build a few new (passing) tests were added and @ignore tests were really ignored; using jtdiff, part of the jtreg package to compute the differences of the tests results:

0: ./b11-langtools/summary.txt  pass: 1,341; fail: 2; error: 6
1: ./b12-langtools/summary.txt  pass: 1,346; fail: 2

0      1      Test
---    pass   tools/javac/6176978/
---    pass   tools/javac/6627362/
error  ---    tools/javac/
error  ---    tools/javac/api/
error  ---    tools/javac/api/
error  ---    tools/javac/api/
error  ---    tools/javac/code/
---    pass   tools/javac/enum/
error  ---    tools/javac/generics/inference/6365166/
---    pass   tools/javac/staticImport/6665223/
---    pass   tools/javac/synthesize/

11 differences

Also for the jdk area, some new passing tests were added and the -ignore:quiet option prevents some tests that knowingly produce an error result from running. The few new failures warrant a bit of investigation.

0: b11-jdk/summary.txt  pass: 2,989; fail: 32; error: 22
1: b12-jdk/summary.txt  pass: 2,985; fail: 36; error: 4

0      1      Test
error  ---    com/sun/crypto/provider/Cipher/DES/
pass   ---    com/sun/org/apache/xml/internal/security/exceptions/
pass   ---    com/sun/org/apache/xml/internal/security/transforms/
error  ---    com/sun/security/auth/callback/DialogCallbackHandler/
error  ---    com/sun/security/auth/callback/TextCallbackHandler/
error  ---    com/sun/security/sasl/gsskerb/
error  ---    com/sun/security/sasl/gsskerb/
error  ---    com/sun/security/sasl/gsskerb/
pass   fail   java/awt/Focus/DeiconifiedFrameLoosesFocus/DeiconifiedFrameLoosesFocus.html
fail   pass   java/awt/Focus/FrameMinimizeTest/
pass   fail   java/awt/TextArea/UsingWithMouse/SelectionAutoscrollTest.html
---    pass   java/awt/Window/PropertyChangeListenerLockSerialization/
error  ---    java/io/
error  ---    java/lang/instrument/
error  ---    java/lang/management/ThreadMXBean/
fail   pass   java/nio/channels/SocketChannel/
pass   ---    java/rmi/activation/Activatable/checkActivateRef/
error  ---    java/util/AbstractList/
---    pass   java/util/EnumSet/
error  ---    java/util/zip/
fail   ---    javax/management/Introspector/
error  ---    javax/management/remote/mandatory/
error  ---    javax/security/auth/kerberos/
---    pass   javax/sound/midi/Gervill/DLSSoundbankReader/
---    pass   javax/sound/midi/Gervill/DLSSoundbankReader/
---    pass   javax/sound/midi/Gervill/DLSSoundbankReader/
---    pass   javax/sound/midi/Gervill/DLSSoundbankReader/
---    pass   javax/sound/midi/Gervill/SF2SoundbankReader/
---    pass   javax/sound/midi/Gervill/SF2SoundbankReader/
---    pass   javax/sound/midi/Gervill/SF2SoundbankReader/
---    pass   javax/sound/midi/Gervill/SF2SoundbankReader/
fail   pass   javax/swing/JColorChooser/
pass   ---    javax/xml/crypto/dsig/
pass   ---    javax/xml/crypto/dsig/
pass   ---    javax/xml/crypto/dsig/keyinfo/KeyInfo/
pass   ---    sun/net/idn/
pass   ---    sun/net/idn/
pass   ---    sun/security/pkcs11/ec/
error  ---    sun/security/provider/PolicyFile/
error  ---    sun/security/provider/PolicyParser/
error  ---    sun/security/provider/PolicyParser/
error  ---    sun/security/provider/PolicyParser/
error  ---    sun/security/provider/PolicyParser/
pass   fail   sun/tools/jps/
pass   fail   sun/tools/jps/
pass   fail   sun/tools/jstatd/
pass   fail   sun/tools/jstatd/
pass   fail   sun/tools/jstatd/
pass   fail   sun/tools/jstatd/

49 differences

I did a quick trial run of the regression tests on windows too, but my new cygwin installs seems to be missing a few pieces and many of the shell tests fail. I assume most of those failures are spurious, so I'll wait until the cygwin install seems configured properly before rerunning and posting windows results.

OpenJDK 6: Sources for b12 published

On September 12, the OpenJDK 6 b12 source bundle was published. Changes of note in this build:

While the changes in this build have corrected a number of JCK 6b test failures, a handful of tests still fail; eliminating those few remaining failures is a goal for the next build or two. (JCK 6b was released several months ago and has now replaced JCK 6a as the relevant test suite to pass for Java SE 6 compatibility.)

Internally, the code for OpenJDK 6 is split into teamware workspaces with a similar structure to the JDK 7 Mercurial repositories (corba, jaxp, jaxws, langtools, jdk, etc.). For a subset of these component workspaces, I've brought over the fixes in corresponding areas from 6u10. OpenJDK 6 build 12 will be one of the last teamware-based OpenJDK 6 build before we transition to a public Mercurial repository. Around the time of the Mercurial transition we will also upgrade the HotSpot in OpenJDK 6 from HotSpot 10 to HotSpot 11; HotSpot 11 is also being used in 6u10. From that point on, the same HotSpot sources will be used for both OpenJDK 6 and the 6 update releases.

I expect build 13 within a few weeks with the Mercurial repositories to follow a few weeks after that.

Wednesday Aug 20, 2008

OpenJDK 6: Some more regression test results for b11

To address a configuration issue with the previously reported regression tests results for OpenJDK 6 b11, I reran the jdk tests with the display set to a virtual framebuffer and 77 more tests pass. In summary now:

  • JDK, 2,989 tests pass, 32 tests fail, 22 tests have errors.

Note that the jtdiff command, included as part of jtreg, can be used to compare the output of different regression test runs.

Thursday Jul 17, 2008

OpenJDK 6: Some regression test results for b11

For comparison purposes, I've uploaded the results of the automated regression tests (-a option to jtreg) from the OpenJDK 6 build 11 source bundle run against a Sun-internal Linux build:

  • HotSpot, 5 tests pass.

  • Langtools, 1,349 tests pass, 2 tests fail, 6 tests have errors.

  • JDK, 2,912 tests pass, 109 tests fail, 22 tests have errors.

I ran the tests remotely and didn't set a usable display so about 72 of the JDK failures are from the AWT tests not having a display; I'll rerun the affected tests and verify they will pass as expected. At least one of the langtools failures seems to be from an overly zealous version check that should be modified.

I've also ran tests on a Solaris SPARC system; the HotSpot and langtools results were identical and there were a few differences in the JDK results.

Tuesday Jul 15, 2008

OpenJDK 6: Sources for b11 published

On July 15, the OpenJDK 6 b11 source bundle was published.

This source bundle includes:

  • All the relevant JDK security fixes that recently went out. More information about these changes is available from the Sun Alerts linked to from one of Sun's security pages. Some of the security fixes only affect closed code or areas otherwise not open sourced as part of OpenJDK 6 so not all the security fixes are reflected in the source bundle.

  • Ports of existing fixes in various areas. In HotSpot, fixes from previous 6 update releases (6497639, 6623167, 6623167, 6497639, 6599425), a build issue (6681796), and a fix for an Eclipse crash (6614100). In the libraries, there were fixes for graphics (6691328, 6608764, 6624717) and monitoring (6685178).

  • Changes to address various of the previously raised concerns on licensing/copyright and binary artifacts. First, I applied Andrew John Hughes's fix to remove jscheme.jar (6695776). Next, Kelly and Iris have fixed many other licensing and binary artifact issues (6565364, 6705945, 6601384, 6601377, 6713083, 6695777 6710791).

  • Miscellaneous other fixes cleaning up regression tests (6621691, 6596323, 6710579) and other matters (6717575, 6665028, 6589868).

I'm in the process of updating the Gervill sound engine in OpenJDK 6 to the current version; that work will be complete by build 12. In the meantime, as an initial step the spacing of the source code has been normalized (6717694).

I'd expect b12 to be available within two weeks; besides the Gervill work, a few more licensing/artifact fixes may be included too. Going forward, I plan to get the OpenJDK 6 sources into a public Mercurial repository by the end of the summer; that will allow fixes put there to be shared more quickly and should also allow more work to go on directly in the upstream code base.

Friday Jun 13, 2008

OpenJDK 6: Sources for b10 published

On May 30, the OpenJDK 6 b10 source bundle was published. Notable fixes in this build include:

With the removal of the binary plug for sound, the only remaining plug in OpenJDK 6 is for SNMP support. Work continues to address the remaining copyright and licensing concerns in the code base. Some regression tests, such as those in networking and javax.script, should probably be modified to more easily accept configuration options without having to modify the source of the tests; discussion on how best to do that has started.

Thursday May 01, 2008

OpenJDK: jtreg and regression tests

Huzzah! Through the dedicated efforts of Jon and others, jtreg is now open sourced! The jtreg program is the test harness used to run the regression tests that come with the JDK sources.

The JCK tests verify properties that should be true of all implementations of a given Java SE specification. The JDK regression tests are different; while many of them test properties that should be true of all implementations, some regression tests look at properties we want to be true of our JDK implementation but are not strictly required by the specification. Therefore, while a failing regression test most likely indicates a problem, in some cases the failure may not be a correctness issue per se. This situation is certainly feasible with ports of the JDK to operating systems sufficiently different than windows, Solaris, and Linux; shell tests are especially susceptible to those OS differences. Creating new shell tests should be avoided if possible and the porting effort may include updating regression tests to make them aware of the new platform.

Monday Apr 14, 2008

OpenJDK 6: Sources for b09 published

On April 11, the OpenJDK 6 b09 source bundle was published.

Notable fixes in this build include:

This will very likely be the last source drop until after JavaOne.

Friday Apr 04, 2008

OpenJDK 6: Sources for b07 and b08 published

The sources for OpenJDK 6 b07 were posted on March 20 followed by b08 on March 27.

The most notable fixes in b07 were:

  • Resolving the last remaining JCK signature test failure (6636951); other JCK failures remain.

  • Making window decorations appear (6586752).

  • Updating to the 1.1 version of the OpenJDK™ trademark notification.

  • Enabling the out-of-the-box build to succeed without any binary plugs being present (6672710). If the plugs aren't used, neither the midi synthesizer nor SNMP will work at runtime.

After b07, Red Hat reported that the source bundle contained some suspect binary artifacts. In b08, we addressed all the make and src artifacts and some of the test ones (6679994). We're reviewing how to make the licensing of the remaining non-source files in the bundle clearer. Additionally, a separate build problem was addressed (6613927)

At present, there is not a particular time line for the next code drop. The fixes in the next drop will include time zone updates (6650748, 6679340).

Thursday Mar 13, 2008

OpenJDK 6 and Distros

There are now OpenJDK 6 packages for the upcoming Hardy Heron Unbuntu Universe and Lilian has announced that OpenJDK 6 will be a part of Fedora 9.

Thursday Mar 06, 2008

Matching JDK and JCK Versions

For those interested in verifying conformance of a JDK, from engineers working on the code base to porters and independent implementors, it is important to pair a JDK with the proper version of the compatibility tests.

Generally in the JCP, a JSR has three deliverables, a specification, a reference implementation, and a technology compatibility kit (TCK). For the Java SE platform umbrella JSR, Sun's JDK is the reference implementation and the Java compatibility Kit (JCK) serves as the TCK.

In this context, the reference implementation refers to a specific binary and, more colloquially, to the code used to produce that binary. The reference implementation of a JSR may or may not change over time. The JDK updates that do come out are improvements to the same specification, with better performance, more bugs fixed, occasional upgrades to select APIs, and additional non-platform APIs, but do not necessarily become the new reference implementation of the Java SE platform specification. Currently, the original JDK 6 remains the reference implementation for Java SE 6, JSR 270, although several JDK 6 updates have been release since then and more are on the way. One pathway for the reference implementation to be upgraded (along with small changes to the specification) is a JCP maintenance update, but to date such a maintenance update has not been done for a Java SE platform JSR. Another way the reference implementation for a specification can be changed is by the release of a new JCK version which cites a different RI. The cited RI must pass the corresponding JCK version. Just as JDK updates are a better implementation of the same specification, JCK updates are a more thorough test of the same specification. For example, when JDK 6 first shipped, JCK 6 was available. Since then, JCK 6a has been developed and one improvement in was the addition of many tests for annotation processing and the language model of JSR 269.

When a new JCK version is shipped, it obsoletes the previous version. JDK implementations which first ship 120 days after a new JCK is released must test against the new version for compliance rather than the old one. For example, JCK 6a was released in June 2007. 120 days after it shipped, it became the JCK suite that must be passed for Java SE 6 compatibility so JCK 6a is now the version of the JCK in force. JCK 6b is in the works and will likewise supplant 6a in due course. Besides adding new tests, JCK updates can also delete invalid tests or place them on the exclude list.

For JCP technologies that are standalone as well as bundled with the platform, the version of the standalone technology in use can be upgraded and with such an upgrade the corresponding TCK may need to be upgraded too. For example, JDK 6 originally shipped with JAX-WS 2.0, but both JDK 6u4 and OpenJDK 6 have been updated to JAX-WS 2.1. There is an alternate bundle with tests specific to JAX-WS 2.1, including signature tests covering new API elements.

So as of March 2008, the proper JCK version to test OpenJDK 6 is JCK 6a with the alternate bundle for JAX-WS 2.1. After JCK 6b ships, we'll start running tests using that version, which assumes JAX-WS 2.1 by default. (The JAX-WS version is simply a configuration option in JCK 6b). If you're interested in running the JCK in context of OpenJDK projects, a license is available.

Friday Feb 29, 2008

OpenJDK 6: Sources for b06 Published

The second code drop for OpenJDK 6, b06, was published earlier today, February 29, 2008, at the usual location.

This drop has several changes of note:

I expect with next drop will be available within two weeks, by March 14, 2008.

Thursday Feb 14, 2008

OpenJDK 6: The End of the Beginning

The initial code drop for OpenJDK 6 is now available. More to follow.




« June 2016

No bookmarks in folder