OpenJDK 6: b21 regression test results

Running with the usual jtreg flags, "-a -ignore:quiet" in all repositories, using generally accessible hosts for the network configuration file, and adding "-s -ea -esa" for langtools, the basic regression test results on Linux for OpenJDK 6 build 21 are:

  • HotSpot, 97 tests passed and 1 failed.

  • Langtools, 1,391 tests passed.

  • JDK, 3,300 tests pass, 29 fail, and 3 have errors.

More HotSpot tests were added and all but one pass; the new failing test is improperly platform-specific:

0: b20-hotspot/summary.txt  pass: 85
1: b21-hotspot/summary.txt  pass: 97; fail: 1

0      1      Test
---    pass   compiler/6431242/Test.java
---    pass   compiler/6894807/IsInstanceTest.java
---    pass   compiler/6932496/Test6932496.java
---    pass   compiler/6946040/TestCharShortByteSwap.java
---    pass   compiler/6958485/Test.java
---    pass   compiler/6973329/Test.java
---    pass   compiler/6982370/Test6982370.java
---    pass   compiler/7002666/Test7002666.java
---    pass   gc/6581734/Test6581734.java
---    pass   runtime/6626217/Test6626217.sh
---    pass   runtime/6888954/vmerrors.sh
---    pass   runtime/6925573/SortMethodsTest.java
---    fail   runtime/6929067/Test6929067.sh

13 differences

In langtools some tests were added and all the tests continue to pass:

0: b20-langtools/summary.txt  pass: 1,365
1: b21-langtools/summary.txt  pass: 1,391

0      1      Test
---    pass   tools/javac/6508981/TestInferBinaryName.java
---    pass   tools/javac/6734819/T6734819a.java
---    pass   tools/javac/6734819/T6734819b.java
---    pass   tools/javac/6734819/T6734819c.java
---    pass   tools/javac/6889255/T6889255.java
---    pass   tools/javac/T6595666.java
---    pass   tools/javac/T6625520.java
---    pass   tools/javac/T6705935.java
---    pass   tools/javac/T6956638.java
---    pass   tools/javac/api/6411310/Test.java
---    pass   tools/javac/api/6440333/T6440333.java
---    pass   tools/javac/api/6733837/T6733837.java
---    pass   tools/javac/api/Sibling.java
---    pass   tools/javac/api/T6483788.java
---    pass   tools/javac/api/T6501502.java
---    pass   tools/javac/api/T6838467.java
---    pass   tools/javac/api/T6877206.java
pass   ---    tools/javac/policy/Test.java
pass   ---    tools/javac/policy/Test.java#id1
pass   ---    tools/javac/policy/Test.java#id2
pass   ---    tools/javac/policy/Test.java#id3
pass   ---    tools/javac/policy/Test.java#id4
pass   ---    tools/javac/policy/Test.java#id5
pass   ---    tools/javac/policy/Test.java#id6
pass   ---    tools/javac/policy/Test.java#id7
---    pass   tools/javac/policy/test1/Test1a.java
---    pass   tools/javac/policy/test1/Test1a.java#id1
---    pass   tools/javac/policy/test1/Test1a.java#id2
---    pass   tools/javac/policy/test1/Test1a.java#id3
---    pass   tools/javac/policy/test1/Test1a.java#id4
---    pass   tools/javac/policy/test1/Test1a.java#id5
---    pass   tools/javac/policy/test1/Test1a.java#id6
---    pass   tools/javac/policy/test1/Test1a.java#id7
---    pass   tools/javac/policy/test1/Test1b.java
---    pass   tools/javac/policy/test1/Test1b.java#id1
---    pass   tools/javac/policy/test1/Test1b.java#id2
---    pass   tools/javac/policy/test1/Test1b.java#id3
---    pass   tools/javac/policy/test2/Test.java
---    pass   tools/javac/policy/test2/Test.java#id1
---    pass   tools/javac/policy/test2/Test.java#id2
---    pass   tools/javac/policy/test2/Test.java#id3
---    pass   tools/javac/policy/test3/Test.java

42 differences

And in jdk, a small percentage of tests were added and the existing tests have generally consistent results:

0: b20-jdk/summary.txt  pass: 3,273; fail: 33; error: 2
1: b21-jdk/summary.txt  pass: 3,300; fail: 29; error: 3

0      1      Test
---    pass   com/sun/java/swing/plaf/gtk/Test6963870.java
fail   pass   java/awt/Focus/NonFocusableWindowTest/NonfocusableOwnerTest.java
---    pass   java/awt/Frame/FrameSize/TestFrameSize.java
fail   pass   java/awt/Frame/MaximizedToIconified/MaximizedToIconified.java
fail   pass   java/awt/Multiscreen/LocationRelativeToTest/LocationRelativeToTest.java
fail   pass   java/awt/TextArea/UsingWithMouse/SelectionAutoscrollTest.html
---    pass   java/awt/font/TextLayout/TestSinhalaChar.java
pass   error  java/lang/management/MemoryMXBean/CollectionUsageThresholdConcMarkSweepGC.sh
---    pass   java/math/BigDecimal/MultiplyTests.java
pass   fail   java/net/InetAddress/IPv4Formats.java
pass   fail   java/net/URL/OpenStream.java
pass   fail   java/net/URLClassLoader/ClassLoad.java
pass   fail   java/rmi/transport/rapidExportUnexport/RapidExportUnexport.java
---    pass   java/util/logging/AnonLoggerWeakRefLeak.sh
---    pass   java/util/logging/LoggerWeakRefLeak.sh
---    pass   javax/imageio/plugins/png/ITXtTest.java
---    pass   javax/imageio/plugins/png/ItxtUtf8Test.java
---    pass   javax/swing/JPopupMenu/6675802/bug6675802.java
---    pass   javax/swing/JPopupMenu/6691503/bug6691503.java
---    pass   javax/swing/Security/6938813/bug6938813.java
---    pass   javax/swing/UIDefaults/6622002/bug6622002.java
---    pass   javax/swing/UIDefaults/6795356/SwingLazyValueTest.java
---    pass   javax/swing/UIDefaults/6795356/TableTest.java
---    pass   javax/swing/UIDefaults/6795356/bug6795356.java
fail   pass   javax/swing/plaf/synth/Test6933784.java
fail   pass   sun/nio/cs/Test4200310.sh
---    pass   sun/security/pkcs11/SecureRandom/TestDeserialization.java
---    pass   sun/security/pkcs11/Signature/TestRSAKeyLength.java
---    pass   sun/security/ssl/com/sun/net/ssl/internal/ssl/SSLSocketImpl/InvalidateServerSessionRenegotiate.java
---    fail   sun/security/ssl/com/sun/net/ssl/internal/www/protocol/https/HttpsURLConnection/CriticalSubjectAltName.java
---    pass   sun/security/ssl/javax/net/ssl/NewAPIs/JSSERenegotiate.java
---    pass   sun/security/ssl/javax/net/ssl/NewAPIs/SSLEngine/CheckStatus.java
---    pass   sun/security/ssl/javax/net/ssl/NewAPIs/SSLEngine/ConnectionTest.java
---    pass   sun/security/ssl/javax/net/ssl/NewAPIs/SSLEngine/NoAuthClientAuth.java
fail   pass   sun/security/ssl/sun/net/www/protocol/https/HttpsURLConnection/DNSIdentities.java
fail   pass   sun/security/ssl/sun/net/www/protocol/https/HttpsURLConnection/IPAddressIPIdentities.java
fail   pass   sun/security/ssl/sun/net/www/protocol/https/HttpsURLConnection/IPIdentities.java
fail   pass   sun/security/ssl/sun/net/www/protocol/https/HttpsURLConnection/Identities.java
pass   fail   sun/security/validator/CertReplace.java
---    pass   sun/tools/common/CommonTests.sh

40 differences

In addition to the basic results, the hotspot and jdk regression test suites were rerun with assertions and system assertions enabled, -ea -esa. (The langtools tests are already run with assertions enabled.) The results for hotspot were identical to running without assertions; in jdk a few additional tests failed:


0: b21-jdk/summary.txt   pass: 3,300; fail: 29; error: 3
1: b21-jdka/summary.txt  pass: 3,295; fail: 33; error: 4

0      1      Test
pass   error  com/sun/jdi/DoubleAgentTest.java
pass   fail   java/awt/Focus/ActualFocusedWindowTest/ActualFocusedWindowBlockingTest.java
pass   fail   java/awt/Focus/ActualFocusedWindowTest/ActualFocusedWindowRetaining.java
pass   fail   java/awt/Focus/TranserFocusToWindow/TranserFocusToWindow.java
fail   pass   java/awt/xembed/server/RunTestXEmbed.java
pass   fail   java/net/Authenticator/B4769350.java
pass   fail   java/net/CookieHandler/TestHttpCookie.java
fail   pass   java/rmi/transport/rapidExportUnexport/RapidExportUnexport.java
pass   fail   java/util/ResourceBundle/Test4300693.java

9 differences

Going forward, the baseline regression test results for all OpenJDK 6 repositories will be reported with assertions and system assertions enabled.

Comments:

Thanx for these kind of overviews!

I was wondering though, if you could comment on old tests that now started to fail, like:

pass fail java/net/URL/OpenStream.java

That sounds like core functionality to me... why is this test failing now?

Posted by Egon Willighagen on January 21, 2011 at 10:50 PM PST #

@Egon,

The test in question is relying on "http://foo.bar.baz" being a bad URL. The test may be failing on my machine due to my ISP's "helpful" behavior of providing a page for a missing URL. I see the same test fail on my laptop when running JDK 7, but the test passes with both OpenJDK 6 and JDK 7 when I disable networking on the box.

In general, unfortunately there can be transient failures on some of the jdk repository's regression tests. There has been some work making those tests more robust for JDK 7, as well as correcting some buggy tests, but not all those changes have been ported to OpenJDK 6.

Posted by Joe Darcy on January 24, 2011 at 03:21 AM PST #

Post a Comment:
Comments are closed for this entry.
About

darcy

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
News

No bookmarks in folder

Blogroll