Tuesday Sep 20, 2016

JavaOne 2016: Slides and video for "JDK 9 Language, Tooling, and Library Features"

The slides and video for my JavaOne 2016 talk about JDK 9 Language, Tooling, and Library Features have now been posted. There are many features in the release in addition to modules!

Tuesday Sep 13, 2016

Project Coin: fully minted

As an addendum to the parting message on coin-dev, one of the most popular candidate coin features not included in JDK 7 was some sort of literal syntax for collections, sets, lists, and/or maps. [1] [2]

Fortunately, this functionality is approximated in JDK 9 via Stuart Marks' JEP 269: Convenience Factory Methods for Collections which added factory methods named "of" that return immutable collections to the List, Set, and Map interfaces. I've enjoyed using this feature in my JDK 9 programming and suggest you give it a try too.

With that, I declare Project Coin well and fully minted! Happy coding.

PS I has heartened to see the improvements to numeric literals included by Project Coin in JDK 7 seem to have partially inspired the features set of an upcoming version of another widely-used language.

Sunday Jan 25, 2015

Advice on removing javac lint warnings

Over the last twelve months or so, one of my projects has been fixing and reviewing fixes of javac lint warnings in the JDK 9 code base (varargs, fallthrough, serial, finally, overrides, deprecation, raw and unchecked) and once a warning category is cleared, making new instances of that category a fatal build error. Ultimately, all the warnings in the jdk repository were resolved and -Xlint:all -Werror is now used in the build.

Being involved in fixing several thousand warnings, I'd like to share some tips for developers who want to undertake an analogous task of cleaning up the technical debt of javac lint warnings in their own code base. First, I recommend tackling the warnings in a way that aligns well with the build system of the project, with a consideration of getting some code protected by the compiler from some warning categories as soon as possible. While the build of the JDK has been re-engineered over the course of the warnings cleanup, to a first approximation the build has been organized around Hg repositories. (At present, in JDK 9 the build is actually arranged around modules. A few years ago, the build was organized around Java packages rather than repositories.) A warnings cleanup isn't really done until introducing new instances of the warning cause a build failure; new warnings are too easy to ignore otherwise. Therefore, for JDK 9, the effort was organized around clearing the whole jdk repository of a warning category and then enabling that warning category in the build as opposed to, say, completely clearing a particular package of all warnings and then moving to the next package.

There are two basic approaches to resolving a warning: suppressing it using the @SuppressWarnings mechanism or actually fixing the code triggering the warning. The first approach is certainly more expedient. While it doesn't directly improve the code base, it can offer an indirect benefit of creating a situation where new warnings can be kept out of the code base by allowing a warning to be turned on in the build sooner. The different warning categories span a range of severity levels and while some warnings are fairly innocuous, others are suspicious enough that I'd recommend always fixing them if a fix is feasible. When resolving warnings in the JDK, generally the non-deprecation warnings categories were fixed while the deprecation warnings were suppressed with a follow-up bug filed. The non-deprecation warnings mostly require Java language expertise to resolve and little area expertise; deprecation warnings are the reverse, often quite deep area expertise is needed to develop and evaluate a true fix.

Tips on addressing specific categories of lint warnings:

[cast]: Warn about use of unnecessary casts.
Since these warnings are generated entirely from the the contents of method bodies, there is no impact to potential callers of the code. Also, the casts analyzed as redundant by javac are easy and safe to remove; fixing cast warnings is essentially a zero-risk change.
[fallthrough]: Warn about falling through from one case of a switch statement to the next.
When such a falling through is not intentional, it can be a very serious bug. All fallthrough switch cases should be examined for correctness. An idiomatic and intentional fallthrough should have two parts: first, the cases in question should be documented in comments explaining that the fallthrough is expected and second, an @SuppressWarnings({"fallthrough"}) annotation should be added to the method containing the switch statement.

See also the discussion of switch statements in Java Puzzlers, Puzzler 23: No Pain, No Gain.

[static]: Warn about accessing a static member using an instance.
This is an unnecessary and misleading coding idiom that should be unconditionally removed. The fix is to simply refer to the static member using the name of the type rather than an instance of the type.

This coding anti-pattern is discussed in Java Puzzlers, Puzzle 48: All I Get Is Static.

[dep-ann]: Warn about items marked as deprecated in JavaDoc but not using the @Deprecated annotation
Since Java SE 5.0, the way to mark an element as deprecated is to modify it with a @Deprecated annotation. While a @deprecated javadoc tag should be used to describe all @Deprecated elements, the javadoc tag is informative only and does not mean the element is treated as deprecated by the compiler.

A element should have an @deprecated javadoc tag in its javadoc if and only if the element is @Deprecated.

Therefore, the fix should be to either remove the @deprecated javadoc tag if the element should not be deprecated or add the @Deprecated annotation if it should be deprecated.

[serial]: Warn about Serializable classes that do not provide a serialVersionUID.
Serialization is a subtle and complex protocol whose compatibility impact on evolving a type should not be underestimated. To check for compatibility between the reader of serial stream data and the writer of the data, besides matching the names of the reader and writer, identification codes of the reader and the writer are also compared and the serial operation fails if the codes don't match. When present, a serialVersionUID field of a class stores the identification code, called a Stream Unique Identifier (SUID) in serialization parlance. When a serialVersionUID field is not present, a particular hashing algorithm is used to compute the SUID instead. The hash algorithm is perturbed by many innocuous changes to a class and can therefore improperly indicate a serial incompatibility when no such incompatibility really exists. To avoid this hazard, a serialVersionUID field should be present on all Serializable classes following the usual cross-version serialization contracts, including Serializable abstract superclasses.

If a Serializable class without a serialVersionUID has already been shipped in a release, running the serialver tool on the type in the shipped release will return the serialVersionUID declaration needed to maintain serial compatibility.

For further discussion, see Effective Java, 2nd Edition, Item 74: Implement Serializable judiciously.

[overrides]: Warn about issues regarding method overrides.
As explained in Effective Java, 2nd Edition, Item 9: Always Override hashCode when you override equals, for objects to behave properly when used in collections, they must have correct equals and hashCode implementations. The invariant checked by javac is more nuanced than the one discussed in Effective Java; javac checks that if a class overrides equals, hashCode has been overriden somewhere in the superclass class chain of the class. It is common for a set of related classes to be able to share a hashCode implementation, say a function of a private field in the root superclass in a set of related types. However, each class will still need to have its own equals method for the usual instanceof check on the argument to equals.
[deprecation]: Warn about use of deprecated items.
Well documented @Deprecated elements suggest a non-deprecated replacement. When using a replacement is not feasible, or no such replacement exists, @SuppressWarnings("deprecation") can be used to acknowledge the situation and remove the warning. A small language change made in JDK 9 makes suppressing deprecation warnings tractable.
[rawtypes]: Warn about use of raw types.
[unchecked]: Warn about unchecked operations.
Both rawtypes and unchecked warnings are linked to the same underlying cause: incomplete generification of APIs and their implementations. Generics shipped in 2004 as part of Java SE 5.0; Java code written and used today should be generics aware! Being generics-aware has two parts, using generics properly in the signature / declaration of a method, constructor, or class and using generics properly in method and constructor bodies. Many uses of generics are straightforward; if you have a list that only contains strings, it should probably be declared as a List<String>. However, some uses of generics can be subtle and are out of scope for this blog entry. However, extensive guides are available with detailed advice. IDEs also provide refactorings for generics; check their documentation for details.

I hope these tips help you make your own Java project warnings-free.

Wednesday Jan 14, 2015

More concise try-with-resources statements in JDK 9

As part of milling Project Coin in JDK 9, the try-with-resources statement has been improved. If you already have a resource as a final or effectively final variable, you can use that variable in the try-with-resources statement without declaring a new variable in the try-with-resources statement.

For example, given resource declarations like

        // A final resource
        final Resource resource1 = new Resource("resource1");
        // An effectively final resource
        Resource resource2 = new Resource("resource2");

the old way to write the code to manager these resources would be something like:

        // Original try-with-resources statement from JDK 7 or 8
        try (Resource r1 = resource1;
             Resource r2 = resource2) {
            // Use of resource1 and resource 2 through r1 and r2.

while the new way can be just

        // New and improved try-with-resources statement in JDK 9
        try (resource1;
             resource2) {
            // Use of resource1 and resource 2.

An initial pass has been made over the java.base module in JDK 9 to update the JDK libraries to use this new language feature.

You can try out these changes in your own code using a JDK 9 snapshot build. Enjoy!

Sunday Dec 08, 2013

JSR 269 Maintenance Review for Java SE 8

The annotation processing API, both the processor-specific portion of the API in javax.annotation.processing and the language modeling portions in javax.lang.model.*, are being updated to support the new language features in Java SE 8. Procedurally, the proposed changes are covered by the second maintenance review of JSR 269: Maintenance Draft Review 2.

As summarized on on the maintenance review page, there are three categories of changes from the version of the API shipped with Java SE 7:

  1. Cleaning up the existing specification without changing its semantics (adding missing javadoc tags, etc.)
  2. API changes to support the language changes being made in Project Lambda /JSR 335. These includes adding javax.lang.model.type.IntersectionType as well as javax.lang.model.element.ExecutableElement.isDefault.
  3. API changes to support the language changes being made under JSR 308, Annotations on Java Types. These include javax.lang.model.AnnotatedConstruct and updating javax.annotation.processing.Processor.

The small repeating annotations language change, discussed on an OpenJDK alias, is also supported by the proposed changes.

A detailed specification difference is available. Please post any comments here or send them to me through email.

Monday May 20, 2013

javax.lang.model backed by core reflection

Back when the javax.lang.model API was being designed as part of JSR 269, while the API was primarily intended for use at compile-time with annotation processing, the expert group also wanted the API to be usable in other contexts as well, including at runtime.

JEP 119, javax.lang.model Implementation Backed by Core Reflection, proposed adding such an alternative runtime implementation of javax.lang.model to JDK 8. Such an implementation has recently been pushed as sample code in the JDK 8 langtools repository.

At a high level, the sample code is at its core repeated uses of the adapter pattern, translating between the core reflection API and the javax.lang.model API. However, there were a number of design issues in what interface the javax.lang.model implementation for core reflection should expose. First, should a core reflection specialization of javax.lang.model expose a more specific interface? There are some advantages to just implementing the base API defined in the existing interfaces; it is the simplest approach and would appear to maximize the ability to inter-operate with other implementations. However, javax.lang.model depends on hidden state to define concepts like equality, so there is intrinsically limited inter-operation between disjoint implementations. Therefore, especially in sample code, it was viewed as worthwhile to experiment with a more specific API in the core reflection case.

To make the javax.lang.model backed by core reflection more specific, for each FooElement interface in javax.lang.model.element, a ReflectionFooElement subinterface was defined, as shown in the diagram below.

The subinterfaces are specialized in a few ways:

  • If a method in the base interface was defined to return a FooElement, a covariant override in the subinteface was defined to return a ReflectionFooElement.
  • If a method in the base interface was defined to return a List< ? extends FooElement>, a covariant override in the subinteface was defined to return a List<ReflectionFooElement>.
  • For getBar(FooElement arg) methods defined in the Elements helper interface, add a getBar() method to the ReflectionFooElement subinterface.
  • Add a getSource method to the root ReflectionElement interface to return the underlying core reflection object being adapted and add covariant overrides in subinterfaces as appropriate.
  • The root ReflectionElement interface implements the full AnnotatedElement interface, not just the subset of its methods found in the base javax.lang.model.element.

A cost of specializing the API is the need to define a new visitor interface as well. Covariant overrides cannot be used as in the element modeling interfaces since the visitor methods take elements in the argument position rather than the return position.

Generally, working on the core reflection specialization proceeded as expected. Equality determinations were generally delegated to core reflection source object; in other words, two ReflectionElement objects are equal if they are instances of the same interface and if their sources are .equals. Writing the sample code highlighted several shortcomings of the core reflection API which have been addressed in Java SE 8, including the addition of an Executable type to abstract over the commonalities of Method and Constructor. The sample code also benefited from other reflection changes in Java SE 8, such as the introduction of a java.lang.reflect.Parameter class and retrofitting TypeVariable to extend AnnotatedElement.

While significantly more testing would be needed to productize javax.lang.model backed by core reflection, even as sample code it validates a number of the technological decisions made in JSR 269 and is an interesting demonstration of how one of the platform's reflective APIs can be bridged to another.

Sunday Apr 14, 2013

Changing Sources and Moving Targets: Evolving the javac command line

As written up in JEP 182: Policy for Retiring javac -source and -target Options, we're implementing a policy to over time clean up the -source and -target portions of javac's command line:

  • Class files going all the way back to version 45.3 generated by JDK 1.0.2 will continue to be recognized by javac when put on the classpath, etc.
  • The never-documented -target options 1.4.1, 1.4.2, and jsr14 have been removed in JDK 8 (8010179).
  • In JDK 8, source and target options for 1.5 and below will be deprecated and a warning will be printed on their use (8011043).
  • In JDK 9, source and target options for 1.5 and below will be removed (8011044).
  • Starting in JDK 9, a "one plus three back" policy will apply to source and target options. JDK 9 will support (9/1.9, 8/1.8, 7/1.7, 6/1.6) and JDK 10 will support (10/1.10, 9/1.9, 8/1.8, 7/1.7) and so on.

Removing support for the old options will ease compiler maintenance in several ways. First, there will be direct benefits from allowing some code to be deleted. Nixing jsr14 allowed about 250 lines of code to be excised. Second, fewer interactions between new language features and old source levels need to be handled in the compiler. The Java Language Specification only deals with a single version of the language and there is no formal specification of many aspects of how a Java compiler is supposed to behave. To use a recent example, there is no specification for how a new-in-8 default method being introduced to the language and core libraries by Project Lambda should appear to code being compiled under, say, -source 6. Limiting the span of source and target version supported reduces the need to define such cross-version interactions. (The practical impact of source cross-version interfaces would be greatly reduced if developers more often follwed the recommended practice of setting the bootclasspath when cross-compiling to an older platform version.)

This policy will help balance stability versus progress and should cover releases having public updates when a new JDK is released.

Thursday Feb 21, 2013

Functional Interfaces

As part of Project Lambda, after discussion with the JSR 335 expert group, we decided to add a FunctionalInterface annotation type to the platform. To a first approximation, functional interfaces are those interface types which define only a single method and are therefore usable in lambda expressions. (There are some tricky details in the definition of a functional interface relating to generics and also some details about excluding from consideration methods defined on java.lang.Object.) The new annotation type allows a library designer to clearly indicate the property of intending an interface type to be used with lambda expressions along with an implied commitment to keep that property true in the future. However, the compiler will allow any interface type meeting the structural properties of a functional interface to be used for a lambda expression regardless of whether or not the interface has a @FunctionalInterface annotation.

They types being added in the java.util.function package are by design functional interfaces and can be annotated with @FunctionalInterface from early days. However, many existing Java SE types are also functional interfaces and we want to identify and annotate those appropriately too. To find those candidate interfaces to be annotated, I ran an annotation processor over the code, using the same methodology as used to find Closeable candidates during JDK 7.

A significant number of candidates were found throughout the JDK. After suitable discussion and review, the first batch of core libraries changes have been pushed. Analogous discussions have been started in the 2D, awt, and swing areas.

For guidance in retrofitting @FunctionalInterface to an existing candidate type, if the type is routinely instantiated using an anonymous class, it is a good candidate for being annotated with @FunctionalInterface.

Monday Oct 29, 2012

JDK bug migration: components and subcomponents

One subtask of the JDK migration from the legacy bug tracking system to JIRA was reclassifying bugs from a three-level taxonomy in the legacy system, (product, category, subcategory), to a fundamentally two-level scheme in our customized JIRA instance, (component, subcomponent). In the JDK JIRA system, there is technically a third project-level classification, but by design a large majority of JDK-related bugs were migrated into a single "JDK" project. In the end, over 450 legacy subcategories were simplified into about 120 subcomponents in JIRA. The 120 subcomponents are distributed among 17 components. A rule of thumb used was that a subcategory had to have at least 50 bugs in it for it to be retained.

Below is a listing the component / subcomponent classification of the JDK JIRA project along with some notes and guidance on which OpenJDK email addresses cover different areas. Eventually, a separate incidents project to host new issues filed at bugs.sun.com will use a slightly simplified version of this scheme.

The preponderance of bugs and subcomponents for the JDK are in library-related areas, with components named foo-libs and subcomponents primarily named after packages. While there was an overall condensation of subcomponents in the migration, in some cases long-standing informal divisions in core libraries based on naming conventions in the description were promoted to formal subcomponents. For example, hundreds of bugs in the java.util subcomponent whose descriptions started with "(coll)" were moved into java.util:collections. Likewise, java.lang bugs starting with "(reflect)" and "(proxy)" were moved into java.lang:reflect.

  • client-libs (Predominantly discussed on 2d-dev and awt-dev and swing-dev.)
    • 2d
    • demo
    • java.awt
    • java.awt:i18n
    • java.beans (See beans-dev.)
    • javax.accessibility
    • javax.imageio
    • javax.sound (See sound-dev.)
    • javax.swing
  • core-libs (See core-libs-dev.)
    • java.io
    • java.io:serialization
    • java.lang
    • java.lang.invoke
    • java.lang:class_loading
    • java.lang:reflect
    • java.math
    • java.net
    • java.nio (Discussed on nio-dev.)
    • java.nio.charsets
    • java.rmi
    • java.sql
    • java.sql:bridge
    • java.text
    • java.util
    • java.util.concurrent
    • java.util.jar
    • java.util.logging
    • java.util.regex
    • java.util:collections
    • java.util:i18n
    • javax.annotation.processing
    • javax.lang.model
    • javax.naming (JNDI)
    • javax.script
    • javax.script:javascript
    • javax.sql
    • org.openjdk.jigsaw (See jigsaw-dev.)
  • security-libs (See security-dev.)
    • java.security
    • javax.crypto (JCE: includes SunJCE/MSCAPI/UCRYPTO/ECC)
    • javax.crypto:pkcs11 (JCE: PKCS11 only)
    • javax.net.ssl (JSSE, includes javax.security.cert)
    • javax.security
    • javax.smartcardio
    • javax.xml.crypto
    • org.ietf.jgss
    • org.ietf.jgss:krb5
  • other-libs
    • corba
    • corba:idl
    • corba:orb
    • corba:rmi-iiop
    • javadb
    • other (When no other subcomponent is more appropriate; use judiciously.)

Most of the subcomponents in the xml component are related to jaxp.

  • xml
    • jax-ws
    • jaxb
    • javax.xml.parsers (JAXP)
    • javax.xml.stream (JAXP)
    • javax.xml.transform (JAXP)
    • javax.xml.validation (JAXP)
    • javax.xml.xpath (JAXP)
    • jaxp (JAXP)
    • org.w3c.dom (JAXP)
    • org.xml.sax (JAXP)

For OpenJDK, most JVM-related bugs are connected to the HotSpot Java virtual machine.

The full JDK bug database contains entries related to legacy virtual machines that predate HotSpot as well as retired APIs.

  • vm-legacy
    • jit (Sun Exact VM)
    • jit_symantec (Symantec VM, before Exact VM)
    • jvmdi (JVM Debug Interface )
    • jvmpi (JVM Profiler Interface )
    • runtime (Exact VM Runtime)

Notable command line tools in the $JDK/bin directory have corresponding subcomponents.

Some aspects of JDK infrastructure directly affect JDK Hg repositories, but other do not.

  • infrastructure
    • build (See build-dev and build-infra-dev.)
    • licensing (Covers updates to the third party readme, licenses, and similar files.)
    • release_eng (Release engineering)
    • staging (Staging of web pages related to JDK releases.)

The specification subcomponent encompasses the formal language and virtual machine specifications.

  • specification
    • language (The Java Language Specification)
    • vm (The Java Virtual Machine Specification)

The code for the deploy and install areas is not currently included in OpenJDK.

  • deploy
    • deployment_toolkit
    • plugin
    • webstart
  • install
    • auto_update
    • install
    • servicetags

In the JDK, there are a number of cross-cutting concerns whose organization is essentially orthogonal to other areas. Since these areas generally have dedicated teams working on them, it is easier to find bugs of interest if these bugs are grouped first by their cross-cutting component rather than by the affected technology.

  • docs
    • doclet
    • guides
    • hotspot
    • release_notes
    • tools
    • tutorial
  • embedded
    • build
    • hotspot
    • libraries
  • globalization
    • locale-data
    • translation
  • performance
    • hotspot
    • libraries

The list of subcomponents will no doubt grow over time, but my inclination is to resist that growth since the addition of each subcomponent makes the system as a whole more complicated and harder to use.

When the system gets closer to being externalized, I plan to post more blog entries describing recommended use of various custom fields in the JDK project.

Thursday Oct 25, 2012

JDK bug migration: bugs.sun.com now backed by JIRA

The JDK bug migration from a Sun legacy system to JIRA has reached another planned milestone: the data displayed on bugs.sun.com is now backed by JIRA rather than by the legacy system. Besides maintaining the URLs to old bugs, bugs filed since the migration to JIRA are now visible too.

The basic information presented about a bug is the same as before, but reformatted and using JIRA terminology:

  • Instead of a "category", a bug now has a "component / subcomponent" classification. As outlined previously, part of the migration effort was reclassifying bugs according to a new classification scheme; I'll write more about the new scheme in a subsequent blog post.

  • Instead of a list of JDK versions a bug is "reported against," there is a list of "affected versions." The names of the JDK versions have largely been regularized; code names like "tiger" and "mantis" have been replaced by the release numbers like "5.0" and "1.4.2".

  • Instead of "release fixed," there are now "Fixed Versions."

  • The legacy system had many fields that could hold a sequence of text entries, including "Description," "Workaround", and "Evaluation." JIRA instead only has two analogous fields labeled as "Description" and a unified stream of "Comments."

Nearly coincident with switching to JIRA, we also enabled an agent which automatically updates a JIRA issue in response to pushes into JDK-related Hg repositories. These comments include the changeset URL, the user making the push, and a time stamp. These comments are first added when a fix is pushed to a team integration repository and then added again when the fix is pushed into the master repository for a release.

We're still in early days of production usage of JIRA for JDK bug tracking, but the transition to production went smoothly and over 1,000 new issues have already been filed. Many other facets of the migration are still in the works, including hosting new incidents filed at bugs.sun.com in a tailored incidents project in JIRA.

Tuesday Oct 02, 2012

JavaOne 2012: Slides for "JDK 7 Action" Posted

Stuart, Mike, and I gave a two-hour tutorial about JDK 7 in Action earlier today at JavaOne. For those moving to JDK 7 soon, this talk covers the language and library changes made in that release, including Project Coin and NIO.2.

Monday Oct 01, 2012

JavaOne 2012: slides for "Lessons from Mathematics" Posted

I've posted the slides for my JavaOne bof on Lessons from Mathematics. I've wanted to put together a talk along these lines for a while and I'm happy to have given the talk at JavaOne this year. I was gratified some stalwart JavaOne attendees choose to attend this late-night talk on such an esoteric topic!

Saturday Sep 29, 2012

Annotation Processing Virtual Mini-Track at JavaOne 2012

Putting together the list of JavaOne talks I'm interested in attending, I noticed there is a virtual mini-track on annotation processing and related technology this year, with a combination of bofs, sessions, and a hands-on-lab:

As the lead engineer on bot apt (rest in peace) in JDK 5 and JSR 269 in JDK 6, I'd be heartened to see greater adoption and use of annotation processing by Java developers.

Wednesday Sep 26, 2012

JDK bug migration milestone: JIRA now the system of record

I'm pleased to announce the OpenJDK bug database migration project has reached a significant milestone: the JDK has switched from the legacy Sun "bugtraq" system to a new internal JIRA instance as the system of record for our bug tracking. This completes the initial phase of the previously described plan of getting OpenJDK onto an externally visible and writable bug tracker. The identities contained in the current system include recognized OpenJDK contributors.

The bug migration effort to date has been sizable in multiple dimensions. There are around 140,000 distinct issues imported into the JDK project of the JIRA instance, nearly 165,000 if backport issues to track multiple-release information are included. Separately, the Code Tools OpenJDK project has its own JIRA project populated with several thousands existing bugs. Once the OpenJDK JIRA instance is externalized, approved OpenJDK projects will be able to request the creation of a JIRA project for issue tracking.

There are many differences in the schema used to model bugs between the legacy bug system and the schema for the new JIRA projects. We've favored simplifications to the existing system where possible and, after much discussion, we've settled on five main states for the OpenJDK JIRA projects:

  • New
  • Open
  • In progress
  • Resolved
  • Closed

The Open and In-progress states can have a substate Understanding field set to track whether the issues has its "Cause Known" or "Fix understood". In the closed state, a Verification field can indicate whether a fix has been verified, unverified, or if the fix has failed.

At the moment, there will be very little externally visible difference between JIRA for OpenJDK and the legacy system it replaces. One difference is that bug numbers for newly filed issues in the JIRA JDK project will be 8000000 and above. If you are working with JDK Hg repositories, update any local copies of jcheck to the latest version which recognizes this expanded bug range. (The bug numbers of existing issues have been preserved on the import into JIRA). Relatively soon, we plan for the pages published on bugs.sun.com to be generated from information in JIRA rather than in the legacy system. When this occurs, there will be some differences in the page display and the terminology used will be revised to reflect JIRA usage, such as referring to the "component/subcomponent" of an issue rather than its "category". The exact timing of this transition will be announced when it is known.

We don't currently have a firm timeline for externalization of the JIRA system. Updates will be provided as they become available. However, that is unlikely to happen before JavaOne next week!

Friday Jun 15, 2012

My integer overfloweth

While certain classes like java.lang.Integer and java.lang.Math have been in the platform since the beginning, that doesn't mean there aren't more enhancements to be made in such places! For example, earlier in JDK 8, library support was added for unsigned integer arithmetic. More recently, my colleague Roger Riggs pushed a changeset to support integer overflow, that is, to provide methods which throw an ArithmeticException on overflow instead of returning a wrapped result. Besides being helpful for various programming tasks in Java, methods like the those for integer overflow can be used to implement runtimes supporting other languages, as has been requested at a past JVM language summit.

This year's language summit is coming up in July and I hope to get some additional suggestions there for helpful library additions as part of the general discussions of the JVM and Java libraries as a platform.

Thursday Jun 14, 2012

JavaOne 2012: Lessons from Mathematics

I was pleased to get notification recently that my bof proposal for Lessons from Mathematics was accepted for JavaOne 2012. This is a bit of a departure from the project-centric JavaOne talks I usually give, but whisps of this kind of material have appeared before.

I'm looking forward to presenting material from linear algebra, stochastics, and numerical optimization that have influence my thinking about technical problems in the JDK and elsewhere.

Tuesday Jun 05, 2012

Moving monarchs and dragons: migrating the JDK bugs to JIRA

Among insects, monarch butterflies and dragonflies have the longest migrations; migrating JDK bugs involves a long journey as well! As previously announced by Mark back in March, we've been working according to a revised plan to transition the JDK bug management from Sun's legacy system to initially an Oracle-internal JIRA instance which is afterward made visible and usable externally. I've been busily working on this project for the last few months and the team has made good progress on many aspects of the effort:

  • JDK bugs will be imported into JIRA regardless of age; bugs will also be imported regardless of state, including closed bugs. Consequently, the JDK bug project will start pre-populated with over 100,000 existing bugs, some dating all the way back to 1994. This will allow a continuity of information and allow new issues to be linked to old ones.

  • Using a custom import process, the Sun bug numbers will be preserved in JIRA. For example, the Sun bug with bug number 4040458 will become "JDK-4040458" in JIRA. In JIRA the project name, "JDK" in our case, is part of the bug's identifier. Bugs created after the JIRA migration will be numbered starting at 8000000; bugs imported from the legacy system have numbers ranging between 1000000 and 79999999.

  • We're working with the bugs.sun.com team to try to maintain continuity of the ability to both read JDK bug information as well as to file new incidents. At least for now, the overall architecture of bugs.sun.com will be the same as it is today: it will be a gateway bridging to an Oracle-internal system, but the internal system will change to JIRA from the legacy database. Generally we are aiming to preserve the visibility of bugs currently viewable on bugs.sun.com; however, bugs in areas not related to the JDK will not be visible after the transition to JIRA. New incoming incidents will be sent to a separate JIRA project for initial triage before possibly being moved into the JDK project.

  • JDK bug management leans heavily on being able to track the state of bugs in multiple releases, especially to coordinate delivering synchronized security releases (known as CPUs, critital patch updates, in Oracle parlance). For a security release, it is common for half a dozen or more release trains to be affected (for example, JDK 5, JDK 6 update, OpenJDK 6, JDK 7 update, JDK 8, virtual releases for HotSpot express, etc.). We've determined we need to track at least the tuple of (release, responsible engineer/assignee for the release, status in the release) for the release trains a fix is going into. To do this in JIRA, we are creating a separate port/backport issue type along with a custom link type to allow the multiple release information to be easily grouped and presented together.

  • The Sun legacy system had a three-level classification scheme, product, category, and subcategory. Out of the box, JIRA only has a one-level classification, component. We've implemented a custom second-level classification, subcomponent. As part of the bug migration we've taken the opportunity to think about how bugs should be grouped under a two-level system and we'll the new system will be simpler and more regular. The main top-level components of the JDK product will include:

    • core-libs
    • client-libs
    • deploy
    • install
    • security-libs
    • other-libs
    • tools
    • hotspot
    For the libs areas, the primary name of the subcomponent will be the package of the API in question. In the core-libs component, there will be subcomponents like:
    • java.lang
    • java.lang.class_loading
    • java.math
    • java.util
    • java.util:i18n
    In the tools component, subcomponents will primarily correspond to command names in $JDK/bin like, jar, javac, and javap.

    The first several bulk imports of the JDK bugs into JIRA have gone well and we're continuing to refine the import to have greater fidelity to the current data, including by reconstructing information not brought over in a structured fashion during the previous large JDK bug system migration back in 2004.

    We don't currently have a firm timeline of when the new system will be usable externally, but as it becomes available, I'll share further information in follow-up blog posts.

Sunday Apr 01, 2012

Goto for the Java Programming Language

Work on JDK 8 is well-underway, but we thought this late-breaking JEP for another language change for the platform couldn't wait another day before being published.

Title: Goto for the Java Programming Language
Author: Joseph D. Darcy
Organization: Oracle.
Created: 2012/04/01
Type: Feature
State: Funded
Exposure: Open
Component: core/lang
Scope: SE
JSR: 901 MR
Discussion: compiler dash dev at openjdk dot java dot net
Start: 2012/Q2
Effort: XS
Duration: S
Template: 1.0
Reviewed-by: Duke
Endorsed-by: Edsger Dijkstra
Funded-by: Blue Sun Corporation


Provide the benefits of the time-testing goto control structure to Java programs. The Java language has a history of adding new control structures over time, the assert statement in 1.4, the enhanced for-loop in 1.5,and try-with-resources in 7. Having support for goto is long-overdue and simple to implement since the JVM already has goto instructions.

Success Metrics

The goto statement will allow inefficient and verbose recursive algorithms and explicit loops to be replaced with more compact code.

The effort will be a success if at least twenty five percent of the JDK's explicit loops are replaced with goto's. Coordination with IDE vendors is expected to help facilitate this goal.


The goto construct offers numerous benefits to the Java platform, from increased expressiveness, to more compact code, to providing new programming paradigms to appeal to a broader demographic.

In JDK 8, there is a renewed focus on using the Java platform on embedded devices with more modest resources than desktop or server environments. In such contexts, static and dynamic memory footprint is a concern. One significant component of footprint is the code attribute of class files and certain classes of important algorithms can be expressed more compactly using goto than using other constructs, saving footprint. For example, to implement state machines recursively, some parties have asked for the JVM to support tail calls, that is, to perform a complex transformation with security implications to turn a method call into a goto. Such complicated machinery should not be assumed for an embedded context. A better solution is just to expose to the programmer the desired functionality, goto.

The web has familiarized users with a model of traversing links among different HTML pages in a free-form fashion with some state being maintained on the side, such as login credentials, to effect behavior. This is exactly the programming model of goto and code. While in the past this has been derided as leading to "spaghetti code," spaghetti is a tasty and nutritious meal for programmers, unlike quiche.

The invokedynamic instruction added by JSR 292 exposes the JVM's linkage operation to programmers. This is a low-level operation that can be leveraged by sophisticated programmers. Likewise, goto is a also a low-level operation that should not be hidden from programmers who can use more efficient idioms.

Some may object that goto was consciously excluded from the original design of Java as one of the removed feature from C and C++. However, the designers of the Java programming languages have revisited these removals before. The enum construct was also left out only to be added in JDK 5 and multiple inheritance was left out, only to be added back by the virtual extension method methods of Project Lambda.

As a living language, the needs of the growing Java community today should be used to judge what features are needed in the platform tomorrow; the language should not be forever bound by the decisions of the past.


From its initial version, the JVM has had two instructions for unconditional transfer of control within a method, goto (0xa7) and goto_w (0xc8). The goto_w instruction is used for larger jumps. All versions of the Java language have supported labeled statements; however, only the break and continue statements were able to specify a particular label as a target with the onerous restriction that the label must be lexically enclosing.

The grammar addition for the goto statement is:

    goto Identifier ;

The new goto statement similar to break except that the target label can be anywhere inside the method and the identifier is mandatory. The compiler simply translates the goto statement into one of the JVM goto instructions targeting the right offset in the method. Therefore, adding the goto statement to the platform is only a small effort since existing compiler and JVM functionality is reused.

Other language changes to support goto include obvious updates to definite assignment analysis, reachability analysis, and exception analysis.

Possible future extensions include a computed goto as found in gcc, which would replace the identifier in the goto statement with an expression having the type of a label.


Since goto will be implemented using largely existing facilities, only light levels of testing are needed.


  • Compatibility: Since goto is already a keyword, there are no source compatibility implications.
  • Performance/scalability: Performance will improve with more compact code. JVMs already need to handle irreducible flow graphs since goto is a VM instruction.

Sunday Mar 11, 2012

Repeating annotations in the works

Work has commenced on the repeating annotations feature discussed in JEP 120: Repeating Annotations.

First, the meta-annotation type used to declare the containing annotation has been added to java.lang.annotation. Updates to the AnnotatedElement interface will be made later. Next, with feedback from other members of the compiler team, I've written a draft specification of the needed language changes:

Updates to JLS 9.6 Annotation Types:

If annotation type T is annotated with an annotation a of type ContainerAnnotation then:

  • The value of the @ContainerAnnotation annotation is called the containing annotation type TC.
  • TC must declare a value() method whose return type is T[], or a compile-time error occurs. (This implies that TC can be the containing annotation type for only a single annotation type.) [Note: Since an annotation type T cannot declare a method that returns T or T[] as a value, that implies that it is a compile time error for an annotation type to specify itself as a container annotation type.]
  • TC may declare methods other than value(). Any such methods must have a default clause, or a compile-time errors occurs. [Note: this is a bit weird "action at a distance" in terms of compiler errors. The error message here is given at the site of the T since the containing annotation might only be present as a class file.]
  • It is a compile time error if the retention of T is not a subset of the retention of TC. This comparison is made taking default retention into consideration.

    In more detail, if the retention of the TC is SOURCE, the retention of T must be SOURCE. If the retention of TC is CLASS, the retention of T may be either CLASS or SOURCE. if the retention of the TC is RUNTIME, the retention of T may be SOURCE, CLASS, or RUNTIME.

  • It is a compile time error if T is @Documented and TC is not @Documented.

    [Note that it is permissible TC to be @Documented and for T to not be @Documented.]

  • It is a compile time error if T is @Inherited and TC is not @Inherited.

    [Note that it is permissible TC to be @Inherited and for T to not be @Inherited.]

  • It is a compile time error if T has target values that are not included in the target set of TC, taking default target values into account.

Note that an annotation type TC may be a container type for annotation type T while also having its own container type TCC.

[However, two annotation types cannot be containers for each other because that would create an illegal cyclic annotation type situation. For example, the following program is rejected by javac:

public @interface Mutual {
    Omaha[] value() default {};

@interface Omaha {
    Mutual[] value() default {};

@Mutual({@Omaha, @Omaha})
@Omaha({@Mutual, @Mutual})
class Foo {}

with the message

Mutual.java:2: error: cyclic annotation element type
    Omaha[] value() default {};
1 error]

Updates to JLS 9.7 Annotations:

"It is a compile-time error if a declaration is annotated with more than one annotation for a given annotation type."

Change to "unless the annotation type is annotated with an @ContainerAnnotation."

Annotation types with an @ContainerAnnotation are known as repeatable annotation types. The presence of an @ContainerAnnotation indicates multiple annotations of that annotation type will be logically replaced with a single annotation synthesized by the compiler. The type of the synthesized annotation, TC, is the type given by the value of the ContainerAnnotation annotation on T. The order of the elements of the value() method of the synthesized TC annotation matches the left to right order of the base repeatable annotations.

If an element only has a single repeatable annotation, the annotation is not replaced.

It is conventional to place the repeatable annotations of a given type contiguously on an element, but this is not required.

It is a compile time error if an element is annotated with both multiple base annotations of a repeatable annotation type T and with one or more annotations of the containing type TC.
[In other words, the collapsing of repeatable annotations to their containers is not recursive.]

Friday Feb 10, 2012

The passing of apt

With a pair of changesets pushed recently, the time for apt to be included in the JDK has drawn to a close, nearly eight years after first being added to the platform. In the Mythical Man-Month sense, apt was always planned to be the one we threw away, we just didn't realize how slow the windup and pitch would be!

The API, but not implementation, of apt were among the first components of the JDK to be released as open source. I learned a lot about technologies and project managment while working on apt, and it was quite satisfying to carry those lessons over to the "second system" of annotation procesing in JSR 269.




« January 2017

No bookmarks in folder