Sunday Dec 08, 2013

JSR 269 Maintenance Review for Java SE 8

The annotation processing API, both the processor-specific portion of the API in javax.annotation.processing and the language modeling portions in javax.lang.model.*, are being updated to support the new language features in Java SE 8. Procedurally, the proposed changes are covered by the second maintenance review of JSR 269: Maintenance Draft Review 2.

As summarized on on the maintenance review page, there are three categories of changes from the version of the API shipped with Java SE 7:

  1. Cleaning up the existing specification without changing its semantics (adding missing javadoc tags, etc.)
  2. API changes to support the language changes being made in Project Lambda /JSR 335. These includes adding javax.lang.model.type.IntersectionType as well as javax.lang.model.element.ExecutableElement.isDefault.
  3. API changes to support the language changes being made under JSR 308, Annotations on Java Types. These include javax.lang.model.AnnotatedConstruct and updating javax.annotation.processing.Processor.

The small repeating annotations language change, discussed on an OpenJDK alias, is also supported by the proposed changes.

A detailed specification difference is available. Please post any comments here or send them to me through email.

Monday May 20, 2013

javax.lang.model backed by core reflection

Back when the javax.lang.model API was being designed as part of JSR 269, while the API was primarily intended for use at compile-time with annotation processing, the expert group also wanted the API to be usable in other contexts as well, including at runtime.

JEP 119, javax.lang.model Implementation Backed by Core Reflection, proposed adding such an alternative runtime implementation of javax.lang.model to JDK 8. Such an implementation has recently been pushed as sample code in the JDK 8 langtools repository.

At a high level, the sample code is at its core repeated uses of the adapter pattern, translating between the core reflection API and the javax.lang.model API. However, there were a number of design issues in what interface the javax.lang.model implementation for core reflection should expose. First, should a core reflection specialization of javax.lang.model expose a more specific interface? There are some advantages to just implementing the base API defined in the existing interfaces; it is the simplest approach and would appear to maximize the ability to inter-operate with other implementations. However, javax.lang.model depends on hidden state to define concepts like equality, so there is intrinsically limited inter-operation between disjoint implementations. Therefore, especially in sample code, it was viewed as worthwhile to experiment with a more specific API in the core reflection case.

To make the javax.lang.model backed by core reflection more specific, for each FooElement interface in javax.lang.model.element, a ReflectionFooElement subinterface was defined, as shown in the diagram below.

The subinterfaces are specialized in a few ways:

  • If a method in the base interface was defined to return a FooElement, a covariant override in the subinteface was defined to return a ReflectionFooElement.
  • If a method in the base interface was defined to return a List< ? extends FooElement>, a covariant override in the subinteface was defined to return a List<ReflectionFooElement>.
  • For getBar(FooElement arg) methods defined in the Elements helper interface, add a getBar() method to the ReflectionFooElement subinterface.
  • Add a getSource method to the root ReflectionElement interface to return the underlying core reflection object being adapted and add covariant overrides in subinterfaces as appropriate.
  • The root ReflectionElement interface implements the full AnnotatedElement interface, not just the subset of its methods found in the base javax.lang.model.element.

A cost of specializing the API is the need to define a new visitor interface as well. Covariant overrides cannot be used as in the element modeling interfaces since the visitor methods take elements in the argument position rather than the return position.

Generally, working on the core reflection specialization proceeded as expected. Equality determinations were generally delegated to core reflection source object; in other words, two ReflectionElement objects are equal if they are instances of the same interface and if their sources are .equals. Writing the sample code highlighted several shortcomings of the core reflection API which have been addressed in Java SE 8, including the addition of an Executable type to abstract over the commonalities of Method and Constructor. The sample code also benefited from other reflection changes in Java SE 8, such as the introduction of a java.lang.reflect.Parameter class and retrofitting TypeVariable to extend AnnotatedElement.

While significantly more testing would be needed to productize javax.lang.model backed by core reflection, even as sample code it validates a number of the technological decisions made in JSR 269 and is an interesting demonstration of how one of the platform's reflective APIs can be bridged to another.

Sunday Apr 14, 2013

Changing Sources and Moving Targets: Evolving the javac command line

As written up in JEP 182: Policy for Retiring javac -source and -target Options, we're implementing a policy to over time clean up the -source and -target portions of javac's command line:

  • Class files going all the way back to version 45.3 generated by JDK 1.0.2 will continue to be recognized by javac when put on the classpath, etc.
  • The never-documented -target options 1.4.1, 1.4.2, and jsr14 have been removed in JDK 8 (8010179).
  • In JDK 8, source and target options for 1.5 and below will be deprecated and a warning will be printed on their use (8011043).
  • In JDK 9, source and target options for 1.5 and below will be removed (8011044).
  • Starting in JDK 9, a "one plus three back" policy will apply to source and target options. JDK 9 will support (9/1.9, 8/1.8, 7/1.7, 6/1.6) and JDK 10 will support (10/1.10, 9/1.9, 8/1.8, 7/1.7) and so on.

Removing support for the old options will ease compiler maintenance in several ways. First, there will be direct benefits from allowing some code to be deleted. Nixing jsr14 allowed about 250 lines of code to be excised. Second, fewer interactions between new language features and old source levels need to be handled in the compiler. The Java Language Specification only deals with a single version of the language and there is no formal specification of many aspects of how a Java compiler is supposed to behave. To use a recent example, there is no specification for how a new-in-8 default method being introduced to the language and core libraries by Project Lambda should appear to code being compiled under, say, -source 6. Limiting the span of source and target version supported reduces the need to define such cross-version interactions. (The practical impact of source cross-version interfaces would be greatly reduced if developers more often follwed the recommended practice of setting the bootclasspath when cross-compiling to an older platform version.)

This policy will help balance stability versus progress and should cover releases having public updates when a new JDK is released.

Thursday Feb 21, 2013

Functional Interfaces

As part of Project Lambda, after discussion with the JSR 335 expert group, we decided to add a FunctionalInterface annotation type to the platform. To a first approximation, functional interfaces are those interface types which define only a single method and are therefore usable in lambda expressions. (There are some tricky details in the definition of a functional interface relating to generics and also some details about excluding from consideration methods defined on java.lang.Object.) The new annotation type allows a library designer to clearly indicate the property of intending an interface type to be used with lambda expressions along with an implied commitment to keep that property true in the future. However, the compiler will allow any interface type meeting the structural properties of a functional interface to be used for a lambda expression regardless of whether or not the interface has a @FunctionalInterface annotation.

They types being added in the java.util.function package are by design functional interfaces and can be annotated with @FunctionalInterface from early days. However, many existing Java SE types are also functional interfaces and we want to identify and annotate those appropriately too. To find those candidate interfaces to be annotated, I ran an annotation processor over the code, using the same methodology as used to find Closeable candidates during JDK 7.

A significant number of candidates were found throughout the JDK. After suitable discussion and review, the first batch of core libraries changes have been pushed. Analogous discussions have been started in the 2D, awt, and swing areas.

For guidance in retrofitting @FunctionalInterface to an existing candidate type, if the type is routinely instantiated using an anonymous class, it is a good candidate for being annotated with @FunctionalInterface.

Friday Jun 15, 2012

My integer overfloweth

While certain classes like java.lang.Integer and java.lang.Math have been in the platform since the beginning, that doesn't mean there aren't more enhancements to be made in such places! For example, earlier in JDK 8, library support was added for unsigned integer arithmetic. More recently, my colleague Roger Riggs pushed a changeset to support integer overflow, that is, to provide methods which throw an ArithmeticException on overflow instead of returning a wrapped result. Besides being helpful for various programming tasks in Java, methods like the those for integer overflow can be used to implement runtimes supporting other languages, as has been requested at a past JVM language summit.

This year's language summit is coming up in July and I hope to get some additional suggestions there for helpful library additions as part of the general discussions of the JVM and Java libraries as a platform.

Sunday Apr 01, 2012

Goto for the Java Programming Language

Work on JDK 8 is well-underway, but we thought this late-breaking JEP for another language change for the platform couldn't wait another day before being published.

Title: Goto for the Java Programming Language
Author: Joseph D. Darcy
Organization: Oracle.
Created: 2012/04/01
Type: Feature
State: Funded
Exposure: Open
Component: core/lang
Scope: SE
JSR: 901 MR
Discussion: compiler dash dev at openjdk dot java dot net
Start: 2012/Q2
Effort: XS
Duration: S
Template: 1.0
Reviewed-by: Duke
Endorsed-by: Edsger Dijkstra
Funded-by: Blue Sun Corporation


Provide the benefits of the time-testing goto control structure to Java programs. The Java language has a history of adding new control structures over time, the assert statement in 1.4, the enhanced for-loop in 1.5,and try-with-resources in 7. Having support for goto is long-overdue and simple to implement since the JVM already has goto instructions.

Success Metrics

The goto statement will allow inefficient and verbose recursive algorithms and explicit loops to be replaced with more compact code.

The effort will be a success if at least twenty five percent of the JDK's explicit loops are replaced with goto's. Coordination with IDE vendors is expected to help facilitate this goal.


The goto construct offers numerous benefits to the Java platform, from increased expressiveness, to more compact code, to providing new programming paradigms to appeal to a broader demographic.

In JDK 8, there is a renewed focus on using the Java platform on embedded devices with more modest resources than desktop or server environments. In such contexts, static and dynamic memory footprint is a concern. One significant component of footprint is the code attribute of class files and certain classes of important algorithms can be expressed more compactly using goto than using other constructs, saving footprint. For example, to implement state machines recursively, some parties have asked for the JVM to support tail calls, that is, to perform a complex transformation with security implications to turn a method call into a goto. Such complicated machinery should not be assumed for an embedded context. A better solution is just to expose to the programmer the desired functionality, goto.

The web has familiarized users with a model of traversing links among different HTML pages in a free-form fashion with some state being maintained on the side, such as login credentials, to effect behavior. This is exactly the programming model of goto and code. While in the past this has been derided as leading to "spaghetti code," spaghetti is a tasty and nutritious meal for programmers, unlike quiche.

The invokedynamic instruction added by JSR 292 exposes the JVM's linkage operation to programmers. This is a low-level operation that can be leveraged by sophisticated programmers. Likewise, goto is a also a low-level operation that should not be hidden from programmers who can use more efficient idioms.

Some may object that goto was consciously excluded from the original design of Java as one of the removed feature from C and C++. However, the designers of the Java programming languages have revisited these removals before. The enum construct was also left out only to be added in JDK 5 and multiple inheritance was left out, only to be added back by the virtual extension method methods of Project Lambda.

As a living language, the needs of the growing Java community today should be used to judge what features are needed in the platform tomorrow; the language should not be forever bound by the decisions of the past.


From its initial version, the JVM has had two instructions for unconditional transfer of control within a method, goto (0xa7) and goto_w (0xc8). The goto_w instruction is used for larger jumps. All versions of the Java language have supported labeled statements; however, only the break and continue statements were able to specify a particular label as a target with the onerous restriction that the label must be lexically enclosing.

The grammar addition for the goto statement is:

    goto Identifier ;

The new goto statement similar to break except that the target label can be anywhere inside the method and the identifier is mandatory. The compiler simply translates the goto statement into one of the JVM goto instructions targeting the right offset in the method. Therefore, adding the goto statement to the platform is only a small effort since existing compiler and JVM functionality is reused.

Other language changes to support goto include obvious updates to definite assignment analysis, reachability analysis, and exception analysis.

Possible future extensions include a computed goto as found in gcc, which would replace the identifier in the goto statement with an expression having the type of a label.


Since goto will be implemented using largely existing facilities, only light levels of testing are needed.


  • Compatibility: Since goto is already a keyword, there are no source compatibility implications.
  • Performance/scalability: Performance will improve with more compact code. JVMs already need to handle irreducible flow graphs since goto is a VM instruction.

Sunday Mar 11, 2012

Repeating annotations in the works

Work has commenced on the repeating annotations feature discussed in JEP 120: Repeating Annotations.

First, the meta-annotation type used to declare the containing annotation has been added to java.lang.annotation. Updates to the AnnotatedElement interface will be made later. Next, with feedback from other members of the compiler team, I've written a draft specification of the needed language changes:

Updates to JLS 9.6 Annotation Types:

If annotation type T is annotated with an annotation a of type ContainerAnnotation then:

  • The value of the @ContainerAnnotation annotation is called the containing annotation type TC.
  • TC must declare a value() method whose return type is T[], or a compile-time error occurs. (This implies that TC can be the containing annotation type for only a single annotation type.) [Note: Since an annotation type T cannot declare a method that returns T or T[] as a value, that implies that it is a compile time error for an annotation type to specify itself as a container annotation type.]
  • TC may declare methods other than value(). Any such methods must have a default clause, or a compile-time errors occurs. [Note: this is a bit weird "action at a distance" in terms of compiler errors. The error message here is given at the site of the T since the containing annotation might only be present as a class file.]
  • It is a compile time error if the retention of T is not a subset of the retention of TC. This comparison is made taking default retention into consideration.

    In more detail, if the retention of the TC is SOURCE, the retention of T must be SOURCE. If the retention of TC is CLASS, the retention of T may be either CLASS or SOURCE. if the retention of the TC is RUNTIME, the retention of T may be SOURCE, CLASS, or RUNTIME.

  • It is a compile time error if T is @Documented and TC is not @Documented.

    [Note that it is permissible TC to be @Documented and for T to not be @Documented.]

  • It is a compile time error if T is @Inherited and TC is not @Inherited.

    [Note that it is permissible TC to be @Inherited and for T to not be @Inherited.]

  • It is a compile time error if T has target values that are not included in the target set of TC, taking default target values into account.

Note that an annotation type TC may be a container type for annotation type T while also having its own container type TCC.

[However, two annotation types cannot be containers for each other because that would create an illegal cyclic annotation type situation. For example, the following program is rejected by javac:

public @interface Mutual {
    Omaha[] value() default {};

@interface Omaha {
    Mutual[] value() default {};

@Mutual({@Omaha, @Omaha})
@Omaha({@Mutual, @Mutual})
class Foo {}

with the message error: cyclic annotation element type
    Omaha[] value() default {};
1 error]

Updates to JLS 9.7 Annotations:

"It is a compile-time error if a declaration is annotated with more than one annotation for a given annotation type."

Change to "unless the annotation type is annotated with an @ContainerAnnotation."

Annotation types with an @ContainerAnnotation are known as repeatable annotation types. The presence of an @ContainerAnnotation indicates multiple annotations of that annotation type will be logically replaced with a single annotation synthesized by the compiler. The type of the synthesized annotation, TC, is the type given by the value of the ContainerAnnotation annotation on T. The order of the elements of the value() method of the synthesized TC annotation matches the left to right order of the base repeatable annotations.

If an element only has a single repeatable annotation, the annotation is not replaced.

It is conventional to place the repeatable annotations of a given type contiguously on an element, but this is not required.

It is a compile time error if an element is annotated with both multiple base annotations of a repeatable annotation type T and with one or more annotations of the containing type TC.
[In other words, the collapsing of repeatable annotations to their containers is not recursive.]

Friday Feb 10, 2012

The passing of apt

With a pair of changesets pushed recently, the time for apt to be included in the JDK has drawn to a close, nearly eight years after first being added to the platform. In the Mythical Man-Month sense, apt was always planned to be the one we threw away, we just didn't realize how slow the windup and pitch would be!

The API, but not implementation, of apt were among the first components of the JDK to be released as open source. I learned a lot about technologies and project managment while working on apt, and it was quite satisfying to carry those lessons over to the "second system" of annotation procesing in JSR 269.

Friday Jan 20, 2012

Unsigned Integer Arithmetic API now in JDK 8

At long last, after due discussion and review, I've just pushed initial API support for unsigned integer arithmetic into JDK 8! The support is implemented via static methods, primarily on java.lang.Integer and java.lang.Long, that:

  • Provide bidirectional conversion between strings and unsigned integers
  • Compare values as unsigned
  • Compute unsigned divide and remainder

Colloquially, "unsigned integer" means a 32-bit int or 64-bit long value where all the bits are interpreted as contributing to the magnitude. In the unsigned realm, the values of an integer type of a given bit-width range from 0 to 2width-1 rather than from -(2width-1) to 2width-1-1. A feature of the two's complement encoding of Java integers is that the bitwise results for add, subtract, and multiply are the same if both inputs are interpreted as signed values or both inputs are interpretted as unsigned values. (Other encodings like one's complement and signed magnitude don't have this properly.) Therefore, of the basic arithmetic operations, only a separate divide method needs to be provided to operate on values interpreted as unsigned.

To avoid dealing with the overhead of boxed values and to allow reuse of the built-in arithmetic operators, the unsigned API support does not introduce new types like UnsignedInt with instance methods to perform addition, subtraction, etc. However, that lack of separate Java-level unsigned types does mean a programmer can accidentally improperly mix signed and unsigned values. However, new unsigned types aren't the only way to mitigate this hazard. For example, a naming convention of adding a trailing "U" or "_U" to variables holding unsigned values could be adopted. A more structured approach would be to add an @Unsigned annotation type to the platform and apply that annotation to variables and fields holding unsigned values. One of the extra-linguistic checkers to be enabled by JSR 308 could then analyze code for signed/unsigned correctness.

I'm glad these methods are finally in the JDK. Later in JDK 8, there may be a few more fun bit-twiddling additions, such as methods to get the high order bits of a full multiply and methods which throw exceptions on integer overflow instead of wrapping around.

Friday Dec 02, 2011

An apt ending draws nigh

I brought you into this world, and I'll take you out!
—Cliff Huxtable

The end of an era draws nigh! After being deprecated in JDK 7, the apt command line tool and the entirely of its associated API is on track to be removed from JDK 8 within the next few months. While apt was fine back in JDK 5, the time has come to transition annotation processing to the superior standardized annotation processing provided by javax.annotation.processing and javax.lang.model.*. These packages were added to Java SE 6 under JSR 269.

This removal effort was discussed in JEP 117: Remove the Annotation-Processing Tool (apt).

Portions of jax-ws in the JDK use apt, but those portions are being rewritten to use the JSR 269 APIs. Once that revised version of jax-ws is being used by the JDK builds, apt will be excised in short order.

As a com.sun.* API, apt is not part of Java SE; it is just a component of the JDK and is thus easier to remove from the platform. While I was the lead in creating apt, lo these many years ago, I'm looking forward to deleting the code from the JDK to encourage use of a better replacement API and to ease maintenance of javac.




« June 2016

No bookmarks in folder