Project Coin: Updated ARM Spec

Starting with Project Coin proposal for Automatic Resource Management (ARM) (Google Docs version), in consultation with Josh Bloch, Maurizio, Jon, and others, Alex and I have produced a specification for ARM blocks that is much closer to Java Language Specification (JLS) style and rigor. The specification involves changes to the existing JLS section §14.20 "The try statement," and will eventually introduce a new subsection §14.20.3 "Execution of try-with-resources," although the specification below is not partitioned as such. Non-normative comments about the specification text below appear inside "[]". Differences between the new specification and the earlier Project Coin proposal for ARM are discussed after the specification.


SYNTAX: The existing set of grammar productions for TryStatement in JLS §14.20 is augmented with:

TryStatement:
try ResourceSpecification Block Catchesopt Finallyopt

Supporting new grammar productions are added:

ResourceSpecification:
( Resources )
Resources:
Resource
Resource ; Resources
Resource:
VariableModifiers Type VariableDeclaratorId = Expression
Expression

[An implication of the combined grammar is that a try statement must have at least one of a catch clause, a finally block, and a resource specification. Furthermore, it is permissible for a try statement to have exactly one of these three components. Note that it is illegal to have a trailing semi-colon in the resource specification.]

A try-with-resources statement has a resource specification that expresses resources to be automatically closed at the end of the Block. A resource specification declares one or more local variables and/or has one or more expressions, each of whose type must be a subtype of AutoCloseable or a compile-time error occurs.

If a resource specification declares a variable, the variable must not have the same name as a variable declared earlier in the resource specification, a local variable, or parameter of the method or initializer block immediately enclosing the try statement, or a compile-time error occurs.

The scope of a variable declared in a resource specification of a try-with-resources statement (§14.20) is from the declaration rightward over the remainder of the resource specification and the entire Block associated with the try. Within the Block of the try, the name of the variable may not be redeclared as a local variable of the directly enclosing method or initializer block, nor may it be redeclared as an exception parameter of a catch clause in a try statement of the directly enclosing method or initializer block, nor may it be redeclared as a variable in the resource specification, or a compile-time error occurs. However, a variable declared in a resource specification may be shadowed (§6.3.1) anywhere inside a class declaration nested within the Block of the try.

The meaning of a try-with-resources statement with a Catches clause or Finally block is given by translation to a try-with-resources statement with no Catches clause or Finally block:

try ResourceSpecification
  Block
Catchesopt
Finallyopt

try {
  try ResourceSpecification
  Block
}
Catchesopt
Finallyopt

In a try-with-resources statement that manages a single resource:

  • If the initialization of the resource completes abruptly because of a throw of a value V, or if the Block of the try-with-resources statement completes abruptly because of a throw of a value V and the automatic closing of the resource completes normally, then the try-with-resources statement completes abruptly because of the throw of value V.

  • If the Block of the try-with-resources statement completes abruptly because of a throw of a value V1, and the automatic closing of the resource completes abruptly because of a throw of a value V2, then the try-with-resources statement completes abruptly because of the throw of value V1, provided that V2 is an Exception. In this case, V2 is added to the suppressed exception list of V1. If V2 is an error (i.e. a Throwable that is not an Exception), then the try-with-resources statement completes abruptly because of the throw of value V2. In this case, V1 is not suppressed by V2.

If a try-with-resources statement that manages multiple resources:

  • If the initialization of a resource completes abruptly because of a throw of a value V, or if the Block of the try-with-resources statement completes abruptly because of a throw of a value V (which implies that the initialization of all resources completed normally) and the automatic closings of all resources completes normally, then the try-with-resources statement completes abruptly because of the throw of value V.

  • If the Block of the try-with-resources statement completes abruptly because of a throw of a value V1, and the automatic closings of one or more resources (that were previously successfully initialized) complete abruptly because of throws of values V2...Vn, then the try-with-resources statement completes abruptly because of the throw of a value Vi (1 ≤ i ≤ n) determined by the translation below.

The exceptions that can be thrown by a try-with-resources statement are the exceptions that can thrown by the Block of the try-with-resources statement plus the union of the exceptions that can be thrown by the automatic closing of the resources themselves. Regardless of the number of resources managed by a try-with-resources statement, it is possible for a Catchesopt clause to catch an exception due to initialization or automatic closing of any resource.

A try-with-resources statement with a ResourceSpecification clause that declares multiple Resources is treated as if it were multiple try-with-resources statements, each of which has a ResourceSpecification clause that declares a single Resource. When a try-with-resources statement with n Resources (n > 1) is translated, the result is a try-with-resources statement with n-1 Resources. After n such translations, there are n nested try-catch-finally statements, and the overall translation is complete.

The meaning of a try-with-resources statement with a ResourceSpecification clause and no Catches clause or Finally block is given by translation to a local variable declaration and a try-catch-finally statement. During translation, if the ResourceSpecification clause declares one Resource, then the try-catch-finally statement is not a try-with-resources statement, and ResourceSpecificationtail is empty. If the ResourceSpecification clause declares n Resources, then the try-catch-finally statement is treated as if it were a try-with-resources-catch-finally statement, where ResourceSpecificationtail is a ResourceSpecification consisting of the 2nd, 3rd, ..., nth Resources in order. The translation is as follows, where the identifiers #primaryException, #t, and #suppressedException are fresh:

try ResourceSpecification
  Block

{
final VariableModifiers_minus_final R #resource = Expression;
Throwable #primaryException = null;

try ResourceSpecificationtail
  Block
catch (final Throwable #t) {
  #primaryException = t;
  throw #t;
} finally {
  if (#primaryException != null) {
    try {
      #resource.close();
    } catch(Exception #suppressedException) {
      #primaryException.addSuppressedException(#suppressedException);
    }
  } else {
    #resource.close();
  }
}

If the Resource being translated declares a variable, then VariableModifiers_minus_final is the set of modifiers on the variable (except for final if present); R is the type of the variable declaration; and #resource is the name of the variable declared in the Resource.

Discussion: Resource declarations in a resource specification are implicitly final. For consistency with existing declarations that have implicit modifiers, it is legal (though discouraged) for a programmer to provide an explicit "final" modifier. By allowing non-final modifiers, annotations such as @SuppressWarnings will be preserved on the translated code. It is unlikely that the Java programming language will ever ascribe a meaning to an explicit final modifier in this location other than the traditional meaning.
[Unlike the new meaning ascribed to a final exception parameter.]

Discussion: Unlike the fresh identifier in the translation of the enhanced-for statement, the #resource variable is in scope in the Block of a try-with-resources statement.

If the Resource being translated is an Expression, then the translation includes an local variable declaration for which VariableModifiers_minus_final is empty; the type R is the type of the Expression (under the condition that the Expression is assigned to a variable of type AutoCloseable); and #resource is a fresh identifier.

Discussion: The method Throwable.addSuppressedException has a parameter of type Throwable, but the translation is such that only an Exception from #resource.close() will be passed for suppression. In the judgment of the designers of the Java programming language, an Error due to automatic closing of a resource is sufficiently serious that it should not be automatically suppressed in favor of an exception from the Block or the initialization or automatic closing of lexically rightward resources.
[However, perhaps such an Error should instead be recorded as suppressing an exception from the Block or other lexically rightward component.]

Discussion: This translation exploits the improved precision of exception analysis now triggered by the rethrow of a final exception parameter.

The reachability and definite assignment rules for the try statement with a resource specification are implicitly specified by the translations above.


Compared to the earlier proposal, this draft specification:

  • Assumes the revised supporting API with java.lang.AutoCloseable as the type indicating participation in the new language feature.

  • Changes the official grammar for a declared resource from
    LocalVariableDeclaration
    to
    VariableModifiers Type VariableDeclaratorId = Expression
    The former syntactically allowed code like
    AutoCloseable a, b, c
    which would not be useful in this context.

  • Preserves modifiers on explicitly declared resources, which implies @SuppressWarnings on a resource should have the intended effect.

  • States how the exception behavior of close methods is accounted for in determining the set of exceptions a try-with-resource statement can throw.

  • Gives a more precise determination of the type used for the local variable holding a resource given as an Expression. This precision is important to allow accurate exception information to be computed.

  • Provides typing constraints so that type inference works as expected if the Expression given as a Resource in a ResourceSpecification is, say, a generic method or null.

Compiler changes implementing this revised specification remain in progress. After experience is gained with the initial implementation, I expect various changes to the feature to be contemplated:

  • Dropping support for a resource to be specified as a general Expression. Nontrivial specification and implementation complexities arise from allowing a general Expression to be used as resource. Allowing a restricted expression that was just a name may provide nearly all the additional flexibility at marginal additional implementation and specification impact.

  • Adjustments to the suppressed exception logic: in the present specification, an incoming primary exception will suppress an Exception thrown by a close method; however, if the close method throws an error, that error is propagated out without suppressing an incoming primary exception. Possible alternatives include having a primary exception in a try-with-resource statement suppress all subsequent Throwables originating in the statement and having a non-Exception thrown by a close suppress any incoming primary exception.

    These alternatives could be implemented by replacing the translated code

        try {
          #resource.close();
        } catch(Exception #suppressedException) {
          #primaryException.addSuppressedException(#suppressedException);
        }

    with

        try {
          #resource.close();
        } catch(Throwable #suppressedException) {
          #primaryException.addSuppressedException(#suppressedException);
        }

    or

        try {
          #resource.close();
        } catch(Exception #suppressedException) {
          #primaryException.addSuppressedException(#suppressedException);
        } catch(Throwable #throwable) {
          #throwable.addSuppressedException(#primaryException);
          throw #throwable;

        }

    respectively.

Comments:

Joe, in regards to your last point of discussion, by catching Throwable and making it a suppressed exception, this has the effect of also suppressing instances of java.lang.Error. A JVM error is much more critical than the resource management of ARM. Should Errors propagate out regardless?

Posted by Paul Benedict on July 17, 2010 at 02:19 AM PDT #

@Paul,

The incoming primary exception can be from a resource initializer, the Block of the try-with-resources statement, or the closing of another resource.

The incoming exception can be a java.lang.Exception, a java.lang.Error, or some other subclass of java.lang.Throwable. In the current translation, regardless of the nature of the incoming exception, if the current close method throws an error, that error get propagated and no record is made of the incoming exception, which itself can be an error.

I think this is a bug in the current translation.

In the worst case, in a try-with-resources statement with n resources, a total of (n+1) locations can throw exceptions (the n calls to close plus the Block). Any or all of these exceptions could potentially be Errors or general Throwables. One could argue the most serious of these (n+1) exceptions should be the one propagated out. However, the decision to suppress or propagate is made without that full context; at a given location, there are two Throwables to compare, but there is not a total ordering of severity among exceptions and I don't think introducing more complicated logic is worthwhile.

Therefore, I favor a translation where any exceptions thrown in the try-with-resources statement that do not propagate out are listed as being suppressed by the one that does propagate out with an easy to understand policy like "first thrown exception wins" or "last thrown exception wins."

Posted by Joe Darcy on July 17, 2010 at 03:51 AM PDT #

What is the status of closures for Java 7? If closures make it in, then it should be possible to make this a library routine rather than an extension to the core language.

Here is how it is done in Scala:

http://scala.sygneca.com/patterns/loan

Perhaps you feel the extra syntactic form has an advantage over the library version, but it seems at least worth a consideration. In general, closures should greatly cut down on the number of control structures that are worth including in the syntax.

Posted by Lex Spoon on August 20, 2010 at 11:21 PM PDT #

@Lex,

For the status of closures in JDK 7 see

http://openjdk.java.net/projects/lambda/

Project Lambda is \*not\* considering closures that support control abstraction. If it was, then try-with-resources could be implemented as a library as you point out.

Posted by Joe Darcy on August 21, 2010 at 03:05 AM PDT #

While I think that ARM is per se a good thing, I am not happy with the syntax. Maybe I haven't fully understood the functionality, but I don't understand why the resource declarations can't be mixed with normal statements. It often happens to me that I need to use multiple resources, but need some preparation between the allocation of them. With the current syntax, one would need multiple nested blocks. Maybe this is naïve, but I would allow a "try <resource> ;" anywhere in any block (including the blocks in a try-catch-finally construct), with the resources deallocated at the end of the block where the try statement occurs, e.g.

{
try InputStream is = new FileInputStream (...);
...
// read some headers or so
...
try OutputStream os = new FileOutputStream (...);
...
}

Posted by Klaus on September 04, 2010 at 06:36 PM PDT #

The definition of AutoCloseable seems too restritive too me. It predefines name of a method as well as its return type. I guess you'd make the design more flexible with:

@Retention(CLASS)
@Target(METHOD)
@interface AutoCloseable {
}

The compiler would then check whether there is at most one annotated method in each class (including inheritance) and whether it has zero arity. The return type would not matter.

This shall not require much more processing overhead in compiler than the original marker interface and it would bring more flexibility to the library writers.

Also, but that may be just personal preference, using @annotation seems more appropriate in language design than hard-coding knowledge of marker interfaces in the compiler.

Posted by Jaroslav Tulach on September 08, 2010 at 10:48 PM PDT #

@Jaroslav,

Yes, that style of solution was considered and discussed on the coin list and ultimately rejected, basically for being an abuse of annotations. A clean-up method named "close" is by far the most common in Java libraries; "dispose" is a distant second. It would be "a small matter of programming" for an analogous AutoDisposable interface to be defined, but that was not judged necessary, at least for the initial version of the feature.

Posted by Joe Darcy on September 09, 2010 at 07:32 AM PDT #

Re. "abuse": I guess that is quite subjective topic. Although I understand that as Czar of annotation processing, you are the one to describe what objective opinion shall be.

Re. "close is the most common". Well it depends whether you are designing this feature for JDK only or for libraries written by all Java developers. For example, how do I use ARM with http://bits.netbeans.org/6.9.1/javadoc/org-openide-awt/org/netbeans/api/actions/Closable.html ? That is an interface with single method close(), but no chance, right?

Re. "analogous AutoDisposable". Nobody can define all "analogous" interfaces as needed by existing Java libraries. Definitely those can't be hard-coded in the javac compiler. Some more open solution would have to be found.

If the goal is to create new language feature to be used primarily by JDK itself, which can't be retrofitted into existing APIs, then the design is OK.

Should there be at least a minimal desire to allow existing APIs to be compatibly evolved to support ARM, the design shall be more flexible.

Btw. compare with generics - their introduction allowed any API to be rewritten to use them and stay compatible. Generics did not prescribe naming or parameter conventions, the design was flexible and adoptable by any Java library existing at that time.

Posted by Jaroslav Tulach on September 09, 2010 at 10:55 PM PDT #

@Jaroslav,

To support methods with other names and signatures, a small adapter can be used as outlined in the design FAQ question about locks in a previous version of the ARM proposal:

https://docs.google.com/View?id=ddv8ts74_3fs7483dp

Posted by Joe Darcy on September 10, 2010 at 02:23 AM PDT #

There is an apparent mistake even in one of the "minimally correct" samples today:

BufferedReader br = new BufferedReader(new FileReader(path));
try {
return br.readLine();
} finally {
br.close();
}

is wrong since the BufferedReader constructor could in principle throw an exception resulting in the FileReader not being closed. This is a very common mistake in existing code (and I am not sure FindBugs even reports it).

What is wanted is perhaps:

Reader r = new FileReader(path);
try {
return new BufferedReader(br).readLine();
} finally {
r.close();
}

Even this would be wrong for FileWriter + PrintWriter, since you need to flush a PrintWriter or you will lose most of the last block of text! I would hope that the proposal will correctly deal with chained initializers, e.g.:

try (Writer w = new FileWriter(path); PrintWriter pw = new PrintWriter(w)) {
w.println("that's all folks");
}

where an exception in the FileWriter constructor would be thrown up, an exception in the PrintWriter constructor (unlikely) would be thrown up after closing the FileWriter, and an exception in println would be thrown up after first closing the PrintWriter and then closing the FileWriter.

Posted by Jesse Glick on September 16, 2010 at 01:21 AM PDT #

@Jesse,

I assume you are referring to the examples given in Josh's previous version of the document (https://docs.google.com/View?id=ddv8ts74_3fs7483dp). In that document, the issue you raised is discussed in design FAQ item 3. "Does the construct work properly with "decorated" resources?"

The suggestion from Josh's document, which is also supported in the implementation, is the chained initialization pattern (decorated resources pattern) you list.

Posted by Joe Darcy on September 17, 2010 at 01:48 AM PDT #

@Joe: right, I missed that FAQ item before. I think its wording may however give a false sense of security. While it is true that the BufferedReader(Reader) constructor is very unlikely to throw an exception, there are many variations on this code in which failure to safely treat decorated resources could cause unacceptably common bugs. For example, a (misguided) attempt to match buffer size to file length:

File ini = new File("C:\\\\whatever.ini");
try (BufferedReader br = new BufferedReader(new FileReader(ini), (int) ini.length()) {
parse(br.readLine());
}

will throw an IllegalArgumentException but leave whatever.ini locked and unusable by other processes if it is of zero length!

Similarly, making an InputStreamReader with a charsetName could throw UnsupportedEncodingException, leaking the underlying InputStream. This could easily happen on a production JVM but never in testing, in case the list of supported encodings differs.

FindBugs could detect this class of mistake, of course, but I think developer documentation introducing the new syntax should be sure to emphasize that it is safest to declare an explicit variable for the most basic object which might possibly hold a native resource (e.g. FileReader), followed by any needed decorators.

I might even suggest a compiler "lint" warning in case an initializer expression in an ARM block contains a subexpression assignable to AutoCloseable.

Posted by Jesse Glick on September 17, 2010 at 03:00 AM PDT #

@Jesse,

I've filed rfe 6986416 "Warn of unprotected AutoCloseables in try-with-resources initializers" to record your suggestion.

Posted by Joe Darcy on September 21, 2010 at 01:38 AM PDT #

Joe, if "try" declares resources and only a "finally" block exists (no "catch" clause), what happens to the suppressed exceptions? Are the resources still closed and the suppressed exceptions thrown away?

Posted by Paul Benedict on September 29, 2010 at 01:43 PM PDT #

@Paul,

From the desugaring,

try(Resource r ...) {...}
finally {...}

gets translated into

try {try(Resource r ...) {...}
} finally {...}

Therefore, all the right close methods get called, and, if thrown in the resource handling code, an exception (possibly with suppressed exceptions) will get propagated out to the finally block. In the finally block, if the code there does not throw an exception, any incoming exception will get propagated out. Otherwise, if the code in the finally throws an exception, any incoming exception is dropped.

Posted by Joe Darcy on October 01, 2010 at 03:23 AM PDT #

Post a Comment:
Comments are closed for this entry.
About

darcy

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
News

No bookmarks in folder

Blogroll