Monday Mar 17, 2014

the isthmus in the VM

This is a good time to consider new options for a “native interconnect” between code managed by the JVM and APIs for libraries not managed by the JVM.

Notably, Charles Nutter has followed up on his JVM Language Summit talk (video on this page) by proposing JEP 191, to provide a new foreign function interface for Java…

This article describes some of the many questions related to native interconnect, along with some approaches for solving them. We will start Project Panama in OpenJDK to air out these questions thoroughly, and do some serious engineering to address them, for the JDK…

the big goal

I think the general, basic, idealistic goal is something like this:

If non-Java programmers find some library useful and easy to access, it should be similarly accessible to Java programmers.

That ideal is easy to state but hard to carry out…[Read More]

Thursday Mar 06, 2014

value types and struct tearing

value types and struct tearing

This note explains how a notion of “value types” for the VM should protect the integrity of that value type’s invariants, and points out important differences, in memory effects, between “struct-like” and “persistent” designs for values.

[Read More]

Monday Apr 02, 2012

Monday at Microsoft Lang.NEXT

We are having a blast at Microsoft Lang.NEXT.

For the record, I posted my talk about Java 8.

[Update 4/06] The videos are already coming out from Channel 9, including my talk on Java 8 (mostly about closures) and an update from Jeroen Frijters on hosting the JVM on .NET.

I recommend Martin Odersky’s keynote ... [Read More]

Wednesday Mar 21, 2012

value types in the vm

value types in the vm

Or, enduring values for a changing world.

Introduction

A value type is a data type which, generally speaking, is designed for being passed by value in and out of methods, and stored by value in data structures. The only value types which the Java language directly supports are the eight primitive types. Java indirectly and approximately supports value types, if they are implemented in terms of classes. For example, both Integer and String may be viewed as value types, especially if their usage is restricted to avoid operations appropriate to Object. In this note, we propose a definition of value types in terms of a design pattern for Java classes, accompanied by a set of usage restrictions. We also sketch the relation of such value types to tuple types (which are a JVM-level notion), and point out JVM optimizations that can apply to value types.

This note is a thought experiment to extend the JVM’s performance model in support of value types. The demonstration has two phases. Initially the extension can simply use design patterns, within the current bytecode architecture, and in today’s Java language. But if the performance model is to be realized in practice, it will probably require new JVM bytecode features, changes to the Java language, or both. We will look at a few possibilities for these new features.

[Posted 3/21/2012, updated as marked 3/24/2012 and 3/05/2014.]

[Read More]

Wednesday Nov 30, 2011

on coding style

I vastly prefer coding to discussing coding style, just as I would prefer to write poetry instead of talking about how it should be written. Sometimes the topic cannot be put off, either because some individual coder is messing up a shared code base and needs to be corrected, or (worse) because some officious soul has decided, "what we really need around here are some strongly enforced style rules!" Neither is the case at the moment, and yet I will venture a post on the subject. The following are not rules, but suggested etiquette. The idea is to allow a coherent style of coding to flourish safely and sanely, as a humane, inductive, social process.

Maxim M1: Observe, respect, and imitate the largest-scale precedents available. (Preserve styles of whitespace, capitalization, punctuation, abbreviation, name choice, code block size, factorization, type of comments, class organization, file naming, etc., etc., etc.)

Maxim M2: Don't add weight to small-scale variations. (Realize that Maxim M1 has been broken many times, but don't take that as license to create further irregularities.)

Maxim M3: Listen to and rely on your reviewers to help you perceive your own coding quirks. (When you review, help the coder do this.)

Maxim M4: When you touch some code, try to leave it more readable than you found it. (When you review such changes, thank the coder for the cleanup. When you plan changes, plan for cleanups.)

On the Hotspot project, which is almost 1.5 decades old, we have often practiced and benefited from such etiquette. The process is, and should be, inductive, not prescriptive. An ounce of neighborliness is better than a pound of police-work.

Reality check: If you actually look at (or live in) the Hotspot code base, you will find we have accumulated many annoying irregularities in our source base. I suppose this is the normal condition of a lived-in space. Unless you want to spend all your time polishing and tidying, you can't live without some smudge and clutter, can you?

Final digression: Grammars and dictionaries and other prescriptive rule books are sometimes useful, but we humans learn and maintain our language by example not grammar. The same applies to style rules. Actually, I think the process of maintaining a clean and pleasant working code base is an instance of a community maintaining its common linguistic identity. BTW, I've been reading and listening to John McWhorter lately with great pleasure.

(If you end with a digression, is it a tail-digression?)

Tuesday Oct 04, 2011

JSR 292 cited by JCP for excellence

This evening in San Francisco I was deeply touched to be recognized at the 9th Annual Awards Nominations of the Java Community Process. On behalf of the JSR 292 Expert Group, I accepted an award for Most Innovative JSR, and another for leading the Expert Group.

These honors arise from a huge amount of teamwork. Together we introduced fundamentally new instructions and data types into the Java Virtual Machine, an undertaking that was both ambitious and complex. Carrying out this work required the dedicated efforts of the JSR 292 Expert Group members, our fellow engineers, corporate sponsors, and the Da Vinci Machine community. It has been and continues to be a pleasure to work alongside such excellent colleagues. Thank you all!

Let's continue to invest in Java standards, building out great computing platforms to run our future on.

Thursday Jul 07, 2011

JSR 292 launched in Java 7

We just finished graduation season for another bunch of students. (Congratulations, Bobby and Joe!) And after about four years of work, it feels to me like JSR 292 has graduated. Oracle launched Java 7 today. I spoke on 292; here are my slides. The webcast will be posted on the Java 7 launch site in a few days.

Thanks, everyone, for your help!

Monday Apr 04, 2011

JSR 292 support in b136

The OpenJDK7 build b136 was recently released for download. This is an important step forward for JSR 292 because the JVM now supports the new package name, java.lang.invoke. Up until recently, the package has been java.dyn, but we changed the name just before the JSR 292 Public Review.

To minimize test failures, we are using a multi-phase process for the API changes:

  1. b130: 7017414 in langtools: Release javac which supports both java.dyn and java.lang.invoke. This allows us to compile either version of JDK code for JSR 292.
  2. b135: 6839872, 7012648 in JVM: Release JVM which supports both java.dyn and java.lang.invoke. This allows us to run either version of JDK code for JSR 292.
  3. b136: 6839872, 7012648 in JDK: Release JDK which supports only java.lang.invoke. This is the JDK code for JSR 292. Here is a preview from mlvm.
  4. b137 (approx): 6817525 in JVM: Turn on JSR 292 functionality by default. This will allow the JVM to support JSR 292 “out of the box.”
  5. b137 (approx): 6981791 in JVM: Release cleaned-up JVM, purging all java.dyn references. Will also include rename support for MethodHandle.invoke.
  6. b137 (approx): 7028405 in langtools: Release cleaned-up javac, purging all java.dyn references.

As the API slowly adjusts under the Public Review, there will be a few additional changes. Here are the ones which are planned or at least under consideration:

  • rename MethodHandle.invokeGeneric to MethodHandle.invoke (the EG has decided on this one)
  • rename or remove the wrapper method MethodHandles.asInstance (possible rename is MethodHandleProxies.asSingleMethodInterface)
  • Possible finality of some classes and methods (to inhibit subclassing and overrides). Might affect SwitchPoint, ConstantCallSite, other call sites, ClassValue.get, ClassValue.remove, etc.
  • Allow ConstantCallSite subclasses to self-bind (i.e., construct with an implicit mh=mh.bindTo(this)).
  • Add non-varargs overloadings to some varargs methods (for efficiency on simple systems). (Could affect insertArguments, dropArguments, filterArguments. Cf. methodType factories.)

There have been very good discussions about JSR 292 on the mlvm-dev and jvm-languages mailing lists, as well as numerous comments from other sources. Since the Public Review, I have updated the working draft in the OpenJDK sources. You can see the current javadoc for JSR 292 via the JDK 7 download page. Here are the components to the JDK7 documentation:

We will continue to update the documentation during and shortly after the Public Review period. Please continue to experiment with the APIs and to share your experiences.

(Note to JDK7 port users: The link to the preview JAR for b136, in the first list above, allows you to run the current JSR 292 under the b135 JVM. This is a spin from the mlvm patch repository of just the JSR 292 classes. If you put it on your boot class path on a b135 JVM, and if you are daring and lucky, you can preview the b136 functionality. This may be useful if you have a b135-level JVM in one of the porting projects, such as the BSD port.)

Tuesday Feb 08, 2011

JSR 292 formal Public Review

If all goes well, there will be a sixty-day formal Public Review of the specification of JSR 292. I expect this to start in about ten days. (Update: The Public Review period is 2/16/2011 through 4/18/2011. The JDK7 implementation appeared in b136.)

For almost two years, I have been keeping an updated preview of the specification here. This is simply a spin of the Javadoc derived from the current patch set in the Da Vinci Machine Project. I have also frozen a copy of this preview, as of the Public Review, here.

This has has been a long time coming. In some ways, it is sad that it has taken so long. (Four years, yikes!) On the other hand, some very exciting improvements have place in recent months, since last year's JVM Language Summit. JSR 292 has not slowed the JDK 7 process, but where JDK 7 has been slowed for other reasons (such as the Oracle acquisition) I have aspired to use the extra time to make JSR 292 even more mature. In that light, I am very glad that the specification has benefited from those extra months of development. The first working implementation of August, 2008 was pretty good (as an API design), but what we have today is far better.

Here are some recent major changes leading up to Public Review:

  • The package for JSR 292 is no longer java.dyn. It is java.lang.invoke.
  • There is a much simpler variable-arity hook. The withTypeHandler method is gone; now we have asVarargsCollector. Varargs method handles are fully integrated with Java's "varargs" feature.
  • There is clear documentation about how compilers are to recognize that the invokeExact and invokeGeneric methods require special processing.
  • The process for decoding a SAM object into a method handle is based on static methods instead of an interface.
  • MethodType is serializable, like Class.
  • There is a documented SecurityManager API for checking reflective method handle lookups.
  • Exceptions produced by method handle lookups are documented and standardized.

And here are a few of the many changes in presentation:

  • There are @throws specifications for all the corner cases of bad arguments.
  • CallSite and its key methods (getTarget, setTarget, dynamicInvoker) are all abstract.
  • The introductory texts for the major classes have been reworked and the examples rechecked.
  • Obsolete, deprecated, and provisional features have been removed. There are some "historic notes" to acknowledge their previous presence.
  • The term "signature" has been replaced by the term "type descriptor" in most places, for compatibility with the JLS. (Exception: The special term "signature polymorphic" is retained.)

A complete (verbose) account of the differences since my last announcement may be found here. As noted in a recent announcement, this specification is not fully implemented yet. Specifically, the package renaming will take a few weeks of maneuvering; this process has already begun with a modification to javac.

Thursday Dec 23, 2010

123, go!

As of today, 12/23, the OpenJDK 7 project just released build b123. This build contains many changes, including an up-to-date version of the JSR 292 API. If you have been experimenting with JSR 292, you’ll want to try this build. See the javadoc for all the gory details.

Here are some of the differences you might notice:

  • The class CallSite is now abstract, causing an InstantiationError if you try to make one directly, or a problem with a super call if you make a subclass. This breaks a bunch of old code, including mine. The Expert Group decided to refactor CallSite into three subclasses: MutableCallSite, ConstantCallSite, and VolatileCallSite. To fix your code, replace CallSite by one of the subclasses, probably MutableCallSite.
  • The invokedynamic instruction should point to a constant pool entry of type 18, not 17 (a tag that is now deprecated) or 12 (the name-and-type required by the Early Draft Review). The first component of this constant pool entry must be an index into the new attribute BootstrapMethods. For the moment, tag 17 is supported, under the JVM option -XX:+AllowTransitionalJSR292.
  • The class java.dyn.Linkage is deprecated and going away. In particular, registerBootstrapMethod is going away, because the class file format now support static, per-instruction registration, via the BootstrapMethods attribute.
  • The class java.dyn.JavaMethodHandle is gone. Use Lookup.findVirtual (cached in a private static final) and MethodHandle.bindTo to convert a random object to a method handle. Use MethodHandles.asInstance to convert a method handle to a random SAM interface.
  • Direct syntax support for generating invokedynamic instructions in Java code will not be part of JDK 7. (Sorry! Use indify instead.) Exotic identifiers are also out for now, as is the annotation-based BSM specification. We will continue to experiment with these things in the mlvm repository, and some may be adopted into the Lambda project. In fact, we hope that Lambda will integrate method handles somehow, at least providing a way to construct a method handle from a literal expression like Object#toString.
  • Some of the documented argument conversions for method handles are not well supported, especially those involving unboxing or boxing, or primitive widening or narrowing. You can run into this easily by passing a primitive value to invokeGeneric. As a workaround, try casting the offending primitive to Object at the call site. This is bug 6939861.
  • Some of the method handle combinators (aka. transformers) will fail if you work with more than about 10 arguments at a time. This is bug 6983728.
  • Your item here: If you notice other changes that disturb your code, please let me know, so I can adjust this list.

The indify tool has been updated to support the new formats. Its default output uses non-transitional formats. An earlier version of indify also appears in the unit test code for OpenJDK.

It is still the case that you must pass the following options to the JVM to use JSR 292: -XX:+UnlockExperimentalVMOptions -XX:+EnableInvokeDynamic.

Here are a few more changes that are very likely in the near future:

  • The JVM flags -XX:+EnableInvokeDynamic and -XX:+EnableMethodHandles will be turned on by default. (This is tracked by bug 6817525.)
  • The JVM flag -XX:+AllowTransitionalJSR292 will be turned off by default, causing constant pool tag 17 to be rejected.
  • The class java.dyn.Switcher will be renamed java.dyn.SwitchPoint.
  • The constant and identity functions of java.dyn.MethodHandles may change, losing the overloadings which take MethodType arguments.
  • The variable-arity “hook” for MethodHandle, withTypeHandler is controversial among JVM implementors and may be adjusted or deleted. (To see what variable arity means for method handles, see the method arityOverload around line 1423 in SIOC.java.)
  • Any remaining stray sun.dyn.\* names in the API are going away, of course. MethodHandle will not have a superclass. (Nor will it have user-callable constructors for building subclasses.)
  • A few items are marked “provisional” in the javadoc; these could still change also.

At year’s end I am deeply grateful to the JSR 292 Expert Group, intrepid early adopters (Dan H., Rémi F., Charlie N., Fredrik O., Jim L., Sundar!), mad scientists (Thomas W., Lukas S., Hiroshi Y.!), conference propagandists (Christian T., Brian G., Alex B., Chanwit K.!) and the whole MLVM community for their enthusiastic, deeply technical contributions to the future of the JVM. Thank you all for the excellent journey so far.

Please accept my personal warm best wishes for a merry Christmas and a happy New Year.

Tuesday Dec 14, 2010

Scheme in one class

One of the secondary design goals of JSR 292 is to give dynamic language implementors freedom to delay bytecode generation until an opportune moment. In other words, we want to encourage mixed-mode execution, bouncing gracefully between interpreted and compiled code... In order to explore this proposition, I decided to build a small Scheme interpreter which could execute interesting programs without dynamically spinning bytecodes. I chose Scheme mainly because of its attractive simplicity (though it has grown over the years). I also want to pay homage to an old exploit called Scheme in One Definition, in which George Carrette coded up a Scheme interpreter in a single file of C code. It is clearly time for Scheme in One Class. [Read More]

Monday Nov 15, 2010

a modest tool for writing JSR 292 code

I have hacked together a classfile transformer, named "indify", which is able to pattern-match some byte-compiled Java programs, and transform them into equivalent programs which use JSR 292 bytecode enhancements, notably invokedynamic... The code is in a Mercurial repository on Kenai.com, all in a single file of less than 1500 lines. [Read More]

Saturday Oct 02, 2010

larval objects in the VM

Or, the ascent from squishy to crunchy.

I want to talk about initialization of immutable data structures on the JVM. But first I want to note that, in good news for Latin lovers (and those with Latin homework), Google has made Latin translation available ad stupor mundi.

That has nothing to do with Java, but it reminds me of a Java design pattern that deserves to be unmasked and recognized: The larval stage. The idea is that every Java data structure goes through a construction phase during which it is incomplete, and at some point becomes mature enough to publish generally. Objects enter the adult stage when their constructor completes. Or, an immutable object’s contents can be built up incrementally, using a builder pattern. In order to make this work better in more cases, I want to give the builder object fully privileged access to the object in its larval state. And I want to these full privileges to extend to the initialization of finals, a privilege which is currently given only to constructors. [Read More]

Wednesday Sep 22, 2010

JavaOne in 2010

This week I gave two talks at JavaOne. Here are my slides. On Monday Brian Goetz and I talked (again, as in 2009) about the future of the JVM in One VM, Many Languages. The room was full of attentive listeners.

On Tuesday evening, I talked about an experiment I did last December combining Rhino with invokedynamic. The talk was Great Thundering Rhinos! (an expedition into JavaScript optimization). That time, the hour was late and the room was not at all full. But the listeners were still attentive. (Thanks, guys!)[Read More]

Wednesday Aug 04, 2010

after the deluge: how to tidy an overflow

Joe Darcy has posted a fine note on the problem of managing integer overflow in Java. This was raised again last week at the JVM Language Summit. Joe’s blog has already attracted some good comments. Here is my comment, which has inflated itself into a blog entry of its own.

The problem is to take two machine words, do a common arithmetic operation on them, and obtain the arithmetically correct results. In particular, it means that at least one bit must be collected besides the machine word of non-overflowing result data. For example (as Joe notes), the extra bit required to report overflow could be implied by an exceptional return. In any case, there are a couple of degrees of freedom in this problem: What we will use the extra bit(s) for, and how to deliever it (or them). After examining a couple of half measures, I will enumerate about ten ways to return normally with a pair of machine words. [Read More]

About

John R. Rose

Java maven, HotSpot developer, Mac user, Scheme refugee.

Once Sun and present Oracle engineer.

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today