Tuesday Nov 10, 2015

Dynamic linker API for the Java platform (JEP 276)

JEP 276 defines dynamic linker API for Java. This JEP provides "facility for linking high-level operations on objects such as "read a property", "write a property", "invoke a callable object", etc., expressed as names in INVOKEDYNAMIC call sites. Provide a default linker for the usual semantics of these operations on plain Java objects, as well as a facility for installing language-specific linkers.

Nashorn JavaScript engine already uses "dynalink" library for linking properties, indexed access, calls on it's (script) objects as well as "foreign"/"host" Java objects (POJOs). With JEP-276, dynalink is exposed as a public (JDK specific) API in the java9 module named "jdk.dynalink".

Right now, the source code for JEP-276 lives in "jdk9 sandbox" OpenJDK repository (http://hg.openjdk.java.net/jdk9/sandbox) in the branched named "JEP-276-branch". This will eventually go into jdk9 repository. If you want to play with "dynalink" API, you can check out this forest and build "JEP-276-branch"

To play with any Java API, these days I use "jshell" (jshell) tool. The following is a jshell repl sample to demonstrate dynalink API from Java code.

import java.lang.invoke.*
// dynalink API lives in these packages
import jdk.dynalink.*
import jdk.dynalink.support.*

// dynamic 'operation' for a callsite. 'length' property
Operation op = new NamedOperation(StandardOperation.GET_PROPERTY, "length")

// method type of operation to be linked - length is 'int' value
MethodType mt = MethodType.methodType(int.class, Object.class)

// callsite descriptor
CallSiteDescriptor desc = new CallSiteDescriptor(MethodHandles.publicLookup(),
     op, mt)

// callsite
SimpleRelinkableCallSite cs = new SimpleRelinkableCallSite(desc)

// create a linker factory
DynamicLinkerFactory fac = new DynamicLinkerFactory()
// create dynalink linker
DynamicLinker linker = fac.createLinker()

// link the callsite

// invoke it!
printf("array size %d\n", (int)cs.getTarget().invoke(new String[10]))

import java.util.ArrayList

// make a list and populate two elements
ArrayList<String> al = new ArrayList<>()

// get 'length' of array list - which is nothing but size
printf("list size %d\n", (int)cs.getTarget().invoke(al))

The above repl prints "array size 10" and "list size 2" respectively. Note that "length" property is relinked automatically to be array length and then ArrayList size (when a different "this" object is passed). This linking (and relinking) of Java objects is handled by the "java beans linker" that comes with the dynalink implementation.

Wednesday Oct 28, 2015

Underscore is a keyword in Java 9, use this script to check your code!

Underscore ("_") is a keyword in Java 9. If you use "_" as an identifier, javac of JDK 8+ issues a warning. javac of JDK 9, issues an error! To check and migrate your Java code to avoid using "_" as an identifier, you can use the following Nashorn script with jjs tool. This script just parses each .java file and reports "_" variables with filename, line and column numbers. This script uses Javac Tool API and Compiler Tree API.

File: find_underscore.js
// Usage: jjs find_underscores.js -- <directory>

if (arguments.length == 0) {
    print("Usage: jjs find_underscores.js -- <directory>");

// Java types used
var File = Java.type("java.io.File");
var Files = Java.type("java.nio.file.Files");
var StringArray = Java.type("java.lang.String[]");
var ToolProvider = Java.type("javax.tools.ToolProvider");
var Tree = Java.type("com.sun.source.tree.Tree");
var Trees = Java.type("com.sun.source.util.Trees");
var TreeScanner = Java.type("com.sun.source.util.TreeScanner");

function findUnderscores() {
    // get the system compiler tool
    var compiler = ToolProvider.systemJavaCompiler;
    // get standard file manager
    var fileMgr = compiler.getStandardFileManager(null, null, null);
    // Using Java.to convert script array (arguments) to a Java String[]
    var compUnits = fileMgr.getJavaFileObjects(Java.to(arguments, StringArray));
    // create a new compilation task
    var task = compiler.getTask(null, fileMgr, null, null, null, compUnits);
    var sourcePositions = Trees.instance(task).sourcePositions;
    // subclass SimpleTreeVisitor - to find underscore variable names
    var UnderscoreFinder = Java.extend(TreeScanner);

    var visitor = new UnderscoreFinder() {
        // override to capture information on current compilation unit
        visitCompilationUnit: function(compUnit, p) {
            this.compUnit = compUnit;
            this.lineMap = compUnit.lineMap;
            this.fileName = compUnit.sourceFile.name;

            return Java.super(visitor).visitCompilationUnit(compUnit, p);

        // override to check variable name
        visitVariable: function(node, p) {
            if (node.name.toString() == "_") {
                var pos = sourcePositions.getStartPosition(this.compUnit, node);
                var line = this.lineMap.getLineNumber(pos);
                var col = this.lineMap.getColumnNumber(pos);
                print(node + " @ " + this.fileName + ":" + line + ":" + col);

            return Java.super(visitor).visitVariable(node, p);

    for each (var cu in task.parse()) {
        cu.accept(visitor, null);

// for each ".java" file in directory (recursively).
function main(dir) {
    var totalCount = 0;
      forEach(function(p) {
          var name = p.toFile().absolutePath;
          if (name.endsWith(".java")) {

main(new File(arguments[0]));

Wednesday Sep 30, 2015

Playing with Java (java9) REPL - an example that uses nashorn engine in REPL

I love "Exploratory Programming" (https://en.wikipedia.org/wiki/Exploratory_programming) tools. If you've not already played with "Java REPL" project, you may want to do so! Check out Kulla project.

Clone kulla forest @ http://hg.openjdk.java.net/kulla and build using these commands:
  1. hg clone http://hg.openjdk.java.net/kulla/dev kulla-dev
  2. cd kulla-dev
  3. sh get_sources.sh
  4. sh configure --with-boot-jdk=/path/to/jdk1.8.0
  5. make clean images
  6. make install

Once build finishes, you can use the Java REPL tool "jshell" ( $kulla-dev/build//images/jdk/bin/jshell). I built on Windows. My jshell path is D:\src\kulla-dev\build\windows-x86_64-normal-server-release\images\jdk\bin\jshell.exe.

Now, two lines to print the squares of integers in the range [0, 100):

|  Welcome to JShell -- Version 1.9.0-internal
|  Type /help for help

-> import java.util.stream.*

-> IntStream.range(0, 100).map(x->x*x).forEach(System.out::println)

Slightly bigger repl sample that pulls weather data in JSON format, uses Nashorn script engine to parse JSON and print the statistics on it using Streams API.

File: weather.repl

import java.net.*
import java.io.*
import java.util.stream.*
import javax.script.*
import java.util.*

// URL to fetch JSON for weather data for Chennai, India
URL u = new URL(

// read text from URL
String getText(URL u) {
    StringBuilder buf = new StringBuilder();
    try (BufferedReader reader =  new BufferedReader(
        new InputStreamReader(u.openStream()))) {
    } catch (IOException exp) {
    return buf.toString();

// create nashorn engine
ScriptEngine e = new ScriptEngineManager().getEngineByName("js")

// expose weather data JSON as global variable to nashorn engine
e.put("str", getText(u))

// massage JSON using nashorn and get the max. temp values
double[] values = (double[]) e.eval(
    "Java.to(JSON.parse(str).list.map(function(val) val.temp.max), Java.type('double[]'))")

// stat on max. temp values

You can evaluate the above source as follows:

jshell  weather.repl
|  Welcome to JShell -- Version 1.9.0-internal
|  Type /help for help
-> DoubleSummaryStatistics{count=7, sum=227.160000, min=30.210000, average=32.451429, max=36.5800

Yep, Chennai is HOT! 32 C average! In Chennai, we have only three seasons - hot, hotter and the hottest :)

Wednesday Oct 21, 2009

Look Ma, javac tells me I am overriding static method wrongly!!

// File: SuperClass.java

public class SuperClass {
    public static int func() {
        return 0;

// File: SubClass.java

public class SubClass extends SuperClass {
    public static boolean func() {
        return false;

$ javac -fullversion
javac full version "1.6.0_15-b03-226"

javac SuperClass.java  SubClass.java 
SubClass.java:2: func() in SubClass cannot override func() in SuperClass; attempting to use incompatible return type
found   : boolean
required: int
    public static boolean func() { 
1 error

The subclass uses a different return type for the same named method with same argument types. So, it is overloading SuperClass.func() and the overloading SubClass.func() differs only in return type. But, I am not sure of the error message....

Tuesday Jul 14, 2009

BTrace "unsafe" mode

We kept saying BTrace is a safe, dynamic tracing tool for Java. To ensure safety, BTrace does not permit many constructs of Java language and also allows calls only to BTraceUtils class.But, not everyone likes safety -- sometimes developers want "unsafe" facility ("I know what I am doing"-style). We have got comments of this nature about BTrace - for example here and here. We have added "unsafe" mode to BTrace - you can edit "btrace" script to change -Dcom.sun.btrace.unsafe=false to -Dcom.sun.btrace.unsafe=true. In "unsafe" mode BTrace permits all Java constructs and also allows calls to arbitrary code. Like every other unsafe facility in programming, you are on own your own with this facility! You may get weird errors while instrumenting target classes and/or running transformed code. But then you know what you are doing, right?

Monday Jul 06, 2009

BTrace project moved to kenai.com

We have recently moved BTrace project to kenai.com. If you are interested in BTrace, please continue to use it and help develop it from kenai.com.

Wednesday Dec 17, 2008

Debugging option for javac and javafxc

I work on JavaFX compiler these days. The command line (debugging) option that I often use is -doe ("dump on error"). This option prints stack trace of the compiler when error message is printed. NOTE: This is an internal option and can be removed any time without notice! But, it is useful for debugging. This option works for javac as well as javafxc. When I misspelled "class" as "clas" and run compiler with -doe option, I got the stack trace below:

$ javac -doe t.java
t.java:1: class, interface, or enum expected
clas t {}
	at com.sun.tools.javac.util.Log.writeDiagnostic(Log.java:565)
	at com.sun.tools.javac.util.Log.report(Log.java:523)
	at com.sun.tools.javac.util.Log.error(Log.java:404)
	at com.sun.tools.javac.parser.Parser.reportSyntaxError(Parser.java:282)
	at com.sun.tools.javac.parser.Parser.syntaxError(Parser.java:267)
	at com.sun.tools.javac.parser.Parser.classOrInterfaceOrEnumDeclaration(Parser.java:2206)
	at com.sun.tools.javac.parser.Parser.typeDeclaration(Parser.java:2180)
	at com.sun.tools.javac.parser.Parser.compilationUnit(Parser.java:2126)
	at com.sun.tools.javac.main.JavaCompiler.parse(JavaCompiler.java:509)
	at com.sun.tools.javac.main.JavaCompiler.parse(JavaCompiler.java:550)
	at com.sun.tools.javac.main.JavaCompiler.parseFiles(JavaCompiler.java:801)
	at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:727)
	at com.sun.tools.javac.main.Main.compile(Main.java:353)
	at com.sun.tools.javac.main.Main.compile(Main.java:279)
	at com.sun.tools.javac.main.Main.compile(Main.java:270)
	at com.sun.tools.javac.Main.compile(Main.java:69)
	at com.sun.tools.javac.Main.main(Main.java:54)
1 error

Friday Aug 08, 2008

Scriptifying BTrace?

One of the issues reported with BTrace is that the trace authors have to write "verbose" code [some people say Java is "verbose"!]. In BTrace, we have to repeat the same set of imports, annotations in every btrace file and all methods have to be "public static void" and so on. Instead of inventing a new scripting language, I've added a simple C preprocessor like step in BTrace compilation. This preprocessor is based on the one in the GlueGen project. Thanks to Ken Russell for this code and for reviewing my changes specific to BTrace project. The preprocessor solution does not rule out a scripting solution in future :-) If you have nice ideas or would like contribute in this area, you are always welcome! But, I think preprocessor solution is simple and will be useful to some.

Simple Example:


To run this sample, the following command can be used:

   btrace -I . <pid-of-the-traced-process> ThreadBean.java

Without the -I option in command line, BTrace skips the preprocessor step.

Friday Jul 25, 2008

Playing with JSqueak

Squeak is a open source implementation of Smalltalk. What is JSqueak? JSqueak is a Squeak interpreter written in Java. You can download JSqueak source code and play with it. I did the following:
  • Expanded the downloaded JSqueak.zip under a directory, say c:\\JSqueak.
  • cd c:\\JSqueak
  • javac -d . \*.java
  • copy mini.image.gz JSqueak
  • java JSqueak.Main
Even if you are not going to learn Smalltalk (why?), you can have the fun of reading Smalltalk VM implemented in Java. If you are lazy and don't want to compile, you can run directly by JNLP link from http://research.sun.com/projects/JSqueak/. Inside the Squeak environment, I wrote the legendary "Hello World" :-) This is how it looks...

Friday Jul 18, 2008

BTrace and JMX

You can dyanamically attach BTrace to a Java process to inject trace code into it. BTrace client classescollect the trace output via a socket -- these client classes are used by BTrace command line client as well as VisualVM plugin for BTrace. How about attaching a JMX client to collect BTrace's trace data? Yes, it is possible to access a BTrace class's static fields as attributes of a MBean with this RFE.

There are two MBean samples in the BTrace repository. I attached both BTrace samples to a "Java2D demo" process. And then I attached VisualVM to view the Mbean registered by these BTrace samples:

  1. ThreadCounterBean.java - this sample instruments java.lang.Thread.start() method to update a counter field. This counter field is accessible by JMX clients.

  2. HistogramBean.java - this sample collects histogram of java.awt.Component objects created by an application and exposes the histogram (map) as MBean attribute.

Thursday Jun 26, 2008

BTrace aggregations - contribution from community

If you have used DTrace, chances are that you have used aggregations. For performance issues, aggregated data is often more useful than individual data points. With BTrace, aggregating data is bit painful (you have to manage using Maps explicitly). It would be nice to have DTrace-style aggregation functions such as sum, max, min and so on. Glencross, Christian M (cited in my previous entry) has contributed code changes, doc and a sample for easy-to-use aggregation facility for BTrace. Please refer to the sample code (JdbcQueries.java) that demonstrates aggregations.

Now something unrelated to aggregations, but related to BTrace : I came to know about another use-case of BTrace. See also http://blog.igorminar.com/2008/06/btrace-dtrace-for-java.html

Monday Jun 16, 2008

BTrace in the real world

In the last few weeks, I came to know about two cases of real world use of BTrace.
  1. Glencross, Christian M (his blog?) wrote about attempting to write a script to track SQL statements executed by a Java application (private email). Thanks to him for permitting me to blog about his BTrace script. I've made few formatting changes to fit his code in this blog and added few explanatory comments (staring with "VERBOSE:").
    import static com.sun.btrace.BTraceUtils.\*;
    import java.sql.Statement;
    import java.util.Map;
    import java.util.concurrent.atomic.AtomicLong;
    import com.sun.btrace.\*;
    import com.sun.btrace.annotations.\*;
     \* BTrace script to print timings for all executed JDBC statements on an event.
     \* <p>
     \* @author Chris Glencross
    public class JdbcQueries {
        private static Map preparedStatementDescriptions = newWeakMap();
        private static Map statementDurations = newHashMap();
        // VERBOSE: @TLS makes the field "thread local" -- sort of like using java.lang.ThreadLocal
        private static String preparingStatement;
        private static long timeStampNanos;
        private static String executingStatement;
         \* If "--stack" is passed on command line, print the Java stack trace of the JDBC statement.
         \* VERBOSE: Command line arguments to BTrace are accessed as $(N) where N is the command line arg position.
         \* Otherwise we print the SQL.
        private static boolean useStackTrace = $(2) != null && strcmp("--stack", $(2)) == 0;
        // The first couple of probes capture whenever prepared statement and callable statements are
        // instantiated, in order to let us track what SQL they contain.
         \* Capture SQL used to create prepared statements.
         \* VERBOSE: +foo in clazz means foo and it's subtypes. Note the use of regular expression
         \* for method names. With that BTrace matches all methods starting with "prepare". The
         \* type "AnyType" matches any Java type.
         \* @param args - the list of method parameters. args[1] is the SQL.
        @OnMethod(clazz = "+java.sql.Connection", method = "/prepare.\*/")
        public static void onPrepare(AnyType[] args) {
            preparingStatement = useStackTrace ? jstackStr() : str(args[1]);
         \* Cache SQL associated with a prepared statement.
         \* VERBOSE: By default, @OnMethod matches method entry points. Modifying with @Location 
         \* annotation to match the method return points.
         \* @param arg - the return value from the prepareXxx() method.
        @OnMethod(clazz = "+java.sql.Connection", method = "/prepare.\*/", location = @Location(Kind.RETURN))
        public static void onPrepareReturn(AnyType arg) {
            if (preparingStatement != null) {
                print("P"); // Debug Prepared
                Statement preparedStatement = (Statement) arg;
                put(preparedStatementDescriptions, preparedStatement, preparingStatement);
                preparingStatement = null;
        // The next couple of probes intercept the execution of a statement. If it execute with no-args,
        // then it must be a prepared statement or callable statement. Get the SQL from the probes up above.
        // Otherwise the SQL is in the first argument.
        @OnMethod(clazz = "+java.sql.Statement", method = "/execute.\*/")
        public static void onExecute(AnyType[] args) {
            timeStampNanos = timeNanos();
            if (args.length == 1) {
                // No SQL argument; lookup the SQL from the prepared statement
                Statement currentStatement = (Statement) args[0]; // this
                executingStatement = get(preparedStatementDescriptions, currentStatement);
            } else {
                // Direct SQL in the first argument
                executingStatement = useStackTrace ? jstackStr() : str(args[1]);
        @OnMethod(clazz = "+java.sql.Statement", method = "/execute.\*/", location = @Location(Kind.RETURN))
        public static void onExecuteReturn() {
            if (executingStatement == null) {
            print("X"); // Debug Executed
            long durationMicros = (timeNanos() - timeStampNanos) / 1000;
            AtomicLong ai = get(statementDurations, executingStatement);
            if (ai == null) {
                ai = newAtomicLong(durationMicros);
                put(statementDurations, executingStatement, ai);
            } else {
                addAndGet(ai, durationMicros);
            executingStatement = null;
        // VERBOSE: @OnEvent probe fires whenever BTrace client sends "event" command.
        // The command line BTrace client sends BTrace events when user pressed Ctrl-C 
        // (more precisely, on receiving SIGINT signal)
        public static void onEvent() {
            printNumberMap("JDBC statement executions / microseconds:", statementDurations);

    And he has expressed few wish lists for BTrace based on his experience with DTrace. We plan to investigate those items in near future.

  2. Binod P.G exchanged private e-mails about BTrace usage to track down a memory leak. Subsequently, he has blogged about the same.

Wednesday May 28, 2008

BTrace JavaOne2008 BOF slides..

I received emails asking for BTrace BOF (JavaOne-2008) slides. Better late than never... I've uploaded PDF of the slides. The BOF was mostly around demos -- slides do not contain much. But, slides have few pointers that may be useful.

Friday May 09, 2008

Thursday May 8, JavaOne

Here are the few highlights from the talks that I attended today:

TS-5428 Java Technology Meets the Real World: Intelligence Everywhere.

This talk is about pervasive computing (a.k.a ubiquitous computing) with products from Sentilla. There was an interesting demo about humidity sensor detecting changes and sending a message to a host. The "motes" run CLDC 1.1 VM (+ proprietary profile for motes). These motes have ports for sensors and actuators and some built-in sensor. There were many interesting suggestions for embedded programming for such small devices (don't allocate in inner loops and there by leading to to GC kick-in, avoid too many static fields, avoid threads whenever possible and so on).

TS-7575 Using Java Technology-Based Class Loaders to design and implementing a Java platform, Micro Edition

The basic idea is to run JavaME applications (developed for different configurations/profiles/subsets of optional packages) on top of JavaSE. The extended JavaSE classes and packages not available in specific profile or optional package set [implemented by a specific phone] should not be made available to JavaME apps targeted. i.e., only the classes available to a specific phone model should be available. If the JavaME app tries to access any other class, it should receive ClassNotFoundException. The speakers explained how to achieve such "containers" by class loader based isolation. The problem is that they seem to solve only the class access. What about extended methods and fields? For example, platform core classes on JavaSE have superset of methods [more methods on the same class available on JavaME - eg. java.util.Hashtable has more methods on JavaSE). The application classes have to bytecode analyzed and instrumented to take care of field/method accces. It seems that their current product that does not address this yet.

PAN-5542 Developing Semantic Web Applications on the Java Platform.

The discussion started with some nice demos. There was a demo with AllegroGraph RDF store, Twine, a demo with using GRDDL and getting RDF triples by a proxy server. i.e., a proxy serves does the GRDDL transformations to get RDF triples from sites [which could be stored/analyzed with RDF stores subsequently] and a demo with FOAF files. Interesting take aways from the discussion include:

  • We don't have to wait for SEMANTIC WEB with full fledged reasoners and so on. Instead, add little semantic bits to existing web (say using RDFa, GRDDL etc.) in your current web projects/pages.
  • There are many Java tools. There is need to standard Java APIs for triple store access etc. Right now, we have to write for Jena, Sesame etc. It was also felt that APIs will need to wait for more usage scenarios.
  • There are tools to expose your existing databases as virtual RDF stores -- for example: D2RQ. Probably, most of the RDF triples could come from existing data.
  • Privacy, security of the information is very important. Work needs to be done in this area.
  • Natural language processing and getting triples out of it is very hard. You may want to refer to systems like DBpedia.

Thursday May 08, 2008

Wednesday May 7, JavaOne

Today Bill, Chihiro, Jaya and I talked on Blu-ray. The talk was centered around the open source project @ http://hdcookbook.dev.java.net - a library and a set of tools to build Blu-ray discs. If you haven't checked out code/docs, you may want to checkout and play with the code. All you need is a laptop with blu-ray drive and a BD-RE disc. Optionally, for added fun you may want to have a hardware bluray player such as PS3 -- so that you can see the output on your TV rather than on a laptop. Other than the session, we also had a very informal BOF on blu-ray, OCAP etc. during the evening. It is good to meet experts in respective technologies in one place!

Other than the the blu-ray stuff, I did attend other talks/BOF. Just after Blu-ray session, I attended "TS-6000 Improving Application Performance with Monitoring and Profiling Tools" talk. This talk was about OS specific tools, JDK tools and third-party tools for profiling and monitoring. Gregg Sporar and Jaroslav Bachorik (NetBeans Profiler team) presented very well. There were many interesting questions/discussions as well. If you haven't done so already, you may want to download VisualVM. If you want bit more fun doing monitoring/profiling, you may want to check out the sources from http://visualvm.dev.java.net and build it yourself. You can build BTrace VisualVM plugin using the command:

    c:\\visualvm\\plugins>ant build

assuming you have checked out VisualVM sources under "c:\\visualvm". If you have already checked out BTrace sources under some other directory, say "c:\\btrace", you can use

    c:\\visualvm\\plugins>ant -Dbtrace.home=c:\\btrace build

To run VisualVM with all the plugins that you built, you can use the following command:

    c:\\visualvm\\plugins>ant -Dbtrace.home=c:\\btrace run

Please let us know what features you'd like to see with BTrace and/or BTrace VisualVM plugin.

I attended and liked the "Class Loader Rearchitected (BOF-6180)" BOF. If you have ever written class loaders, chances are that you have faced mysterious deadlocks or ClassCastException that said "ClassCastException: Foo cannot be cast to Foo" or having to decide between overriding loadClass and findclass, you probably should have attended this talk and gave your opinions/suggestions/ideas :-) If I understood properly, I think there was a suggestion to add class loader info. to the ClassCastException (something like class-loader-class-name@identity-HashCode style string?) so that one can quickly see it is a class loader issue. Also, there were many questions on loading classes from jar files. Looks like there will be changes to class loader API and class loading in VM for JDK 7.




« November 2015

No bookmarks in folder


No bookmarks in folder


No bookmarks in folder