Friday Jan 29, 2016

Accessing Python objects from Nashorn JavaScript

Dynamic linker API for the Java platform (JEP 276) in JDK 9 Early Access defines dynamic linker API for the Java platform. Using this API, different language runtimes implemented for the Java platform can interoperate with each other.

Nashorn JavaScript engine has been a part of JDK platform since JDK 8. Jython is an implementation of the Python programming language designed to run on the Java platform. In this blog entry, I'll demonstrate seamless access of Python objects from a Nashorn script!

Steps:

My directory structure looks as follows:

With the above files in place, we just to have run the sample with this command:

jjs jython_linker.js

The output looks as follows:

With Dynalink, different language runtimes can interoperate with each other! Have fun with mixed language programming on the Java platform!

Tuesday Jan 05, 2016

Printing parse trees using nashorn directive prologues

ECMAScript specification allows for "directive prologues" (http://www.ecma-international.org/ecma-262/5.1/#sec-14.1). A directive prologue in an instruction to the ECMAScript engine. Apart from the standard specified "use strict" directive (which makes the particular program or function "strict"), ECMAScript specification allows implementation defined directive prologues as well.

Implementations may define implementation specific meanings for ExpressionStatement productions which are not a Use Strict Directive and which occur in a Directive Prologue..

Nashorn supports few directives apart from the standard "use strict" in nashorn debug mode. All nashorn directives start with the "nashorn" word. To trigger the nashorn "debug mode", you need to set the Java System property "nashorn.debug".

Two of the nashorn specific directives are "nashorn print ast" and "nashorn print lower ast". If you use these directives at the start of the program or a function, Nashorn prints abstract syntax tree (AST) of the program or the function in the nashorn debug mode. In the non-debug (default) mode, Nashorn specific directives are just ignored. "lower ast" is the AST after nashorn processes AST for non-reachable statements, inlined "finally" blocks are so on - after "lowering" the AST!

I'll demonstrate the use of these directives with a simple "automatic semicolon insertion" example. You perhaps omit semicolons and expect the ECMAScript engine to insert semicolons for you. While this seems to work in most cases, there are corner cases.

In the following example, where do you think the semicolon is inserted?

function func() {
    // return an object literal with one property or undefined?
    // in other words, semicolon inserted after "return" or after "object literal"?
    return
    { x: 44 }
}

func()

How about printing the AST to see what happens? The same example with the nashorn directive to print AST inserted:

function func() {
    "nashorn print ast"; // ask nashorn to print AST

    return
    { x: 44 }
}

func()

jjs -J-Dnashorn.debug=true file.js
[function root { @0x057e1b0c]
    [block body { @0x0ea1a8d5]
        [statements[0..3]]
            [expression statements[0] string @0x30a3107a]
                [literal expression = '"nashorn print ast"' @0x7a765367]
            [return statements[1] = 'return' [Terminal] @0x17d677df]
            [block statements[2] { @0x78e67e0a]
                [block block { @0x0bd8db5a]
                    [statements[0..1]]
                        [label statements[0] ident @0x4b553d26]
                            [block body decimal @0x069a3d1d]
                                [statements[0..1]]
                                    [expression statements[0] decimal @0x086be70a]
                                        [literal expression = '44' @0x2a556333]

As you see, semicolon is inserted after the "return" keyword. So, effectively you've a "return;" - which is return undefined! And there is an unreachable statement after that return statement. That is not object literal expression at all. That is a block statement that has one labeled statement - the labeled statement is labels a literal expression statement!

You can print "lowered" AST - AST after a bit of processing by Nashorn - in particular, unreachable statements removed.


function func() {
    "nashorn print lower ast";

    return
    { x: 44 }
}

func()

jjs -J-Dnashorn.debug=true file.js
Lower AST for: 'func'
[function root { @0x7181ae3f]
    [block body { [Terminal] @0x1188e820]
        [statements[0..2]]
            [expression statements[0] string @0x679b62af]
                [literal expression = '"nashorn print lower ast"' @0x799d4f69]
            [return statements[1] = 'return' [Terminal] @0x290dbf45]

Now, you can see that the unreachable block statement is removed by Nashorn! Apart from having some debugging fun with Nashorn, the moral of the story is: better to stay away from automatic semicolon insertion as much as possible. Typing semicolons is not that hard really ;)

Monday Nov 30, 2015

Writing a pluggable dynalink linker and using it with Nashorn

JEP-276 (JEP 276: Dynamic Linking of Language-Defined Object Models) adds a facility for linking high-level operations on objects such as "read a property", "write a property", "invoke a callable object", etc., expressed as names in INVOKEDYNAMIC call sites. Nashorn JavaScript engine in OpenJDK uses dynalink for linking script and other objects.

JEP 276 allows user written ("pluggable") linkers to be loaded via service loader mechanism. Because of Dynalink, Nashorn also allows user-written dynalink linkers to be used along from nashorn scripts! You can write your own linker(s) and drop it as a "jar" in jjs -classpath. Your scripts can make use of those pluggable linkers to handle "special linkages".

If this sounds little abstract, you may want to look at the sample dynalink linker in nashorn OpenJDK repository - http://hg.openjdk.java.net/jdk9/dev/nashorn/rev/e9c4c02337cc. This sample adds a simple DOM Element linker DOMLinkerExporter.java. This linker allows scripts to access DOM child elements by child element tag name - rather than having to use DOM Java API to get child Node list and iterate to get the matching Element(s). dom_linker_gutenberg.js script uses "DOM linker". The script accesses child DOM elements by samply "_" followed by child element tag name.

You can imagine many other such useful dynalink linkers for your own Java classes or even JDK platform classes.How about a dynalink linker that supports easier access for SQL ResultSet objects - perhaps column names as properties? How about a dynalink linker that handles your domain object model complexity and exposes a simpler script access model -- without having to write any Java or script wrappers around it! All complexity is hidden in the linker code! No Java or script wrapper implies that easy script access does not result in reduced performance - because linking is just (mostly!) one time activity per callsite.

Friday Nov 27, 2015

Using Dynalink API with invokedynamic - a Java assembler example

Dynalink API JEP-276

Dynalink API (http://openjdk.java.net/jeps/276) provides a facility for linking high-level operations on objects such as "read a property", "write a property", "invoke a callable object", etc., expressed as names in INVOKEDYNAMIC call sites. Dynalink also provides a default linker for the usual semantics of these operations on plain Java objects (Java POJO "beans" linker), as well as a facility for installing your own language-specific linkers.

Dynalink has been used by Nashorn JavaScript engine internally starting from JDK 8. But, Dynalink is exposed as a public API starting from JDK 9. You can use dynalink to simplify invokedynamic generation when compiling your language for the Java platform.

To learn Dynalink API from Java code, you can check our dynalink API tests in nashorn repo. test/src/jdk/dynalink directory in Nashorn repo http://hg.openjdk.java.net/jdk9/dev/nashorn

There is a simple OpenJDK Java assembler based sample in nashorn repostory - $nashorn/samples/Main.asm - http://hg.openjdk.java.net/jdk9/dev/nashorn/file/376a63a077ee/samples/Main.asm. This example shows how "length" property can be linked using invokedynamic and dynalink. The Java beans linker that comes with Dynalink can link "length" property for Java arrays (array length) and Java Lists (List.size()). The example shows how "length" works for arrays and lists uniformly.

Wednesday Nov 25, 2015

Remote debugging of nashorn scripts with NetBeans IDE using "debugger" statements

You can debug nashorn scripts - even if you do not create any NetBeans project or even open the JavaScript files in NetBeans! You just need to connect NetBeans debugger to a remote Java process that evaluates Nashorn scripts!

You can use ECMAScript debugger statements to stop the script execution at the places of interest.

Start the process running your script with debugger agent

I'm using "jjs" tool to run a sample script - but you may use any java application that evaluates Nashorn scripts via javax.script API.

File: test.js

print("in test script");
debugger;
var obj = { foo: 2 };
obj.foo++;
print(obj.foo);

jjs command with debugger turned on


jjs -J-agentlib:jdwp=server=y,suspend=y,transport=dt_socket test
Listening for transport dt_socket at address: 50349

Set breakpoint in nashorn's internal DEBUGGER method

Nashorn compiles ECMAScript "debugger" statements as calls to jdk.nashorn.internal.runtime.ScriptRuntime.DEBUGGER method.

Use Debug->New Breakpoint... menu to create a new breakpoint. Add breakpoint in jdk.nashorn.internal.runtime.ScriptRuntime.DEBUGGER method.

Attach NetBeans to remote process

Use Debug->Attach Debugger... menu to attach NetBeans to the debuggee process

Getting to 'debugger' statements

Whenever script executes ECMAScript "debugger" statement, breakpoint hits into that nashorn DEBUGGER method. On pressing F8, execution control reaches the "debugger" statement in script file. NetBeans automatically opens the script file and stops at the "debugger" statement!

Tuesday Nov 10, 2015

Dynamic linker API for the Java platform (JEP 276)

JEP 276 defines dynamic linker API for Java. This JEP provides "facility for linking high-level operations on objects such as "read a property", "write a property", "invoke a callable object", etc., expressed as names in INVOKEDYNAMIC call sites. Provide a default linker for the usual semantics of these operations on plain Java objects, as well as a facility for installing language-specific linkers.

Nashorn JavaScript engine already uses "dynalink" library for linking properties, indexed access, calls on it's (script) objects as well as "foreign"/"host" Java objects (POJOs). With JEP-276, dynalink is exposed as a public (JDK specific) API in the java9 module named "jdk.dynalink".

Right now, the source code for JEP-276 lives in "jdk9 sandbox" OpenJDK repository (http://hg.openjdk.java.net/jdk9/sandbox) in the branched named "JEP-276-branch". This will eventually go into jdk9 repository. If you want to play with "dynalink" API, you can check out this forest and build "JEP-276-branch"

To play with any Java API, these days I use "jshell" (jshell) tool. The following is a jshell repl sample to demonstrate dynalink API from Java code.


import java.lang.invoke.*
// dynalink API lives in these packages
import jdk.dynalink.*
import jdk.dynalink.support.*

// dynamic 'operation' for a callsite. 'length' property
Operation op = new NamedOperation(StandardOperation.GET_PROPERTY, "length")

// method type of operation to be linked - length is 'int' value
MethodType mt = MethodType.methodType(int.class, Object.class)

// callsite descriptor
CallSiteDescriptor desc = new CallSiteDescriptor(MethodHandles.publicLookup(),
     op, mt)

// callsite
SimpleRelinkableCallSite cs = new SimpleRelinkableCallSite(desc)

// create a linker factory
DynamicLinkerFactory fac = new DynamicLinkerFactory()
// create dynalink linker
DynamicLinker linker = fac.createLinker()

// link the callsite
linker.link(cs)

// invoke it!
printf("array size %d\n", (int)cs.getTarget().invoke(new String[10]))

import java.util.ArrayList

// make a list and populate two elements
ArrayList<String> al = new ArrayList<>()
al.add("hello")
al.add("world")

// get 'length' of array list - which is nothing but size
printf("list size %d\n", (int)cs.getTarget().invoke(al))

The above repl prints "array size 10" and "list size 2" respectively. Note that "length" property is relinked automatically to be array length and then ArrayList size (when a different "this" object is passed). This linking (and relinking) of Java objects is handled by the "java beans linker" that comes with the dynalink implementation.

Wednesday Oct 28, 2015

Underscore is a keyword in Java 9, use this script to check your code!

Underscore ("_") is a keyword in Java 9. If you use "_" as an identifier, javac of JDK 8+ issues a warning. javac of JDK 9, issues an error! To check and migrate your Java code to avoid using "_" as an identifier, you can use the following Nashorn script with jjs tool. This script just parses each .java file and reports "_" variables with filename, line and column numbers. This script uses Javac Tool API and Compiler Tree API.

File: find_underscore.js
// Usage: jjs find_underscores.js -- <directory>

if (arguments.length == 0) {
    print("Usage: jjs find_underscores.js -- <directory>");
    exit(1);
}

// Java types used
var File = Java.type("java.io.File");
var Files = Java.type("java.nio.file.Files");
var StringArray = Java.type("java.lang.String[]");
var ToolProvider = Java.type("javax.tools.ToolProvider");
var Tree = Java.type("com.sun.source.tree.Tree");
var Trees = Java.type("com.sun.source.util.Trees");
var TreeScanner = Java.type("com.sun.source.util.TreeScanner");

function findUnderscores() {
    // get the system compiler tool
    var compiler = ToolProvider.systemJavaCompiler;
    // get standard file manager
    var fileMgr = compiler.getStandardFileManager(null, null, null);
    // Using Java.to convert script array (arguments) to a Java String[]
    var compUnits = fileMgr.getJavaFileObjects(Java.to(arguments, StringArray));
    // create a new compilation task
    var task = compiler.getTask(null, fileMgr, null, null, null, compUnits);
    var sourcePositions = Trees.instance(task).sourcePositions;
    // subclass SimpleTreeVisitor - to find underscore variable names
    var UnderscoreFinder = Java.extend(TreeScanner);

    var visitor = new UnderscoreFinder() {
        // override to capture information on current compilation unit
        visitCompilationUnit: function(compUnit, p) {
            this.compUnit = compUnit;
            this.lineMap = compUnit.lineMap;
            this.fileName = compUnit.sourceFile.name;

            return Java.super(visitor).visitCompilationUnit(compUnit, p);
        },

        // override to check variable name
        visitVariable: function(node, p) {
            if (node.name.toString() == "_") {
                var pos = sourcePositions.getStartPosition(this.compUnit, node);
                var line = this.lineMap.getLineNumber(pos);
                var col = this.lineMap.getColumnNumber(pos);
                print(node + " @ " + this.fileName + ":" + line + ":" + col);
            }

            return Java.super(visitor).visitVariable(node, p);
        }
    }

    for each (var cu in task.parse()) {
        cu.accept(visitor, null);
    }
}

// for each ".java" file in directory (recursively).
function main(dir) {
    var totalCount = 0;
    Files.walk(dir.toPath()).
      forEach(function(p) {
          var name = p.toFile().absolutePath;
          if (name.endsWith(".java")) {
              findUnderscores(p);
          }
      });
}

main(new File(arguments[0]));

Saturday Oct 10, 2015

Thursday Oct 08, 2015

Fun with NetLogo using Nashorn JavaScript engine

NetLogo is a multi-agent programmable modeling environment running for the Java platform. In addition to being a GUI program to edit and run NetLogo models, NetLogo supports Controlling API so that NetLogo can be invoked and controlled by a program running on JVM. The "controlling api" supports both GUI and 'headless' mode.

I used Nashorn javascript engine to use NetLogo "controlling API" from a script. I ran the bundled Maxwell's demon model in "headless mode" using the following Nashorn script:

File: nl_maxwell_demon.js

// NetLogo "controlling API" example in Nashorn

// java classes used
var HeadlessWorkspace = org.nlogo.headless.HeadlessWorkspace;
var Thread = java.lang.Thread;

// set the thread context loader for correct working of NetLogo!

Thread.currentThread().contextClassLoader = HeadlessWorkspace.class.classLoader;

// new headless workspace
var ws = HeadlessWorkspace.newInstance();

// change this as per your installation directory!
// My NetLogo installation dir is "C:\Program Files (x86)\NetLogo 5.2.1"

var NL = "C:\\Program Files (x86)\\NetLogo 5.2.1\\";
ws.open(NL + "models\\Curricular Models\\GasLab\\GasLab Maxwells Demon.nlogo");

// setup and call "go" (100 iterations)
ws.command("setup");
ws.command("repeat 100 [ go ]");

// print average left and right speeds
print("average speed (left) = " + ws.report("avg-speed-left"));
print("average speec (right) = " +ws.report("avg-speed-right"));

// We're done!
ws.dispose();

I used the jjs nashorn shell command to run the above script:


$ jjs  -cp "C:\\Program Files (x86)\\NetLogo 5.2.1\\NetLogo.jar" nl_maxwell_demon.js
average speed (left) = 8.712072244000128
average speec (right) = 9.08120463863075

Saturday Oct 03, 2015

Standalone JavaFX FXML app with nashorn scripts for event handling - no java code!

In nashorn openjdk repository http://hg.openjdk.java.net/jdk9/dev/nashorn, there is a script called "fxmlrunner.js" in the "samples" directory. This script can accept any FXML file as command line argument and "run it". If that FXML file has event handlers written in (nashorn) javascript, then we can have complete runnable application without having to write any Java code!

When I came across this FXML example https://github.com/halcat0x15a/calculator/blob/master/src/calculator/calculator.fxml, I wanted to replace Java event handling + actual calculator implementation with nashorn scripts. The "ported" FMXL+nashorn script app. uses nashorn scripts for action handlers and uses "eval" to implement calculator back-end.

Calculator App in FXML + nashorn scripts

Download these files: Now, you can the app with the following command:

jjs -fx fxmlrunner.js -- calc.fxml

You should see a calculator app window that looks like this:

Wednesday Sep 30, 2015

Playing with Java (java9) REPL - an example that uses nashorn engine in REPL

I love "Exploratory Programming" (https://en.wikipedia.org/wiki/Exploratory_programming) tools. If you've not already played with "Java REPL" project, you may want to do so! Check out Kulla project.

Clone kulla forest @ http://hg.openjdk.java.net/kulla and build using these commands:
  1. hg clone http://hg.openjdk.java.net/kulla/dev kulla-dev
  2. cd kulla-dev
  3. sh get_sources.sh
  4. sh configure --with-boot-jdk=/path/to/jdk1.8.0
  5. make clean images
  6. make install

Once build finishes, you can use the Java REPL tool "jshell" ( $kulla-dev/build//images/jdk/bin/jshell). I built on Windows. My jshell path is D:\src\kulla-dev\build\windows-x86_64-normal-server-release\images\jdk\bin\jshell.exe.

Now, two lines to print the squares of integers in the range [0, 100):


D:\src\kulla-dev\build\windows-x86_64-normal-server-release\images\jdk\bin>jshell
|  Welcome to JShell -- Version 1.9.0-internal
|  Type /help for help

-> import java.util.stream.*

-> IntStream.range(0, 100).map(x->x*x).forEach(System.out::println)

Slightly bigger repl sample that pulls weather data in JSON format, uses Nashorn script engine to parse JSON and print the statistics on it using Streams API.

File: weather.repl

import java.net.*
import java.io.*
import java.util.stream.*
import javax.script.*
import java.util.*

// URL to fetch JSON for weather data for Chennai, India
URL u = new URL(
  "http://api.openweathermap.org/data/2.5/forecast/daily?q=Chennai&mode=json&units=metric&cnt=7")

// read text from URL
String getText(URL u) {
    StringBuilder buf = new StringBuilder();
    try (BufferedReader reader =  new BufferedReader(
        new InputStreamReader(u.openStream()))) {
        reader.lines().forEach(l->buf.append(l));
    } catch (IOException exp) {
        exp.printStackTrace();
    }
    return buf.toString();
}

// create nashorn engine
ScriptEngine e = new ScriptEngineManager().getEngineByName("js")

// expose weather data JSON as global variable to nashorn engine
e.put("str", getText(u))

// massage JSON using nashorn and get the max. temp values
double[] values = (double[]) e.eval(
    "Java.to(JSON.parse(str).list.map(function(val) val.temp.max), Java.type('double[]'))")

// stat on max. temp values
printf(DoubleStream.of(values).summaryStatistics().toString())


You can evaluate the above source as follows:

jshell  weather.repl
|  Welcome to JShell -- Version 1.9.0-internal
|  Type /help for help
-> DoubleSummaryStatistics{count=7, sum=227.160000, min=30.210000, average=32.451429, max=36.5800
00}

Yep, Chennai is HOT! 32 C average! In Chennai, we have only three seasons - hot, hotter and the hottest :)

Tuesday Sep 22, 2015

Playing with Nerdamer symbolic math JS library with Nashorn jjs

Using third-party JavaScript libraries with nashorn javascript shell. Using Nerdamer Symbolic math library with nashorn.[Read More]

Sunday Dec 13, 2009

Graphs, GXL, dot and Graphviz

Sometimes you may want to quickly generate graphs programmatically and view/analyze those. Examples include, inheritance/type relation diagrams of an object oriented program, function call graphs and any other domain specific graphs (reporting chain of your organization chart for example). I find GXL very useful for this. GXL stands for Graph eXchange Language. It is a simple XML format to specify graphs. A simple graph stating that "JavaFX" language is related to "Java" language is as follows:

File: Test.gxl

<gxl>

<!-- edgemode tells this is directed or undirected graph -->
<graph id="langs" edgemode="directed">

<!-- write your nodes -->
<node id="java"/>
<node id="javafx"/>

<-- now write your edges -->
<edge from="java" to="javafx"/>

</graph>
</gxl>

You can also add number of "attributes" to nodes and edges - like color of the edge, style of the edge and so on. For example, "red" color can be specified for an edge as follows:


<edge from="java" to="javafx">
  <attr name="color"><string>red</string></attr>
</edge>

Now that we have written a simple graph with two nodes and a single edge between them, we may want to view it. There are number of tools/libraries to view GXL documents -- I've used Graphviz. Graphviz displays it's own native format called ".dot". Graphviz comes with a set of command line tools. One such tool is "gxl2dot", which as you'd have guessed, can be used to convert a .gxl file to a .dot file.


    gxl2dot Test.gxl > Test.dot

Once converted the .dot file can be opened in Graphviz GUI and we can export it to .pdf/.jpg/.png and so on. This way you can email the graphs to others and/or publish in your blogs/webpages easily.

The converted .pdf file for the above simple graph is here: test.pdf

I've used GXL graphs in a recent debugging tool related to JavaFX compiler. More on that later...

Wednesday Oct 21, 2009

Look Ma, javac tells me I am overriding static method wrongly!!


// File: SuperClass.java

public class SuperClass {
    public static int func() {
        return 0;
    }
}


// File: SubClass.java

public class SubClass extends SuperClass {
    public static boolean func() {
        return false;
    }
}

$ javac -fullversion
javac full version "1.6.0_15-b03-226"

javac SuperClass.java  SubClass.java 
SubClass.java:2: func() in SubClass cannot override func() in SuperClass; attempting to use incompatible return type
found   : boolean
required: int
    public static boolean func() { 
                          \^
1 error

The subclass uses a different return type for the same named method with same argument types. So, it is overloading SuperClass.func() and the overloading SubClass.func() differs only in return type. But, I am not sure of the error message....

Thursday Sep 17, 2009

"helloworld" with Maxine JVM on Mac

Last night, I listened to this nice podcast on Maxine JVM from Software Engineering Radio. Maxine is a Java Virtual Machine implemented in the Java programming language.

I wanted to try out Maxine on my MacBook Pro running Mac OS X 10.5.8. I followed these steps to try it out:

  • Get the Maxine sources
    
        hg clone https://kenai.com/hg/maxine~maxine maxine
    
    
  • Setup the environment
    
        export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home
        export JUNIT4_CP=<full-path-of junit-4.4.jar>
    
    
  • Build Maxine
    
       cd $MAXINE_HOME/bin
       ./max build
       ./max image   
    
    
  • Run the "helloworld"
    
        ./max helloworld
    
    

Wow! Maxine VM printed "Hello World!". Now, it is time to explore Maxine's inspector and other cool stuff ...

About

sundararajan

Search

Archives
« February 2016
SunMonTueWedThuFriSat
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
     
       
Today
Bookmarks
Links

No bookmarks in folder

Blogroll

No bookmarks in folder

News

No bookmarks in folder