Monday Apr 20, 2015

Flexible JSON with Nashorn Parser API (JDK9)

[Posted by Sundar] 

I recently came across Hjson - "the Human JSON - A configuration file format that caters to humans and helps reduce the errors they make". See also: http://hjson.org.

I wanted to see if I can use Nashorn Parser API (jdk9) to support similar flexible JSON extension with Nashorn. In the following FlexiJSON.parse implementation, Nashorn Parser API is used to validate that the extendable flexi JSON is "data only" (That is, no executable code) and then 'eval'uated to make an object out of it.

FlexiJSON allows the following:

  • single and mutliple line comments anywhere
  • non-quoted property names and values
  • regexp literal values
  • omitting trailing comma

When nashorn -scripting mode is enabled, FlexiJSON supports these as well:

  • shell style # comments
  • multiple line (Unix heredoc style) string values
File: flexijson.js
/*
 * Copyright (c) 2015, Oracle and/or its affiliates. All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 *   - Redistributions of source code must retain the above copyright
 *     notice, this list of conditions and the following disclaimer.
 *
 *   - Redistributions in binary form must reproduce the above copyright
 *     notice, this list of conditions and the following disclaimer in the
 *     documentation and/or other materials provided with the distribution.
 *
 *   - Neither the name of Oracle nor the names of its
 *     contributors may be used to endorse or promote products derived
 *     from this software without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
 * IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
 * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
 * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
 * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
 * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
 * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */

"use strict";

function FlexiJSON() {}

// helper to locate Nashorn Parser API classes
FlexiJSON.treeType = function(name) {
    return Java.type("jdk.nashorn.api.tree." + name);
}

// Nashorn Parser API classes used
FlexiJSON.ArrayLiteral = FlexiJSON.treeType("ArrayLiteralTree");
FlexiJSON.ExpressionStatement = FlexiJSON.treeType("ExpressionStatementTree");
FlexiJSON.ObjectLiteral = FlexiJSON.treeType("ObjectLiteralTree");
FlexiJSON.RegExpLiteral = FlexiJSON.treeType("RegExpLiteralTree");
FlexiJSON.Literal = FlexiJSON.treeType("LiteralTree");
FlexiJSON.Parser = FlexiJSON.treeType("Parser");
FlexiJSON.SimpleTreeVisitor = FlexiJSON.treeType("SimpleTreeVisitorES5_1");

// FlexiJSON.parse API

FlexiJSON.parse = function(str) {
    var parser = (typeof $OPTIONS == "undefined")? 
        FlexiJSON.Parser.create() :
        FlexiJSON.Parser.create("-scripting");

    // force the string to be an expression by putting it inside (, )
    str = "(" + str + ")";
    var ast = parser.parse("", str, null);
    // Should not happen. parse would have thrown syntax error
    if (!ast) {
        return undefined;
    }

    // allowed 'literal' values in flexi JSON
    function isLiteral(node) {
        return node instanceof FlexiJSON.ArrayLiteral ||
            node instanceof FlexiJSON.Literal ||
            node instanceof FlexiJSON.ObjectLiteral ||
            node instanceof FlexiJSON.RegExpLiteral;
    }

    var visitor;
    ast.accept(visitor = new (Java.extend(FlexiJSON.SimpleTreeVisitor)) {
         lineMap: null,

         throwError: function(msg, node) {
             if (this.lineMap) {
                 var pos = node.startPosition;
                 var line = this.lineMap.getLineNumber(pos);
                 var column = this.lineMap.getColumnNumber(pos);
                 // we introduced extra '(' at start. So, adjust column number
                 msg = msg + " @ " + line + ":" + (column - 1);
             }
             throw new TypeError(msg);
         },

         visitLiteral: function(node, extra) {
             print(node.value);
         },

         visitExpressionStatement: function(node, extra) {
             var expr = node.expression;
             if (isLiteral(expr)) {
                 expr.accept(visitor, extra);
             } else {
                 this.throwError("only literals can occur", expr);
             }
         },

         visitArrayLiteral: function(node, extra) {
             for each (var elem in node.elements) {
                 if (isLiteral(elem)) {
                     elem.accept(visitor, extra);
                 } else {
                     this.throwError("only literal array element value allowed", elem);
                 }
             }
         },

         visitObjectLiteral: function(node, extra) {
             for each (var prop in node.properties) {
                 if (prop.getter != null || prop.setter != null) {
                     this.throwError("getter/setter property not allowed", node);
                 }

                 var value = prop.value;
                 if (isLiteral(value)) {
                     value.accept(visitor, extra);
                 } else {
                     this.throwError("only literal property value allowed", value);
                 }
             }
         },

         visitCompilationUnit: function(node, extra) {
             this.lineMap = node.lineMap;
             var elements = node.sourceElements;
             if (elements.length > 1) {
                 this.throwError("more than one top level expression", node.sourceElements[1]);
             } 
             var stat = node.sourceElements[0];
             if (! (stat instanceof FlexiJSON.ExpressionStatement)) {
                 this.throwError("only one top level expresion allowed", stat);
             }
             stat.accept(visitor, extra);
         },
    }, null);

    // safe to eval given string as flexi JSON!
    return eval(str);
}

Sample simple usage of FlexiJSON.parse would be as follows:

File: fjson.js
var obj = FlexiJSON.parse(<<EOF
// this is a comment
{
    foo: 23,
    bar: [ 34, 454, 54,],
    // inline comment here

    /** multi line
        comments are fine too! */

    # shell style line comment is fine!

    regex: /gdfg/i, // regexp literal

    str: <<END
Multiple line strings via nashorn 
-scripting mode extension as well
END
}
EOF)

print(obj.foo);
print(obj.bar);
print(obj.regex);
print(obj.str);

Tuesday Feb 10, 2015

JavaScript Stored Procedures and Node.js Applications with Oracle Database 12c

This is a reposting of an article written by Kuassi Mensah JavaScript Stored Procedures and Node.js Applications with Oracle Database 12c

Introduction

Node.js and server-side JavaScript are hot and trendy; per the latest “RedMonk Programming Languages Rankings[1], JavaScript and Java are the top two programming languages. For most developers building modern Web, mobile, and cloud based applications, the ability to use the same language across all tiers (client, middle, and database) feels like Nirvana but the IT landscape is not a green field; enterprises have invested a lot in Java (or other platforms for that matter) therefore, the integration of JavaScript with it becomes imperative. WebSockets and RESTful services enable loose integration however, the advent of JavaScript engines on the JVM (Rhino, Nashorn, DynJS), and Node.js APIs on the JVM (Avatar.js, Nodyn, Trireme), make possible and very tempting to co-locate Java and Node applications on the same JVM. 

This paper describes the steps for running JavaScript stored procedures[2] directly on the embedded JVM in Oracle database 12c and the steps for running Node.js applications on the JVM against Orace database 12c, using Avatar.js, JDBC and UCP.          

JavaScript and the Evolution of Web Applications Architecture               

At the beginning, once upon a time, long time ago, JavaScript was a browser-only thing while business logic, back-end services and even presentations where handled/produced in middle-tiers using Java or other platforms and frameworks. Then JavaScript engines (Google’s V8, Rhino) leave the browsers and gave birth to server-side JavaScript frameworks and Node.js.

Node Programming Model

Node.js and similar frameworks bring ease of development rapid prototyping, event-driven, and non-blocking programming model[3] to JavaScript. This model is praised for its scalability and good enough performance however, unlike Java, Node lacks standardization in many areas such as database access i.e., JDBC equivalent, and may lead, without discipline, to the so called “callback hell[4]”.

Nonetheless, Node is popular and has a vibrant community and a large set of frameworks[5].

Node Impact on Web Applications Architecture

With the advent of Node, REST and Web Sockets, the architecture of Web applications has evolved into 

(i) plain JavaScript on browsers (mobiles, tablets, and desktops); 

(ii) server-side JavaScript modules (i.e., Node.js, ORM frameworks) interacting with Java business logic and databases.

The new proposal for Web applications architecture is the integration of Node.js and Java on the JVM.  Let’s discuss the enabling technologies: JavaScript engine on the JVM and Node API on the JVM and describe typical use cases with Oracle database 12c.  

JavaScript on the JVM

Why implement a JavaScript engine and run JavaScript on the JVM? For starters, i highly recommend Mark Swartz ‘s http://moduscreate.com/javascript-and-the-jvm/ and Steve Yegge’s  http://steve-yegge.blogspot.com/2008/06/rhinos-and-tigers.html blog posts.

In summary, the JVM brings (i) portability; (ii) manageability; (iii) Java tools; (iv) Java libraries/technologies such as JDBC, Hadoop; and (v) the preservation of investments in Java. 

There are several implementations/projects of Java based JavaScript engines including Rhino, DynJS and Nashorn.

Rhino

First JavaScript engine entirery written in Java; started at NetScape in 1997 then, became an open-source Mozilla project [6]. Was for quite some time the default JavaScript engine in Java SE, now  replaced by Nashorn in Java SE 8.

DynJS

DynJS is another open-source JavaScript engine for the JVM. Here is the project homepage http://dynjs.org/.

Nashorn

Introduced in Java 7 but “production” in Java 8[7], the goal of project Nashorn (JEP 174), is to enhance the performance and security of the Rhino JavaScript engine on the JVM. It integrates with javax.script API (JSR 223) and allows seamless interaction between Java and JavaScript (i.e., invoking Nashorn from Java and invoking Java from Nashorn).

To illustrate the reach of Nashorn on the JVM and the interaction between Java and JavaScript, let’s run some JavaScript directly on the database-embedded JVM in Oracle database 12c.

JavaScript Stored Procedures with Oracle database 12c Using Nashorn

Why would anyone run JavaScript in the database? For the same reasons you’d run Java in Oracle database. Then you might ask why run Java in the database, in the first place? As discussed in my book[8], the primary motivations are: 

(i) reuse skills and code, i.e., which programming languages are your new hire knowledgeable of or willing to learn; 

(ii) avoid data shipping[9] i.e., in-place processing of billions of data/documents; 

(iii) combine SQL with foreign libraries to achieve new database capability thereby extending SQL and the reach of the RDBMS, e.g., Web Services callout, in-database container for Hadoop[10]

Some developers/architects prefer a tight separation between the RDBMS and applications therefore, no programming language in the database[11]but there are many pragmatic developers/architects who run code near data, whenever it is more efficient than shipping data to external infrastructure.

Co-locating functions with data on the same compute engine is shared by many programming models such as Hadoop. With the surge and prevalence of Cloud computing, RESTful service based architecture is the new norm. Data-bound services can be secured and protected by the REST infrastructure, running outside the RDBMS. Typical use case: a JavaScript stored procedures service would process millions/billions of JSON documents in the Oracle database and would return the result sets to the service invoker.

To conclude, running Java, JRuby, Python, JavaScript, Scala, or other programming language on the JVM in the database is a sound architectural choice. The best practices consist in: (i) partitioning applications into data-bound and compute-bound modules or services; (ii) data-bound services are good candidates for running in the database; (iii) understand DEFINER’s vs INVOKER’s right[12] and grant only the necessary privilege and/or permission. 

The Steps

The following steps allow implementing JavaScipt stored procedure  running in Oracle database; these steps represent an enhancement from the ones presented at JavaOne and OOW 2014 -- which consisted in reading the JavaScript from the database file system; such approach required granting extra privileges to the database schema for reading from RDBMS file system something not recommended from security perspective. Here is a safer approach:

1. Nashorn is part of Java 8 but early editions can be built for Java 7; the embedded JavaVM in Oracle database 12c supports Java 6 (the default) or Java 7. For this proof of concept, install Oracle database 12c with Java SE 7 [13]

2. Build a standard Nashorn.jar[14]; (ii) modify the Shell code to interpret the given script name as an OJVM resource; this consists mainly in invoking getResourceAsStream() on the current thread's context class loader ; (iii) rebuild Nashorn.jar with the modified Shell

3. Load the modified Nashorn jar into an Oracle database shema e.g., HR

loadjava -v -r -u hr/ nashorn.jar

4. Create a new dbms_javascript package for invoking Nashorn’s Shell with a script name as parameter

create or replace package dbms_javascript as
  procedure run(script varchar2);
end;
/
create or replace package body dbms_javascript as
  procedure run(script varchar2) as
  language java name 'com.oracle.nashorn.tools.Shell.main(java.lang.String[])';
end;
/ 

Then call dbms_javascript,run(‘myscript.js’) from SQL which will invoke Nashorn  Shell to execute the previously loaded myscript.js .

5. Create a custom role, we will name it NASHORN, as follows, connected as SYSTEM

SQL> create role nashorn;
SQL> call dbms_java.grant_permission('NASHORN', 'SYS:java.lang.RuntimePermission', 'createClassLoader', '' );
SQL> call dbms_java.grant_permission('NASHORN', 'SYS:java.lang.RuntimePermission', 'getClassLoader', '' );
SQL> call dbms_java.grant_permission('NASHORN', 'SYS:java.util.logging.LoggingPermission', 'control', '' );

Best practice: insert those statements in a nash-role.sql file and run the script as SYSTEM

6. Grant the NASHORN role created above to the HR schema as follows (connected as SYSTEM):

SQL> grant NASHORN to HR;

7. Insert the following JavaScript code in a file e.g., database.js stored on your client machine’s (i.e., a machine from which you will invoke loadjava as explained in the next step).

This script illustrates using JavaScript and Java as it uses the server-side JDBC driver to execute a PreparedStatement to retrieve the first and last names from the EMPLOYEES table.

var Driver = Packages.oracle.jdbc.OracleDriver;
var oracleDriver = new Driver();
var url = "jdbc:default:connection:";   // server-side JDBC driver
var query ="SELECT first_name, last_name from employees";
// Establish a JDBC connection
var connection = oracleDriver.defaultConnection();
// Prepare statement
var preparedStatement = connection.prepareStatement(query);
// execute Query
var resultSet = preparedStatement.executeQuery();
// display results
     while(resultSet.next()) {
     print(resultSet.getString(1) + "== " + resultSet.getString(2) + " " );
     }
// cleanup
resultSet.close();
preparedStatement.close();
connection.close();

8. Load database.js in the database as a Java resource (not a vanilla class)

loadjava –v –r –u hr/ database.js

9. To run the loaded script

sqlplus hr/
SQL>set serveroutput on
SQL>call dbms_java.set_output(80000)
SQL>call dbms_javascript.run(‘database.js’);

The Nashorn Shell reads ‘database.js’ script stored as Java Resource from internal table; the JavaScript in its turn invokes JDBC to execute a PreparedStatement and the result set is displayed on the console. The message “ORA=29515: exit called from Java code with status 0” is due to the invocation of java.lang.Runtime.exitInternal; and status 0 means normal exit (i.e., no error). The fix is to remove that call from Nashorn. 

Node.js on the JVM

As discussed earlier, Node.js is becoming the man-in-the-middle between Web applications front ends and back-end legacy components and since companies have invested a lot in Java, it is highly desirable to co-locate Node.js and Java components on the same JVM for better integration thereby eliminating the communication overhead. There are several projects re-implementing Node.js APIs on the JVM including: Avatar.js, Nodyn, and Trireme. This paper will only discuss Oracle’s Avatar.js.

Project Avatar.js[15]

The goal of project Avatar.js is to furnish “Node.js on the JVM”; in other words, an implementation of Node.js APIs, which runs on top of Nashorn and enables the co-location of Node.js programs and Java components. It has been outsourced by Oracle under GPL license[16]. Many Node frameworks and/or applications have been certified to run unchanged or slightly patched, on Avatar.js.

There are binary distributions for Oracle Enterprise Linux, Windows and MacOS (64-bits). These builds can be downloaded from https://maven.java.net/index.html#welcome. Search for avatar-js.jar and platform specific libavatar-js libraries (.dll, .so, dylib). Get the latest and rename the jar and the specific native libary accordingly. For example: on  Linux, rename the libary to avatar-js.so; on Windows, rename the dll to avatar-js.dll and add its location to your PATH (or use -Djava.library.path=).

RDBMSes in general and Oracle database in particular remain the most popular persistence engines and there are RDBMS specific Node drivers[17] as well as ORMs frameworks. However, as we will demonstrate in the following section, with Avatar.js, we can simply reuse existing Java APIs including JDBC and UCP for database access.

Node Programming with Oracle Database using Avatar.js, JDBC and UCP

The goal of this proof of concept is to illustrate the co-location of a Node.js application, the Avatar.js library, the Oracle JDBC driver and the Oracle Universal Connection Pool (UCP) on the same Java 8 VM.

The sample application consists in a Node.js application which performs the following actions:

(i) Request a JDBC-Thin connection from the Java pool (UCP)

(ii)Create a PreparedStatement object for “SELECT FIRST_NAME, LAST_NAME FROM EMPLOYEES

(iii)Execute the statement and return the ResultSet in a callback

(iv)Retrieve the rows and display in browser on port 4000

(v) Perform all steps above in a non-blocking fashion – this is Node.js’s raison d’être. The demo also uses Apache ab load generator to simulate concurrent users running the same application in the same/single JVM instance.For the Node application to scale in the absence of asynchronous JDBC APIs, we need to turn synchronous calls into non-blocking ones and retrieve the result set via callback.

Turning Synchronous JDBC Calls into Non-Blocking Calls

We will use the following wrapper functions to turn any JDBC call into a non-blocking call i.e., put the JDBC call into a thread pool and free up the Node event loop thread.

var makeExecutecallback = function(userCallback) {
 return function(name, args){
      ...
      userCallback(undefined, args[1]);
  }
}
 function submit(task, callback, msg) {
    var handle = evtloop.acquire();
    try {    var ret = task();
               evtloop.post(new EventType(msg, callback, null, ret)); {catch{}
    evtloop.submit(r);
}

Let’s apply these wrapper functions to executeQuery JDBC call, to illustrate the concept


exports.connect = function(userCallback) {..} // JDBC and UCP settings
Statement.prototype.executeQuery = function(query, userCallback) { 
          var statement = this._statement; 
          var task = function() { 
          return statement.executeQuery(query); 
           submit(task, makeExecutecallback(userCallback), "jdbc.executeQuery"); 
}

Similarly the same technique will be applied to other JDBC statement APIs.

Connection.prototype.getConnection = function() {…}
Connection.prototype.createStatement = function() {..}
Connection.prototype.prepareCall = function(storedprocedure) {..}
Statement.prototype.executeUpdate = function(query, userCallback) {..}

Returning Query ResultSet through a Callback

The application code fragment hereafter shows how: for every HTTP request: (i) a connection is requested, (ii) the PreparedStatement is executed, and (iii) the result set printed on port 4000.

   ...
var ConnProvider = require('./connprovider').ConnProvider;
var connProvider = new ConnProvider(function(err, connection){.. });
var server = http.createServer(function(request, response) { 
  connProvider.getConn(function(name,data){..});      
  connProvider.prepStat(function(resultset) {
                while (resultset.next()) {
                   response.write(resultset.getString(1) + " --" + resultset.getString(2));
                   response.write('
');
    }
    response.write('
');
    response.end();    
}
server.listen(4000, '127.0.0.1');

Using Apache AB, we were able to scale to hundreds of simultaneous invocations of the Node application. Each instance grabs a Java connection from The Universal Connection Pool (UCP), executes the SQL statements through JDBC then return the result set via a Callbak on port 4000.

Conclusions

Through this paper, i discussed the rise of JavaScript for server-side programming and how Java is supporting such evolution; then – something we set out to demonstrate – furnished step by step details for implementing and running JavaScript stored procedures in Oracle database 12c using Nashorn as well as running Node.js applications using Avata.js, Oracle JDBC, UCP against Oracle database 12c.

As server-side JavaScript (typified by Node.js) gains in popularity it’ll have to integrate with existing components (COBOL is still alive!!). Developers, architects will have to look into co-locating JavaScript with Java, across middle and database tiers.

[1] http://redmonk.com/sogrady/2015/01/14/language-rankings-1-15/

[2] I’ll discuss the rationale for running programming languages in the database, later in this paper.

[3] Request for I/O and resource intensive components run in separate process then invoke a Callback in the main/single Node  thread, when done.

[4] http://callbackhell.com/

[5] Search the web for “Node.js frameworks

[6] https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Rhino

[7] Performance being one of the most important aspect

[8] http://www.amazon.com/exec/obidos/ASIN/1555583296

[9] Rule of thumb: when processing more than ~20-25% of target data, do it in-place, where data resides (i.e., function shipping).

[10] In-database Container for Hadoop is not available, as of this writing.

[11] Other than database’s specific procedural language, e.g.,  Oracle’s PL/SQL

[12] I discuss this in chapter 2 of my book; see also Oracle database docs.

[13] See Multiple JDK Support in http://docs.oracle.com/database/121/JJDEV/E50793-03.pdf

[14] Oracle does not furnish a public download of Nashorn.jar for Java 7; search “Nashorn.jar for Java 7”.

[15]  https://avatar-js.java.net/

[16] https://avatar-js.java.net/license.html

[17] The upcoming Oracle Node.js driver was presented at OOW 2014. 

Friday Jan 09, 2015

Using Nashorn with IntelliJ

Hi! 

The Nashorn repository already contains a Netbeans project, so that you can seamlessly work with Netbeans as your IDE when using Nashorn.

There are some tricks, for example, the IDE needs to build and access the pre-generated Nasgen classes that represent the builtin JavaScript objects: Array, Date, Number, String etc. These are part of the build step, and are generated if you just execute "ant clean jar" from the command line. Nashorn, furthermore, is now a jimage in Java 9, as part of the JDK modularization process. This makes it a bit tricky to configure an IDE in a straightforward way, so that you can transparently build and develop Nashorn. Not everyone uses Netbeans, however, so one of our third party contributors, Dmitry Alexandrov, has done a great job doing an extensive writeup on how to integrate Nashorn with IntelliJ. 

Thanks a lot Dmitry! This truly rocks. 

The blog post can be found here.

Do we have any volunteers for Eclipse? ;-)

Keep on hacking! 

Friday Dec 12, 2014

Nashorn Architecture and Performance Improvements in the Upcoming JDK 8u40 Release

Hello everyone!

We've been bad at blogging here for a while. Apologies for that. I thought it would be prudent to talk a little bit about OpenJDK 8u40 which is now code frozen, and what enhancements we have made for Nashorn.

8u40 includes a total rewrite of the Nashorn code generator, which now contains the optimistic type system.

JavaScript, or any dynamic language for that matter, doesn't provide enough compile time information to easily generate performant code. This is why, for 8u40, we have redesigned the Nashorn Type System, to produce better, faster code.  The output of the Nashorn compiler, Java byte code, is by design strongly typed and JavaScript isn't. There are significant problems translating a dynamic language AST to optimal bytecode. Attila Szegedi, Hannes Wallnöfer and myself have spent significant time researching and implementing a solution to this problem over the last year.

Background: Conservatively, when implementing a JavaScript runtime in Java, anything known to be a number can be represented in Java as a double, and everything else can be represented as an Object. This includes numbers, ints, longs and other primitive types that aren't statically provable. Needless to say, this approach leads to a lot of internal boxing, which is quite a bottleneck for dynamic language execution speed on the JVM. The JVM is very good at optimizing Java-like bytecode, and this is not Java-like bytecode. 

Type transitions in the optimistic type system

Another novelty in 8u40 is that Nashorn actually performs extensive intra-function static type analysis, so the generated code ends up being much better than in the above conservative scenario. It still can't prove all the types, though, and that's where optimistic type system comes in. Our optimistic type system works by assuming that any statically unprovable type is an int, the narrowest possible of all types. If this turns out to be wrong at runtime, e.g. we load a field from memory/scope that turned out to be an Object or a double instead, or if we perform a 32-bit addition that overflows, we will replace the executing code with a more conservative version, that is regenerated on the fly. Nashorn supports simple continuations to facilitate code replacement at runtime. This is a feature that can have multiple other uses in the future, for example going back and forth between a hypothetical interpreter layer and compiled bytecode.

We also specialize/guess how bytecode will look depending on what arguments are passed to a callsite at link time, with guards against these changing in the future.


Octane benchmark scores, normalized on JDK 8. 

This makes the bytecode that we produce (when stable) contain optimal Java types for speedy execution. The cost for this (everything has a cost) is that we spend more time generating code, which affects things like warmup slightly negatively. Warmup is the major R &D area for JDK 9, and we are confident that we can solve this, and also customize the workload we give HotSpot to compile better, so that it spends its cycles much more optimally. 

In our efforts to turn Nashorn into a generic platform/runtime for dynamic languages for the JVM, the optimistic type system integrates nicely with whatever AST Nashorn is asked to generate code for. These plans will also solidify more for future Java releases, and we hope that Nashorn eventually will turn into a kind of "LLVM for dynamic languages on the JVM". The optimistic typing problem comes back to every dynamic language. To prove this, our excellent thesis student Andreas Gabrielsson has successfully implemented almost everything required to run TypeScript on Nashorn. At the time of this writing, modules in the only large feature missing. Nashorn goes directly to bytecode from TypeScript without generating JavaScript in a mid-level compiler tier. This is pretty awesome and shows how extensible Nashorn already is. We have thought about Ruby and Groovy running in a similar environment. Nashorn TypeScript is still not open sourced, but we hope we have the chance to do it soon.

To run Nashorn with 8u40 and optimistic types, use the argument --optimistic-types=true. As we haven't had time to check in a solution to the warmup issues yet (it doesn't affect everything, but where issues remain it can be annoying), the optimistic type system is not yet the default behavior for Nashorn. This might be overly conservative, but we didn't want to break anyone's warmup unnecessarily.

We can report several orders of magnitude of performance increase on things like the Octane benchmarks suite compared to previous releases. We can also report near native or native performance on the REST and HTTP benchmarks for the Avatar project. We feel pretty good about this. In several cases, JavaScript applications on Nashorn run as fast as the same application on a native runtime such as SpiderMonkey or v8. Needless to say, we are pretty proud of our work.

Furthermore: Nashorn in JDK 8u40 also supports caching generated code and type information deduced at runtime to disk. This means that only the first iteration of a largish application with long warmup will be an issue. Consecutive runs of the same code will stabilize very quickly with the same good performance.

Please try it out and tell us what you find! If you have questions, please read the documentation and feel free to join the nashorn-dev@openjdk.java.net list and talk to us. You can also follow the Nashorn team on Twitter:

  • Jim Laskey: @wickund 
  • Sundararajan Athijegannathan: @sundararajan_a
  • Marcus Lagergren: @lagergren
  • Attila Szegedi: @asz
  • Hannes Wallnöfer: @hannesw 

We'll try to blog more here in the near future. Stay tuned and sorry for the hiatus.

Proudly yours,
The Nashorn team through 
Marcus Lagergren (dynamic language performance architect)

P.S. If you are interested in the internal workings of the new type system or more information as the JVM as a polyglot multilanguage runtime and Nashorn as its facilitating framework, here are some links to presentations on the subject, that you may find interesting:

Friday Oct 24, 2014

Porting from the Browser to Nashorn/JavaFX (Part I)

During the JavaOne Nashorn: JavaScript for the JVM session, I showed a couple examples of converting JavaScript browser examples to use Nashorn and JavaFX. The first example was using CKEditor, a rich text/HTML editor. This editor is written entirely in JavaScript using the browser DOM interface. In this case, it did not really make sense to convert all the DOM code over to FX. Instead, I used the JavaFX WebView which supports all the DOM needed.

So let's start with a simple example to bring up a WebView (run with jjs -fx -scripting example1.js);
// example1.js

var Scene = Java.type("javafx.scene.Scene");
var WebView = Java.type("javafx.scene.web.WebView");

var site = "http://www.oracle.com";
var webView = new WebView();
webView.engine.load(site);

$STAGE.title = site;
$STAGE.scene = new Scene(webView, 800, 600);
$STAGE.show();
The next example brings up a WebView with a CKEditor contained within. You can download the CKEditor toolkit from http://ckeditor.com/download. Place the ckeditor folder in your working directory.
<!-- example2.html -->

<!DOCTYPE html>
<html>
<head>
  <script src="./ckeditor/ckeditor.js"></script>
</head>
<body>
  <form id="MainForm" method="post">
    <textarea id="MainEditor">
      &lt;h1&gt;Example&lt;/h1&gt;
      &lt;p&gt;Some content for the Demo&lt;/p&gt;
    </textarea>
    <script>
      CKEDITOR.replace("MainEditor");
    </script>
  </form>
</body>
</html>

// example2.js

var Scene = Java.type("javafx.scene.Scene");
var WebView = Java.type("javafx.scene.web.WebView");

var link = "file:${$ENV.PWD}/example2.html";
var webView = new WebView();
webView.engine.load(link);

$STAGE.title = "CKEditor";
$STAGE.scene = new Scene(webView, 1024, 360);
$STAGE.show();

  

For the next part, it gets a little confusing, but you should be able to catch on to the weirdness (after all, you're a JavaScript developer.) First, you have to be aware of the fact that there is a separate JavaScriptCore engine inside the WebView. The example shows interaction with this engine by asking it to load an URL.

Second, we want to be able to have JSObject objects passed back from the WebView engine and have them behave like Nashorn objects. To accomplish this we need to "wrap" the JSObject objects in Nashorn JSAdaptor objects.
function wrap(jsobject) {
  return new JSAdapter({
    __get__ : function (key) {
      return jsobject.getMember(key);
    },
    __has__ : function (key) {
      return jsobject.call("hasOwnProperty", key);
    },
    __put__ : function (key, value) {
      return jsobject.setMember(key, value);
    },
    __call__ : function (name) {
      var args = Array.prototype.slice.call(arguments);
      args.shift();
      args = Java.to(args, ObjectArray);
      return jsobject.call(name, args);
    },
    __new__ : function () {
      return wrap(jsobject.eval("new this()"));
    },
    __delete__ : function (key) {
      jsobject.removeMember(key);
      return true;
    },
    __getIds__ : function () {
      return jsobject.eval("Object.keys(this)");
    }
  });
}
The wrap function creates a closure containing "jsobject" and the adaptor functions specified. This will allow you to do things like;
var window = wrap(webview.engine.executeScript("window"));
var document = wrap(window.document);
The adaptor above is very basic. The complete code might look more like;
var ObjectArray = Java.type("java.lang.Object[]");
var NetscapeJSObject = Java.type("netscape.javascript.JSObject");
var EventHandler = Java.type("javafx.event.EventHandler");

// Lambda functions for 0, 1, or 2 arguments (no need for external java code.)
var Supplier = Java.type("java.util.function.Supplier");
var UniFunction = Java.type("java.util.function.Function");
var BiFunction = Java.type("java.util.function.BiFunction");

function wrap(jsobject) {
// If not a JSObject type (primitives) then don't wrap.
  if (!(jsobject instanceof NetscapeJSObject)) {
    return jsobject;
  }

// Construct a SAM to call back into Nashorn (event handling)
  function callback(func) {
    if (typeof func == 'function') {
      switch (func.length) {
      case 0:
        return new (Java.extend(Supplier, { get: function() { return func(); } }))();
      case 1:
        return new (Java.extend(UniFunction, { apply: function(arg) { return func(wrap(arg)); } }))();
      case 2:
        return new (Java.extend(BiFunction, { apply: function(arg1, arg2) { return func(wrap(arg1), wrap(arg2)); }}))();
      }
    }
    throw "Callbacks can only have zero, one or two arguments";
  }

  // Potentially wrap arguments for a call.
  function wrapArgs(args) {
    for (var i = 0; i < args.length; i++) {
      var arg = args[i];
      if (arg) {
        if (arg.unwrap) {
          args[i] = arg.unwrap();
        } else if (typeof arg == "function") {
          args[i] = callback(arg);
        }
      }
    }
  }

  return new JSAdapter({
    __get__ : function (key) {
      // special case unwrap to return the jsobject
      if (key == "unwrap") {
        return function() { return jsobject };
      }
      // Handle number indexing
      var value = typeof key == "number" ? jsobject.getSlot(key) : jsobject.getMember(key);

      // Wrap the result
      return wrap(value);
    },
    __has__ : function (key) {
      return jsobject.call("hasOwnProperty", key);
    },
    __put__ : function (key, value) {
      // Wrap functions as callback SAMs
      if (typeof value == "function") {
        value = callback(value);
      }

      // Handle number indexing
      return typeof key == "number" ? jsobject.setSlot(key, value) : jsobject.setMember(key, value);
    },
    __call__ : function (name) {
      // Special case unwrap to return the jsobject
      if (name == "unwrap") {
        return jsobject;
      }

      var args = Array.prototype.slice.call(arguments);
      args.shift();
      args = Java.to(args, ObjectArray);
      wrapArgs(args);

      return jsobject.call(name, args);
    },
    __new__ : function () {
      return wrap(jsobject.eval("new this()"));
    },
    __delete__ : function (key) {
      jsobject.removeMember(key);
      return true;
    },
    __getIds__ : function () {
      // Convert the Node collection to a JS array
      var keys = jsobject.eval("Object.keys(this)");
      var length = keys.getMember("length");
      var ids = [];
      for (var i = 0; i < length; i++) {
        ids.push(keys.getSlot(i));
      }
      return ids;
    }
  });
}
I know this code looks complex, but you only have to write once and include (load) in WebView related projects. Note that the code snippet above then becomes;
var window = wrap(webview.engine.executeScript("window"));
var document = window.document;
In the example I also created a special helper wrapper for the WebView to simplify accessing the DOM.
function WebViewWrapper(onload) {
  var This = this;
  var WebView = Java.type("javafx.scene.web.WebView");
  var webview = new WebView();

  This.webview = webview;
  This.engine = webview.engine;
  This.window = undefined;
  This.document = undefined;

  // Make sure the JavaScript is enabled.
  This.engine.javaScriptEnabled = true;

  // Complete initialization when page is loaded.
  This.engine.loadWorker.stateProperty().addListener(new ChangeListener() {
    changed: function(value, oldState, newState) {
      if (newState == Worker.State.SUCCEEDED) {
        This.document = wrap(This.engine.executeScript("document"));
        This.window = wrap(This.engine.executeScript("window"));

        // Call users onload function.
        if (onload) {
          onload(This);
        }
      }
    }
  });

  // Divert alert message to print.
  This.engine.onAlert = new EventHandler() {
    handle: function(evt) {
      print(evt.data);
    }
  };

  // Load page from URL.
  This.load = function(url) {
    This.engine.load(url);
  }

  // Load page from text.
  This.loadContent = function(text) {
    This.engine.loadContent(text);
  }
}
So now the main script looks as follows;
var Scene = Java.type("javafx.scene.Scene");
var link = "file:${$ENV.PWD}/example2.html";
var wvw = WebViewWrapper();
wvw.load(link);

$STAGE.title = "CKEditor";
$STAGE.scene = new Scene(wvw.webview, 1024, 360);
$STAGE.show();
Getting the content from the CKEditor;
var window = wvw.window;
var CKEDITOR = window.CKEDITOR;
var mainEditor = CKEDITOR.instances.MainEditor;
var text = mainEditor.getData();
Setting the content in the CKEditor;
var window = wvw.window;
var CKEDITOR = window.CKEDITOR;
var mainEditor = CKEDITOR.instances.MainEditor;
mainEditor.setData(text);

Thursday May 08, 2014

Popularity of Nashorn

I know it's unseemly to toot one's own horn, but after the release I did a Topsy to see how we were faring.


I did some deeper analysis of the tweets and determined that about a quarter of the Nashorn tweets were about zoos and tanks.  The exceptions were when Nashorn crossed above JDK8/Java8.  The first peak was when people started tweeting about the Nashorn release video.  The second was when Benjamin Winterberg announced his tutorial (40.7K hits to date.)

I've also been monitoring the activity on the article Nashorn: JavaScript made great in Java 8 .  It's been hit over 20,000 times.

JavaScript is the language rising star GitHub language trends and the fragmenting landscape. Make Nashorn part of it. Get the word out.

Tuesday May 06, 2014

Documents and Articles on Nashorn

I've decided to move this entry to the main Nashorn wiki site to allow for "living document" additions.  Nashorn: Articles, Documents, Slides and Videos

Monday Mar 17, 2014

Latest news from the Nashorn team

Things have been busy on the Nashorn front, getting ready for the JDK 8 rollout and all.  March 18 is the magic day (EclipseCon.)

 I see that JetBrains has added Nashorn debugging to IntelliJ. This rounds out the debugging story with support in NetBeans and Eclipse.

 The Nashorn team has been working hard on performance improvements for the next round.  The first set involves caching of compiled scripts.  This will make a huge difference for reoccurring scripts (think servers.)  The second set of changes involve optimistic typing, where code generated for functions assumes optimal data types in expressions (ex. integers) and falls back to broader types (ex. double) if it doesn't work out.  This provides a huge performance win, since most of the time the assumptions prove correct.  These fixes are in staging repos moving to the JDK9-dev and JDK8u20-dev repos in the next few weeks.  Both these changes should be in the JDK 8u20 update targeted for August.

Finally, on the Node.js front, Node has found a permanent home in the Java EE world.  The project has been renamed to Avatar.js to tie in with the large Avatar project and is taking full advantage of multithreading.

Thursday Sep 26, 2013

setInterval and setTimeout JavaScript Functions

With people porting HTML to Nashorn + JavaFX more frequently, I often get asked the question "how do you port functions like setInterval and setTimeout" (setInterval is used to have a function repeated at regular intervals, where setTimeout is used to have a function delayed.) The following code duplicates the functionality of the each of the setInterval family of functions, when used in a JavaFX application.

var Platform = Java.type("javafx.application.Platform");
var Timer    = Java.type("java.util.Timer");

function setInterval(func, milliseconds) {
	// New timer, run as daemon so the application can quit
	var timer = new Timer("setInterval", true);
	timer.schedule(function() Platform.runLater(func), milliseconds, milliseconds);	
	return timer;
}

function clearInterval(timer) {
	timer.cancel();
}

function setTimeout(func, milliseconds) {
	// New timer, run as daemon so the application can quit
	var timer = new Timer("setTimeout", true);
	timer.schedule(function() Platform.runLater(func), milliseconds);	
	return timer;
}

function clearTimeout(timer) {
	timer.cancel();
}

<Update March 21, 2014>

I was requested by a reader to implement setInterval, setTimeout and setImmediate with the same arguments as defined in DOM.

var Platform = Java.type("javafx.application.Platform");
var Timer    = Java.type("java.util.Timer");

function setTimerRequest(handler, delay, interval, args) {
    handler = handler || function() {};
    delay = delay || 0;
    interval = interval || 0;

    var applyHandler = function() handler.apply(this, args);
    var runLater = function() Platform.runLater(applyHandler);

    var timer = new Timer("setTimerRequest", true);

    if (interval > 0) {
        timer.schedule(runLater, delay, interval);
    } else {
        timer.schedule(runLater, delay);
    }

    return timer;
}

function clearTimerRequest(timer) {
    timer.cancel();
}

function setInterval() {
    var args = Array.prototype.slice.call(arguments);
    var handler = args.shift();
    var ms = args.shift();

    return setTimerRequest(handler, ms, ms, args);
}

function clearInterval(timer) {
    clearTimerRequest(timer);
}

function setTimeout() {
    var args = Array.prototype.slice.call(arguments);
    var handler = args.shift();
    var ms = args.shift();

    return setTimerRequest(handler, ms, 0, args);
}

function clearTimeout(timer) {
    clearTimerRequest(timer);
}

function setImmediate() {
    var args = Array.prototype.slice.call(arguments);
    var handler = args.shift();

    return setTimerRequest(handler, 0, 0, args);
}

function clearImmediate(timer) {
    clearTimerRequest(timer);
}

Monday Sep 09, 2013

Nashorn + FX 350fps vs. Chrome 120fps

I was reading an article written by Felix Bembrick where he took a Chrome canvas example and ported it to Nashorn with JavaFX.  I ran the two tests myself and was surprised to see that Nashorn + FX ran at 350fps where Chrome ran at 120fps (iMac, 3.4 GHz Intel Core i7, AMD Radeon HD 6970M 2048M.)  Much of the credit for that is JavaFX's rendering engine, but Nashorn allowed Felix to leverage that.  The original article is at JavaFX with Nashorn Canvas example

Wednesday Sep 04, 2013

JavaOne: JavaScript Sessions

These are the sessions I found related to JavaScript.  There are likely others.  If you spot others, send a comment and I'll update this entry.  The session on the list is mine and the second is Attila's. Teaching the Java Platform with Nashorn is done with Ben Evans of the London JUG.  Come to the Script Bowl and vote for the only first time entry (Nashorn.)

Nashorn: JavaScript on the JVM [CON7835]

  • Monday, Sep 23, 11:30 AM - 12:30 PM - Hilton - Yosemite B/C


Nashorn: Java and JavaScript—Shaken, Not Stirred [BOF5793]

  • Monday, Sep 23, 6:30 PM - 7:15 PM - Hilton - Yosemite B/C


Teaching the Java Platform with Nashorn [BOF5225]

  • Tuesday, Sep 24, 4:30 PM - 5:15 PM - Hilton - Yosemite B/C


The Curious Case of JavaScript on the JVM [CON2952]

  • Tuesday, Sep 24, 1:00 PM - 2:00 PM - Hilton - Yosemite B/C


Embedding JVM Scripting Languages [CON2585]

  • Tuesday, Sep 24, 3:00 PM - 4:00 PM - Hilton - Yosemite B/C


Modular JavaScript [CON2959]

  • Wednesday, Sep 25, 3:00 PM - 4:00 PM - Hilton - Yosemite B/C


MySQL User-Defined Functions....in JavaScript! [CON1738]

  • Saturday, Sep 21, 1:00 PM - 2:00 PM - Hilton - Powell


Effective Use of HTML5 JavaScript for Java EE 7 Web Applications [BOF7838]

  • Monday, Sep 23, 4:30 PM - 5:15 PM - Parc 55 - Cyril Magnin II/III


Script Bowl 2013: The Battle for Supremacy and Survival [CON2653]

  • Wednesday, Sep 25, 4:30 PM - 5:30 PM - Hilton - Imperial Ballroom B


Developing JavaScript Applications for Node.js with MySQL and NoSQL [CON6805]

  • Sunday, Sep 22, 4:00 PM - 5:00 PM - Hilton - Powell

JavaOne: What a Difference a Year Makes

I haven't been blogging for a while.  JavaOne examples and slides are taking up all my time.  Marcus and Attila leaked out that I was doing a demo of the Nashorn Debugger prototype, but I still have some surprises up my sleeve.  I was reviewing examples from last year and felt that they needed some updating.  The FX Fireworks example had all kinds of Java code to support applications, timers and callbacks.  This year's Fireworks version is pure Nashorn.  I posted source and backdrop here.  Just run with jjs -fx -scripting fireworks.js  Enjoy.

var ArrayList      = Java.type("java.util.ArrayList");
var File           = Java.type("java.io.File");

var AnimationTimer = Java.type("javafx.animation.AnimationTimer");
var BlendMode      = Java.type("javafx.scene.effect.BlendMode");
var Canvas         = Java.type("javafx.scene.canvas.Canvas");
var Color          = Java.type("javafx.scene.paint.Color");
var CycleMethod    = Java.type("javafx.scene.paint.CycleMethod");
var Group          = Java.type("javafx.scene.Group");
var ImageView      = Java.type("javafx.scene.image.ImageView");
var Pane           = Java.type("javafx.scene.layout.Pane");
var RadialGradient = Java.type("javafx.scene.paint.RadialGradient");
var Reflection     = Java.type("javafx.scene.effect.Reflection");
var Scene          = Java.type("javafx.scene.Scene");
var Stop           = Java.type("javafx.scene.paint.Stop");

var AnimationTimerExtend = Java.extend(AnimationTimer);

var doubleArray    = Java.type("double[]");

var GRAVITY = 0.06;

function Particle(posX, posY, velX, velY, targetX, targetY, color, size, usePhysics, shouldExplodeChildren, hasTail) {
    this.posX                  = posX;
    this.posY                  = posY;
    this.velX                  = velX;
    this.velY                  = velY;
    this.targetX               = targetX;
    this.targetY               = targetY;
    this.color                 = color;
    this.size                  = size;
    this.usePhysics            = usePhysics;
    this.shouldExplodeChildren = shouldExplodeChildren;
    this.hasTail               = hasTail;
    this.alpha                 = 1;
    this.easing                = Math.random() * 0.02;
    this.fade                  = Math.random() * 0.1;
}

Particle.prototype.update = function() {
    this.lastPosX = this.posX;
    this.lastPosY = this.posY;

    if(this.usePhysics) { // on way down
        this.velY += GRAVITY;
        this.posY += this.velY;
        this.alpha -= this.fade; // fade out particle
    } else { // on way up
        var distance = (this.targetY - this.posY);
        // ease the position
        this.posY += distance * (0.03 + this.easing);
        // cap to 1
        this.alpha = Math.min(distance * distance * 0.00005, 1);
    }

    this.posX += this.velX;

    return this.alpha < 0.005;
}


Particle.prototype.draw = function(context) {
    var x    = Math.round(this.posX);
    var y    = Math.round(this.posY);
    var xVel = (x - this.lastPosX) * -5;
    var yVel = (y - this.lastPosY) * -5;

    // set the opacity for all drawing of this particle
    context.globalAlpha = Math.random() * this.alpha;
    // draw particle
    context.fill = this.color;
    context.fillOval(x - this.size, y - this.size, this.size + this.size, this.size + this.size);

    // draw the arrow triangle from where we were to where we are now
    if (this.hasTail) {
        context.fill = Color.rgb(255, 255, 255, 0.3);
        var x = new doubleArray(3);
        var y = new doubleArray(3);
        x[0] = this.posX + 1.5;  y[0] = this.posY;
        x[1] = this.posX + xVel; y[1] = this.posY + yVel;
        x[2] = this.posX - 1.5;  y[2] = this.posY;
        context.fillPolygon(x, y, 3);
    }
}

function drawFireworks(gc) {
    var iter = particles.iterator();
    var newParticles = new ArrayList();

    while(iter.hasNext()) {
        var firework = iter.next();

        // if the update returns true then particle has expired
        if(firework.update()) {
            // remove particle from those drawn
            iter.remove();

            // check if it should be exploded
            if(firework.shouldExplodeChildren) {
                if(firework.size == 9) {
                    explodeCircle(firework, newParticles);
                } else if(firework.size == 8) {
                    explodeSmallCircle(firework, newParticles);
                }
            }
        }

        firework.draw(gc);
    }

    particles.addAll(newParticles);
}

function fireParticle() {
    particles.add(new Particle(
        surface.width * 0.5, surface.height + 10,
        Math.random() * 5 - 2.5, 0,
        0, 150 + Math.random() * 100,
        colors[0], 9,
        false, true, true));
}

function explodeCircle(firework, newParticles) {
    var count = 20 + (60 * Math.random());
    var shouldExplodeChildren = Math.random() > 0.5;
    var angle = (Math.PI * 2) / count;
    var color = (Math.random() * (colors.length - 1));

    for(var i=count; i>0; i--) {
        var randomVelocity = 4 + Math.random() * 4;
        var particleAngle = i * angle;

        newParticles.add(
            new Particle(
                firework.posX, firework.posY,
                Math.cos(particleAngle) * randomVelocity, Math.sin(particleAngle) * randomVelocity,
                0, 0,
                colors[Math.ceil(color)],
                8,
                true, shouldExplodeChildren, true));
    }
}

function explodeSmallCircle(firework, newParticles) {
    var angle = (Math.PI * 2) / 12;

    for(var count = 12; count > 0; count--) {
        var randomVelocity = 2 + Math.random() * 2;
        var particleAngle = count * angle;
        newParticles.add(
            new Particle(
                firework.posX, firework.posY,
                Math.cos(particleAngle) * randomVelocity, Math.sin(particleAngle) * randomVelocity,
                0, 0,
                firework.color,
                4,
                true, false, false));
    }
}

function fileToURL(file) {
    return new File(file).toURI().toURL().toExternalForm();
}

var timer = new AnimationTimerExtend() {
    handle: function handle(now) {
        var gc = surface.graphicsContext2D;
        // clear area with transparent black
        gc.fill = Color.rgb(0, 0, 0, 0.2);
        gc.fillRect(0, 0, 1024, 708);
        // draw fireworks
        drawFireworks(gc);

        // countdown to launching the next firework
        if (countDownTillNextFirework <= 0) {
            countDownTillNextFirework = 10 + (Math.random() * 30);
            fireParticle();
        }

        countDownTillNextFirework--;
    }
};

// Kill timer before exiting.
function stop() {
    timer.stop();
}

var particles = new ArrayList();
var countDownTillNextFirework = 40;

// create a color palette of 180 colors
var colors = new Array(181);
var stops = new ArrayList();
stops.add(new Stop(0, Color.WHITE));
stops.add(new Stop(0.2, Color.hsb(59, 0.38, 1)));
stops.add(new Stop(0.6, Color.hsb(59, 0.38, 1,0.1)));
stops.add(new Stop(1, Color.hsb(59, 0.38, 1,0)));
colors[0] = new RadialGradient(0, 0, 0.5, 0.5, 0.5, true, CycleMethod.NO_CYCLE, stops);

for (var h = 0; h < 360; h += 2) {
    var stops2 = new ArrayList();
    stops2.add(new Stop(0, Color.WHITE));
    stops2.add(new Stop(0.2, Color.hsb(h, 1, 1)));
    stops2.add(new Stop(0.6, Color.hsb(h, 1, 1,0.1)));
    stops2.add(new Stop(1, Color.hsb(h, 1, 1,0)));
    colors[1 + (h / 2)] = new RadialGradient(0, 0, 0.5, 0.5, 0.5, true, CycleMethod.NO_CYCLE, stops2);
}

var surface = new Canvas(1024, 500);
surface.blendMode = BlendMode.ADD;
surface.effect = new Reflection(0, 0.4, 0.15, 0);

var imageUrl = fileToURL("sf.jpg");
var background = new ImageView(imageUrl);

var pane = new Pane();
pane.children.add(background);
pane.children.add(surface);

var root = new Group();
root.children.add(pane);
$STAGE.scene = new Scene(root);

timer.start();


Thursday Aug 08, 2013

Repost: A simple Real-time Web App built with Nashorn Scripting

Peter Hausel posted a new blog entry on Nashorn.  Cool example.  A simple Real-time Web App built with Nashorn Scripting

Tuesday Jul 09, 2013

Nashorn Multithreading and MT-safety

Tobias Schlotte asked the question (via twitter @tobsch), "Where is thread-safety on your agenda for Nashorn?"  The quick answer is that multithreading is very high on the list, but MT-safety is a very complicated issue. I'll attempt to explain our plan of action.

As Attila responded http://mail.openjdk.java.net/pipermail/nashorn-dev/2013-July/001567.html "ECMAScript 5.1 language specification doesn't define multithreading semantics for programs written in the language; they are inherently single threaded. If we were to make them thread safe, we'd be sacrificing single threaded performance for a behaviour that falls outside of the specification."

Why is performance affected?  Well, generally, the internal structure of a script object is complex.  Any modification to the object would have to be done with mutex locks (synchronize) or compare/swap.  As an example, take obj.x = 1.  In a static language, this code would likely generate a simple native store that can be executed, somewhat, atomically.  With multiple threads, obj.x = ? would simply be "last thread that stores, wins" (ignoring coherency et al) and the application would continue to run.  In dynamic languages like JavaScript, what happens when property "x" doesn't exist yet?  Internally, the script object has to be restructured to include the new property before the store takes place.  Now add the fact that another thread could be adding or removing properties on the same object; objects would have to be locked for (nearly) every operation. Then add in the thread coherency issue…  Slowness, reigns.

On the other hand, having a language on the JVM and not supporting multithreading is just plain daft.  Especially, since our target platforms include multi-core servers.

So, our agenda is two fold.  The first is to provide a "workers" library (timeline is not tied to JDK8) which uses an onevent model that JavaScripters are familiar with.  No synchronization/locking constructs to be added to the language.  Communication between threads (and potentially nodes/servers) is done using JSON (under the covers.) Going this route allows object isolation between threads, but also allows maximal use of CPU capacity.

The second part is, we will not guarantee MT-safe structures, but we have to face reality.  Developers will naturally be drawn to using threads.  Many of the Java APIs require use of threads.  So, the best we can do is provide guidelines on how to not shoot yourself in the foot.  These guidelines will evolve and we'll post them 'somewhere' after we think them through.  In the meantime,  I follow some basic rules;
  • Avoid sharing script objects or script arrays across threads (this includes global.)  Sharing script objects is asking for trouble.  Share only primitive data types, or Java objects.
  • If you want to pass objects or arrays across threads, use JSON.stringify(obj) and JSON.parse(string) to transport using strings.
  • If you really really feel you have to pass a script object, treat the object as a constant and only pass the object to new threads (coherency.)  Consider using Object.freeze(obj).
  • If you really really really feel you have to share a script object, make sure the object's properties are stabilized.  No adding or removing properties after sharing starts.  Consider using Object.seal(obj).
  • Given enough time, any other use of a shared script object will eventually cause your app to fail.
The number of reallys in a rule reflects the uncertainty of the outcome.

Note in the first rule, I stated that you shouldn't share global.  The reason is simple, almost everything adds properties to global.  How do you avoid share modifying global?  Simply give each thread a new global.

Nashorn has two builtin functions for starting scripts; load and loadWithNewGlobal.  Each takes an arg specifying which script to load and evaluate; either an URL/file string or an object with a name and script properties (name is for source location, script is body of script.)  Additional arguments are passed as the arguments global when evaluating.  Unlike load, loadWithNewGlobal creates a fresh global before loading the script.  This isolates evaluation from the current global and thus suppresses sharing.

Going back to Tobias' example (the original is at https://gist.github.com/tobsch/5955518);

import javax.script.CompiledScript;
import javax.script.Compilable;
import javax.script.ScriptException;
 
import jdk.nashorn.api.scripting.NashornScriptEngineFactory;
import jdk.nashorn.api.scripting.NashornScriptEngine;
 
import java.util.concurrent.Future;
import java.util.concurrent.Executors;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.Callable;
 
import java.util.ArrayList;
 
public class SSCCE {
  private static final NashornScriptEngineFactory engineFactory = new NashornScriptEngineFactory();
   
  public static void main(String[] args) throws ScriptException, InterruptedException, ExecutionException {
    Compilable engine = (Compilable) engineFactory.getScriptEngine();
     
    String script = new StringBuilder("i = 0;")
      .append("i += 1;")
      .append("shortly_later = new Date()/1000 + Math.random;") // 0..1 sec later
      .append("while( (new Date()/1000) < shortly_later) { Math.random() };") //prevent optimizations
      .append("i += 1;")
      .toString();
     
    final CompiledScript onePlusOne = engine.compile(script);
     
    Callable<Double> addition = new Callable<Double>() {
      @Override
      public Double call() {
        try {
          return (Double) onePlusOne.eval();
        }
        catch(ScriptException e) {
          throw new RuntimeException(e);
        }
      }
    };
     
    ExecutorService executor = Executors.newCachedThreadPool();
    ArrayList<Future<Double>> results = new ArrayList<Future<Double>>();
     
    for(int i = 0; i < 50; i++) {
      results.add(executor.submit(addition));
    }
     
    int miscalculations = 0;
    for(Future<Double> result : results) {
      int jsResult = result.get().intValue();
       
      if(jsResult != 2) {
        System.out.println("Incorrect result from js, expected 1 + 1 = 2, but got " + jsResult);
        miscalculations += 1;
      }
    }
     
    executor.awaitTermination(1, TimeUnit.SECONDS);
    executor.shutdownNow();
     
    System.out.println("Overall: " + miscalculations + " wrong values for 1 + 1.");
  }
}

Output;

Incorrect result from js, expected 1 + 1 = 2, but got 11
Incorrect result from js, expected 1 + 1 = 2, but got 4
Incorrect result from js, expected 1 + 1 = 2, but got 9
Incorrect result from js, expected 1 + 1 = 2, but got 6
Incorrect result from js, expected 1 + 1 = 2, but got 4
Incorrect result from js, expected 1 + 1 = 2, but got 4
Incorrect result from js, expected 1 + 1 = 2, but got 6
Incorrect result from js, expected 1 + 1 = 2, but got 5
Incorrect result from js, expected 1 + 1 = 2, but got 6
Incorrect result from js, expected 1 + 1 = 2, but got 3
Incorrect result from js, expected 1 + 1 = 2, but got 3
Incorrect result from js, expected 1 + 1 = 2, but got 13
Incorrect result from js, expected 1 + 1 = 2, but got 10
Incorrect result from js, expected 1 + 1 = 2, but got 6
Incorrect result from js, expected 1 + 1 = 2, but got 13
Incorrect result from js, expected 1 + 1 = 2, but got 5
Incorrect result from js, expected 1 + 1 = 2, but got 6
Incorrect result from js, expected 1 + 1 = 2, but got 11
Incorrect result from js, expected 1 + 1 = 2, but got 9
Incorrect result from js, expected 1 + 1 = 2, but got 7
Incorrect result from js, expected 1 + 1 = 2, but got 11
Incorrect result from js, expected 1 + 1 = 2, but got 11
Incorrect result from js, expected 1 + 1 = 2, but got 7
Incorrect result from js, expected 1 + 1 = 2, but got 9
Incorrect result from js, expected 1 + 1 = 2, but got 3
Incorrect result from js, expected 1 + 1 = 2, but got 3
Incorrect result from js, expected 1 + 1 = 2, but got 13
Incorrect result from js, expected 1 + 1 = 2, but got 11
Incorrect result from js, expected 1 + 1 = 2, but got 12
Incorrect result from js, expected 1 + 1 = 2, but got 8
Incorrect result from js, expected 1 + 1 = 2, but got 4
Incorrect result from js, expected 1 + 1 = 2, but got 4
Incorrect result from js, expected 1 + 1 = 2, but got 4
Incorrect result from js, expected 1 + 1 = 2, but got 8
Incorrect result from js, expected 1 + 1 = 2, but got 9
Incorrect result from js, expected 1 + 1 = 2, but got 3
Incorrect result from js, expected 1 + 1 = 2, but got 5
Incorrect result from js, expected 1 + 1 = 2, but got 3
Incorrect result from js, expected 1 + 1 = 2, but got 9
Incorrect result from js, expected 1 + 1 = 2, but got 6
Incorrect result from js, expected 1 + 1 = 2, but got 7
Overall: 41 wrong values for 1 + 1.

I rewrote in JavaScript and introduced loadWithNewGlobal;

#!/usr/bin/env jjs -scripting

var Executors = java.util.concurrent.Executors;
var TimeUnit  = java.util.concurrent.TimeUnit;
var ArrayList = java.util.ArrayList;

var script = <<EOD
    i = 0;
    i += 1;
    shortly_later = new Date()/1000 + Math.random;
    while( (new Date()/1000) < shortly_later) { Math.random() };
    i += 1;
EOD

function addition() {
    return loadWithNewGlobal({ name: "addition", script: script });
}

var executor = Executors.newCachedThreadPool();
var results = new ArrayList();

for(var i = 0; i < 50; i++) {
    // Clarify Runnable versus Callable
    results.add(executor["submit(java.util.concurrent.Callable)"](addition));
}

var miscalculations = 0;
for each (var result in results) {
    var jsResult = result.get().intValue();
    if (jsResult != 2) {
        print("Incorrect result from js, expected 1 + 1 = 2, but got " + jsResult);
        miscalculations += 1;
    }
}

executor.awaitTermination(1, TimeUnit.SECONDS);
executor.shutdownNow();
print("Overall: " + miscalculations + " wrong values for 1 + 1.");

Works as expected;

Overall: 0 wrong values for 1 + 1.

Wednesday Jun 19, 2013

Nashorn and Lambda, What the Hey!

Yesterday, Brian Goetz (Lambda architect) suggested that I produce an example of Lambda being used from Nashorn.  Since, I've been heads down in Nashorn, I really haven't played with Lambda that much.  After looking at some examples from Stuart Marks,  I figured, what the hey, it doesn't look that hard.  Details of the Lambda APIs are available at JDK 8 b92 API .

Many constructs are familiar to JavaScript developers.  The main things to note;
  • where you can use a Lambda you can use a JavaScript function
  • JavaScript arrays need to be converted to Java collections
  • JavaScript syntax requires the '.' end a phrase, not start one (forces continuation)
Other than that, the JavaScript source and Java source looks very much the same.

#!/usr/bin/env jjs -scripting

var copyright = <<<EOS;
/*
 * Copyright (c) 2010, 2013, Oracle and/or its affiliates. All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 *   - Redistributions of source code must retain the above copyright
 *     notice, this list of conditions and the following disclaimer.
 *
 *   - Redistributions in binary form must reproduce the above copyright
 *     notice, this list of conditions and the following disclaimer in the
 *     documentation and/or other materials provided with the distribution.
 *
 *   - Neither the name of Oracle nor the names of its
 *     contributors may be used to endorse or promote products derived
 *     from this software without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
 * IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
 * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
 * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
 * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
 * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
 * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */
EOS

var Collectors = java.util.stream.Collectors;

// Break the copyright into tokens.
var tokens = copyright.split(/\s+/);

// Convert to ArrayList.
var list = new java.util.ArrayList();
tokens.map(function(e) list.add(e));

// The JavaScript collection for the result.
var result = [];

// Parallelize some of the activity.
list.parallelStream().
    // Select only words.
    filter(function(t) t.match(/^[A-Za-z]+$/)).
    // Make case comparable.
    map(function(t) t.toLowerCase()).
    // Fold duplicates.
    collect(Collectors.groupingBy(function(t) t)).
    // Move results to JavaScript collection.
    forEach(function(t) result.push(t));

// Sort the result.
result.sort();

print(result);
The result;
a,above,advised,all,and,any,are,arising,be,binary,business,but,by,caused,code,
conditions,consequential,contributors,copyright,damages,derived,disclaimer,
documentation,endorse,even,event,express,fitness,following,for,form,from,
goods,holders,however,if,implied,in,is,its,liable,limited,list,loss,materials,
may,merchantability,must,name,names,negligence,neither,no,nor,not,of,on,or,
oracle,other,out,owner,particular,permitted,possibility,prior,procurement,
products,promote,provided,purpose,redistribution,redistributions,reproduce,
retain,rights,shall,software,source,specific,strict,substitute,such,that,
the,theory,this,to,tort,use,used,warranties,way,whether,with,without,written

About

Technical discussions and status of the Nashorn JavaScript Project.

Search

Categories
Archives
« July 2015
SunMonTueWedThuFriSat
   
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
 
       
Today