Nashorn Multithreading and MT-safety

Tobias Schlotte asked the question (via twitter @tobsch), "Where is thread-safety on your agenda for Nashorn?"  The quick answer is that multithreading is very high on the list, but MT-safety is a very complicated issue. I'll attempt to explain our plan of action.

As Attila responded http://mail.openjdk.java.net/pipermail/nashorn-dev/2013-July/001567.html "ECMAScript 5.1 language specification doesn't define multithreading semantics for programs written in the language; they are inherently single threaded. If we were to make them thread safe, we'd be sacrificing single threaded performance for a behaviour that falls outside of the specification."

Why is performance affected?  Well, generally, the internal structure of a script object is complex.  Any modification to the object would have to be done with mutex locks (synchronize) or compare/swap.  As an example, take obj.x = 1.  In a static language, this code would likely generate a simple native store that can be executed, somewhat, atomically.  With multiple threads, obj.x = ? would simply be "last thread that stores, wins" (ignoring coherency et al) and the application would continue to run.  In dynamic languages like JavaScript, what happens when property "x" doesn't exist yet?  Internally, the script object has to be restructured to include the new property before the store takes place.  Now add the fact that another thread could be adding or removing properties on the same object; objects would have to be locked for (nearly) every operation. Then add in the thread coherency issue…  Slowness, reigns.

On the other hand, having a language on the JVM and not supporting multithreading is just plain daft.  Especially, since our target platforms include multi-core servers.

So, our agenda is two fold.  The first is to provide a "workers" library (timeline is not tied to JDK8) which uses an onevent model that JavaScripters are familiar with.  No synchronization/locking constructs to be added to the language.  Communication between threads (and potentially nodes/servers) is done using JSON (under the covers.) Going this route allows object isolation between threads, but also allows maximal use of CPU capacity.

The second part is, we will not guarantee MT-safe structures, but we have to face reality.  Developers will naturally be drawn to using threads.  Many of the Java APIs require use of threads.  So, the best we can do is provide guidelines on how to not shoot yourself in the foot.  These guidelines will evolve and we'll post them 'somewhere' after we think them through.  In the meantime,  I follow some basic rules;
  • Avoid sharing script objects or script arrays across threads (this includes global.)  Sharing script objects is asking for trouble.  Share only primitive data types, or Java objects.
  • If you want to pass objects or arrays across threads, use JSON.stringify(obj) and JSON.parse(string) to transport using strings.
  • If you really really feel you have to pass a script object, treat the object as a constant and only pass the object to new threads (coherency.)  Consider using Object.freeze(obj).
  • If you really really really feel you have to share a script object, make sure the object's properties are stabilized.  No adding or removing properties after sharing starts.  Consider using Object.seal(obj).
  • Given enough time, any other use of a shared script object will eventually cause your app to fail.
The number of reallys in a rule reflects the uncertainty of the outcome.

Note in the first rule, I stated that you shouldn't share global.  The reason is simple, almost everything adds properties to global.  How do you avoid share modifying global?  Simply give each thread a new global.

Nashorn has two builtin functions for starting scripts; load and loadWithNewGlobal.  Each takes an arg specifying which script to load and evaluate; either an URL/file string or an object with a name and script properties (name is for source location, script is body of script.)  Additional arguments are passed as the arguments global when evaluating.  Unlike load, loadWithNewGlobal creates a fresh global before loading the script.  This isolates evaluation from the current global and thus suppresses sharing.

Going back to Tobias' example (the original is at https://gist.github.com/tobsch/5955518);

import javax.script.CompiledScript;
import javax.script.Compilable;
import javax.script.ScriptException;
 
import jdk.nashorn.api.scripting.NashornScriptEngineFactory;
import jdk.nashorn.api.scripting.NashornScriptEngine;
 
import java.util.concurrent.Future;
import java.util.concurrent.Executors;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.Callable;
 
import java.util.ArrayList;
 
public class SSCCE {
  private static final NashornScriptEngineFactory engineFactory = new NashornScriptEngineFactory();
   
  public static void main(String[] args) throws ScriptException, InterruptedException, ExecutionException {
    Compilable engine = (Compilable) engineFactory.getScriptEngine();
     
    String script = new StringBuilder("i = 0;")
      .append("i += 1;")
      .append("shortly_later = new Date()/1000 + Math.random;") // 0..1 sec later
      .append("while( (new Date()/1000) < shortly_later) { Math.random() };") //prevent optimizations
      .append("i += 1;")
      .toString();
     
    final CompiledScript onePlusOne = engine.compile(script);
     
    Callable<Double> addition = new Callable<Double>() {
      @Override
      public Double call() {
        try {
          return (Double) onePlusOne.eval();
        }
        catch(ScriptException e) {
          throw new RuntimeException(e);
        }
      }
    };
     
    ExecutorService executor = Executors.newCachedThreadPool();
    ArrayList<Future<Double>> results = new ArrayList<Future<Double>>();
     
    for(int i = 0; i < 50; i++) {
      results.add(executor.submit(addition));
    }
     
    int miscalculations = 0;
    for(Future<Double> result : results) {
      int jsResult = result.get().intValue();
       
      if(jsResult != 2) {
        System.out.println("Incorrect result from js, expected 1 + 1 = 2, but got " + jsResult);
        miscalculations += 1;
      }
    }
     
    executor.awaitTermination(1, TimeUnit.SECONDS);
    executor.shutdownNow();
     
    System.out.println("Overall: " + miscalculations + " wrong values for 1 + 1.");
  }
}

Output;

Incorrect result from js, expected 1 + 1 = 2, but got 11
Incorrect result from js, expected 1 + 1 = 2, but got 4
Incorrect result from js, expected 1 + 1 = 2, but got 9
Incorrect result from js, expected 1 + 1 = 2, but got 6
Incorrect result from js, expected 1 + 1 = 2, but got 4
Incorrect result from js, expected 1 + 1 = 2, but got 4
Incorrect result from js, expected 1 + 1 = 2, but got 6
Incorrect result from js, expected 1 + 1 = 2, but got 5
Incorrect result from js, expected 1 + 1 = 2, but got 6
Incorrect result from js, expected 1 + 1 = 2, but got 3
Incorrect result from js, expected 1 + 1 = 2, but got 3
Incorrect result from js, expected 1 + 1 = 2, but got 13
Incorrect result from js, expected 1 + 1 = 2, but got 10
Incorrect result from js, expected 1 + 1 = 2, but got 6
Incorrect result from js, expected 1 + 1 = 2, but got 13
Incorrect result from js, expected 1 + 1 = 2, but got 5
Incorrect result from js, expected 1 + 1 = 2, but got 6
Incorrect result from js, expected 1 + 1 = 2, but got 11
Incorrect result from js, expected 1 + 1 = 2, but got 9
Incorrect result from js, expected 1 + 1 = 2, but got 7
Incorrect result from js, expected 1 + 1 = 2, but got 11
Incorrect result from js, expected 1 + 1 = 2, but got 11
Incorrect result from js, expected 1 + 1 = 2, but got 7
Incorrect result from js, expected 1 + 1 = 2, but got 9
Incorrect result from js, expected 1 + 1 = 2, but got 3
Incorrect result from js, expected 1 + 1 = 2, but got 3
Incorrect result from js, expected 1 + 1 = 2, but got 13
Incorrect result from js, expected 1 + 1 = 2, but got 11
Incorrect result from js, expected 1 + 1 = 2, but got 12
Incorrect result from js, expected 1 + 1 = 2, but got 8
Incorrect result from js, expected 1 + 1 = 2, but got 4
Incorrect result from js, expected 1 + 1 = 2, but got 4
Incorrect result from js, expected 1 + 1 = 2, but got 4
Incorrect result from js, expected 1 + 1 = 2, but got 8
Incorrect result from js, expected 1 + 1 = 2, but got 9
Incorrect result from js, expected 1 + 1 = 2, but got 3
Incorrect result from js, expected 1 + 1 = 2, but got 5
Incorrect result from js, expected 1 + 1 = 2, but got 3
Incorrect result from js, expected 1 + 1 = 2, but got 9
Incorrect result from js, expected 1 + 1 = 2, but got 6
Incorrect result from js, expected 1 + 1 = 2, but got 7
Overall: 41 wrong values for 1 + 1.

I rewrote in JavaScript and introduced loadWithNewGlobal;

#!/usr/bin/env jjs -scripting

var Executors = java.util.concurrent.Executors;
var TimeUnit  = java.util.concurrent.TimeUnit;
var ArrayList = java.util.ArrayList;

var script = <<EOD
    i = 0;
    i += 1;
    shortly_later = new Date()/1000 + Math.random;
    while( (new Date()/1000) < shortly_later) { Math.random() };
    i += 1;
EOD

function addition() {
    return loadWithNewGlobal({ name: "addition", script: script });
}

var executor = Executors.newCachedThreadPool();
var results = new ArrayList();

for(var i = 0; i < 50; i++) {
    // Clarify Runnable versus Callable
    results.add(executor["submit(java.util.concurrent.Callable)"](addition));
}

var miscalculations = 0;
for each (var result in results) {
    var jsResult = result.get().intValue();
    if (jsResult != 2) {
        print("Incorrect result from js, expected 1 + 1 = 2, but got " + jsResult);
        miscalculations += 1;
    }
}

executor.awaitTermination(1, TimeUnit.SECONDS);
executor.shutdownNow();
print("Overall: " + miscalculations + " wrong values for 1 + 1.");

Works as expected;

Overall: 0 wrong values for 1 + 1.

Comments:

Will your worker API be consistent with proposals for web workers from WHATWG? I appreciate that these aren't set in stone yet, but do you plan to try and stick with their API or are you going in your own direction?

Posted by Richard Warburton on July 09, 2013 at 11:47 PM PDT #

[Note, this is all experimental at this point and should not be construed as a product announcement.] The current prototype has some similarity to WHATWG workers (postmessage, onmessage). However, the goals of Nashorn workers are slightly different. WHATWG workers are geared toward HTML/DOM. Nashorn workers are geared toward generic application use. The underlying mechanism we are currently using is JSON-RPC based. This gives us quite a bit of flexibility for multithread communication and also multi-process, multi-node communication. The goal is to make whether the worker is running locally or remotely transparent to the invoker.

Posted by jlaskey on July 10, 2013 at 06:41 AM PDT #

I much rather prefer being able to instantiate java Threads from within JavaScript programs as you could with Rhino.

Rhino provided a handy function called sync() that returns a synchronized function so you can implement thread safe accesses to shared variables.

Thread isolation is awful for server applications. Instead of just accessing an in-memory variable, you have to either duplicate those variables in separate engine instances (waste of memory), or constantly access (slow) some intermediate cache like memcache.

NodeJS is bound by the lack of thread ability in V8. If they could have used threads, they would have. But since they don't have them, they're stuck with the awful event loop paradigm. Awful? Do an array.sort() of 250 integers and watch the server's req/second get cut in half. If they had threads, a worker could do that sort without a noticeable hit in req/second.

Please don't turn the JVM, which has native threading, into V8.

(see http://silkjs.net)

Posted by guest on October 19, 2013 at 08:36 AM PDT #

You are free to instantiate threads straight up if you are mindful of the issues described in the article.
Many of the examples I've posted around the net use threads as is.

The Worker object is for those who want to use threads without stumbling into thread safety issues. Those that propose Nashorn Workers, claim that Workers are more in the spirit of JavaScript. So, we provide that option.

Rhino could get away with multi-threading because of its interpreter engine and internal synchronization. Nashorn didn't want to sacrifice performance by doing the same kinds synchronization every step of the way.

sync() was recently added to mozilla_compat.js.

Thread isolation was a requirement passed down from the EE folks. When you have 1000s of cores firing aware, any synchronization/coherence point is a major performance hit. Copying is negligible in those contexts.

Avatar.js (Node.js under Nashorn) uses threading internal for multiple event queues and background tasks. The JavaOne demo showed dozens of thumbnail generations from pictures posted from a picture upload website, all done multithreaded.

Again, you are free to use thread/fork/concurrent packages, just be mindful of the gotchas.

Posted by jlaskey on October 21, 2013 at 05:02 AM PDT #

I was playing with this code and observed an interesting side effect related to

executor["submit(java.util.concurrent.Callable)"]

which has an internal type SimpleDynamicMethod and although reports itself as a 'function' does not work if you pass it as function-argument.
There is a detailed question on SF:
(http://stackoverflow.com/questions/19575653/dereferencing-method-variant-in-java-nashorn)
Could you please comment? Thank you.

Posted by guest on November 13, 2013 at 11:58 AM PST #

loadWithNewGlobal() is apparently a Nashorn specific feature; I'm trying to develop Java code that executes scripts in a generic way that knows when to re-use a thread-safe CompiledScript and when it needs to pool a re-usable set of non-thread-safe ones. I can detect if it's thread-safe with this:
Object threading = engine.getFactory().getParameter("THREADING");
and seeing if it's non-null. Rhino is non-null, Nashorn is null. I was hoping that CompiledScript might be Cloneable but it isn't; so I need to re-compile for each new CompiledScript using Nashorn :-(

Does this make sense or is there a better way? I really think CompiledScript should be Cloneable and should have explicit Javadocs alerting readers to the need to check thread-safety before using it in a thread-safe manner.

Posted by David Smiley on May 05, 2014 at 08:41 PM PDT #

CompileScript is a no-op for Nashorn, since Nashorn compiles on demand. I.E., you get the same performance from a straight eval.

There was some debate about how to deliver thread isolation in Nashorn. loadWithNewGlobal was a compromise to allow 'experts' to use Nashorn without the overhead of new globals.

The approach I would recommend in this case is to have a prefix script that is dependent on the engine name. That way you could introduce a 'simple pass through' loadWithNewGlobal into rhino and keep common code otherwise.

Posted by jlaskey on May 06, 2014 at 05:18 AM PDT #

I should note that --global-per-engine will provide Rhino like behaviour.

Posted by jlaskey on May 06, 2014 at 05:32 AM PDT #

Thanks for responding. What do you mean by a prefix-script? A script that I run before the script I really want to run?

What I'm finding frustrating, is that it's very unclear and perhaps impossible (!) to use the javax.script API to work with script engines in a generic sense. Isn't that the point? Otherwise, why even bother with a common API if for each language I need to introduce separate hacks? I'm not talking about a case where concurrent thread executions need to cooperate (which arguably might be advanced/expert) -- they are separate and work with application objects that are already thread-safe or only local to the current thread/execution that I add to the script binding. I'm trying to generically understand how to use the javax.script API to know where I need to synchronize, where I need to pool, or neither, so that the Script object (be it "compiled" or not) and/or "engine" is used safely and efficiently.

Posted by David Smiley on May 06, 2014 at 05:42 AM PDT #

Please check out JSR-223 notes on nashorn @ https://wiki.openjdk.java.net/display/Nashorn/Nashorn+jsr223+engine+notes as well.

Posted by sundararajan on May 06, 2014 at 06:07 AM PDT #

Yes, "A script that I run before the script I really want to run". We do the same internally.

I was discussing your issue with Sundar (hence his link.) My understanding from that conversation is that if you get null from engine.getFactory().getParameter("THREADING") then you should use a separate engine for each thread. This is true for all engines. Obviously, this has performance implications and why we came up with tools like loadWithNewGlobal.

Posted by jlaskey on May 06, 2014 at 07:08 AM PDT #

If you create a function first and keep the variables local to the function you don't have any threading/sharing issues. In your example if you replace the script with:

var script = <<EOD
function go() {
var i = 0;
i += 1;
shortly_later = new Date()/1000 + Math.random;
while( (new Date()/1000) < shortly_later) { Math.random() };
i += 1;

return i;
}

go();
EOD

then it works fine.

Posted by guest on June 13, 2014 at 06:06 AM PDT #

In response to "guest": Doesn't that assume that no other script also defines go() differently? If "go" is global then there is a race condition in which at the time you run go() it may have been changed. I suppose an automated system might randomize the wrapping function name. If all this is true, I claim Nashorn should do this automatically and then get the thread-safety we all want without having to do engine-specific hacks like this.

Posted by David Smiley on June 13, 2014 at 09:14 AM PDT #

How do you think, if one whants to use JVM multithreading in Nashorn, can he use read-write lock to execute new scripts in write locked code block and to execute old scripts in read locked code blocks againsted the same global object? It's considered that read locked code blocks will be run rapidly and write locked code blocks will be run from time to time. Is it effective strategy for isolation of modifying global object process from processes that use it only for read? To guarantee read only use of global, one chould be able to freeze global and then after holding of write lock unfreeze it to freeze it again later. It seems, that it is impossible, but i am very interested in your comments about such approach.

Posted by Marat Gainullin on January 02, 2015 at 03:56 AM PST #

The main issue with a global lock is performance. Having all your threads contending with a global lock will eventually bite you. It would be better to use AtomicReference http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/atomic/package-summary.html.

Posted by jlaskey on January 05, 2015 at 04:46 AM PST #

Ii is considered, that executing new scripts puts some new content into global object (global variables and functions), thus reorganizing it's internal structure. And that is why calls of already executed scripts should be synchronized with executing of new ones. Previous posts, talking about "go" function push me to think about global object reorganization process as of thread-safe already. Because otherwise it will be incorrect to talk about eventually removing of "go" function at all. If global object internal structure is not thread-safe, than any MT manlpulations on a global object is harmful (even executing new scripts). Is it correct?

Posted by Marat Gainullin on January 05, 2015 at 10:31 AM PST #

The global object is never thread safe. It's trivially easy to create a new global (missing 'var'). That is why we recommend the use of loadWithNewGlobal per thread. However, you can define a shared AtomicReference that would always be read ready across all threads.

Posted by jlaskey on January 05, 2015 at 10:41 AM PST #

It seems, that some misunderstood happend. Attila have wrote in OpenJDK mailing list (http://mail.openjdk.java.net/pipermail/nashorn-dev/2013-July/001567.html), that internal structures of library nashorn are thread safe, but javascript programs are not. Is it true? And what if somebody will execute scripts concurrently in the same Global and the same engine. Will the Global scope object itself(not global variables) been destroyed because of concurrently loading scripts, thus concurrently modifying Global's internal structures (propertyMaps, etc)?

Posted by Marat Gainullin on January 06, 2015 at 07:30 AM PST #

What Attila was stating (and information may be dated, not recently checked) was that you can update Nashorn objects concurrently (no crashes), but there are no guarantees of outcome. x.a = 10; on one thread and x.b = 20; on another may or may not result in x having an "a" AND a "b".

The rules described in this article will hold indefinitely. At some point in the future, changes to allow concurrent updating.

Posted by jlaskey on January 06, 2015 at 07:40 AM PST #

Simple fix here (at least for the simple example):

Replace:
onePlusOne.eval();

With:
onePlusOne.eval(new SimpleBindings());

Or, for a slighter more comprehensive approach:
onePlusOne.eval(new SimpleScriptContext());

Granted, once situations are involved that require sharing of state between threads, things get more complicated - but please note that for simple cases where things need to be completely isolated anyway, the above will suffice.

Posted by Mark A. Ziesemer on April 12, 2015 at 09:29 PM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Technical discussions and status of the Nashorn JavaScript Project.

Search

Categories
Archives
« April 2015
SunMonTueWedThuFriSat
   
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
21
22
23
24
25
26
27
28
29
30
  
       
Today