X

Recent Posts

Nashorn

Java source name pattern checker with a nashorn script

Ken Fogel recently tweeted:a,b,c,d,e,f,g,h,I,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y & z are not identifiers. IDE should show an error when an identifier is less than 3 char.While I don't have any IDE solution here, the following Nashorn script may be customized to check class, method and variable name patterns in Java sources. This script uses Javac Tree API to parse java sources. Then it uses a TreeScanner subclass in script to call checkXYZ methods to check class, method and variable names. If any checkXYZName returns false, this script prints a message in standard output with source filename, line number and column number.Perhaps customized version of this script may be used to grade student sources :)/* * Copyright (c) 2016, Oracle and/or its affiliates. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * - Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * - Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * - Neither the name of Oracle nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS * IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */// Usage: jjs checknames.js -- <file-or-directory>if (arguments.length == 0) { print("Usage: jjs checknames.js -- <file-or-directory>"); exit(1);}// Java types usedvar File = Java.type("java.io.File");var Files = Java.type("java.nio.file.Files");var StringArray = Java.type("java.lang.String[]");var ToolProvider = Java.type("javax.tools.ToolProvider");var Tree = Java.type("com.sun.source.tree.Tree");var Trees = Java.type("com.sun.source.util.Trees");var TreeScanner = Java.type("com.sun.source.util.TreeScanner");// customize these functions with checks that you'd want!function checkClassName(name) { return name.length < 3;}function checkMethodName(name) { return name.length < 3;}function checkVarName(name) { return name.length < 3;}function checkNames() { // get the system compiler tool var compiler = ToolProvider.systemJavaCompiler; // get standard file manager var fileMgr = compiler.getStandardFileManager(null, null, null); // Using Java.to convert script array (arguments) to a Java String[] var compUnits = fileMgr.getJavaFileObjects(Java.to(arguments, StringArray)); // create a new compilation task var task = compiler.getTask(null, fileMgr, null, null, null, compUnits); var sourcePositions = Trees.instance(task).sourcePositions; // subclass SimpleTreeVisitor var NameChecker = Java.extend(TreeScanner); var visitor = new NameChecker() { report: function(node) { var pos = sourcePositions.getStartPosition(this.compUnit, node); var line = this.lineMap.getLineNumber(pos); var col = this.lineMap.getColumnNumber(pos); print("Too short name: " + node.name + " @ " + this.fileName + ":" + line + ":" + col); }, // override to capture information on current compilation unit visitCompilationUnit: function(compUnit, p) { this.compUnit = compUnit; this.lineMap = compUnit.lineMap; this.fileName = compUnit.sourceFile.name; return Java.super(visitor).visitCompilationUnit(compUnit, p); }, // override to check class name visitClass: function(node, p) { if (checkClassName(node.simpleName.toString())) { this.report(node); } return Java.super(visitor).visitClass(node, p); }, // override to check method name visitMethod: function(node, p) { if (checkMethodName(node.name.toString())) { this.report(node); } return Java.super(visitor).visitMethod(node, p); }, // override to check variable name visitVariable: function(node, p) { if (checkVarName(node.name.toString())) { this.report(node); } return Java.super(visitor).visitVariable(node, p); } } for each (var cu in task.parse()) { cu.accept(visitor, null); }}// for each ".java" file in directory (recursively).function main(dir) { var totalCount = 0; Files.walk(dir.toPath()). forEach(function(p) { var name = p.toFile().absolutePath; if (name.endsWith(".java")) { checkNames(p); } });}main(new File(arguments[0]));

Ken Fogel recently tweeted: a,b,c,d,e,f,g,h,I,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y & z are not identifiers. IDE should show an error when an identifier is less than 3 char. While I don't have any IDE...

Nashorn

Extracting a single .class file from java9 platform jimage modules file

We can use java FileSystem API from Nashorn JavaScript to extract a single .class file from java9 platform jimage modules file.The following simple Nashorn script extracts .class of the given class name:File: jextract.js// This script extracts .class for a single class// from the platform jimage ($JDK9/lib/modules) file.// Specify module/class like java.base/java.util.Vectorif (arguments.length == 0) { print("Usage: jjs jextract -- <module>/<classname>") exit(1)}// Java types used from file system and net APIvar FileSystems = Java.type("java.nio.file.FileSystems")var Files = Java.type("java.nio.file.Files")var URI = Java.type("java.net.URI")var name = arguments[0]var moduleName = name.substring(0, name.indexOf('/'))var className = name.substring(name.indexOf('/') + 1)var pathName = "/modules/" + moduleName + "/" + className.replace(/\./g, '/') + ".class"// get jrt fs file system instancevar fs = FileSystems.getFileSystem(URI.create("jrt:/"))// read .class contentvar content = Files.readAllBytes(fs.getPath(pathName))// write to a file in current dir (using default fs)// simple class namevar simpleName = className.substring(className.lastIndexOf('.') + 1)Files.write(FileSystems.default.getPath(simpleName + ".class"), content)Example commands to use the above script:$ jjs jextract.js -- java.base/java.util.Vector$ jjs jextract.js -- jdk.scripting.nashorn/jdk.nashorn.api.scripting.ScriptObjectMirror

We can use java FileSystem API from Nashorn JavaScript to extract a single .class file from java9 platform jimage modules file. The following simple Nashorn script extracts .class of the given class...

Java

Accessing Python objects from Nashorn JavaScript

Dynamic linker API for the Java platform (JEP 276) in JDK 9 Early Access defines dynamic linker API for the Java platform. Using this API, different language runtimes implemented for the Java platform can interoperate with each other. Nashorn JavaScript engine has been a part of JDK platform since JDK 8. Jython is an implementation of the Python programming language designed to run on the Java platform. In this blog entry, I'll demonstrate seamless access of Python objects from a Nashorn script!Steps:I downloaded Jython 2.7 standalone jar from jython.org. In Jython, all objects known to the Jython runtime are represented by instances of the class PyObject or one of its subclasses. So, we've to write a pluggable dynalink linker that can link objects of PyObject and subclasses for easier access. JythonLinkerExporter.javadynalink uses java.util.ServiceLoader mechanism to loadpluggable linkers. We've to write a META-INF/services file jdk.dynalink.linker.GuardingDynamicLinkerExporter and put it linker jar.I wrote a nashorn script to compile the linker source and make a jar jython_linker.js In addition, jython_linker.js also runs a sample mixed language script in which Nashorn script accesses Python objects - jython_sample.jsMy directory structure looks as follows:With the above files in place, we just to have run the sample with this command:jjs jython_linker.jsThe output looks as follows:With Dynalink, different language runtimes can interoperate with each other! Have fun with mixed language programming on the Java platform!

Dynamic linker API for the Java platform (JEP 276) in JDK 9 Early Access defines dynamic linker API for the Java platform. Using this API, different language runtimes implemented for the Java platform...

Nashorn

Printing parse trees using nashorn directive prologues

ECMAScript specification allows for "directive prologues" (http://www.ecma-international.org/ecma-262/5.1/#sec-14.1). A directive prologue in an instruction to the ECMAScript engine. Apart from the standard specified "use strict" directive (which makes the particular program or function "strict"), ECMAScript specification allows implementation defined directive prologues as well.Implementations may define implementation specific meanings for ExpressionStatement productions which are not a Use Strict Directive and which occur in a Directive Prologue.. Nashorn supports few directives apart from the standard "use strict" in nashorn debug mode. All nashorn directives start with the "nashorn" word. To trigger the nashorn "debug mode", you need to set the Java System property "nashorn.debug".Two of the nashorn specific directives are "nashorn print ast" and "nashorn print lower ast". If you use these directives at the start of the program or a function, Nashorn prints abstract syntax tree (AST) of the program or the function in the nashorn debug mode. In the non-debug (default) mode, Nashorn specific directives are just ignored. "lower ast" is the AST after nashorn processes AST for non-reachable statements, inlined "finally" blocks are so on - after "lowering" the AST!I'll demonstrate the use of these directives with a simple "automatic semicolon insertion" example. You perhaps omit semicolons and expect the ECMAScript engine to insert semicolons for you. While this seems to work in most cases, there are corner cases.In the following example, where do you think the semicolon is inserted?function func() { // return an object literal with one property or undefined? // in other words, semicolon inserted after "return" or after "object literal"? return { x: 44 }}func()How about printing the AST to see what happens? The same example with the nashorn directive to print AST inserted:function func() { "nashorn print ast"; // ask nashorn to print AST return { x: 44 }}func()jjs -J-Dnashorn.debug=true file.js[function root { @0x057e1b0c] [block body { @0x0ea1a8d5] [statements[0..3]] [expression statements[0] string @0x30a3107a] [literal expression = '"nashorn print ast"' @0x7a765367] [return statements[1] = 'return' [Terminal] @0x17d677df] [block statements[2] { @0x78e67e0a] [block block { @0x0bd8db5a] [statements[0..1]] [label statements[0] ident @0x4b553d26] [block body decimal @0x069a3d1d] [statements[0..1]] [expression statements[0] decimal @0x086be70a] [literal expression = '44' @0x2a556333]As you see, semicolon is inserted after the "return" keyword. So, effectively you've a "return;" - which is return undefined! And there is an unreachable statement after that return statement. That is not object literal expression at all. That is a block statement that has one labeled statement - the labeled statement is labels a literal expression statement!You can print "lowered" AST - AST after a bit of processing by Nashorn - in particular, unreachable statements removed.function func() { "nashorn print lower ast"; return { x: 44 }}func()jjs -J-Dnashorn.debug=true file.jsLower AST for: 'func'[function root { @0x7181ae3f] [block body { [Terminal] @0x1188e820] [statements[0..2]] [expression statements[0] string @0x679b62af] [literal expression = '"nashorn print lower ast"' @0x799d4f69] [return statements[1] = 'return' [Terminal] @0x290dbf45]Now, you can see that the unreachable block statement is removed by Nashorn! Apart from having some debugging fun with Nashorn, the moral of the story is: better to stay away from automatic semicolon insertion as much as possible. Typing semicolons is not that hard really ;)

ECMAScript specification allows for "directive prologues" (http://www.ecma-international.org/ecma-262/5.1/#sec-14.1). A directive prologue in an instruction to the ECMAScript engine. Apart from the...

Java

Writing a pluggable dynalink linker and using it with Nashorn

JEP-276 (JEP 276: Dynamic Linking of Language-Defined Object Models) adds a facility for linking high-level operations on objects such as "read a property", "write a property", "invoke a callable object", etc., expressed as names in INVOKEDYNAMIC call sites. Nashorn JavaScript engine in OpenJDK uses dynalink for linking script and other objects. JEP 276 allows user written ("pluggable") linkers to be loaded via service loader mechanism. Because of Dynalink, Nashorn also allows user-written dynalink linkers to be used along from nashorn scripts! You can write your own linker(s) and drop it as a "jar" in jjs -classpath. Your scripts can make use of those pluggable linkers to handle "special linkages".If this sounds little abstract, you may want to look at the sample dynalink linker in nashorn OpenJDK repository - http://hg.openjdk.java.net/jdk9/dev/nashorn/rev/e9c4c02337cc. This sample adds a simple DOM Element linker DOMLinkerExporter.java. This linker allows scripts to access DOM child elements by child element tag name - rather than having to use DOM Java API to get child Node list and iterate to get the matching Element(s). dom_linker_gutenberg.js script uses "DOM linker". The script accesses child DOM elements by samply "_" followed by child element tag name.You can imagine many other such useful dynalink linkers for your own Java classes or even JDK platform classes.How about a dynalink linker that supports easier access for SQL ResultSet objects - perhaps column names as properties? How about a dynalink linker that handles your domain object model complexity and exposes a simpler script access model -- without having to write any Java or script wrappers around it! All complexity is hidden in the linker code! No Java or script wrapper implies that easy script access does not result in reduced performance - because linking is just (mostly!) one time activity per callsite.

JEP-276 (JEP 276: Dynamic Linking of Language-Defined Object Models) adds a facility for linking high-level operations on objects such as "read a property", "write a property", "invoke a callable...

Java

Using Dynalink API with invokedynamic - a Java assembler example

Dynalink API JEP-276Dynalink API (http://openjdk.java.net/jeps/276) provides a facility for linking high-level operations on objects such as "read a property", "write a property", "invoke a callable object", etc., expressed as names in INVOKEDYNAMIC call sites. Dynalink also provides a default linker for the usual semantics of these operations on plain Java objects (Java POJO "beans" linker), as well as a facility for installing your own language-specific linkers.Dynalink has been used by Nashorn JavaScript engine internally starting from JDK 8. But, Dynalink is exposed as a public API starting from JDK 9. You can use dynalink to simplify invokedynamic generation when compiling your language for the Java platform.To learn Dynalink API from Java code, you can check our dynalink API tests in nashorn repo. test/src/jdk/dynalink directory in Nashorn repo http://hg.openjdk.java.net/jdk9/dev/nashornThere is a simple OpenJDK Java assembler based sample in nashorn repostory - $nashorn/samples/Main.asm - http://hg.openjdk.java.net/jdk9/dev/nashorn/file/376a63a077ee/samples/Main.asm. This example shows how "length" property can be linked using invokedynamic and dynalink. The Java beans linker that comes with Dynalink can link "length" property for Java arrays (array length) and Java Lists (List.size()). The example shows how "length" works for arrays and lists uniformly.

Dynalink API JEP-276 Dynalink API (http://openjdk.java.net/jeps/276) provides a facility for linking high-level operations on objects such as "read a property", "write a property", "invoke a...

Nashorn

Remote debugging of nashorn scripts with NetBeans IDE using "debugger" statements

You can debug nashorn scripts - even if you do not create any NetBeans project or even open the JavaScript files in NetBeans! You just need to connect NetBeans debugger to a remote Java process that evaluates Nashorn scripts! You can use ECMAScript debugger statements to stop the script execution at the places of interest. Start the process running your script with debugger agentI'm using "jjs" tool to run a sample script - but you may use any java application that evaluates Nashorn scripts via javax.script API.File: test.jsprint("in test script");debugger;var obj = { foo: 2 };obj.foo++;print(obj.foo);jjs command with debugger turned onjjs -J-agentlib:jdwp=server=y,suspend=y,transport=dt_socket testListening for transport dt_socket at address: 50349Set breakpoint in nashorn's internal DEBUGGER methodNashorn compiles ECMAScript "debugger" statements as calls to jdk.nashorn.internal.runtime.ScriptRuntime.DEBUGGER method.Use Debug->New Breakpoint... menu to create a new breakpoint. Add breakpoint in jdk.nashorn.internal.runtime.ScriptRuntime.DEBUGGER method.Attach NetBeans to remote processUse Debug->Attach Debugger... menu to attach NetBeans to the debuggee processGetting to 'debugger' statementsWhenever script executes ECMAScript "debugger" statement, breakpoint hits into that nashorn DEBUGGER method. On pressing F8, execution control reaches the "debugger" statement in script file. NetBeans automatically opens the script file and stops at the "debugger" statement!

You can debug nashorn scripts - even if you do not create any NetBeans project or even open the JavaScript files in NetBeans! You just need to connect NetBeans debugger to a remote Java process that...

Java

Dynamic linker API for the Java platform (JEP 276)

JEP 276 defines dynamic linker API for Java. This JEP provides "facility for linking high-level operations on objects such as "read a property", "write a property", "invoke a callable object", etc., expressed as names in INVOKEDYNAMIC call sites. Provide a default linker for the usual semantics of these operations on plain Java objects, as well as a facility for installing language-specific linkers.Nashorn JavaScript engine already uses "dynalink" library for linking properties, indexed access, calls on it's (script) objects as well as "foreign"/"host" Java objects (POJOs). With JEP-276, dynalink is exposed as a public (JDK specific) API in the java9 module named "jdk.dynalink".Right now, the source code for JEP-276 lives in "jdk9 sandbox" OpenJDK repository (http://hg.openjdk.java.net/jdk9/sandbox) in the branched named "JEP-276-branch". This will eventually go into jdk9 repository. If you want to play with "dynalink" API, you can check out this forest and build "JEP-276-branch"To play with any Java API, these days I use "jshell" (jshell) tool. The following is a jshell repl sample to demonstrate dynalink API from Java code.import java.lang.invoke.*// dynalink API lives in these packagesimport jdk.dynalink.*import jdk.dynalink.support.*// dynamic 'operation' for a callsite. 'length' propertyOperation op = new NamedOperation(StandardOperation.GET_PROPERTY, "length")// method type of operation to be linked - length is 'int' valueMethodType mt = MethodType.methodType(int.class, Object.class)// callsite descriptorCallSiteDescriptor desc = new CallSiteDescriptor(MethodHandles.publicLookup(), op, mt)// callsiteSimpleRelinkableCallSite cs = new SimpleRelinkableCallSite(desc)// create a linker factoryDynamicLinkerFactory fac = new DynamicLinkerFactory()// create dynalink linkerDynamicLinker linker = fac.createLinker()// link the callsitelinker.link(cs)// invoke it!printf("array size %d\n", (int)cs.getTarget().invoke(new String[10]))import java.util.ArrayList// make a list and populate two elementsArrayList<String> al = new ArrayList<>()al.add("hello")al.add("world")// get 'length' of array list - which is nothing but sizeprintf("list size %d\n", (int)cs.getTarget().invoke(al))The above repl prints "array size 10" and "list size 2" respectively. Note that "length" property is relinked automatically to be array length and then ArrayList size (when a different "this" object is passed). This linking (and relinking) of Java objects is handled by the "java beans linker" that comes with the dynalink implementation.

JEP 276 defines dynamic linker API for Java. This JEP provides "facility for linking high-level operations on objects such as "read a property", "write a property", "invoke a callable object", etc.,...

Java

Underscore is a keyword in Java 9, use this script to check your code!

Underscore ("_") is a keyword in Java 9. If you use "_" as an identifier, javac of JDK 8+ issues a warning. javac of JDK 9, issues an error! To check and migrate your Java code to avoid using "_" as an identifier, you can use the following Nashorn script with jjs tool. This script just parses each .java file and reports "_" variables with filename, line and column numbers. This script uses Javac Tool API and Compiler Tree API.File: find_underscore.js// Usage: jjs find_underscores.js -- <directory>if (arguments.length == 0) { print("Usage: jjs find_underscores.js -- <directory>"); exit(1);}// Java types usedvar File = Java.type("java.io.File");var Files = Java.type("java.nio.file.Files");var StringArray = Java.type("java.lang.String[]");var ToolProvider = Java.type("javax.tools.ToolProvider");var Tree = Java.type("com.sun.source.tree.Tree");var Trees = Java.type("com.sun.source.util.Trees");var TreeScanner = Java.type("com.sun.source.util.TreeScanner");function findUnderscores() { // get the system compiler tool var compiler = ToolProvider.systemJavaCompiler; // get standard file manager var fileMgr = compiler.getStandardFileManager(null, null, null); // Using Java.to convert script array (arguments) to a Java String[] var compUnits = fileMgr.getJavaFileObjects(Java.to(arguments, StringArray)); // create a new compilation task var task = compiler.getTask(null, fileMgr, null, null, null, compUnits); var sourcePositions = Trees.instance(task).sourcePositions; // subclass SimpleTreeVisitor - to find underscore variable names var UnderscoreFinder = Java.extend(TreeScanner); var visitor = new UnderscoreFinder() { // override to capture information on current compilation unit visitCompilationUnit: function(compUnit, p) { this.compUnit = compUnit; this.lineMap = compUnit.lineMap; this.fileName = compUnit.sourceFile.name; return Java.super(visitor).visitCompilationUnit(compUnit, p); }, // override to check variable name visitVariable: function(node, p) { if (node.name.toString() == "_") { var pos = sourcePositions.getStartPosition(this.compUnit, node); var line = this.lineMap.getLineNumber(pos); var col = this.lineMap.getColumnNumber(pos); print(node + " @ " + this.fileName + ":" + line + ":" + col); } return Java.super(visitor).visitVariable(node, p); } } for each (var cu in task.parse()) { cu.accept(visitor, null); }}// for each ".java" file in directory (recursively).function main(dir) { var totalCount = 0; Files.walk(dir.toPath()). forEach(function(p) { var name = p.toFile().absolutePath; if (name.endsWith(".java")) { findUnderscores(p); } });}main(new File(arguments[0]));

Underscore ("_") is a keyword in Java 9. If you use "_" as an identifier, javac of JDK 8+ issues a warning. javac of JDK 9, issues an error! To check and migrate your Java code to avoid using "_" as...

Nashorn

Nashorn js extension for NetLogo - implementing NetLogo commands, reporters in scripts!

Earlier I blogged about playing with NetLogo using Nashorn. NetLogo, as you'd recall, is a multi-agent programmable modeling environment for the Java platform. In addition to supporting NetLogo programming language, NetLogo supports extensions written in JVM languages such as Java, Scala. To write extensions, NetLogo supports an Extensions API. Writing a NetLogo extension in Java or Scala or any compiled JVM language means that you've to compile and make a jar to extend NetLogo. But, how about a NetLogo extension commands and reports in JavaScript? If that is supported, user can write and load JavaScript files to extend NetLogo. No "compile and make jar" etc. Just load script files that implement extension commands, reporters!I used NetLogo Extension API to write a simple Nashorn engine baed "js" extension. This extension supports just few NetLogo reports such as - "eval" to evaluate a given String as Nashorn script and "load" to load script file (or URL). The loaded scripts can add more NetLogo commands, reporters using scripting API for extensions. This allows arbitrary scripts to implement NetLogo extension commands and extensions. To use "js" extension, you've to use JDK 8+ to run NetLogo. NetLogo executable uses bundled JRE to run which is not JDK 8. So, we've to use "java -jar NetLogo.jar" to run NetLogo with JDK8+ java.Note: This is just a proof-of-concept/weekend hack :) This is not production quality code. I'd leave that to NetLogo community :)File: NashornExtension.javaimport org.nlogo.api.*;import javax.script.*;import java.util.Objects;import java.io.InputStreamReader;// Simple NetLogo JavaScript extension// Supports reporters and commands defined by scriptpublic class NashornExtension extends DefaultClassManager { public void load(PrimitiveManager primitiveManager) { // create ScriptEngine and expose primitiveManager as a global variable final ScriptEngineManager m = new ScriptEngineManager(); final ScriptEngine e = m.getEngineByName("nashorn"); // expose primitive manager as a global variable to scripts e.put("primitiveManager", primitiveManager); // load "init" script to expose script friendly functions to define // reporters and commands in loaded scripts! User can load additional // scripts using "eval" or "load" reporters defined in init.js try { e.eval(new InputStreamReader(NashornExtension.class.getResourceAsStream("init.js"))); } catch (Exception exp) { throw new RuntimeException(exp); } }}File: init.js// init script that is run when NetLogo 'js' extension is loaded// useful classes from NetLogo APIvar DefaultCommand = Java.type("org.nlogo.api.DefaultCommand");var DefaultReporter = Java.type("org.nlogo.api.DefaultReporter");var Syntax = Java.type("org.nlogo.api.Syntax");// helper to define a new NetLogo reporterfunction addReporter(name, argTypes, returnType, callback) { primitiveManager.addPrimitive(name, new (Java.extend(DefaultReporter))() { getSyntax: function() { return Syntax.reporterSyntax(argTypes, returnType); }, report: callback });}// helper to define a new NetLogo commandfunction addCommand(name, argTypes, callback) { primitiveManager.addPrimitive(name, new (Java.extend(DefaultCommand))() { getSyntax: function() { return Syntax.commandSyntax(argTypes); }, perform: callback });}// "eval" reporter that evaluates JS code and returns result as a StringaddReporter("eval", [ Syntax.StringType() ], Syntax.StringType(), function (args, ctx) { return String(eval(args[0].string)); });// "eval" reporter that evaluates JS code and returns result as a NumberaddReporter("eval-num", [ Syntax.StringType() ], Syntax.NumberType(), function (args, ctx) { return Number(eval(args[0].string)); });// reporter to load a JS script File OR URLaddReporter("load", [ Syntax.StringType() ], Syntax.StringType(), function (args, ctx) { return String(load(args[0].string)); });// commands and reporters to put/get JS global variable as number/string(function () { var global = this; addCommand("put-num", [ Syntax.StringType(), Syntax.NumberType() ], function (args, ctx) { global[args[0].string] = args[1].doubleValue; }); addCommand("put-str", [ Syntax.StringType(), Syntax.StringType() ], function (args, ctx) { global[args[0].string] = args[1].string; }); addReporter("get-num", [ Syntax.StringType() ], Syntax.NumberType(), function (args, ctx) { return Number(global[args[0].string]); }); addReporter("get-str", [ Syntax.StringType() ], Syntax.StringType(), function (args, ctx) { return String(global[args[0].string]); });})();File: sample.js// simple sample implementing reporters and commands// export Math functions as NetLogo reporters(function() { addReporter("random", [], Syntax.NumberType(), Math.random); var funcs = [ "abs","acos","asin","atan","ceil", "cos", "floor","log", "round", "sin", "sqrt", "tan" ]; function makeReporter(func) { return function(args, cx) { return func(args[0].doubleValue); } } for each (var f in funcs) { addReporter(f, [ Syntax.NumberType() ], Syntax.NumberType(), makeReporter(Math[f])); } function makeReporter2(func) { return function(args, cx) { return func(args[0].doubleValue, args[1].doubleValue); } } var funcs2 = [ "atan2", "max", "min", "pow" ]; for each (var f in funcs2) { addReporter(f, [ Syntax.NumberType(), Syntax.NumberType() ], Syntax.NumberType(), makeReporter2(Math[f])); }})();// a 'dump' command to dump stack traceaddCommand("dump", [], Error.dumpStack);// 'exit' commandaddCommand("exit", [], exit);"loaded sample.js";If you want to play with this Nashorn "js" extension, you may want to download netlogoext.zip. I tested this on Windows using JDK 8u60 and NetLogo 5.2.1. YMMV :)Sample NetLogo session with 'js' extension usage:

Earlier I blogged about playing with NetLogo using Nashorn. NetLogo, as you'd recall, is a multi-agent programmable modeling environment for the Java platform. In addition to supporting NetLogo...

Nashorn

Fun with NetLogo using Nashorn JavaScript engine

NetLogo is a multi-agent programmable modeling environment running for the Java platform. In addition to being a GUI program to edit and run NetLogo models, NetLogo supports Controlling API so that NetLogo can be invoked and controlled by a program running on JVM. The "controlling api" supports both GUI and 'headless' mode.I used Nashorn javascript engine to use NetLogo "controlling API" from a script. I ran the bundled Maxwell's demon model in "headless mode" using the following Nashorn script:File: nl_maxwell_demon.js// NetLogo "controlling API" example in Nashorn// java classes usedvar HeadlessWorkspace = org.nlogo.headless.HeadlessWorkspace;var Thread = java.lang.Thread;// set the thread context loader for correct working of NetLogo!Thread.currentThread().contextClassLoader = HeadlessWorkspace.class.classLoader;// new headless workspacevar ws = HeadlessWorkspace.newInstance();// change this as per your installation directory!// My NetLogo installation dir is "C:\Program Files (x86)\NetLogo 5.2.1"var NL = "C:\\Program Files (x86)\\NetLogo 5.2.1\\";ws.open(NL + "models\\Curricular Models\\GasLab\\GasLab Maxwells Demon.nlogo");// setup and call "go" (100 iterations)ws.command("setup");ws.command("repeat 100 [ go ]");// print average left and right speedsprint("average speed (left) = " + ws.report("avg-speed-left"));print("average speec (right) = " +ws.report("avg-speed-right"));// We're done!ws.dispose();I used the jjs nashorn shell command to run the above script:$ jjs -cp "C:\\Program Files (x86)\\NetLogo 5.2.1\\NetLogo.jar" nl_maxwell_demon.jsaverage speed (left) = 8.712072244000128average speec (right) = 9.08120463863075

NetLogo is a multi-agent programmable modeling environment running for the Java platform. In addition to being a GUI program to edit and run NetLogo models, NetLogo supports Controlling API so that...

Java

Playing with Java (java9) REPL - an example that uses nashorn engine in REPL

I love "Exploratory Programming" (https://en.wikipedia.org/wiki/Exploratory_programming) tools. If you've not already played with "Java REPL" project, you may want to do so! Check out Kulla project.Clone kulla forest @ http://hg.openjdk.java.net/kulla and build using these commands:hg clone http://hg.openjdk.java.net/kulla/dev kulla-devcd kulla-devsh get_sources.shsh configure --with-boot-jdk=/path/to/jdk1.8.0make clean imagesmake install Once build finishes, you can use the Java REPL tool "jshell" ( $kulla-dev/build//images/jdk/bin/jshell). I built on Windows. My jshell path is D:\src\kulla-dev\build\windows-x86_64-normal-server-release\images\jdk\bin\jshell.exe.Now, two lines to print the squares of integers in the range [0, 100):D:\src\kulla-dev\build\windows-x86_64-normal-server-release\images\jdk\bin>jshell| Welcome to JShell -- Version 1.9.0-internal| Type /help for help-> import java.util.stream.*-> IntStream.range(0, 100).map(x->x*x).forEach(System.out::println)Slightly bigger repl sample that pulls weather data in JSON format, uses Nashorn script engine to parse JSON and print the statistics on it using Streams API.File: weather.replimport java.net.*import java.io.*import java.util.stream.*import javax.script.*import java.util.*// URL to fetch JSON for weather data for Chennai, IndiaURL u = new URL( "http://api.openweathermap.org/data/2.5/forecast/daily?q=Chennai&mode=json&units=metric&cnt=7")// read text from URLString getText(URL u) { StringBuilder buf = new StringBuilder(); try (BufferedReader reader = new BufferedReader( new InputStreamReader(u.openStream()))) { reader.lines().forEach(l->buf.append(l)); } catch (IOException exp) { exp.printStackTrace(); } return buf.toString();}// create nashorn engineScriptEngine e = new ScriptEngineManager().getEngineByName("js")// expose weather data JSON as global variable to nashorn enginee.put("str", getText(u))// massage JSON using nashorn and get the max. temp valuesdouble[] values = (double[]) e.eval( "Java.to(JSON.parse(str).list.map(function(val) val.temp.max), Java.type('double[]'))")// stat on max. temp valuesprintf(DoubleStream.of(values).summaryStatistics().toString())You can evaluate the above source as follows:jshell weather.repl| Welcome to JShell -- Version 1.9.0-internal| Type /help for help-> DoubleSummaryStatistics{count=7, sum=227.160000, min=30.210000, average=32.451429, max=36.580000}Yep, Chennai is HOT! 32 C average! In Chennai, we have only three seasons - hot, hotter and the hottest :)

I love "Exploratory Programming" (https://en.wikipedia.org/wiki/Exploratory_programming) tools. If you've not already played with "Java REPL" project, you may want to do so! Check out Kulla project. Clo...

Nashorn

Playing with Nerdamer symbolic math JS library with Nashorn jjs

I was searching for a self-contained, simple JS library for symbolic math to check homework solutions of my son. On googling "symbolic math javascript", the first hit I got was http://www.nerdamer.com/I git cloned the source from the git repo:$ cd ~/src$ git clone https://github.com/jiggzson/nerdamer$ cd nerdamerThere is a nice documentation for nerdamer @ http://www.nerdamer.com/documentation I started Nashorn jjs shell. The following is from my jjs session with nerdamer.$ lsAlgebra.js changelog.md nerdamer.core.js Solve.jsCalculus.js license.txt README.mdNote: I used rlwrap - http://freecode.com/projects/rlwrap - for better line editing support. Note that rlwrap is not needed for jdk9 builds.jjs in jdk9 build has integrated better line editing support, tab-completion etc using "jline". "rlwrap" is suggested only for jdk8u builds.$ rlwrap jjs -scriptingjjs> load("nerdamer.core.js")jjs> load("Algebra.js")jjs> load("Calculus.js")jjs> load("Solve.js")jjs> nerdamer('sqrt(x)*sqrt(x)-2').evaluate({x:2})0jjs> nerdamer("diff(tan(x), x)")sec(x)^2jjs> nerdamer.solveEquations("x^2 + 1 = 0", "x")i,-ivar f = nerdamer("x^3 + sin(x)").buildFunction()jjs> ffunction (x) {return (Math.pow(x,3)+Math.sin(x));}jjs> f(Math.PI)31.006276680299816I really like this self-contained JS library. No browser specific stuff (window.call_crap_function() --so that I didn't have to introduce fake helper functions! And no "require('foo.js')" module stuff -- so that I didn't have to fake a require using nashorn's "load" function. Nerdamer works as advertised and runs smooth on Nashorn!

I was searching for a self-contained, simple JS library for symbolic math to check homework solutions of my son. On googling "symbolic math javascript", the first hit I got was http://www.nerdamer.com/...

General

Graphs, GXL, dot and Graphviz

Sometimes you may want to quickly generate graphs programmatically and view/analyze those. Examples include, inheritance/type relation diagrams of an object oriented program, function call graphs and any other domain specific graphs (reporting chain of your organization chart for example). I find GXL very useful for this. GXL stands for Graph eXchange Language. It is a simple XML format to specify graphs. A simple graph stating that "JavaFX" language is related to "Java" language is as follows:File: Test.gxl<gxl><!-- edgemode tells this is directed or undirected graph --><graph id="langs" edgemode="directed"><!-- write your nodes --><node id="java"/><node id="javafx"/><-- now write your edges --><edge from="java" to="javafx"/></graph></gxl> You can also add number of "attributes" to nodes and edges - like color of the edge, style of the edge and so on. For example, "red" color can be specified for an edge as follows:<edge from="java" to="javafx"> <attr name="color"><string>red</string></attr></edge>Now that we have written a simple graph with two nodes and a single edge between them, we may want to view it. There are number of tools/libraries to view GXL documents -- I've used Graphviz. Graphviz displays it's own native format called ".dot". Graphviz comes with a set of command line tools. One such tool is "gxl2dot", which as you'd have guessed, can be used to convert a .gxl file to a .dot file. gxl2dot Test.gxl > Test.dotOnce converted the .dot file can be opened in Graphviz GUI and we can export it to .pdf/.jpg/.png and so on. This way you can email the graphs to others and/or publish in your blogs/webpages easily.The converted .pdf file for the above simple graph is here:test.pdfI've used GXL graphs in a recent debugging tool related to JavaFX compiler. More on that later...

Sometimes you may want to quickly generate graphs programmatically and view/analyze those. Examples include, inheritance/type relation diagrams of an object oriented program, function call graphs and...

JavaFX

java.lang.NoClassDefFoundError: com/sun/javafx/runtime/FXBase

I created a simple JavaFX applet and compiled it with "javafxc" and created a jar "HelloApplet.jar". Then, I created a simple HTML file as follows:<script src="http://dl.javafx.com/dtfx.js"></script><script> javafx( { archive: "HelloApplet.jar", draggable: true, width: 150, height: 100, code: "hello.HelloApplet", name: "HelloApplet" } );</script>When viewing the HTML, I did not see the expected applet content in it - just a gray rectangle panel :-( Then, I turned on the "Show Java Console" option. I saw the following exception trace in Java console:JNLPAppletLauncher: static initializerException in thread "AWT-EventQueue-2" java.lang.NoClassDefFoundError: com/sun/javafx/runtime/FXBaseat java.lang.ClassLoader.defineClass1(Native Method)at java.lang.ClassLoader.defineClass(ClassLoader.java:675)at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)at java.net.URLClassLoader.access$100(URLClassLoader.java:56)at java.net.URLClassLoader$1.run(URLClassLoader.java:195)[.. more stack frames deleted for brevity ..]Aha! com.sun.javafx.runtime.FXBase is new since JavaFX 1.2. Yes, I had compiled my applet with JavaFX 1.2 (full version "1.2.0_b233"). So, the JavaFX runtime used turned out to be old one! I changed the HTML script URL from http://dl.javafx.com/dtfx.js to http://dl.javafx.com/1.2/dtfx.js. The applet worked as expected :-)

I created a simple JavaFX applet and compiled it with "javafxc" and created a jar "HelloApplet.jar". Then, I created a simple HTML file as follows: <script src="http://dl.javafx.com/dtfx.js"></script><s...

education

Teaching programming to kids

I've been experimenting/playingwith programming tools for kids. I've two kids - 8 and 5 year old.ScratchGreat multimedia rich introduction to programming for kids. My son keeps playing with it! If you are after ease-of-use, this is probably the bestchoice. Available for download on Mac and Windows. If you haveSqueak installed, you can get Scratchrunning on Linux as well (possibly with some audio problems). It is possibleto export your Scratch projects as applets from your websiteWebsite: http://scratch.mit.eduTutorials:http://learnscratch.org/ hashas nice screen-cast tutorials that cover basics very well.http://www.redware.com/scratch/ has nice tutorials and teaching resources that eventually leadskids to develop interesting gamesExamples/Repository: Register @ scratch.mit.edu to download lots of great samples!You can view scratch programs online if you have Java plugin installed.EToysEToys is an tool to teach children a multimedia rich authoring environment and visual programming system. If you have OLPC laptop (or emulation environment for OLPC), it comes pre-installed with EToys. Or else you can download EToys for yourplatform. It is avaliable for Linux, Mac and Windows.Website: www.squeakland.orgTutorials:Excellent step-by-step video tutorials @ http://freemor.wordpress.com/tag/etoys/http://www.squeakland.org/tutorials/Examples: http://www.etoysillinois.org/StarLogoTNGAlthough StarLogoTNG is described as "tool to create and understand simulations of complex systems", it can be used as a tool to teach programming.It supports 3D and it is cross-platform a (Java) tool.Website: http://education.mit.edu/drupal/starlogo-tngTutorial: http://education.mit.edu/drupal/starlogo-tng/learnTutorial video at http://education.mit.edu/starlogo-tng/tutorial-videos/. These videosdescribe slightly different interface. Menus/buttons have changed a bit inthe recent version but it is still very useful.Examples: I am looking at Starlogo TNG Treasure Game CurriculumAliceThis is an environment to teach programming in a 3D environment. Thisis a cross-platform (Java) tool.Website: http://www.alice.org.Tutorial: There is a very nice tutorial series of worlds within the tool.http://www.aliceprogramming.net/Turtle ArtTurtle Art activity is bundled with

I've been experimenting/playing with programming tools for kids. I've two kids - 8 and 5 year old. Scratch Great multimedia rich introduction to programming for kids. My son keepsplaying with it! If you...

JavaFX

JavaFX interactive shell

JavaFX compiler has a built-in script shell - Per Bothner has implemented a read-eval-print loop facility for JavaFX. The script shell class is com.sun.tools.javafx.script.ScriptShell.Note:This is in the openjfx-compiler repository and not in the JavaFX 1.0 binary.A sample JavaFX session is as follows:$ java -cp dist/lib/shared/javafxc.jar com.sun.tools.javafx.script.ScriptShell /\*fx1\*/ "hello"hello/\*fx2\*/ 2 + 46/\*fx3\*/ function greet (name) { println("Hello, {name}") }/\*fx4\*/ greet("Sundar")Hello, Sundar/\*fx5\*/ var s = [ "Sunday", "Monday", "Tuesday" ][ Sunday, Monday, Tuesday ]/\*fx6\*/ for (i in s) println(i) SundayMondayTuesday/\*fx7\*/ class Person { public var name: String; } /\*fx8\*/ var p = Person { name: "Sundar" }fx7$Person@13fba1/\*fx9\*/ p.nameSundar/\*fx10\*/ import javax.swing.\*;/\*fx11\*/ var f = new JFrame("hello");fx11:1: cannot find symbolsymbol : class JFramelocation: class fx11/\*fx12\*/ /\*fx13\*/ var f = new javax.swing.JFrame("hello")javax.swing.JFrame[frame0,0,0,0x0,invalid,hidden,layout=java.awt.BorderLayout,title=hello,resizable,normal,defaultCloseOperation=HIDE_ON_CLOSE,rootPane=javax.swing.JRootPane[,0,0,0x0,invalid,layout=javax.swing.JRootPane$RootLayout,alignmentX=0.0,alignmentY=0.0,border=,flags=16777673,maximumSize=,minimumSize=,preferredSize=],rootPaneCheckingEnabled=true]/\*fx14\*/ f.setSize(300, 300)/\*fx15\*/ f.setVisible(true)While there is a generic shell called "jrunscript" for languages that implement jsr-223, it does not work very well for statically typed languages (which can be said about jsr-223 itself). But, the above mentioned JavaFX script shell is independent of jsr-223 interface. Besides, jrunscript crashes for JavaFX!$ jrunscript -cp dist/lib/shared/javafxc.jar -l javafxfx> "hello"Exception in thread "main" java.lang.NoSuchMethodError: com.sun.tools.javac.main.RecognizedOptions.getJavacFileManagerOptions(Lcom/sun/tools/javac/main/RecognizedOptions$OptionHelper;)[Lcom/sun/tools/javac/main/JavacOption$Option;at com.sun.tools.javac.util.JavacFileManager.(JavacFileManager.java:973)at com.sun.tools.javafx.api.JavafxcTool.getStandardFileManager(JavafxcTool.java:102)at com.sun.tools.javafx.script.JavaFXScriptCompiler.(JavaFXScriptCompiler.java:87)at com.sun.tools.javafx.script.JavaFXScriptContext.(JavaFXScriptContext.java:45)at com.sun.tools.javafx.script.JavaFXScriptEngineImpl.getJavaFXScriptContext(JavaFXScriptEngineImpl.java:61)at com.sun.tools.javafx.script.JavaFXScriptEngineImpl.getJavaFXScriptContext(JavaFXScriptEngineImpl.java:55)at com.sun.tools.javafx.script.JavaFXScriptEngineImpl.parse(JavaFXScriptEngineImpl.java:220)at com.sun.tools.javafx.script.JavaFXScriptEngineImpl.eval(JavaFXScriptEngineImpl.java:173)at com.sun.tools.javafx.script.JavaFXScriptEngineImpl.eval(JavaFXScriptEngineImpl.java:164)at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:247)at com.sun.tools.script.shell.Main.evaluateString(Main.java:280)at com.sun.tools.script.shell.Main.processSource(Main.java:249)at com.sun.tools.script.shell.Main.access$100(Main.java:19)at com.sun.tools.script.shell.Main$1.run(Main.java:165)at com.sun.tools.script.shell.Main.main(Main.java:30)This is because JavaFX compiler code depends on modified javac classes. The code for jrunscript lives in tools.jar along with the unmodified javac classes! These unmodified javac classes are loaded which results in exception from javafxc code! We have to use the bootstrap path workaround as shown below:$ jrunscript -J-Xbootclasspath/p:./dist/lib/shared/javafxc.jar -l javafxfx> 233 + 334567fx> for (i in [0..4]) println(i)01234fx> But, now we have a much better ScriptShell for JavaFX :-) We don't need to resort to this hack...

JavaFX compiler has a built-in script shell - Per Bothner has implemented a read-eval-print loop facility for JavaFX. The script shell class is com.sun.tools.javafx.script.ScriptShell.Note:This is in...

General

Installed Ubuntu 8.10 on my PS3

Ubuntu on PS3I wanted to install Ubuntu on my PlayStation 3.My SetupPS3 is NTSC 60 GB hard disk version - updated with firmware version 2.42.PS3 is connected to 32 inch 720p Sony LCD TV via HDMI.PS3 is connected to wireless network.Stuff needed in addition to the aboveUSB keyboard.USB mouse.CD burned with Ubuntu powerpc iso (ubuntu-8.10-alternate-powerpc+ps3.iso).Preparing PS3Backup your PS3 hard disk using [Settings] -> [System Settings] -> [Backup Utility] menu.I didn't bother to backup the hard disk.Go to [Settings] -> [System Settings] > [Format Utility] menu.Select [Format Hard Disk] and click [Yes].Choose [Custom] and [Allot 10GB to the Other OS].Select [Quick Format] and confirm with [Yes].Installing UbuntuConnect USB keyboard and mouse to PS3.Insert the disk with Ubuntu iso image into PS3.Go to [Settings] -> [System Settings] > [Install Other OS]. PS3 will detect the install CD and copy filesand instruct you to ..Select [Settings] -> [System Settings] -> [Default System] -> [Other OS].This will boot PS3 with other OS. From then onwards, follow the Ubuntu installation instructions.Small hiccupThe installation was smooth except for one small issue - the installation seemed to hang in "Select and install software" step. After 6% the progress bar did not increase at all! Fortunately, this seems to be a known issue with text mode installer. Please refer to Ubuntu 8.10 release notes and bug 290234. I pressed Alt-F4 and Alt-F1 to toggle between logging console and main screen to check the progress. Eventually, the installation completed! While installing I configured network as well -- i.e., giving WEP password etc. -- not sure if this is mandatory, but in my case I have wireless connectivity and so I supplied the configuration values for the same.Switching between Operating SystemsFrom Ubuntu to Game OS, use the command sudo boot-game-osFrom Game OS to Ubuntu, use the menu [Settings] -> [System Settings] -> [Default System] -> [Other OS]Few ScreenshotsInstall ScreenLogin ScreenGNU chessNetwork update

Ubuntu on PS3 I wanted to install Ubuntu on my PlayStation 3. My Setup PS3 is NTSC 60 GB hard disk version - updated with firmware version 2.42. PS3 is connected to 32 inch 720p Sony LCD TV via HDMI. PS3...

Java

Debugging option for javac and javafxc

I work on JavaFX compiler these days. The command line (debugging) option that I often use is -doe ("dump on error"). This option prints stack trace of the compiler when error message is printed. NOTE: This is an internal option and can be removed any time without notice! But, it is useful for debugging. This option works for javac as well as javafxc. When I misspelled "class" as "clas" and run compiler with -doe option, I got the stack trace below: $ javac -doe t.javat.java:1: class, interface, or enum expectedclas t {}\^java.lang.RuntimeExceptionat com.sun.tools.javac.util.Log.writeDiagnostic(Log.java:565)at com.sun.tools.javac.util.Log.report(Log.java:523)at com.sun.tools.javac.util.Log.error(Log.java:404)at com.sun.tools.javac.parser.Parser.reportSyntaxError(Parser.java:282)at com.sun.tools.javac.parser.Parser.syntaxError(Parser.java:267)at com.sun.tools.javac.parser.Parser.classOrInterfaceOrEnumDeclaration(Parser.java:2206)at com.sun.tools.javac.parser.Parser.typeDeclaration(Parser.java:2180)at com.sun.tools.javac.parser.Parser.compilationUnit(Parser.java:2126)at com.sun.tools.javac.main.JavaCompiler.parse(JavaCompiler.java:509)at com.sun.tools.javac.main.JavaCompiler.parse(JavaCompiler.java:550)at com.sun.tools.javac.main.JavaCompiler.parseFiles(JavaCompiler.java:801)at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:727)at com.sun.tools.javac.main.Main.compile(Main.java:353)at com.sun.tools.javac.main.Main.compile(Main.java:279)at com.sun.tools.javac.main.Main.compile(Main.java:270)at com.sun.tools.javac.Main.compile(Main.java:69)at com.sun.tools.javac.Main.main(Main.java:54)1 error

I work on JavaFX compiler these days. The command line (debugging) option that I often use is -doe ("dump on error"). This option prints stack trace of the compiler when error message is printed. NOTE:...

Java

BTrace in the real world

In the last few weeks, I came to know about two cases of real world use of BTrace. Glencross, Christian M (his blog?) wrote about attempting to write a script to track SQL statements executed by a Java application (private email). Thanks to him for permitting me to blog about his BTrace script. I've made few formatting changes to fit his code in this blog and added few explanatory comments (staring with "VERBOSE:").import static com.sun.btrace.BTraceUtils.\*;import java.sql.Statement;import java.util.Map;import java.util.concurrent.atomic.AtomicLong;import com.sun.btrace.\*;import com.sun.btrace.annotations.\*;/\*\* \* BTrace script to print timings for all executed JDBC statements on an event. \* <p> \* \* @author Chris Glencross \*/@BTracepublic class JdbcQueries { private static Map preparedStatementDescriptions = newWeakMap(); private static Map statementDurations = newHashMap(); // VERBOSE: @TLS makes the field "thread local" -- sort of like using java.lang.ThreadLocal @TLS private static String preparingStatement; @TLS private static long timeStampNanos; @TLS private static String executingStatement; /\*\* \* If "--stack" is passed on command line, print the Java stack trace of the JDBC statement. \* \* VERBOSE: Command line arguments to BTrace are accessed as $(N) where N is the command line arg position. \* \* Otherwise we print the SQL. \*/ private static boolean useStackTrace = $(2) != null && strcmp("--stack", $(2)) == 0; // The first couple of probes capture whenever prepared statement and callable statements are // instantiated, in order to let us track what SQL they contain. /\*\* \* Capture SQL used to create prepared statements. \* \* VERBOSE: +foo in clazz means foo and it's subtypes. Note the use of regular expression \* for method names. With that BTrace matches all methods starting with "prepare". The \* type "AnyType" matches any Java type. \* \* @param args - the list of method parameters. args[1] is the SQL. \*/ @OnMethod(clazz = "+java.sql.Connection", method = "/prepare.\*/") public static void onPrepare(AnyType[] args) { preparingStatement = useStackTrace ? jstackStr() : str(args[1]); } /\*\* \* Cache SQL associated with a prepared statement. \* \* VERBOSE: By default, @OnMethod matches method entry points. Modifying with @Location \* annotation to match the method return points. \* \* @param arg - the return value from the prepareXxx() method. \*/ @OnMethod(clazz = "+java.sql.Connection", method = "/prepare.\*/", location = @Location(Kind.RETURN)) public static void onPrepareReturn(AnyType arg) { if (preparingStatement != null) { print("P"); // Debug Prepared Statement preparedStatement = (Statement) arg; put(preparedStatementDescriptions, preparedStatement, preparingStatement); preparingStatement = null; } } // The next couple of probes intercept the execution of a statement. If it execute with no-args, // then it must be a prepared statement or callable statement. Get the SQL from the probes up above. // Otherwise the SQL is in the first argument. @OnMethod(clazz = "+java.sql.Statement", method = "/execute.\*/") public static void onExecute(AnyType[] args) { timeStampNanos = timeNanos(); if (args.length == 1) { // No SQL argument; lookup the SQL from the prepared statement Statement currentStatement = (Statement) args[0]; // this executingStatement = get(preparedStatementDescriptions, currentStatement); } else { // Direct SQL in the first argument executingStatement = useStackTrace ? jstackStr() : str(args[1]); } } @OnMethod(clazz = "+java.sql.Statement", method = "/execute.\*/", location = @Location(Kind.RETURN)) public static void onExecuteReturn() { if (executingStatement == null) { return; } print("X"); // Debug Executed long durationMicros = (timeNanos() - timeStampNanos) / 1000; AtomicLong ai = get(statementDurations, executingStatement); if (ai == null) { ai = newAtomicLong(durationMicros); put(statementDurations, executingStatement, ai); } else { addAndGet(ai, durationMicros); } executingStatement = null; } // VERBOSE: @OnEvent probe fires whenever BTrace client sends "event" command. // The command line BTrace client sends BTrace events when user pressed Ctrl-C // (more precisely, on receiving SIGINT signal) @OnEvent public static void onEvent() { println("---------------------------------------------"); printNumberMap("JDBC statement executions / microseconds:", statementDurations); println("---------------------------------------------"); }}And he has expressed few wish lists for BTrace based on his experience with DTrace. We plan to investigate those items in near future.Binod P.G exchanged private e-mails about BTrace usage to track down a memory leak. Subsequently, he has blogged about the same.

In the last few weeks, I came to know about two cases of real world use of BTrace.Glencross, Christian M (his blog?) wrote about attempting to write a script to track SQL statements executed by a Java...

Java

Thursday May 8, JavaOne

Here are the few highlights from the talks that I attended today: TS-5428 Java Technology Meets the Real World: Intelligence Everywhere.This talk is about pervasive computing (a.k.a ubiquitous computing) with products from Sentilla. There was an interesting demo about humidity sensor detecting changes and sending a message to a host. The "motes" run CLDC 1.1 VM (+ proprietary profile for motes). These motes have ports for sensors and actuators and some built-in sensor. There were many interesting suggestions for embedded programming for such small devices (don't allocate in inner loops and there by leading to to GC kick-in, avoid too many static fields, avoid threads whenever possible and so on).TS-7575 Using Java Technology-Based Class Loaders to design and implementing a Java platform, Micro EditionThe basic idea is to run JavaME applications (developed for different configurations/profiles/subsets of optional packages) on top of JavaSE. The extended JavaSE classes and packages not available in specific profile or optional package set [implemented by a specific phone] should not be made available to JavaME apps targeted. i.e., only the classes available to a specific phone model should be available. If the JavaME app tries to access any other class, it should receive ClassNotFoundException. The speakers explained how to achieve such "containers" by class loader based isolation. The problem is that they seem to solve only the class access. What about extended methods and fields? For example, platform core classes on JavaSE have superset of methods [more methods on the same class available on JavaME - eg. java.util.Hashtable has more methods on JavaSE). The application classes have to bytecode analyzed and instrumented to take care of field/method accces. It seems that their current product that does not address this yet. PAN-5542 Developing Semantic Web Applications on the Java Platform.The discussion started with some nice demos. There was a demo with AllegroGraph RDF store, Twine, a demo with using GRDDL and getting RDF triples by a proxy server. i.e., a proxy serves does the GRDDL transformations to get RDF triples from sites [which could be stored/analyzed with RDF stores subsequently] and a demo with FOAF files. Interesting take aways from the discussion include:We don't have to wait for SEMANTIC WEB with full fledged reasoners and so on. Instead, add little semantic bits to existing web (say using RDFa, GRDDL etc.) in your current web projects/pages. There are many Java tools. There is need to standard Java APIs for triple store access etc. Right now, we have to write for Jena, Sesame etc. It was also felt that APIs will need to wait for more usage scenarios.There are tools to expose your existing databases as virtual RDF stores -- for example: D2RQ. Probably, most of the RDF triples could come from existing data.Privacy, security of the information is very important. Work needs to be done in this area.Natural language processing and getting triples out of it is very hard. You may want to refer to systems like DBpedia.

Here are the few highlights from the talks that I attended today: TS-5428 Java Technology Meets the Real World: Intelligence Everywhere. This talk is about pervasive computing (a.k.a ubiquitous...

Java

Wednesday May 7, JavaOne

Today Bill, Chihiro, Jaya and I talked on Blu-ray. The talk was centered around the open source project @ http://hdcookbook.dev.java.net - a library and a set of tools to build Blu-ray discs. If you haven't checked out code/docs, you may want to checkout and play with the code. All you need is a laptop with blu-ray drive and a BD-RE disc. Optionally, for added fun you may want to have a hardware bluray player such as PS3 -- so that you can see the output on your TV rather than on a laptop. Other than the session, we also had a very informal BOF on blu-ray, OCAP etc. during the evening. It is good to meet experts in respective technologies in one place!Other than the the blu-ray stuff, I did attend other talks/BOF. Just after Blu-ray session, I attended "TS-6000Improving Application Performance with Monitoring and Profiling Tools" talk. This talk was about OS specific tools, JDK tools and third-party tools for profiling and monitoring. Gregg Sporar and Jaroslav Bachorik (NetBeans Profiler team) presented very well. There were many interesting questions/discussions as well. If you haven't done so already, you may want to download VisualVM. If you want bit more fun doing monitoring/profiling, you may want to check out the sources from http://visualvm.dev.java.net and build it yourself. You can build BTrace VisualVM plugin using the command: c:\\visualvm\\plugins>ant buildassuming you have checked out VisualVM sources under "c:\\visualvm". If you have already checked out BTrace sources under some other directory, say "c:\\btrace", you can use c:\\visualvm\\plugins>ant -Dbtrace.home=c:\\btrace build To run VisualVM with all the plugins that you built, you can use the following command: c:\\visualvm\\plugins>ant -Dbtrace.home=c:\\btrace runPlease let us know what features you'd like to see with BTrace and/or BTrace VisualVM plugin.I attended and liked the "Class Loader Rearchitected (BOF-6180)" BOF. If you have ever written class loaders, chances are that you have faced mysterious deadlocks or ClassCastException that said "ClassCastException: Foo cannot be cast to Foo" or having to decide between overriding loadClass and findclass, you probably should have attended this talk and gave your opinions/suggestions/ideas :-) If I understood properly, I think there was a suggestion to add class loader info. to the ClassCastException (something like class-loader-class-name@identity-HashCode style string?) so that one can quickly see it is a class loader issue. Also, there were many questions on loading classes from jar files. Looks like there will be changes to class loader API and class loading in VM for JDK 7.

Today Bill, Chihiro, Jaya and I talked on Blu-ray. The talk was centered around the open source project @ http://hdcookbook.dev.java.net - a library and a set of tools to build Blu-ray discs. If you...

Java

JVM Languages @ JavaOne 2008

In JavaOne 2008, there are many intesting sessions on "other" JVM languages covering both dynamically typed languages (JavaScript, Groovy, JRuby) and statically typed languages (JavaFX, Scala). As usual, there are many sessions covering application aspects -- like using scripting on Glassfish, Grials (Groovy), Rails (JRuby) and so on. But, my interest is mostly on the programming language aspects and JVM implementation issues. Here is a table of sessions covering those:SessionID SessionTitle SessionType Speakers and CompanySpeakers and CompanyVenue- Room TS-5152Overviewof the JavaFX™ Script Programming LanguageTechnical SessionChristopherOliver, Sun Microsystems, Inc.Tuesday May 0610:50 - 11:50Moscone Center -Gateway 104TS-5416 JRuby:Why, What, How...Do It NowTechnical SessionThomasEnebo, Sun Microsystems, Inc. ; CharlesNutter, Sun Microsystems, Inc.Tuesday May 0610:50 - 11:50Moscone Center -Esplanade 307-310 TS-4794AJavaFX™ Script Programming Language TutorialTechnical SessionJamesWeaver, LATTuesday May 0612:10 - 13:10Moscone Center -Esplanade 305TS-4986JavaScript™Programming Language: The Language Everybody Loves to HateTechnical SessionRobertoChinnici, Sun Microsystems, Inc.Tuesday May 0615:20 - 16:20Moscone Center -Esplanade 307-310PAN-5435TheScript Bowl: A Rapid-Fire Comparison of Scripting LanguagesPanel SessionGuillaumeLaforge, G2One, Inc.; CharlesNutter, Sun Microsystems, Inc. ; JorgeOrtiz, Stanford; RaghavanSrinivas, Sun Microsystems, Inc.; FrankWierzbicki, Sun MicrosystemsWednesday May 0709:30 - 10:30Moscone Center -Gateway 104TS-5572Groovy,the Red Pill: Metaprogramming--How to Blow the Mind of Developerson the Java™ PlatformTechnical SessionScottDavis, Davisworld Consulting, Inc.Wednesday May 0709:30 - 10:30Moscone Center -North Mtg-121/122/124/125TS-5165Programmingwith Functional Objects in ScalaTechnical SessionMartinOdersky, EPFLThursday May 0813:30 - 14:30Moscone Center -Gateway 104TS-5693WritingYour Own JSR-Compliant, Domain-Specific Scripting LanguageTechnical SessionJohn Colosi, VeriSign, Inc.; DavidSmith, VeriSign Inc.Thursday May 0813:30 - 14:30Moscone Center -Esplanade 301TS-6050ComparingJRuby and GroovyTechnical SessionNealFord, ThoughtWorks Inc.Friday May 0913:30 - 14:30Moscone Center -Esplanade 303 TS-6039Jython- Implementing Dynamic Language Features for the Java™Platform EcosystemTechnical SessionJimBaker, Zyasoft; TobiasIvarsson, Neo Technology Friday May 0914:50 - 15:50Moscone Center -Esplanade 305

In JavaOne 2008, there are many intesting sessions on "other" JVM languages covering both dynamically typed languages (JavaScript, Groovy, JRuby) and statically typed languages (JavaFX, Scala)....

education

A tale of many OSes without reboot/partitioning

In my previous post, I talked about having to use more than OS for education/entertainment purpose. The problem is that I've quite a few ISO images and LiveCDs with those images. Two problems:I need to keep searching right LiveCD I want or I need to install more than one OSes on the same/different boxes.Need to re-start the machine if I've use a single machine with many OSes.For kids, all they want is the best stuff available across all operating systems. They don't wait for the machine to reboot from another partition or from a LiveCD. Besides, while they can switch between applications easily, restarting a different OS is bit much to ask :-) [although I won't be surprised if they do that!]. Kids want to switch between interesting applications/games/edutainment stuff rather quickly. Also, we don't seem to get all the good stuff on the same OS! For example, I need to Scratch on Windows or Mac -- no official Linux binary yet :-( While it is possible to hack to run Scratch on Linux by taking the Squeak image, there are issues with MIDI etc. There are many good stuff in Edubuntu and OLPC too. The point is that you want to run the best set of applications/games across operating systems -- without having to reboot.This is where Virtualization helps! As Tim Marland said, you don't have to be a hypervisor/virtualization expert to use it. I downloaded VirtualBox for Windows XP [just run .msi file to install] and edubuntu [this later was very easy -- I just to use Add/Remove programs menu and look for "VirtualBox". VirtualBox is very easy to configure -- mostly point-n-click stuff. Now, I can run OLPC, Edubuntu and Belenix on my Windows XP latop without having to partition my hard-disk or rebooting. Kids can switch between their favorite applications across Operating Systems easily. If you are curious how it looks, here are some screen-shots:OLPC on Windows XPEdubuntu on Windows XPBelenix on Windows XPEdubuntu on Edubuntu! - Apparently OS geeks run the same OS on itself during OS development. No, I won't go there because I am a simple "application" programmer :-)

In my previous post, I talked about having to use more than OS for education/entertainment purpose. The problem is that I've quite a few ISO images and LiveCDs with those images. Two problems: I need...

education

A tale of two operating systems for kids..

When I introduced computers to my kids, as like many other kids they started with games on the net. Mostly playing simple games like tom-and-jerry chase, bob the builder etc. My mother tongue is Tamil and so I came across the kids section of the Tamil Virtual University site. Then, I started experimenting with Squeak and EToys. Later on, I moved to use Scratch. After some time, I learned about GCompris, Tux Math, Alice, Robomind etc. During that period, I still used the non-open-source OS that came with my laptop. GCompris version on that OS does not include all the activities -- to encourage the usage of open source operating systems! So, I looked for operating systems for kids. I've started using the following operating systems. Edubuntu. Venkateswara TV of Solaris sustaining team suggested this to me when I asked him about a Linux distro with GCompris. Wow! Edubuntu is very nice. In addition to what edubuntu comes with (lot of good stuff!), I installed more programs like Childsplay etc.LiveCD for OLPC (One-Laptop-Per-Child). The user interface definitely appeals to kids. Kids seem to enjoy the text editing, painting, Turtle (LOGO) and EToys...!!Recently, I came to know about OpenSolaris in Tamil -- one of these days, I'll try a LiveCD. Need to check if there is a Tamil version of Belenix out there...

When I introduced computers to my kids, as like many other kids they started with games on the net. Mostly playing simple games like tom-and-jerry chase, bob the builder etc. My mother tongue is Tamil...

cricket

No evidence, but still guilty?

Harbhajan banned for three-matches for alleged "racial abuse". Umpires haven't heard or seen anything. Mike Procter himself came on camera and said that the umpires told him they have not heard or seen anything. Did he see anything from the TV footage? It does not seem so. If there is any evidence, we have the right to know it. Aussie players claim something. Indian players claim otherwise. Two parties are not agreeing on what really happened. Under the circumstances, you expect the ICC referee to dismiss the charge on the lack of evidence. Instead, he punished Harbhajan. Did he go by just the word from Aussie players alone? Now, by this process anyone can be punished. Any team can gang against the better players of the opponent team and claim racial abuse or anything that can get a ban! If a Muttiah Muralidaran bowls well, previously they claimed chucking, shouted "no ball", wrote "expert" columns about his bowling action and so on. Now, it seems that there is even a new trick - claim "racial abuse", get a ban for 2/3 matches and then you can bag the series!! If BCCI believes injustice has been done, then they should just pull out and send a strong message to ICC. It is ICC that needs India and it's large cricket crazy population. Not the other way around. It is economically viable to have our own tournaments, entertain public, have fun and make money. BCCI has to learn from the ICL example and start something within India and expand.

Harbhajan banned for three-matches for alleged "racial abuse". Umpires haven't heard or seen anything. Mike Procter himself came on camera and said that the umpires told him they have not heard or...

cricket

Aussies can win any match....

Observations based on the second test:Aussies lost 6 wickets fairly quickly in the first inning. It could have been easily 200 all out. But, one player [Symonds] gets multiple lives! He goes as not out.So long as Steve Bucknor officiates matches India has no chance. He does not ask for third umpire at times when Indian team needs most.Even the third umpire seems to "get it wrong" [what was he doing other than watching TV?]Umpire (Mark Benson) asks the Aussie fielder to check whether Ganguly was out or not!! Why not call out third umpire or ask the batsman as well? [never mind even the third umpire gets wrong to help the Aussies!]They can play so called "mind games". Even a bunch of former players and local media bat and bowl for their team. Other teams don't have this luxury or haven't learned the "art". They write/talk about so called "weakness" of opponents, "talk a lot" during fielding/batting. But, claim racial abuse when the opponent talks. I don't believe Aussies are talking the truth here. Why should anyone believe Aussie players? They claim catch after picking up the ball from the ground. Aussie players don't walk when they know they are out. ( "I was out when I was 30 - given not out. I can sit here and tell you about some bad decisions as well, but I won't. That's the game."Andrew Symonds tells). That talks a lot about their honesty. Why should anyone believe them? I believe (like many in India) this may be one of the tricks by Aussies. Unless the umpires have seen/heard something or TV has captured something, it is not fair to punish Harbhajan Singh based on the words from Aussies.

Observations based on the second test:Aussies lost 6 wickets fairly quickly in the first inning. It could have been easily 200 all out. But, one player [Symonds] gets multiple lives! He goes as...

Java

Week-end fun with the java compiler source code

@page { size: 8.5inch 11inch; margin-top: 0.7874inch; margin-bottom: 0.7874inch; margin-left: 0.7874inch; margin-right: 0.7874inch }table { border-collapse:collapse; border-spacing:0; empty-cells:show }td, th { vertical-align:top; }h1, h2, h3, h4, h5, h6 { clear:both }ol, ul { padding:0; }\* { margin:0; }\*.Caption { font-family:Thorndale; font-size:12pt; margin-top:0.0835in; margin-bottom:0.0835in; font-style:italic; }\*.Heading { font-family:Albany; font-size:14pt; margin-top:0.1665in; margin-bottom:0.0835in; }\*.Index { font-family:Thorndale; font-size:12pt; }\*.List { font-family:Thorndale; font-size:12pt; margin-top:0in; margin-bottom:0.0835in; }\*.P1 { font-family:Thorndale; font-size:12pt; color:#ff0000; }\*.P2 { font-family:Thorndale; font-size:12pt; }\*.P3 { font-family:Thorndale; font-size:12pt; color:#000000; }\*.P4 { font-family:Thorndale; font-size:12pt; color:#000000; }\*.P5 { font-family:Thorndale; font-size:12pt; color:#000000; }\*.P6 { font-family:Thorndale; font-size:12pt; color:#000000; font-weight:normal; }\*.Standard { font-family:Thorndale; font-size:12pt; }\*.Textbody { font-family:Thorndale; font-size:12pt; margin-top:0in; margin-bottom:0.0835in; }\*.BulletSymbols { font-family:StarSymbol; font-size:9pt; }\*.Internetlink { color:#000080; text-decoration:underline; }\*.NumberingSymbols { }\*.SourceText { font-family:Cumberland; }\*.T1 { font-weight:bold; }\*.T2 { font-style:normal; font-weight:normal; }\*.T3 { font-style:normal; font-weight:bold; text-decoration:underline; }\*.T4 { color:#ff0000; }\*.T5 { color:#000000; }\*.T6 { color:#000080; font-weight:bold; }\*.T7 { font-weight:bold; }\*.T8 { font-weight:normal; }\*.VisitedInternetLink { color:#800000; text-decoration:underline; }I downloaded java compiler (javac) source code from the JDK 7 site. I did not download entire JDK – I just downloaded compiler-7-ea-src-b15-05_jul_2007.zip I've installed JDK 6 and NetBeans 5.0I extracted the source zip file into c:\\javac directory. From NetBeans IDE, File->Open Project menu, I chose c:\\javac\\compiler directory. Then, I build the project – I scrolled the build output log to the end and I saw: Building jar: C:\\javac\\compiler\\dist\\lib\\javac.jar build-bin.javac: Copying 1 file to C:\\javac\\compiler\\dist\\bin build: BUILD SUCCESSFUL (total time: 8 seconds) So, I tried to run the newly compiled java compiler. I attempted to compile a simple “Hello World” program. I got the following error: C:\\javac\\compiler\\dist\\lib>java -jar javac.jar Hello.java Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/tools/javac/Main What happened? I looked at the build log again. I missed the following lines – because I had seen only at the end!! Copying 7 files to C:\\javac\\compiler\\build\\bootclasses recompiling compiler with itself javac: invalid flag: C:\\javac\\compiler\\build\\classes Usage: javac <options> <source files> use -help for a list of possible options Java Result: 2 Copying 7 files to C:\\javac\\compiler\\build\\classes Copying 1 file to C:\\javac\\compiler\\build\\classes\\com\\sun\\tools\\javac\\resources Building jar: C:\\javac\\compiler\\dist\\lib\\javac.jar build-bin.javac: Copying 1 file to C:\\javac\\compiler\\dist\\bin build: BUILD SUCCESSFUL (total time: 8 seconds) Looks like there is a build error. The compiler in built in two steps: The sources are built with javac in JDK 6 (on which my NetBeans IDE ran) Then, compiler sources are built again – but this time with the new compiler binary generated by step (1). Looks we got error in the step (2) [see above: recompiling compiler with itself] . I searched the ant script used to build for “recompiling compiler with itself”. The following is the fragment after that: <echo message="recompiling compiler with itself"/> <pathconvert pathsep=" " property="src.javac.files"> <path> <fileset dir="${src.classes}"> <patternset refid="src.javac"/> </fileset> </path> </pathconvert> <java fork="true" classpath="${build.bootclasses}" classname="com.sun.tools.javac.Main"> <arg value="-sourcepath"/> <arg value=""/> <arg value="-d"/> <arg file="${build.classes}"/> <arg value="-g:source,lines"/> <arg line="${src.javac.files}"/> </java> The problem seems to be with “java” command above. Empty string is set as value for -sourcepath option. I changed that to the following: <arg value="-sourcepath"/> <arg value="${src.classes}"/> When I re-built the compiler after the above change, there were no errors – yes, I scrolled the build output to check it :-) And newly compiled javac could compile “Hello World” program. Now, I wanted to make some to “interesting” but simple change to the compiler source. From a “doc” page, I came to know that there is a hidden javac option called “-printflat”. It appears that with -printflat option javac prints source code after doing transformations for generic types, inner classes, enhanced for-loops, assertions etc. It would be great to visualize the kind of transformations done by javac. So, I wanted to make “hidden” option available. I searched for “printflat” in the project. I got three hits: JavaCompiler.java RecognizedOptions.java java.properties As usual, I am impatient – wanted to enable printflat option always [regardless of what the command line is]. So, I changed the following line in JavaCompiler.java printFlat = options.get("-printflat") != null; to printFlat = true; // options.get("-printflat") != null; so that the secret option is enabled always. After rebuilding the compiler, I tried compiling my “Hello World” program. Surprise! I got the following error: C:\\javac\\compiler\\dist\\lib>java -jar javac.jar Hello.java Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/tools/javac/Main When I checked “javac.jar” by “jar tvf javac.jar”, I saw only “.java” files instead of “.class” files! Remember I mentioned that javac is recompiled by itself (step (2) above)? Apparently with “-printflat” option, javac just write transformed files but does not generate .class files! Because I had hardcoded printflat to be true always, during the second bootstrap compilation javac did not generate .class files. Looks like my lazy way does not work! I need to find how to really change the code to accept printflat command line option explicitly. I cut the story shot and just summarize the changes I made: added a enum value to com.sun.tools.javac.main.OptionName – PRINTFLAT("-printflat"); In com.sun.tools.javac.main.RecognizedOptions class, I added PRINTFLAT to “static Set<OptionName> javacOptions” initialization value. In public static Option[] getAll(final OptionHelper helper) method of RecognizedOptions class, I added “new HiddenOption(PRINTFLAT)” as an element in the returned Option[]. I managed to compile and run the compiler after the above changes! Now when I can pass “printflat” option!! I compiled the following simple Book.java:class Book { private String name; public Book(String name) { this.name = name; } class Order { private int quantity; public Order(int quantity) { this.quantity = quantity; } }}with the following command: c:\\javac\\compiler\\dist\\lib\\>java -jar javac.jar -printflat c:\\Book.java Now, I can see the generated Book.java and Book$Order.java in the current directory where java compiler was run: class Book { private String name; public Book(String name) { super(); this.name = name; } { }}class Book$Order { /\*synthetic\*/ final Book this$0; private int quantity; public Book$Order(/\*synthetic\*/ final Book this$0, int quantity) { this.this$0 = this$0; super(); this.quantity = quantity; }}Wow! I can see how java compiler generates a hidden synthetic parameter for the outer class object and so on. Note that the java compiler does not overwrite your original source files. You need to run the compiler in a different directory – compiler generates new files [which is good, you won't accidentally overwrite your original code with generics, inner classes and so on]. Now, you can experiment with constructs like asserts, inner class methods accessing outer's private methods/fields, anonymous/local classes, local class accessing final parameters/locals of enclosing method, generics, enhanced for-loop and so on and see how java compiler transforms those constructs to generate good-old “flat” classes without these features. Have fun!!

I downloaded java compiler (javac) source code from the JDK 7 site. I did not download entire JDK – I just downloaded compiler-7-ea-src-b15-05_jul_2007.zip I've installed JDK 6 and NetBeans 5.0 I extracted...

Java

JavaScript, JSON and JavaFX Script

JavaFX Script, asyou may know already, is a declarative and statically typed scripting language. It has first-class functions, declarativesyntax, list-comprehensions, and incremental dependency-based evaluation.JavaFX Script syntax includesobject literals- a declarative syntax to construct objects. If you knowJavaScript,you may know about JavaScriptObject Literals and it's subset called JSON- which is a lightweight data-interchange format (considered to be analternative for XML in certain use-cases). There are some parallels between JavaScript object literal syntax and that of the JavaFX Script. Of course there are differences. The following is an attempt tosummarize the differences.TypesJavaFX Script is statically typed. But, user may omit types in manyplaces. JavaFX can infer types from the context. Whiledefining object literals, the type of the object is specified.For example:class Person { attribute name: String; attribute children: Person\*;}var chris = Person { name: "Chris" children: [Person { name: "Dee" }, Person { name: "Candice" }] };But except for the "top most" object, we can omit types elsewhereand JavaFX will infer types. For example, the above may also bewritten as shown below:class Person { attribute name: String; attribute children: Person\*;}var chris = Person { name: "Chris" children: [ Person { name: "Dee" },Person { name: "Candice" }] };JavaScript is dynamically typed. So, we don't and can't specifythe type of the object. [And yes, I do know aboutJavaScript 2.0- which is optionally typed. But, I am talking about the currentimplementations]. We write the above object in JavaScript asvar chris = { name: "Chris", children: [ { name: "Dee" }, { name: "Candice" }] };No comma between property valuesJavaFX object literal examples do not include comma between property specifications. For example,class Person { attribute name: String; attribute children: Person\*;}var chris = Person { name: "Chris" children: [Person { name: "Dee" }, Person { name: "Candice" }] };Please note that there is comma between elements of the array butthere is no comma between properties (between name and children in above). But, JavaScript object literals(and JSON) have comma between properties as well. But,JavaFX implementation seems to accept comma. So, the followingvar chris = Person { name: "Chris", //

JavaFX Script, as you may know already, is a declarative and statically typed scripting language. It has first-class functions, declarativesyntax, list-comprehensions, and incremental...

Java

A project idea for OpenJDK...

You probably checked out the OpenJDK project and even built it on your favorite platform. And you are wondering how can you do a small, but interesting project with OpenJDK. In Nov 2006, I mentioned about HotSpot Serviceability Agent (SA) which is a set of APIs and tools for debugging HotSpot Virtual Machine processes and core dumps. You may want to look at HotSpot Serviceability page and HotSpot SA sources in the hotspot/agent/ directory and the subdirectories. One of the components of HotSpot SA is (pure Java) disassemblers for SPARC and x86 processors. These are used to view HotSpot compiled code, HotSpot interpreter code (recall that hotspot Java bytecode interpreter is generated at the start of the virtual machine) as well as the compiled C/C++ code of the virtual machine. You may want to refer to the asmand subdirectories. But, there is no disassembler yet for AMD64. Maxwell Assembler Framework has assemblers and disassemblers for SPARC, PowerPC, AMD64 and IA32. From the project web page:The architecture of our system is largely based on the Klein assembler system, which has been developed in the Klein project. Its main idea is to generate assembler methods that are very well tested against existing textual assemblers. How about using the Maxwell framework with HotSpot SA? It will get us AMD64 disassembler and we can replace the existing disassemblers for SPARC and x86 as well! I think this is a small enough, but interesting project to work on.

You probably checked out the OpenJDK project and even built it on your favoriteplatform. And you are wondering how can you do a small, but interesting project with OpenJDK. In Nov 2006, I mentioned...

Java

Programmatically dumping heap from Java applications

In the troubleshooting BOF, we demonstrated how to use thejmap -dump option to dump heap dump of a running application and showed how to browse/analyze the resulting binary heap dump using the jhat tool.One of the questions was how to programmatically dump the heap from applications. For example, you may want to dump multiple heap snapshots from your application at various points in time and analyze those off line using jhat. Yes, you can dump heap from your application -- but you have to do a bit of programming as shown below:import javax.management.MBeanServer;import java.lang.management.ManagementFactory;import com.sun.management.HotSpotDiagnosticMXBean;public class HeapDumper { // This is the name of the HotSpot Diagnostic MBean private static final String HOTSPOT_BEAN_NAME = "com.sun.management:type=HotSpotDiagnostic"; // field to store the hotspot diagnostic MBean private static volatile HotSpotDiagnosticMXBean hotspotMBean; /\*\* \* Call this method from your application whenever you \* want to dump the heap snapshot into a file. \* \* @param fileName name of the heap dump file \* @param live flag that tells whether to dump \* only the live objects \*/ static void dumpHeap(String fileName, boolean live) { // initialize hotspot diagnostic MBean initHotspotMBean(); try { hotspotMBean.dumpHeap(fileName, live); } catch (RuntimeException re) { throw re; } catch (Exception exp) { throw new RuntimeException(exp); } } // initialize the hotspot diagnostic MBean field private static void initHotspotMBean() { if (hotspotMBean == null) { synchronized (HeapDumper.class) { if (hotspotMBean == null) { hotspotMBean = getHotspotMBean(); } } } } // get the hotspot diagnostic MBean from the // platform MBean server private static HotSpotDiagnosticMXBean getHotspotMBean() { try { MBeanServer server = ManagementFactory.getPlatformMBeanServer(); HotSpotDiagnosticMXBean bean = ManagementFactory.newPlatformMXBeanProxy(server, HOTSPOT_BEAN_NAME, HotSpotDiagnosticMXBean.class); return bean; } catch (RuntimeException re) { throw re; } catch (Exception exp) { throw new RuntimeException(exp); } } public static void main(String[] args) { // default heap dump file name String fileName = "heap.bin"; // by default dump only the live objects boolean live = true; // simple command line options switch (args.length) { case 2: live = args[1].equals("true"); case 1: fileName = args[0]; } // dump the heap dumpHeap(fileName, live); }}By including the above class in your application, you can call HeapDumper.dumpHeap whenever you want to dump the heap as shown in the sample main.Important note: while you can dump multiple heap snapshots from your application, you can not correlate the objects from multiple dumps. jmap tool uses object addresses as object identifiers - which vary between garbage collections [recall that GC may move objects which changes the object addresses]. But, you can correlate by aggregate stat such as histogram etc.

In the troubleshooting BOF, we demonstrated how to use the jmap -dump option to dump heap dump of a running application and showed how to browse/analyze the resulting binary heap dump using the jhat t...

Java

JavaOne'07 -- my summay.

After 19 hours of flight, I finally reached Chennai from SFO (on Sunday morning). I'm still going through the timezone change. I wish for a "Star Trek" like transport :-)This JavaOne turned to be a "device" JavaOne for me -- I attended talks on SunSPOT, Java ME, Blu-ray, OCAP etc. In some sense, it is device JavaOne anyway -- after all one of the big announcements is about JavaFX mobile. I wanted to attend, but missed the talk on "x86 (JPC?) Java emulator".Random bits from this year's JavaOne:Our troubleshooting BOF went well. As usual there were many questions to the point that they would hardly let me finish jhat demo :-). I was at the performance and diagnostics POD for some time - mostly demonstrating jconsole, jhat to few folks.I felt that there was more crowd this time. JavaOne is getting bigger and bigger. Long queues everywhere ...On Friday, I had a chat with Fat Cat Air. The chat was mostly about the flight that he is building and not about "tiered compilation" :-)I had to travel to L.A for a day. I stayed with my friend Gurumoorthy over the week end. Now, I've a new friend "Akhilan" (Gurumoorthy's son).I had to explain quite a few folks how I work at Chennai when there is no Sun office at Chennai. Yes, I work from home. If you are curious, you may want to check out OpenWork.That's all....

After 19 hours of flight, I finally reached Chennai from SFO (on Sunday morning). I'm still going through the timezone change. I wish for a "Star Trek" like transport :-) This JavaOne turned to be a...

Java

JavaOne'07 Wed (May 9)

I attended the following sessions/BOFs (in no specific order):Building PhoneME project and applications on it. The speakers (Stuart Marks and Hinkmond Wong) demonstrated how to SVN checkout, build the platform - both Phone ME feature (CLDC) and Phone ME advanced (CDC). A sample protocol handler (called "upcase" protocol) was added to the platform and the platform was built. Also, the speakers showed how to build CLDC/CDC apps using NetBeans IDE. Stuart Marks made the IDE to use the Phone ME feature build rather than the bundled wireless toolkit version of CLDC [He didn't tell how to do so. You may want to watch out for Terrence Barr's blog.Vincent Hardy's talk on using JSR 226, 287 and 290 (SVG/XHTML/CSS/ECMAScript) on phones. It was interesting talk. I knew a bit about SVG format - but I learned about how to use it on Java ME platform.JavaFX talk by Chris Oliver. Seeing is believing -- you got to see the stuff - you may want to download and try it out!. I attended the session on developing flashy graphics for Java ME platform -- it was mostly the "common sense" stuff ((like don't call System.gc() explicitly, use double-buffering etc). But, sometimes it is worthwhile reminding the "obvious".I attended the Java SE performance BOF - it was an interactive Q & A session. The "performance gurus" of Sun answered the questions. Most questions were about GC - as one would expect in such sessions :-)

I attended the following sessions/BOFs (in no specific order): Building PhoneME project and applications on it. The speakers (Stuart Marks and Hinkmond Wong) demonstrated how to SVN checkout, build...

Java

JavaOne'07 Tuesday (May 8)

Here is what I did on the first day of JavaOne 2007.I attended the keynote session - main messages are:"JavaFX script" language - a dynamic, statically typed language for GUI/Java2D/Swing type applications (can be used for general purpose as well). It was formerly called F3. In future, there will be an authoring tool that emits JavaFX script.JavaFX/Mobile - complete software platform for mobible - sort of like "Java OS" for mobile phones. You have Java SE, yes - you read it right, Java SE for mobile phones! And JavaFX script wil run on this platform too.OpenJDKNetBeans 6.0I attended the following talks:Language oriented programming - this was very nice presentation on DSLs (Domain Specific Languages). Very nice examples for internal DSLs (where you stay within the general purpose language) in Groovy, Java and JRuby - then followed by brief description of external DSLs (do it in the hard way - create your own lexer, parser) and tools for the same. Interesting talk!Then, I attended the talk on SunSPOT. The talk was around constructing virtual reality/game/3D stuff using SunSPOT devices and desktop. SunSPOT runs Squawk - VM that runs on the bare ARM-9 hardware and provides low-level operating system support, as well as application isolation (isolates). Over the week end, I played on Wii console [which my friend bought recently]. After attending this session on SunSPOT, I appreciate how difficult it is to create such a system! Cool stuff!At 8.00 PM, Alan, Mandy and I presented the usual JDK troubleshooting tools BOF. There were lots of questions. We wanted it to be interactive - it was so interactive that I could barely complete the demo on jhat tool :-) If you are keen on improving heap dumps, you may want to join and contribute http://heap-snapshot.dev.java.net project. One important suggestion was to create a web page of tools and products that import hprof heap dump format files. After the talk, we discussed on importing the heap dump to a database and allowing SQL queries -- so that we can handle very large heap dumps. We will have to revisit this. But, now that we have Open JDK, you can contribute with new ideas and implementations/improvements!

Here is what I did on the first day of JavaOne 2007. I attended the keynote session - main messages are: "JavaFX script" language - a dynamic, statically typed language for GUI/Java2D/Swing type...

Java

Desktop scripting applications with NetBeans 5.5

How about developing desktop Java applications using scripting, XML and database withNetBeans? This afternoon I played with scripting using NetBeans 5.5.Before proceeding further, I will describe the devepment environment:JDK 6NetBeans 5.5 withNetBeans Enterprise PackNetBeans Scripting plugins from the Phobos projectFor desktop apps, you probably need only the NetBeans JavaScript Editor. But, it does not hurt to have everything :-)In addition, I've CVS checked out the following java.net projects:http://scripting.dev.java.nethttp://phobos.dev.java.netI started with a simple Java program project. Added a script filewith the name "hello.js" to the project:I added the following code using the JavaScript editor:JavaScript editor supports syntax highlighting etc. The "importPackage" built-in function imports a Java package to script."importClass"imports a specific Java class. Everything above is straight forward -- except maybe the listener for the button. JavaScript engine takes care of wrapping the supplied JavaScript function to ActionListener interface. Also, JavaScript engines supportsJavaBean convention so that we can write f.visible = true instead of f.setVisible(true).I added the following code to the main class:package helloscript;import java.io.InputStream;import java.io.InputStreamReader;import java.io.Reader;import javax.script.ScriptEngine;import javax.script.ScriptEngineManager;public class Main { public static void main(String[] args) throws Exception { // create manager ScriptEngineManager m = new ScriptEngineManager(); // create javascript script engine ScriptEngine js = m.getEngineByName("javascript"); // evaluate "hello.js" InputStream strm = Main.class.getResourceAsStream("/hello.js"); Reader r = new InputStreamReader(strm); js.eval(r); } }Note that NetBeans copies "hello.js" to build/classes directoryand packages it into dist/HelloScript.jar as well. So, the abovegetResourceAsStream works as expected. When running the project,I got this:Now, how about throwing some database access and XML? I'm too lazy to use JDBCAPI directly. I wanted to try out the persistence API. So, I followed the steps below to generate entity classes from the sample database [shipped withNetBeans enterprise pack? I've no idea, as long as it works... :-) ]. You probablyknow this already: You do not need to use Java EE to use the persistence API.You can use it within Java SEPassword is also "app". Don't forget to choose "Remember password during this session" option!I added all tables from the sample db and I just selected package name to be "helloscript".Created "persistence unit" as suggested!Now, I need to add Java code to fetch all "Customer" instances from DB. So, I addedthe following to Main.java:package helloscript;import java.io.InputStream;import java.io.InputStreamReader;import java.io.Reader;import java.util.List;import javax.persistence.EntityManager;import javax.persistence.EntityManagerFactory;import javax.persistence.Persistence;import javax.persistence.Query;import javax.script.ScriptEngine;import javax.script.ScriptEngineManager;public class Main { public static void main(String[] args) throws Exception { // create JavaScript engine ScriptEngineManager m = new ScriptEngineManager(); ScriptEngine js = m.getEngineByName("javascript"); // Create Entity manager factory - "HelloScriptPU" // is the name given when creating persistence unit EntityManagerFactory emf = Persistence.createEntityManagerFactory("HelloScriptPU"); // create Entity manager EntityManager em = emf.createEntityManager(); // select all Customer instances Query q = em.createQuery("SELECT c FROM Customer c"); // get the result list (of Customers) List l = q.getResultList(); // expose the Customer list to script // as a global variable by the name "customers" js.put("customers", l); // now evaluate script -- the script may use // "customers" variable InputStream strm = Main.class.getResourceAsStream("/hello.js"); Reader r = new InputStreamReader(strm); js.eval(r); } }Now, I added logic to display the customer list in the JavaScript code:Again, I've used JavaBean convention support to access Customer name andemail -- so that I wrote c.name and c.email insteadof calling c.getName() and c.getEmail() on the entityinstances. I used JEditorPane to display the generated HTML tableinside the JFrame. When I ran the above code. I got the followingerror:Exception in thread "main" Local Exception Stack: Exception [TOPLINK-4003] (Oracle TopLink Essentials - 2006.8 (Build 060830)): oracle.toplink.essentials.exceptions.DatabaseExceptionException Description: Configuration error. Class [org.apache.derby.jdbc.ClientDriver] not found. at oracle.toplink.essentials.exceptions.DatabaseException.configurationErrorClassNotFound(DatabaseException.java:86) at oracle.toplink.essentials.sessions.DefaultConnector.loadDriver(DefaultConnector.java:168) at oracle.toplink.essentials.sessions.DefaultConnector.connect(DefaultConnector.java:83) at oracle.toplink.essentials.sessions.DatasourceLogin.connectToDatasource(DatasourceLogin.java:170) at oracle.toplink.essentials.internal.sessions.DatabaseSessionImpl.loginAndDetectDatasource(DatabaseSessionImpl.java:537) at oracle.toplink.essentials.ejb.cmp3.EntityManagerFactoryProvider.login(EntityManagerFactoryProvider.java:180) at oracle.toplink.essentials.internal.ejb.cmp3.EntityManagerSetupImpl.deploy(EntityManagerSetupImpl.java:230) at oracle.toplink.essentials.internal.ejb.cmp3.base.EntityManagerFactoryImpl.getServerSession(EntityManagerFactoryImpl.java:78) at oracle.toplink.essentials.internal.ejb.cmp3.base.EntityManagerFactoryImpl.createEntityManagerImpl(EntityManagerFactoryImpl.java:113) at oracle.toplink.essentials.internal.ejb.cmp3.base.EntityManagerFactoryImpl.createEntityManagerImpl(EntityManagerFactoryImpl.java:107) at oracle.toplink.essentials.internal.ejb.cmp3.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:76) at helloscript.Main.main(Main.java:20)Java Result: 1Seems like persistence runtime did not find the driver classes. I addedderbyclient.jar from $JDK/db/lib directory to the project's "Libraries". After that, it ran as expected!But, I mentioned XML. Where is it? Well, I generated (X)HTML for table :-) Howabout using E4X? (easier interface to work with XML).Although Mozilla Rhino JavaScriptsupports E4X, the bundled version in JDK 6 does not support E4X. Rhino's E4Ximplementation depends on Apache XML Beans. We do have a jsr-223 script engine for "full" Rhino with E4X support in the http://scripting.dev.java.net project. I added js-engine.jarfrom this project, js.jar [Rhino jar - either download it from Mozilla site or from CVS workspace of scripting project] and xmlbeans.jar[which I got from the Phobos project $PHOBOS/phobos/dist/lib/xmlbeans.jar] to theproject:Then, I modified the "hello.js" script to use E4X:Note that the HTML table generation does not involve strings. I use E4Xto generate it -- I've expressions inside { } for the 'dynamic' portions of the (X)HTML. Also, in the Java code, I made asmall change: ScriptEngine js = m.getEngineByName("rhino-nonjdk");Note that I used "rhino-jdk" as the engine name instead of "javascript" so thatthe manager would choose the jsr-223 engine that is bundled with JDK 6. Withoutthis, it would choose the not bundled jsr-223 engine in rt.jar. With the changeabove, E4X works! I learned a bit about NetBeans 5.5 JavaScript support, persistence API and E4X. Hmm..., some day I need to try out the bleeding edge stuff inNetBeans 6...

How about developing desktop Java applications using scripting, XML and database with NetBeans? This afternoon I played with scripting using NetBeans 5.5.Before proceeding further, I will describe the...

Java

SML subset for TEDI?

I came across "Do we need a new kind of schema language?" from Tim Bray's blog. After reading it, I can't resist this: why not use a subset of Standard ML for this purpose?Basic types - SML has int, real, bool, char, stringTuples - SML has t1\*t2\*t3... (for example, int\*bool, int\*string\*real and so on)Lists - 'a list - int list, string list and so onRecords - {name1:t1, name2:t2 } (for example, {name:string, age:int})type for type abbreviations - useful when defining "bigger" schema.datatype for algebraic data typesFor larger schemas, we could use parameterized data types.For larger schemas, we could also use SML structures to group related types.In the above, we have every type expression except for function types! (IMO, it is not a big list!). Annotations for bindings may be specified in SML comments [this needs more thought].The example in James Clark's page { url: String, width: Integer?, height: Integer?, title: String } would become the following in SML: { url: string, width: Maybe int, height: Maybe int, title: string }where Maybe is datatype 'a Maybe = Just of 'a | NothingPros:We can have parametric types and module system - may be useful, for defining larger, generic schema. Tuples, Lists, Records could be mapped to parametrized classes (like in Scala, Java etc.) or native data types in scripting languages.sum-of-product types can be translated as classes (like case classes in Scala)Type expressions are proper set of one particular language - at least few people would feel at home :-) JSON is proper subset of JavaScript object literals, functional folks could have their turn :-). BTW, I am okay with Haskell as well. Personally, I've played with SML little bit more :-)If we want to have schema specified default values of various elements of data, we can include proper subset of value definitions in SML.

I came across "Do we need a new kind of schema language?" from Tim Bray's blog. After reading it, I can't resist this: why not use a subset of Standard ML for this purpose? Basic types - SML has int,...

Java

Retrieving .class files from a Java core dump

In my last blog entry, I explained how to retrieve .class files from a running Java application. How about retrieving .class files from a Java core dump? You may have got a core dump from JVM due toa bug in some JNI code [you may want to run your application with the -Xcheck:jni flag during development]a bug in JVM. You got core dump and hs_err_pid<NNN> file [which you can decode online!]or you may have forced core dump by a tool such as gcoreWhen you debug native core dumps, you may have asked your customer for the matching shared objects [or DLLs]. This is because text (a.k.a "code") pages are not dumped into core dump in most operating systems [although you could configure any page to be dumped into core on Solaris 10+]In JVM, the equivalent of "text" section is the JVM internal data structures created by parsing the loaded .class files. Such data structures are stored in an area of the Java heap called "permanent generation". Because the Java heap is included into the core dumps, it should be possible to access these from Java core dumps. If so, how about reconstructing .class files from core dumps? As it turns out, there is already a tool to do exactly that!HotSpot Serviceability Agent (SA) is a postmortem/snapshot debugger for HotSpot Java Virtual Machine. SA has a number of tools that are not bundled with JDK/JRE. But, now that HotSpot JVM is a open source, you can look/modify/build HotSpot SA. HotSpot SA lives under the $HOTSPOT/agent directory. There are tools under $HOTSPOT/agent/make directory. To use the SA tools, you need to build SA. There is a gnumake file for SA under $HOTSPOT/agent/make directory. HotSpot SA uses Mozilla Rhino JavaScript engine to implement certain features:So, you need Mozilla Rhino JavaScript engine version 1.5R5 which is available for download @ ftp://ftp.mozilla.org/pub/mozilla.org/js/rhino1_5R5.zipCopy "js.jar" under $HOTSPOT/agent/src/share/lib directory.Note: This requirement will go away after JDK 7 build 12 or beyond (SA now uses jsr-223 interface for JavaScript engine - thereby direct dependency on Rhino jar file is avoided). If you wait till JDK 7 build 12, you need to do this download/copy step.After building HotSpot SA, you can run any of the tools under $HOTSPOT/agent/make. There are shell scripts the SA tools - jcoreproc.sh is the tool to retrieve .class files from core dumps. There is another tool called clhsdbproc.sh that supports dbx/gdb like command line interface. This tool supports dumpclass command to retrieve .class files from core dump. There is some documentation of SA tools under $HOTSPOT/agent/doc directory. Happy core dump debugging!

In my last blog entry, I explained how to retrieve .class files from a running Java application. How about retrieving .class files from a Java core dump? You may have got a core dump from JVM due to a...

Java

Retrieving .class files from a running app

Many Java applications generate .class files on-the-fly at runtime. Some applications modify loaded Java classes either at class load time or even later using hotswap. Few examples:Scripting language implementations that generate .class files - like BeanShell, GroovyOR mapping tools such as Toplink EssentialsCompilation of XSL stylesheets to .classes (assuming compilation is done at runtime rather than at build).Profilers that use byte code instrumentation technique - like NetBeans profiler.It becomes difficult to debug such applications - for example, you may get verifier error because of a bug in the .class generator somewhere. So, sometimes it is better to dump .class files of generated/modified classes for off-line debugging - for example, we may want to view such classes using tools like jclasslib. Instead of modifying each and every component that generates/modifies .class files, we would want to have a generic solution. The solution below uses attach-on-demand facility in JDK 6.File: ClassDumperAgent.javaimport java.lang.instrument.\*;import java.io.File;import java.io.FileOutputStream;import java.security.ProtectionDomain;import java.util.List;import java.util.ArrayList;import java.util.regex.Pattern;/\*\* \* This is a java.lang.instrument agent to dump .class files \* from a running Java application. \*/public class ClassDumperAgent implements ClassFileTransformer { // directory where we would write .class files private static String dumpDir; // classes with name matching this pattern // will be dumped private static Pattern classes; public static void premain(String agentArgs, Instrumentation inst) { agentmain(agentArgs, inst); } public static void agentmain(String agentArgs, Instrumentation inst) { parseArgs(agentArgs); inst.addTransformer(new ClassDumperAgent(), true); // by the time we are attached, the classes to be // dumped may have been loaded already. So, check // for candidates in the loaded classes. Class[] classes = inst.getAllLoadedClasses(); List<Class> candidates = new ArrayList<Class>(); for (Class c : classes) { if (isCandidate(c.getName())) { candidates.add(c); } } try { // if we have matching candidates, then // retransform those classes so that we // will get callback to transform. if (! candidates.isEmpty()) { inst.retransformClasses(candidates.toArray(new Class[0])); } } catch (UnmodifiableClassException uce) { } } public byte[] transform(ClassLoader loader, String className, Class redefinedClass, ProtectionDomain protDomain, byte[] classBytes) { // check and dump .class file if (isCandidate(className)) { dumpClass(className, classBytes); } // we don't mess with .class file, just // return null return null; } private static boolean isCandidate(String className) { // ignore array classes if (className.charAt(0) == '[') { return false; } // convert the class name to external name className = className.replace('/', '.'); // check for name pattern match return classes.matcher(className).matches(); } private static void dumpClass(String className, byte[] classBuf) { try { // create package directories if needed className = className.replace("/", File.separator); StringBuilder buf = new StringBuilder(); buf.append(dumpDir); buf.append(File.separatorChar); int index = className.lastIndexOf(File.separatorChar); if (index != -1) { buf.append(className.substring(0, index)); } String dir = buf.toString(); new File(dir).mkdirs(); // write .class file String fileName = dumpDir + File.separator + className + ".class"; FileOutputStream fos = new FileOutputStream(fileName); fos.write(classBuf); fos.close(); } catch (Exception exp) { exp.printStackTrace(); } } // parse agent args of the form arg1=value1,arg2=value2 private static void parseArgs(String agentArgs) { if (agentArgs != null) { String[] args = agentArgs.split(","); for (String arg: args) { String[] tmp = arg.split("="); if (tmp.length == 2) { String name = tmp[0]; String value = tmp[1]; if (name.equals("dumpDir")) { dumpDir = value; } else if (name.equals("classes")) { classes = Pattern.compile(value); } } } } if (dumpDir == null) { dumpDir = "."; } if (classes == null) { classes = Pattern.compile(".\*"); } }}File: manifest.mfPremain-Class: ClassDumperAgentAgent-Class: ClassDumperAgentCan-Redefine-Classes: trueCan-Retransform-Classes: trueFile: Attacher.javaimport com.sun.tools.attach.\*;/\*\* \* Simple attach-on-demand client tool that \* loads the given agent into the given Java process. \*/public class Attacher { public static void main(String[] args) throws Exception { if (args.length < 2) { System.out.println("usage: java Attach <pid> <agent-jar-full-path> [<agent-args>]"); System.exit(1); } // JVM is identified by process id (pid). VirtualMachine vm = VirtualMachine.attach(args[0]); String agentArgs = (args.length > 2)? args[2] : null; // load a specified agent onto the JVM vm.loadAgent(args[1], agentArgs); }}Steps to build class dumper:javac ClassDumperAgent.javajar cvfm classdumper.jar manifest.mf ClassDumperAgent.classjavac -cp $JAVA_HOME/lib/tools.jar Attacher.javaSteps to run class dumper:start your target process -- I used java2d demo application in JDKfind the process id of your process using "jps" tooljava -cp $JAVA_HOME/lib/tools.jar:. Attacher <pid> <full-path-of-classdumper.jar> dumpDir=<dir>,classes=<name-pattern>The above command will dump all classes matching the given name (regex) pattern into the given directory. The default dump directory is the current working directory (of the target application!). The default pattern is ".\*" i.e., match all classes loaded/will be loaded in the target application.Now, how about a nice GUI thatshows all the Java processes lets you can choose dump directoryOptionally, shows all the classes of a selected applicationAs usual, that is left as an exercise to the reader :-)

Many Java applications generate .class files on-the-fly at runtime. Some applications modify loaded Java classes either at class load time or even later using hotswap. Few examples: Scripting language...

Java

Script Beans - part 2

This is continuation of my earlier post onscript beans.On rethinking, I've made few (minor) changes to ScriptObject and Callable. Thechanges are:Using varargs instead of Object[] arguments in few places.Added invoke method in ScriptObject-- this is used to directly invoke a method on a script object insteadof getting a method/function valued property using getand then invoking Callable.call method. Why? Many dynamically typedlanguages have Smalltalk's doesNotUnderstand equivalent. i.e., if amethod is not found, then a method missing handler is invoked. - like JRuby's method_missing and Groovy's invokeMethod.ScriptObject.invokemay be implemented by calling ScriptObject.get and Callable.call on result andpossibly incorporating language specific method missing method as well.ScriptObject.put returns boolean to tell whether the putwas successful or not (may be read-only property was assigned or property additionis not possible and so on).ScriptObjectimport javax.script.ScriptException;/\*\* \* Any Java object supporting this interface can be \* accessed from scripts with "simpler" access pattern. \* For example, a script engine may support natural \* property/field access syntax for the properties exposed \* via this interface. We use this interface so that we \* can dynamically add/delete/modify fields exposed to \* scripts. Also, script engines may expose this interface for \* it's own objects. \*/public interface ScriptObject { // special value to denote no-result -- so that // null could be used as proper result value public static final Object UNDEFINED = new Object(); // empty object array public static final Object[] EMPTY_ARRAY = new Object[0]; /\* \* Returns all property names supported by this object. \* Property "name" is either a String or an Integer". \*/ public Object[] getIds(); /\*\* \* Get the value of the named property. \*/ public Object get(String name); /\*\* \* Get the value of the "indexed" property. \* Returns UNDEFINED if the property does not exist. \*/ public Object get(int index); /\*\* \* Set the value of the named property. Returns \* whether the put was successful or not. \*/ public boolean put(String name, Object value); /\*\* \* Set the value of the indexed property. Returns \* whether the put was successful or not. \*/ public boolean put(int index, Object value); /\*\* \* Returns whether the named property exists or not. \*/ public boolean has(String name); /\*\* \* Returns whether the indexed property exists or not. \*/ public boolean has(int index); /\*\* \* Deletes the named property. Returns true on success. \*/ public boolean delete(String name); /\*\* \* Deletes the indexed property. Returns true on success. \*/ public boolean delete(int index); /\*\* \* Call the named method on this script object. \*/ public Object invoke(String name, Object... arguments) throws NoSuchMethodException, ScriptException;}Callable.javaimport javax.script.ScriptException;/\*\* \* This interface is used to represent "function/method" valued \* properties in ScriptObjects. \*/public interface Callable { /\*\* \* Call the underlying function passing the given \* arguments and return the result. \*/ public Object call(Object... args) throws ScriptException;}AbstractScriptObject.javaimport javax.script.ScriptException;/\*\* \* Simple dummy implementation of ScriptObject. \*/public abstract class AbstractScriptObject implements ScriptObject { public Object[] getIds() { return EMPTY_ARRAY; } public Object get(String name) { return UNDEFINED; } public Object get(int index) { return UNDEFINED; } public boolean put(String name, Object value) { return false; } public boolean put(int index, Object value) { return false; } public boolean has(String name) { return false; } public boolean has(int index) { return false; } public boolean delete(String name) { return false; } public boolean delete(int index) { return false; } public Object invoke(String name, Object... arguments) throws NoSuchMethodException, ScriptException { Object value = get(name); if (value instanceof Callable) { return ((Callable)value).call(arguments); } else { throw new NoSuchMethodException(name); } }}

This is continuation of my earlier post onscript beans. On rethinking, I've made few (minor) changes to ScriptObject and Callable. The changes are: Using varargs instead of Object[] arguments in few...

Java

Script Beans?

When we expose Java objects (or other language objects!) to a scripting language, we may want flexibility. Most scripting languages on the Java platform support JavaBean conventions. For example, JavaScript allows propertystyle access for "getXXX" methods. Some language engines (for example, Groovy) treat java.util.Map's speciallyto provide map.key_name style access. But, we may want more flexiblity in addition to bean conventions. For example, we may want java.sql.ResultSet to be accessed withthe natual obj.column_name syntax. But, instead of such ad-hoc special cases,we may want to have a generic way. Also, while jsr-223 has API support to call a specific script function or method, there isno way to "reflect" on script objects. For example, you can't find out the all methods andproperties supported by a specific script object. There is no engine independentway to reflect script objects - you have to use Scriptable interface for Rhino, GroovyObject interface for Groovy and IRubyObject interface for JRuby and so on.HotSpot Serviceability Agent (SA) is a core dump/hung process debugger for HotSpot JVM. It supports JavaScript based command line interface - much like dbx/gdb's shell-like scripting interface. I had used MozillaRhino API directly to implement that -- because it was done before the advent of jsr-223.I am porting this to use jsr-223 API. Previously, I had used Scriptable, ScriptableObjectand Function interfaces from Mozilla Rhino. I had changed this to use the following interfaces:ScriptObject/\*\* \* Any Java object supporting this interface can be \* accessed from scripts with "simpler" access pattern. \* For example, a script engine may support natural \* property/field access syntax for the properties exposed \* via this interface. We use this interface so that we \* can dynamically add/delete/modify fields exposed to \* scripts. Also, script engines may expose this interface for \* it's own objects. \*/public interface ScriptObject { // special value to denote no-result -- so that // null could be used as proper result value public static final Object UNDEFINED = new Object(); // empty object array public static final Object[] EMPTY_ARRAY = new Object[0]; /\* \* Returns all property names supported by this object. \* Property "name" is either a String or an Integer". \*/ public Object[] getIds(); /\*\* \* Get the value of the named property. \*/ public Object get(String name); /\*\* \* Get the value of the "indexed" property. \* Returns UNDEFINED if the property does not exist. \*/ public Object get(int index); /\*\* \* Set the value of the named property. \*/ public void put(String name, Object value); /\*\* \* Set the value of the indexed property. \*/ public void put(int index, Object value); /\*\* \* Returns whether the named property exists or not. \*/ public boolean has(String name); /\*\* \* Returns whether the indexed property exists or not. \*/ public boolean has(int index); /\*\* \* Deletes the named property. Returns true on success. \*/ public boolean delete(String name); /\*\* \* Deletes the indexed property. Returns true on success. \*/ public boolean delete(int index);}Callable.java/\*\* \* This interface is used to represent "function/method" valued \* properties in ScriptObjects. \*/public interface Callable { /\*\* \* Call the underlying function passing the given \* arguments and return the result. \*/ public Object call(Object[] args) throws ScriptException;}Just to be sure: please note that I am thinking of ScriptObject and Callable astwo way interfaces. Script engines would expose it's objects (all or some of it's objects)as ScriptObjects. Also, from Java code we can expose ScriptObject objectsto script engines and the engines will support special obj.fieldand obj.method(...) syntax in the respective language.Why can't we just use BeanUtils and avoid reinventing the wheel?For example, we can probably make use of BeanUtils API to expose objects to scripts and requires jsr-223 script engine implementers to treat DynaBeans specially to provide easier syntax. Andoptionally expose (some or all of) their own objects with DynaBean interface.beanutils has the notion of DynaClass.Every DynaBean isqueried to return it's DynaClass and from the DynaClass we get all properties supported.I think we can (and should?) avoid this when we have to deal with multiple languages. We would like to make minimal assumptions about objects -- allow prototype/delegationlanguages like JavaScript (where there is no notion of class) and allow fordynamically changing metaclass (like in Groovy, JRuby). So, in the above ScriptObject interface, we querythe ScriptObject itself to return it's properties by calling getIds method.That returns the current snapshot of the properties of the object.Also, beanutils API seems to concentrate only on "properties" only. There is nothing mentioned about "methods". (Did I miss anything in BeanUtils API that deals methods?).We have "Callable" - which is wrapper for any script method/function. ScriptObject.get(String name) would return a Callable object for method/function valued properties of the script object. beanutils brings much more than what we need - we want to add just enough abstractions.The ScriptObject and Callable interfaces could help in inter scripting language communication as well -- thereis no need to specify a strongly typed Java interface to communicate b/w twodynamically typed languages (using say, Invocable.getInterface()). For example, you can access Groovy object from a JRuby script andvice versa and both sides could use natural syntax to access ScriptObjects.Pleaselet me know your comments. We can probably try this out with script engines @ scripting.dev.java.net

When we expose Java objects (or other language objects!) to a scripting language, wemay want flexibility. Most scripting languages on the Java platform support JavaBean conventions. For example,...

Java

JVM Talks...

Recently, I gave a series 4 talks (consecutive saturdays about 2 to 2.5 hours) at CS department of Anna Universityon Java Virtual Machine. The aim was to give understanding JVM at specification leveland not much on any specific implementation of it. I did not create slides on my own. Also, I didn't stick only to slides - I didmostly marker-pen-and-board as well.Initial reading to introduce the concept of virtual machines:First chapter of Virtual Machines: Versatile Platforms for Systems and Processes book.P-code machine for PascalBecause we want to study a machine, it is useful to look at disassembled code.I used the following to show disassembled .class files:javap -comes with JDK. Run with -p (private as well) and -c (code) options.jclasslib Bytecode Viewer- very nice "visual" bytecode viewer. Allows us to nativagte various elements in a .classfile by hyperlinks. There is also a NetBeansmodule available for jclasslib.I used/referred the teaching materials from these sites/courses as well:Virtual Execution Environments - Jon Vitekhttp://www.jopdesign.com/teaching.jspArtima.com's http://www.artima.com/java/ - has very niceillustrative applets showing various aspects of the JVMWriting code using a Java assembler gives very good understanding. I usedjasmin assembler to showvarious JVM instructions. Also, we can see JVM's bytecode verifier in action:.class public Adder.super java/lang/Object; standard constructor.method public <init>()V aload_0 invokespecial java/lang/Object/()V return.end method.method public static main([Ljava/lang/String;)V .limit stack 2 .limit locals 2 ; nothing on stack, try using "iadd"iadd return.end methodYou can assmeble the above code by the command java -jar jasmin.jar Adder.jWhen you run the "Adder" class, you get the following verifier error:Exception in thread "main" java.lang.VerifyError: (class: Adder, method: main signature: ([Ljava/lang/String;)V) Unable to pop operand off an empty stackThere are also Java libraries to work with .class files:Byte Code Engineering Library (http://jakarta.apache.org/bcel/)ASM (http://asm.objectweb.org)Both the libraries can be used to read/analyze existing .class files and generate/modify .class files. BCELiferclass can be supplied with a .class file. BCELifier generates a Java program to generate the given .class file. By looking at the generated program, we can learn how to use BCEL API. Command to generate a Java source program using BCELifier for a given "HelloWorld.class" (The generated Java program can be compiled and run to (re-)create HelloWorld.class).java -cp bcel-5.1.jar org.apache.bcel.util.BCELifier HelloWorld.classASM library also comes with similar ASMifierClassVisitorclass to generate Java source.java -cp $ASM_HOME/lib/asm-util-3.0.jar:$ASM_HOME/lib/asm-3.0.jar org.objectweb.asm.util.ASMifierClassVisitor HelloWorld.classTutorials on BCEL and ASM:BCEL reportASM tutorial

Recently, I gave a series 4 talks (consecutive saturdays about 2 to 2.5 hours) at CS department of Anna University on Java Virtual Machine. The aim was to give understanding JVM at specification levela...

Java

(mis)understaning Scala's singleton types?

.keyword { color: red }I've been reading/playing with Scala. Scala has the concept of "singleton types". If types are considered as sets, singleton types are sets with only one object. A singleton type is expressed in Scala using object.type. To refer to the singleton type the expression for object has to be "stable". Informally, this means the expression that denotes the object should evaluate to the "same object reference" every time it is evaluated. A simple Scala interactive session to understand singleton types:This is an interpreter for Scala.Type in expressions to have them evaluated.Type :help for more information.scala> val s = "hello"s: java.lang.String = helloscala> val x : s.type = sx: s.type = helloscala> val p : s.type = "ss"<console>:5: error: type mismatch; found : java.lang.String("ss") required: s.type val p : s.type = "ss" \^The last command resulted in error because value "p" can not be initialized with any other String other than the one referred by "s" -- because p is of type s.type and not java.lang.String.Following is another session to understand "stable" path:This is an interpreter for Scala.Type in expressions to have them evaluated.Type :help for more information.scala> var s = "hello"s: java.lang.String = helloscala> val p : s.type = "ss":5: error: stable identifier required, but line8$object.s found. val p : s.type = "ss" \^"s.type" is not allowed as a valid type because "s" is not stable - because s is a variable (and not a value) it could be assigned with some other String reference and so "s.type" can not be defined.So far so good! How about creating a new instance of singleton type? What that could really mean? Let us try another interpreter session:This is an interpreter for Scala.Type in expressions to have them evaluated.Type :help for more information.scala> val s = "hello"s: java.lang.String = helloscala> var p = new s.type()Exception in thread "main" java.lang.AssertionError: assertion failed: s at scala.Predef$.assert(Predef.scala:89) at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:619) at scala.tools.nsc.ast.Trees$Transformer.transform(Trees.scala:1183) at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:635) at scala.tools.nsc.ast.Trees$Transformer.transform(Trees.scala:1161) at scala.tools.nsc.typechecker.RefChecks$RefCheckTransformer.transform(RefChecks.scala:635) at scala.tools.nsc.ast.Trees$Transformer.transform(Trees.scala:1173) [.... more frames deleted ...]Okay, the interpreter "crashed". How about the compiler?object test { def main(args: Array[String]) { val s = "hello"; var x = new s.type(); Console.println(x.getClass()); }}Surprise! The above code compiled and "x" refers to a (new) empty String! Now, either the interpreter or the compiler is wrong. And so my understanding of singleton types??. Update: Creating objects of singleton types means nothing (please see Martin Odersky's comment below). So, it is an implementation bug.

I've been reading/playing with Scala. Scala has the concept of "singleton types". If types are considered as sets, singleton types are sets with only one object. A singleton type is expressed in Scala...

Java

Scala for Java programmers - Part 2

.keyword { color : red }This is continuation of my last blog entry titled Scala for Java programmers.FeatureJavaScalaApplicationAny Java class with public static void main(String[])method can be run as an application.public class Hello {public static void main(String[] args) { System.out.println("hello world"); }}Any object with def main(args: Array[String]) : unit methodcan be run as an application. [Note: Scala does not have static members - youhave to use singleton objects.]object Hello {def main(args: Array[String]) : unit = { Console.println("hello world"); } }Also, you could extend Application class and have just code inside object likeinobject Hello extends Application { Console.println("hello world");}But, if you want command line parameters, you have to use the first option.Sequence-comprehensionis an easier syntax to build a new sequence from existing sequence(s)by filtering, mapping or combination of those.Use Java's for-eachloop and build a new list.General form:for(<generators>; <filter>) yield <expression>Example:object Main extends Application { val list = List("Fortran", "Java", "Scala", "JavaScript", "ML"); val shortList = for (val e e.length() < 6) .map(e=>e.toUpperCase()); Console.println(shortList);}See also: Sequence Comprehensions.Method referencesNo. Use java.lang.reflect.Method (but, you'll miss static type checking!)Method references can be passed as argument whenever a function type valueis expected.object Main { def func(s: String) : boolean = { return s.startsWith("J"); } def print(s: String): unit = { Console.println(s); } def main(args: Array[String]) : unit = { var list = List("hello", "world", "Java"); // pass method reference as argument list = list.filter(func); Console.println(list); // explicitly mentions "Main" object // in the method reference list.foreach(Main.print); }}Also, remember operators are methods too and numbers are objects too!So, the following works:object Main { def main(args: Array[String]) : unit = { val list = List(3, 54, -1, -324, 45); // print negative numbers only Console.println(list.filter(0 >)); }}Pattern Matchingis a techique to classify objects by their run-time type, accessing theirmebers or some other characteristic of group of objects.No. Following "Matching Objects with Patterns",in Java you could do:Object-oriented decompositionVisitor patternType-Test/Type-Cast (instanceof and cast)Scala has "match" expression. Think of it as a generalization of Java'sswitch statement. While Java's switch statement works only for integers andenums,Scala's pattern matching is more general. Simple switch statement equivalent inScala is as follows:object t { def main(args: Array[String]) : unit = { args.length match {case 1 => Console.println("1 argument");case 2 => Console.println("2 arguments");case _ => Console.println("more than 2!"); } }}More complex match example: def matchTest(x: Any): Any = x match { // matching integer value "1" case 1 => "one" // matches string "two" // x is a string with "two" as value case "two" => 2 // x is any int type but with not equal to 1 case y:Int => "scala.Int" }Even more complex pattern matchings involve case classes.Case classes are classes with "case" modifier. With "case" classes, you cancreate objects of case classes without "new" operator - use function call-like syntaxcase classes have automatic accessor generators for constructor parameter.case classes can be used in pattern matching.Example:abstract class Expression;case class Number(n:int) extends Expression;case class Add(left:Expression, right:Expression) extends Expression;case class Var(name: String) extends Expression;case class Multiply(left:Expression, right:Expression) extends Expression;// sample match expression that uses above case classes// This expression simplifies a given expression "e"// We use x + 0 = x and x \* 1 = x rules to simplifydef simplify(e: Expression) : Expression = { e match { // matches if expression is a Multiply object // and right operand is Number 1case Multiply(x, Number(1)) => x // matches if expression is a Add object // and right operand is Number 0case Add(x, Number(0)) => x // matches any other expression - no simplification // just return the original expressioncase _ => e }}References:In Defense of Pattern MatchingMatching Objects with PatternsSee also: Scala case classesand Scala pattern matching.Throwing exceptionsthrow new RuntimeException();throw new IOException();throw new RuntimeException();throw new IOException();This is same as Java except that in Scala, there are no checked exceptions.User is not forced to have "throws" clauses as in Java.Catching Exceptionstry { // ...} catch(IOException ieExp) { // ...} catch(RuntimeException re) { // ...} finally { // ...}where finally is optional.try { // ...} catch {case ioExp:IOException => { // ... }case re:RuntimeException => { // ... }} finally { // ...}Notes:Pattern matching is used to match aparticular exception type within a single catch clause - unlikeseparate catch clauses in Java.finally clause is optional here as well.Because catch uses case clauses, it is possibleto match values - for example, if your exception class isa case class, then you can match constructor parameters ofyour exception class as well.case class MyException(s:String) extends Exception(s) {}object Hello extends Application { try { throw new MyException("hello"); } catch { // MyException with message equal to "hello" case MyException("hello") => Console.println("world!"); // MyException with message not equal to "hello" case MyException(_) => Console.println("no way!"); // any other exception case _ => Console.println("any other exception"); }}Packagespackage com.acme;With the above statement as first line in a compilation unit,everything in the compilation unit goes into the packagepackage com.acme;With the above statement as first line in a compilation unit,everything in the compilation unit goes into the package --as in Java. But, Scala supports "packaging" statements as well.The following example definespackage com.acme { class X {} class Y {}}package com.sun { class X {} package P { class Y {} }}Notes:Several packages may be declared in the same compilation unit.Packages can include other packages.The above example defines com.acme, com.sun, com.sun.P packages.See also: Scala packages.Import Statements// import whole packageimport java.io.\*;// import a specific (public) class from a packageimport java.net.URL;// import whole packageimport java.io._;// import a specific (public) class from a packageimport java.net.URL;Static Import// import all static members of Math classimport static java.lang.Math.\*No statics in Scala -- we use singleton objects. But, we can import all members of singleton objects.object t { def sayHello() : unit = { Console.println("hello"); }}object Main {import t._; def main(args: Array[String]): unit = { sayHello(); }}Also, every Java class is exposed a Scala class withoutstatic members and an object that contains only the staticmembers of the Java class. So, it is possible to import allstatic members of a Java class using the object import asfollows:import java.lang.Math._;object t { def main(a: Array[String]) : unit = { Console.println(sin(3.14/2)); } }typedefs(a.k.a type aliasing).No (but may be in future?)Generic pattern is type <name> = <type-expression>Examples:type StringList = List[String];var sl: StringList = List("hello", "world");type voidFunc = ()=>unit;var v : voidFunc = ()=>Console.println("hello");v();Abstract Types(using abstract type members in classes in addition to abstract value members)No. Use generics!In Scala, classes can have type members in addition to value members. Also,like value members (say a method being abstract), type members could be abstractas well. Example:abstract class Cell { // an abstract type member // subclass can define ittype T // abstract value member var element: T}abstract class ValueCell extends Cell { // define T to be some subtype of AnyVal type T

This is continuation of my last blog entry titled Scala for Java programmers. Feature Java Scala Application Any Java class with public static void main(String[])method can be run as an...

Java

Scala for Java programmers

.keyword { color : red }There are many languages that target to the Java Virtual Machine (JVM)-- not just the scripting and dynamically typed ones. How about learning another statically typed languagethat is compiled JVM and seamlessly integrates to the Java platform? The latest mantra is "One Great VM, Many Languages", Right? :-)We will look at - Scala? - a functional,object-oriented and concurrent language that runs on the Java platform. Note:this is not a language comparison exercise [i.e., X versus Y comparison to "conclude" which one is better!]. Rather, it is an attempt to give head-start to the Java programmerswho want to learn other languages that run on the Java platform. So, the aim is same as it was for my blog past entries such as Java, Groovy and (J)Ruby,Java, JavaScript and Jython - except that we are now looking at a statically typed language...FeatureJavaScalaStatic TypingYesYesObject Oriented programmingYesYesFunctional programmingNoYesVariable declaration// type var_name = init_value;int i = 0;// var var_name: type = init_value;var i : int = 0;Constant declaration// final type var_name = init_value;final int i = 0;// val var_name: type = init_value;val i : int = 0;Class declarationclass Person { // members here}class Person { // members here}Methods// RetType name(PType1 pName1, PType2 pName2...)class Person { // members here public String getName() { // code here.. } public void setAge(int age) { // code here.. }}// def name(pName: PType1, pName2: PType2...) : RetType class Person { // members heredef getName() : String = { // code here.. }def setAge(age: int) : unit = { // code here.. }}ConstructorsMethod(s) with same name as class name with no return type.class Person { public Person(String name, int age) { // initialization here.. } public Person(String name) { // call the other constructorthis(name, 1); }}Constructor parameters are specified in class declaration itself.Example:class Person(name: String, age: int) { // any method can access "name" or "age" parameter}You can have "secondary constructors" as well. Example:class Person(name: String, age: int) { // any method can access "name" or "age" parameter def this(name: String) { // call the "primary" constructorthis(name, 1); }}Operator overloadingNo. Except that string concatenation can be done using "+" - which gets transformed as "concat" calls.Yes. Just use operator as method name. Example:class Complex { def + (other: Complex) : Complex = { //.... }}Also, any single parameter method can be used as an infix operator.Example:// call System.exit(int) using infix notationSystem exit 0// call Thread.sleep(int) using infix notationThread sleep 10See also: Scala operatorsStatic fields and methodsclass Person { private static Person president = .... private static Person getPresident() { return president; }}No static members. Use singletons.SingletonsNo language support. You can simulate something like this:class President extends Person { // make the constructor private - so that // instance can not be created outside of this class private President() {} // create the singleton object here.. private static President thePresident = new President(); // provide accessor for singleton // and add other methods here..}object President extends Person { // have methods of the singleton object}Singleton "objects" are in effect "modules" in Scala. Also, every Java classis viewed like any other Scala class without static methods and fieldsand a singleton object [whose name is same as Java class name] that contains only the static methods and fields of Java class as it's (non-static) members.This allows accesing static methods of Java classes but at the same timenot having static members in Scala.Inheritanceclass Person {}class Graduate extends Person {}class Person {}class Graduate extends Person {}See also: Scala subclassing.But, this reference needs update. Now, Java (since JDK 5.0) supports covariant return types.Calling super class constructorclass Graduate extends Person { public Graduate(String name, int age, String degree) {super(name, age); }}You call super class constructor in class declaration itself.class Graduate(name: String , age: int, degree: String) extends Person(name, age) {}Method Overridingclass Graduate extends Person {@Override public String getName() { // code here... } }@Override is optional - but helps with detecting errors (you may be thinkingyou are overriding - but in fact you may actually be overloading a super class method).class Graduate extends Person {override def getName(): String = { // code here... }}The keyword "override" is mandatory for non-abstract method overrides.Abstract classes and methodsabstract class Person {abstract public String getName();}abstract class Person { def getName(): String;}No "abstract" keyword for methods. Only for classes.Root of all reference typesjava.lang.Objectscala.AnyRef- which is same as java.lang.Object in JVM implementation of ScalaUniform object orientation? (i.e., is everything object?)No. There are primitive types and reference types. But withautoboxing/unboxing, you don't need to explicitly convert b/w primitive types and corresponding box types.Yes. "int" is an alias to "scala.Int" and so on. A bit explanation oftype hierarchy. scala.Any is supertype of all types. scala.AnyVal is supertype of value types (such as "int" etc.). scala.AnyRef is super type ofall "reference" types. But, method calls etc. work on value types as well.Examples:44.+(4) is same as 44 + 4 where "+" is a method on scala.Int class.Also, single argument methods can be called using "infix" notation just likeoperator methods are. Example: System exit 4 is same as System.exit(4)interfacesinterface Runnable { void run();}Use traits. Traits are like interface but can have method bodies (i.e., not justmethod declarations). See also: multiple inheritance.trait Runnable { def run(): unit;}Multiple inheritanceNo. Only multiple interfaces may be implemented. Only single class maybe extended.Multiple classes can not be extended. Multiple traits can be.And traits can have code -- not just declarations.trait Runnable { // parameter accepting run method def run(obj: AnyRef) : unit; // no params - assuming "null" instead def run() : unit = { System.out.println("assuming null..."); run(null); }}// extend a single class, but can "extend"// multiple traits with "with"class MyClass extends AnyRef with Runnable { // just implement run with param. // Other "run" is inherited from Runnable def run(obj: AnyRef) : unit = { System.out.println("got " + obj); } }See also: Scala mixins,mixins paper,traits paper.Inner classespublic class Book { private String name; // ... public Order newOrder() { ... } public class Order { private Date orderDate; private int quantity; // ... } }class Book(name: String) { class Order(orderDate: Date, quantity : int) { } def newOrder() : Order { return new Order(new Date(), 1); }}Unlike Java where such inner classes are members of the enclosing class, in Scala inner classes are bound to the outer object. With the above classes, in Java youcan write// JavaBook freakonomics = new Book("Freakonomics");Book letUsKillGandhi = new Book("Let us kill Gandhi");Book.Order o1 = freakonomics.newOrder();Book.Order o2 = letUsKillGandhi.newOrder();// You can assign any Book.Order // to any other Book.Ordero2 = o1;// Not in Scala!val freakonomics: Book = new Book("freakonomics");val letUsKillGandhi : Book = new Book("Let us kill Gandhi");var o1 = freakonomics.newOrder();var o2 = letUsKillGandhi.newOrder();// this line below will not compile!// type of "o1" is "freakonomics.Order"// and type of "o2" is "letUsKillGandhi.Order"o2 = o1;In other words, order for "Freakonomics" book can not be treated as instance of order for "Let us kill Gandhi"! If you really want to referto inner class type as in Java, you can use the following:var o1 : Book#Order = freakonomics.newOrder();var o2 : Book#Order = letUsKillGandhi.newOrder();o1 = o2;See also: Scala inner classes.Class literals[section 15.8.2 of JLS].String.classObject.classclassOf[String]classOf[AnyRef]See also: Scala classOf.Dynamic type checkx instanceof Stringn instanceof Numberx.isInstanceOf[String]n.isInstanceOf[Number]Dynamic type castString x = (String) obj;Number n = (Number) obj;x : String = obj.asInstanceOf[String];n : Number = obj.asInstanceOf[Number];Generics// an interface with a type parameterinterface Stack<E> { void push(T t); T pop(); boolean isEmpty();}// a class with two type parametersclass Pair<K, V> { private K key; private V value; public Pair(K key, V value) { this.key = key; this.value = value; } // more code...}// a trait with a type parametertrait Stack[E] { def push(o: E) : unit; def pop() : E; def isEmpty(): boolean;}// a class with two type parametersclass Pair[K, V](key: K, value: V) { // code here...}See also: Scala generics.Generic methodsclass Test { public <T> List<T> dup(T t, int n) { //.... }}class Test { def dup[T](x: T, n: Int): List[T] = { if (n == 0) Nil else x :: dup(x, n - 1) }}See also: Scala polymorphic methods.Bounded type parameter// upper boundclass Foo<E extends SomeType> { // ... code here}// lower boundclass Foo<E super SomeType> { // ...}// type parameter in boundclass Stack<E extends Comparable<E>> { // ...}// upper boundclass Foo[E : SomeType] { // ...}// type parameter in boundclass Stack[E extends Comparable[E]] { // ...}Wildcards// upper boundArrayList<? extends Number> l = ...// lower boundArrayList<? super Number> l = ...// wildcard without boundList<?> list = ...See also: Wildcards paperIn Java, variance annotations are specified by "clients" [a.k.a use-site]. InScala, if you want Foo[A] to be subtype of Foo[B] whenever A is subtype of B,you have to declare Foo as// annotation +T declares type T to be used // only in covariant positions.class Foo[+A] { // ...}// annotation -T declares type T to be used // only in contravariant positions.class Foo[-A] {}Example: In Scala, Lists are immutable and List has covariantannotation. So, List[B] is subtype of List[A] whenever B is subtype of A. See also: Scala variancesArraysString[] s = new String[10];var s : Array[String] = new Array[String](10);Array is a generic type in Scala (much like "Vector", "List" etc.in Java are). Note: Arrays do not follow the covariant subtype rule.i.e., Array[Graduate] is not a subtype of Array[Person] even if Graduate is subtype of Person - this is unlike Java language in whichGraduate[] is a subtype of Person[].Array element access, updateint[] a = new int[3];a[0] = 3;System.out.println(a[0]);var a : Array[int] = new Array[int](3);a(0) = 3;System.out.println(a(0));Array element access and update are actually methodcalls on Array[T]. You may be wondering how method callappears on the right side (for element update). No,Scala does not have C++ style reference (&) types.a(i) = 10 is translated as a.update(i, 10)Varargsclass Calc { public static int sum(int... values) { int res; for (int i in values) { res += i; } return res; } }// with the above definition, caller can useCalc.sum(3, 44)Calc.sum(4, 4, 5,45, 45);With '...' after type of last parameter, your method becomes variadic. The typeof "values" in above example is int[]. In general, thetype would be T[], if the last param type was T....object Calc { def sum(values: int\*): int = { var res: int = 0; for (val v unit): unit = { while (true) { callback(); Thread.sleep(1000);}def printHello() = { System.out.println("hello world");}// pass printHello as parameteroncePerSecond(printHello);// define anonymous function and call it((i:int, j:int) => i+j)(3, 4)// anonymous function passed as parameteroncePerSecond(()=>System.out.println("hello"));// nested functions toodef outer() = { def inner() = { System.out.println("I am inner"); } inner();}// inner functions can access outer's locals// and argumentsdef outer(s: String) = { def inner() = { System.out.println("outer's 's': " + s); } inner();}// Java closure proposals talk about converting// closures to interfaces automatically. I think// we can use views in Scala.// define conversion from any parameterless function // to java.lang.Runnableimplicit def asRunnable(func : ()=>unit) : Runnable = { new Runnable() { def run() { func() } }}def main ( args : Array[String] ) = { // create a new thread - // passing an anonymous function // for Runnable var t = new Thread(()=>Console.println("hello")); t.start(); // you can now initialize Runnable // with any function var r : Runnable = ()=>Console.println("I am running!") r.run();} See also: Scala views.List supportUse java.util.List and classes implementing it. // create a List with "List" function (which is actually// an object with "apply")var names : List[String] = List("Java", "JavaScript")// :: is the cons operator// creates a new list with first element as "Scala"// and rest of the elements from "names" listval l = "Scala" :: names// :: becomes method call on List -- unlike // other operators :: takes right-side value // as list as target object to call method on!Lists in Scala are immutable. There are methodson List class - like map, filter etc. that acceptclosure arguments. Examples:// make a new list that contains only short names// shortNames is of type List[String]var shortNames = names.filter(n=>n.length() < 6)// make a new list that has lengths of names// "lengths" is of type List[int]var lengths = names.map(n=>n.length())scala.List has covariant type parameter [note: Lists are immutable].So, List[Graduate] is subtype of List[Person] if Graduate is subtype of Person.val graduates:List[Graduate] =...val persons:List[Person] = graduates;There is much more to Scala - pattern matching, for-comprehension, exception handling, packages, abstract types, XML literals, currying, implicit parameters etc. But, we have had enough for a single blog entry :-)To be continued.. stay tuned!

There are many languages that target to the Java Virtual Machine (JVM) -- not just the scripting and dynamically typed ones. How about learning another statically typed languagethat is compiled JVM...

Java

Sun Tech Days Hyderabad 2007 - trip report and slides

As I mentioned earlier, I spoke at Sun Tech Days 2007. About 10000 folks attended Sun Tech Days. Approximately about 2000 folks attended each talk (at least in the ones in the "main" room where key notes talks were given). Very nice to see so many developers at one place! I think over the years, Sun Tech Days is growing to become like JavaOne. Kannan and I traveled from Chennai to Hyderabad to attend this event. This year I gave two talks. In the talk titled "Java SE Language Features - JDK 5.0, 6 and Future", I covered mostly the language features of JDK 5.0. Because there are no Java language features in JDK 6, I covered javac's APIs (JSR 199, JSR 269 and Tree API). For JDK 7, I gave few web references (on closures etc.). The scripting talk is the one we (Mike Grogan and I) gave many times :-) Good news - we won't be talking on scripting in JavaOne 2007 :-). Q&AI didn't have time for Q&A after the scripting talk. We [Charles Nutter and I] answered few scripting questions outside the lecture hall. Summary of the Q & A:What are the typical use cases of Scripting API?JSR-223 is applicable where "Java code calls scripts". Say, you have a big Java application. You want to support customization of it. Instead of inventing an ad-hoc, home-grown (and possibly expression-only!) language for that purpose, you can use any JSR-223 scripting language. Your user will have the choice of scripting language. Examples: Scripting SVG with JSR-223. Another example: The Phobos project uses scripting API to support any jsr-223 language in it's web application framework. Yet another example: jconsole script shell plug-in.Should I've to learn more languages in addition to Java?It depends on your project! Most projects already involve more than one language (Java, XML, SQL dialect(s?), XPath/XQuery/XSLT ....) anyway. If your project requires, you may add scripting to the mix! I just prefer to use another language on the Java platform. Do I still have to use JSR-223?No. JSR-223 is meant for the use cases where you want to call/eval script from your Java code. If you just want to code in another language and still want to use Java platform (for portability, platform API or for any other reason], you can do so without getting into the JSR-223 API. For example, you can use JRuby or Groovy or any other language that runs on the Java platform.How does the invokedynamic (JSR 292) relate to JSR-223?A scripting or dynamically typed language may be implemented in many ways: interpreting the abstract syntax tree directly, interpreting it's own bytecode [like JavaScript] or generating JVM bytecodes. If a language is implemented by generating (JVM) bytecode (like Groovy, Pnuts ...), invokedynamic would help there. Whether or not the language runtime implements JSR-223 interface is another (unrelated) matter.SlidesThe slides are usually posted at Sun Tech Days site few days after the event. But if you are impatient, you can download the slides for the talks that I gave :-)Java SE Language Features - JDK 5.0, 6 and FutureJava Scripting: One VM, Many Languages

As I mentioned earlier, I spoke at Sun Tech Days 2007. About 10000 folks attended Sun Tech Days. Approximately about 2000 folks attended each talk (at least in the ones in the "main" room where key...

Java

JSR-223 variable access in Scheme, Tcl and F3

jsr-223 defines scripting API for the Java platform -- so that Java programs can execute scripts written in various scripting languages. As you'd probably know already, JDK 6 includes javax.script API and JavaScript engine in it. One of the facilities of javax.script API is script Bindings. Through the javax.script.Bindings interface, you can expose your application objects as global variables to your scripts. Ideally, you want to access global variables by the same name as key names in Bindings. But, there are some language specific complications in exposing global variables. Not every language has global scope - for example, Java. So, the script engine for Java looks for special setScriptContext static method to expose current context to the "script". In JRuby, global variables are always "$" prefixed - the prefixing is done by jsr-223 script engine for JRuby - so that your Bindings may have "foo" as key and your JRuby script may use "$foo". In addition, there are also some implementation specific complications in implementing JSR-223 Bindings. We will look at some specific cases below.JSR-223 Bindings implementation for SchemeSISC implementation has beenused to implement jsr-223 engine for Scheme. The way jsr-223 Bindings works in Scheme script engine is as follows:Scheme user has to use "var" procedure to access/update the variables exposed by jsr-223 bindings. To access variable "foo" from Scheme, you use (var 'foo). To update variable "foo", you need to use (var 'foo <value>). For example, you can use the following expression to print the current file name. (display (var 'javax.script.filename))The following expression (var 'context) can be used to access the current script context from Scheme.JSR-223 Bindings implementation for Tcl (Jacl)TCL (Jacl) user has to use "var" procedure to access/update the variables exposed by jsr-223 bindings. To access variable "foo" from Tcl, you need to use [var "foo"]. To update variable "foo", you need to use var "foo" <value>; For example, you can use the following expression to print the current file name. echo [var "javax.script.filename"]Similarly, the following expression var "context"can be used to access the current script context from Tcl.JSR-223 Bindings implementation for F3F3 is statically typed language. You may want to refer to F3 and JSR-223to check how jsr-223 Bindings is implemented in F3.

jsr-223 defines scripting API for the Java platform -- so that Java programs can execute scripts written in various scripting languages. As you'd probably know already, JDK 6 includes javax.script API...

Java

jhat's JavaScript interface

In one of the recent e-mails, Frank Kieviet suggested an improvement for jhat tool in JDK 6. He said something like:Sundar: another idea to extend jhat: when looking at a memory dump and looking at the routes to a particular classloader, there are sometimes hundreds of references. [...] jhat could print some histogram data so that it’s immediately apparent which object reference is suspect.The above could be generalized to any Java object (not just a classloader object). "routes to a particular classloader" probably requires explanation. Every Java object is live because there are one or more paths or routes (or chain of references) from the GC root set [recall that the root set is the initial object references from which garbage collector starts and constructs transitive closure of all objects reachable from there - such transitively reachable objects are alive and hence not eligible for collection]. Frank wants a facility in jhat to view histogram of objects across the liveness paths of a given object.Being lazy, I don't want to change jhat for this :-) [also users may not be willing to move to a different release of JDK just for this jhat change!] Fortunately, jhat has a query interface called OQL.OQL is built on top of the JavaScript engine bundled in JDK 6. Actually, you can even run any JavaScript code (and not just select...from..where style OQL queries) in jhat's OQL window! If the string entered in OQL window does not start with the "select" keyword, it is treated as JavaScript code and eval'ed as is using the JavaScript script engine bundled in JDK 6.OQL has number of built-in JavaScript functions and objects. There is one object called "heap" which has useful methods such as "livepaths", "findObject". "livepaths" methods returns an array of paths by which given Java object is alive. Each path is another array of objects [objects in that liveness path chain]. There is another method called "findObject" which returns a specific object given it's object identifier [note that jhat shows object ids in hex format - so the hex format is used by "findObject" as well]. Using these we can write something like this:(function() { var histo = new Object(); map(heap.livepaths(heap.findObject('0x0f054b90')), function(path) { for (var i = 0; i < path.length; i++) { var desc = toHtml(path[i]); if (desc in histo) { histo[desc]++; } else { histo[desc] = 1; } } }); var str = "<table>"; for (i in histo) { str += "<tr>"; str += "<td>" + i + "</td><td>" + histo[i] + "</td></tr>"; } str += "</table>"; return str;})();In the above code, the object id '0x0f054b90' used as argument to heap.findObject method is an object of interest in a specific heap dump. This is an object whose liveness paths we are interested on [turns out to be a classloader object]. When I entered the above script in jhat's OQL window, I got a nice histogram in HTML table form!

In one of the recent e-mails, Frank Kieviet suggested an improvement for jhat tool in JDK 6. He said something like:Sundar: another idea to extend jhat: when looking at a memory dump and looking at...

Oracle

Integrated Cloud Applications & Platform Services