Hotspot vs Adobe Tamarin VM? No contest.

Several people have told me of Adobe boasting about their Tamarin JavaScript VM, so I decided to look into it myself. I ran 100 iterations of the same Takeuchi Benchmark from my previous post with both Tamarin and Java SE 1.6 Hotspot VM and for this case I found Hotspot more than 25 times faster (see below).

Surprisingly (or maybe not) Adobe's ActionScript compiler (which generates the byte-code for Tamarin) itself is actually a Java application.

Here's Tamarin:


function tak(x:Number, y:Number , z:Number) {
    return y >= x ? z : tak(tak(x-1, y, z),
                            tak(y-1, z, x),
                            tak(z-1, x, y));
}

var i = 0;
while (i < 100) {
  tak(24, 16, 8);
  i++;
}

$ java -jar asc.jar -import builtin.abc tak.as
$ time ./avmplus tak.abc

real    0m58.587s
user    0m57.900s
sys     0m0.130s

And here's Java:

public class Tak {
    public static double tak(double x, double y, double z) {
        return y >= x ? z : tak(tak(x-1, y, z),
                                tak(y-1, z, x),
                                tak(z-1, x, y));
    }
    public static void main(String argv[]) {
        for (int i = 0; i < 100; i++) {
            tak(24, 16, 8);
        }
    }
}


$ time java -cp . Tak

real    0m2.231s
user    0m2.143s
sys     0m0.061s

Interestingly, I also tried the Mozilla Rhino JavaScript engine (which I contributed to years ago), which is a pure Java implementation of JavaScript with this result:
$ time java -jar js.jar -opt 9 tak.js

real    0m31.944s
user    0m31.718s
sys     0m0.181s
Comments:

Chris, according to your previous post, the JavaFX version was running even faster than Java. It would also be nice to see the memory usage and startup time for a fair comparison.

Posted by holycow on July 26, 2007 at 06:06 PM PDT #

FWIW Chris' benchmarks do include startup time (double check the command lines he used to run the benchmarks). More importantly startup time is in most cases irrelevant - when was the last time you worked on an application where startup time dominated (or was even a simple majority) of the total runtime of the system? OTOH memory usage would be interesting to see.

Posted by startup time is irrelevant on July 27, 2007 at 02:17 AM PDT #

No contest indeed, Tamarin is optimized towards small footprint and instantaneous startup, compared to Hotspot. However, it can do better. If I add a return type annotation, and place a package {} declaration around the test (triggering early binding), Tamarin avoids boxing overhead, and the test results drop from 56s to 11s on my T60p, solidly inbetween Rhino-JS and pure Java.

Posted by Edwin Smith on July 27, 2007 at 03:16 AM PDT #

To add to Ed's post, we're of course very interested in making Tamarin optimize unannotated JS as well as annotated JS.

The missing return type annotation in the original Tak AS3 code, though, should be added by itself, just for apples-to-apples comparison without the package wrapping. (The package wrapping is also something we will relieve programmers from having to write, since the web is full of untyped and package-less JS.)

Hotspot is an amazing piece of work, years (decade plus, right?) older, and enjoying the benefit of a great many hands, compared to Tamarin. So no one should expect miracles. On the other hand, the design point for Tamarin is well-aligned with the web browser JS engine's requirements, so we are going to push Tamarin as far as we can.

My keynote from The Ajax Experience West talks more about what we're doing with Tamarin, and why.

/be

Posted by Brendan Eich on July 27, 2007 at 04:25 AM PDT #

Chris: your question answers itself. Java in the browser is effectively dead, killed by a combination of poor single-vendor stewardship (IMHO -- and whether the vendor was Netscape or is Sun doesn't matter, I'm not pointing a finger here), Microsoft giving it the boot, and the rise of Flash. It is years and megabytes of download late to the Firefox party, for instance.

Moreover, for Mozilla at least, we absolutely cannot depend on closed source, and we require a non-copyleft BSD license, or at most MPL/GPL/LGPL. Java was not even open source until recently (I don't remember the date; it was preannounced one too many times :-/), well after we had to make our own plans and commitments.

Finally, in spite of the prospects with JRuby, the JVM really is about Java first and last. Tamarin is about an ECMAScript variant, so it's a better target now, and more likely to evolve to support JS1 and JS2 in a first class way, than the JVM.

Compilation heroics can help, but the browser will remain an environment where compilation must be very fast. Wherefore our forthcoming work on a trace-based JIT.

/be

Posted by Brendan Eich on July 27, 2007 at 04:34 AM PDT #

By the way, I find it hard to believe that any Adobe engineer has ever claimed that Tamarin was faster than the Sun JVM. As Brendan points out, such a comparison would be foolhardy given the difference in target environment and programming language. The engineers who built AVM are justifiably proud of its performance as an ECMAScript engine, though, and it may have been this kind of "bragging" you hard about.

Posted by Andrew Shebanow on July 27, 2007 at 05:22 AM PDT #

make that "heard about"...

Posted by Andrew Shebanow on July 27, 2007 at 05:22 AM PDT #

More sour grapes from the Java crowd. Applets are dead and never coming back, contrived benchmarks won't change that.

Posted by Slava Pestov on July 27, 2007 at 05:56 AM PDT #

"Java in the browser is effectively dead". This is as far from the truth as it could possibly be. Sure, you may not see coke.com using a applet on their home page, but enterprises that are building applications that drive their business or power knowledge workers use Java because of the centeral deployment coupled with the power of Javas JIT compiler. The company I work at use a Java based solution to help people build RIA applications. Java in the browser did have some issues as did the browse and JavaScript for building applications. Just like Ajax libraries implementors build abstractions to help people. Vendors are doing the same thing with Java.

Posted by Bob Buffone on July 27, 2007 at 06:54 AM PDT #

Bob: enterprises use various "RIA" approaches; many still use ActiveX plugins. An enterprise can mandate and internally distribute whatever runtimes it pleases.

My point was in the context of the Web as the contested ground, not intranets; and the Web is where Flash has effectively vanquished Java. You can find real estate virtual tours that use Java, but the defaults are always Flash, always load faster, and always perform better. Same with Flash games vs. Java games. The ubiquity of the Flash plugin compared to an up-to-date JRE is another data point.

/be

Posted by Brendan Eich on July 27, 2007 at 04:50 PM PDT #

SADLY the Mozilla-organization is more interested in supporting Microsoft .NET then Java (see the ongoing efforts on mozilla.org for supporting IronPython). Apparently Brendan Eich and friends hate everything related to Java-technology and they prefer to reinvent the wheels. Very sad indeed. There's only one problem: Java-JVM are developed by IBM and Intel. Those companies know a lot about multi- and many-core chips. So the JVM is likely to win.

Posted by F. Simoni on July 27, 2007 at 06:10 PM PDT #

F. Simoni: who are you to impugn hatred to me or Mozilla? We've made our reasoning crystal clear, and until this year, without open source Java, there was \*no way\* to integrate a JVM as the VM underlying JavaScript in Firefox or any Mozilla product. We don't depend on closed source in-process code, period. I've also mentioned the license requirements, not that it mattered due to the lateness of Java's open source transition (GPL alone is not ok).

I spoke about Mozilla's VM needs at an IBM hosted "VEE" conference almost three years ago. When one of the IBM researchers said "why not Jikes?" his colleague quickly said "no way, not ready, not complete, way too many megs of code footprint."

It seems the first resort of those who divide the world into "Java haterz" and ".NET lovers" is to make emotional accusations and misattributions. Mozilla can't afford such content-free drama. We actually kept Firefox 1 under 5MB in download size for good reasons. No way could we have used a JVM for its needs, even if the open source license and Java-only focus problems were fixed. Wake up!

/be

Posted by Brendan Eich on July 28, 2007 at 07:12 AM PDT #

Oh, I forgot to deal with this bizarre aside:
(see the ongoing efforts on mozilla.org for supporting IronPython)
You must, by "ongoing", mean the C-Python integration work over the years by Mark Hammond. Clue: C-Python is the original Python interpreter, written in C; it has no relation to IronPython or .NET. This use of "ongoing" cannot possibly refer to the IronPython work that I announced just three days ago at the Ajax Experience West. Check your facts, please.

/be

Posted by Brendan Eich on July 28, 2007 at 08:18 AM PDT #

Brendan,

Please don't feel obligated to defend your actions, especially here on my blog. I agree the decisions you've made seem reasonable, including exploring using Tamarin.

However, as you point out there are some recent developments with the Java platform, namely open-sourcing, and also JavaFX that may change the picture in the future.

If JavaFX is going to be successful, then we'll have to solve the footprint issues with the Java platform regardless. These issues have little to do with the VM, but rather are due to the inflexible and monolothic libraries packaged with Java SE, and the sloppy and unnecessary initialization of various subsystems during the startup sequence. The JavaME stacks demonstrate this point conclusively.

The underlying requirements of JavaFX are largely similar in terms of footprint, VM performance, graphics quality, graphics performance, to that of an HTML-DOM + JavaScript implementation.

Of course, this is all speculative at this point, however, if we end up solving these problems, then possibly most of the past roadblocks to using the Java platform may have been removed.

Note: I had no involvement in this area until 2005. So I can't speak (nor do I want to speak) about past sins.

The point of my original post was simply to point out that the purely technical benefits of the Hotspot VM seem to be being ignored, by handwaving about orthogonal issues.

Posted by Chris Oliver on July 29, 2007 at 02:58 AM PDT #

No need to defend, no need to offend, yet there's clearly bad blood and some amount of impugning (I misused that earlier, meant impute ;-) of Mozilla's record. It would be great to get back to technical brass tacks.

In that vein, did you have a chance to add just the return type annotation to the AS3 Tak and see what difference that made?

/be

Posted by Brendan Eich on July 29, 2007 at 05:52 AM PDT #

I just tried it, and it seems that specifying the return type has no impact on performance in this case.

Is Adobe's stuff actually available in open-source form yet?

I looked around and all I see are the press releases.

Hopefully, this won't scare them away from open-sourcing it, but I think it'd be interesting to look into retargeting asc to the JVM.

Chris

Posted by Chris Oliver on July 29, 2007 at 08:24 AM PDT #

Tamarin was open-sourced without pre-announcement last November (easy to find via Google for "Tamarin VM" -- first hit), here.

The source is in Mozilla's new Mercurial repository.

The asc compiler is part of Flex.

/be

Posted by Brendan Eich on July 30, 2007 at 03:18 AM PDT #

Yes, I found the Mozilla-hosted Tamarin stuff without a problem, previously. However, Adobe's press release states that they're open-sourcing "Flex SDK" which seemed to include asc. Apparently, that hasn't happened yet.

Once it does, I'll take another look.

Chris

Posted by Chris Oliver on July 30, 2007 at 06:58 AM PDT #

Flex SDK is a free download with source, just not yet MPL-licensed. You can grab and use it any time. The relicensed source in a public repository is scheduled for December, but Flex is free as in beer already.

/be

Posted by Brendan Eich on July 31, 2007 at 08:25 AM PDT #

Hello,

I ran the previous test in Mono and extrapolating the values it would end up running in 10 seconds.

This is a known problem in Mono's floating point support on x86 where we are still using the x87 floating point stack for floating point operations instead of the new XMM-based operations.

Mono's VM is already under an acceptable license for Mozilla (LGPL), it can be shrunk down to 5 megs (uncompressed) and it runs the DLR, IronPython and IronRuby out of the box and supports more platforms than Tamarin does at this point.

Miguel.

Posted by Miguel de Icaza on August 06, 2007 at 09:54 AM PDT #

But Mozilla doesn't have the benefit of a $108M patent deal like the one Novell made with Microsoft.

Donald

Posted by Donald on August 06, 2007 at 12:15 PM PDT #

Troll-nald,

The pieces of Mono that are required for Mozilla are merely a subset of the ECMA subset, read the Mono FAQ for more details.

Miguel.

Posted by Miguel de Icaza on August 06, 2007 at 02:22 PM PDT #

Brendan,

>Chris: your question answers itself. Java in the browser is effectively dead, killed by a combination of poor single-vendor stewardship

I hate to be rude, but the stupidity of your comment, esp considering your line of work just boggles the mind.

Have you forgetten that javascript for all intents and purposes was dead until google rolled out google maps?

Sun has some issues to overcome, but they are certainly not impossible to achieve.

It's obvious from your posts that you're in bed w/ the Adobe crowd via the flex is free as in beer comment. The cost of developing in flex is the cost of developing in .net. and i don't think anyone would ever say .net development is 'free as in beer.'

Posted by thefacts on August 07, 2007 at 01:17 AM PDT #

thefacts (how about identifying yourself, so we can assess which bed you're in?): you protest too much. You do not hate to be rude, you obviously prefer it to making a rational argument.

The situation with Java on the client side of the Web speaks for itself. Flash is dominant, faster to start, faster to render, while Java is in decline and still slow to load and render. There could be a Java come-back some year (who knows?), but you're misleading yourself if you think the question is merely one of Sun -- by itself -- overcoming "some issues".

Name-calling and ranting aside, you make a claim that deserves a response.

My comment about Flex used standard language ("free as in beer") meaning free-to-download, nothing more. If you can calm down for a minute and read for context, you'll see my words about Flex being free to download were in reply to Chris's question about how to run the ASC compiler to benchmark Tamarin in a more apples-to-apples fashion. I was not touting Flex as some better brand X (than what? Here, I'll recommend Nicolas Cannasse's MTASC compiler as well -- am I in bed with him too? Yeesh).

I hold no brief for Flex in particular, compared to LZX from Laszlo or XAML from MS. I'm glad Laszlo is already open source, that Java is finally open source, and that Flex is being open-sourced. I've blogged about how everyone is putting on faux "open" clothes already, and how the real test of being "open" depends on community goals trumping any single business's agenda. By that measure, neither Laszlo nor Java nor the DLR/IronPython/IronRuby is truly open yet.

Mozilla's platform is built on Web standards and XUL, not on Flex or the Flash Player, so it's pretty silly (I wouldn't say stupid) to claim we're in the Flex bed with Adobe. We are pushing XUL into the Web standards along with others (e.g. Safari, which already implements XUL box layout via CSS extensions). Adobe is hosting the Flex code on their own, and good for them, but it has nothing to do with our platform.

Our common interest in ECMAScript had resulted in shared work on Tamarin. That's the kind of free as in speech and free as an beer software that moves the Web forward and consolidates platforms (Flash and browser in this case, and yes: it's only a first step, but a big one) instead of introducing new and incompatible competing platforms.

Contrast with Silverlight, a Flash-killer encrusted with patent hazards and MS FUD, where there's no path (sorry, Miguel -- 5MB is way too big, even if we ignore the lack of a general license instead of a Novell-MS bilateral deal) for browsers to embed the needed VM code by default.

/be

Posted by Brendan Eich on August 07, 2007 at 03:28 AM PDT #

Brendan,

I was not suggesting that you use Silverlight as your VM. Why would I suggest something like that? That is a proprietary VM with a proprietary interface and is barely available anywhere.

I said \*Mono\* not Silverlight.

For the size, 4 megs is what you get just by trimming files without touching anything. You can further reduce the size using a linker (http://www.mono-project.com/Linker) or removing pieces from the runtime that are not necessary.

The DLR would increase your download, but it would do it with or without Mono.

Miguel.

Posted by Miguel de Icaza on August 07, 2007 at 07:33 AM PDT #

It's worth nothing that with the Rhino benchmark, you've turned optimisation right up to level 9, so I take it the reported time would include both the parsing of the script, the generating and loading of the classfiles, and then finally bytecode execution by HotSpot. This is very significant since level 9 performs the most complex optimization, so the classfile generation stage would be fairly slow.

However, when measuring Tamarin speed, it would appear you pre-generated Tamrin bytecode, and measured only bytecode execution time. So the Rhino test did a lot more work during the measured time than the Tamarin test.

Posted by Ben Loud on August 07, 2007 at 02:59 PM PDT #

be, i obviously must have struck a nerve to elicit such a ranting response.

it's not even worth a response - you defend yourself like you got caught cheating on your boyfriend...that speaks for itself

Posted by thefacts on August 08, 2007 at 03:54 AM PDT #

Miguel: where did I say you suggested Silverlight rather than Mono? My parenthetical aside was about Mono, or anything like it including the CLR that's in Silverlight 1.1.

You still are not responding to my reasons for us not using Mono. From your blog, it's also clear you are treating our VM exercise as a "let's support multiple languages" project, to some level of Mono-like completeness and performance. That would be a suicidal detour from fielding a competitive browser, not only due to footprint and IP problems, but also due to JS performance and compatibility regressions. Browsers have to do JS first, and \*fast\* -- they have to load and evaluate top level script, compiling functions, without losing pageload benchmark wars with competitors. Sorry, "Rhyno" won't cut it.

And beside this point, there are still showstoppers for footprint: 4MB or 5MB, doesn't matter -- still too big; and patents: you definitely didn't address this, and I think anyone working for Novell will have a hard time, unless there's a general open-source patent license to all of the IP that MS might claim that Mono contains.

I had a comment pending at your blog, I'll follow up there if it made it past moderation.

thefacts: nice echoing of my words ("ranting", "speaks for itself"). I'm happy to let readers judge who ranted. At this point you re-hash from the anonymous coward bucket into the troll bucket, so I'll not feed you further -- but feel free to prove me wrong by de-cloaking and responding to the reasons I've given here.

/be

Posted by Brendan Eich on August 08, 2007 at 06:42 AM PDT #

What a fun thread! Hopefully the Mono and Java hacker's will take Tamarin's existance as affrontery and go on a serious diet. Tamarin is <200kb and getting smaller. Viva competition! The world would be a better place with more clean small VM's with small mandatory libraries. More developers/apps would use managed/virtual systems and software would be better. Java and C# are missing too many boats b/c they'd sink them if they got on. Its Tamarin's raison d'etre.

Posted by Tommy Reilly on August 08, 2007 at 09:33 AM PDT #

What a funny discussion.

Brendan always mixes (on purpose?) the JDK ("download size", "startup"), applet ("dead on the client", "Flex", "Flash") and Java VM topics, when this post is only about the VM.

What a textbook example of

http://en.wikipedia.org/wiki/Straw_man

Could we please focus on VM technologies?

Peace
-stephan

--
Stephan Schmidt :: stephan@reposita.org
Reposita Open Source - Monitor your software development
http://www.reposita.org
Blog at http://stephan.reposita.org - No signal. No noise.

Posted by Stephan Schmidt on August 08, 2007 at 05:41 PM PDT #

"...when this post is only about the VM."

Stephen,

While Chris' post is about the performance about VM performance, Brenden is trying to emphasize this is an apples-to-oranges comparison because Tamarin is confined to such a small foot-print while targeting a dynamically typed language.

If you think that point doesn't need to be emphasized because the majority of readers are too perceptive not to that type of critical nuanced point, then you're not as perceptive as you think.

Lastly, Brenden is just trying to explain the rational behind his decision. He's not muddying the waters with straw-man attacks as you say. (BTW, You didn't bother to actually city Brenden in your own straw-man attack).

Maybe you should take your own advice and focus on the VM technologies rather than further inflame a sensitive topic. I'd certainly like to hear some informed opinions on the subject of performance of small-footprint VMs that target fast-start up times for dynamic language.

I'm serious. I'd rather not read all this other bullshit.

Steve

Posted by Stephen Goguen on August 09, 2007 at 01:00 AM PDT #

Hey Stephan,

Careful with that straw man charge (with a wikipedia link, wow). When you see several arguments in a row, it's a bit of a diversion on your part to level that charge instead of dealing with each argument in turn:

[Preamble] I pointed out that Java is all but dead in the browser to argue that we have no _a priori_ cause to build anything like the JRE into Firefox.

[Size] Even if we ripped out applets and LiveConnect (some argue for this, given spoofing opportunities they create: http://hackademix.net/2007/08/07/java-evil-popups/ -- but we don't want to break a few popular game sites), we can't afford the JRE. The JRE requires megabytes (7.1 according to http://www.java.com/en/download/manual.jsp) of uncompressed code size.

[Compatibility] Ok, next argument: don't use the JRE, use something smaller. There are small JVMs out there, some even open source now. But, and we've been over this too, the JVM is for Java first and always. Jython is dead (MS hired Hugunin), and Rhino is not ready to replace browser interpreters.

[Focus] On that last point: who would use a browser that supported Java or Jython but had sub-standard JS support? The point I've hammered on repeatedly here seems lost: Python and other languages are gravy, and a hope for the future. For now, browsers must interpret source JavaScript, with both web compatibility and only ever increasing performance. This does not come "for free" with a JVM. It's not clear that Firefox could even get there from here in any reasonable number of years.

[Timing] And anyway, the door closed for us last year. Mozilla is not going to reset its plans and throw away the work we've already done with Tamarin, and take another suicidal detour into what would likely be a zero-sum-game of re-engineering and shrinking of a JVM to meet browser requirements including "JS first and fast."

[Bird in the hand] Tamarin is already small; it already does most of "JS first and fast".

[License] Mozilla can't contain GPL-only code, we require MPL/GPL/LGPL or a non-copyleft license such as BSD.

I hope this bulleted-list form helps. Nothing is straw here. Please pick apart the concrete reasoning if you can, don't just try to score "gotcha" points with wikipedia fallacy links. Doing that just dodges the substantive arguments, as well as ascribes bad intent.

It's not my intent to waste time putting up straw men. If I'm wrong in anything I wrote here, show me the error and I'll thank you for the correction.

/be

Posted by Brendan Eich on August 09, 2007 at 04:26 AM PDT #

Putting aside all the (slightly colorful) history for a moment, some comments:

\* I find that for my web applications, Javascript code in Firefox is 2-3 times slower (or worse) than the same code running in IE. We all wish this were better (at least so I would hope).

\* It seems there is quite a significant amount of work yet to do before Tamarin can be incorporated in the browser. At least that is my impression.

\* For somewhat obvious reasons (not immediately relevant here), the regular Java JVM is a bit "fat" for inclusion in the browser. Most folk would like to not bulk up the browser more than necessary.

\* Java in the browser is not the immediate issue. The issue is speed and stability for the increasing amount of Javascript code running in the browser.

I believe the above boils down to a small set of questions.

\* How does a "small" JVM (Java ME?) compare in footprint to Tamarin?

\* How does Javascript (Rhino) on the "small" JVM compare in terms of footprint and performance?

\* Is there a JVM with an appropriate license model for Mozilla? Given Sun's ongoing (if extended) efforts to open up Java, are there folk at Sun willing to insure a suitable license (if not presently suitable)?

\* How does the needed work compare for incorporating Rhino versus Tamarin?

Naturally, as a developer I am most interested in performance and stability. We might expect better stability out of a JVM, given longer and wider use than Tamarin. A performance comparison between Tamarin and Rhino (on a suitable JVM) would be interesting.

The question here is simply how the two VMs compare.

Posted by Preston L. Bannister on August 09, 2007 at 07:37 AM PDT #

Brendan,

Our patent policy is well articulated here:

http://www.mono-project.com/FAQ:_Licensing

It is not any different than any other open source project (or any other software project for that matter): if we are notified of a patent violation we will swiftly remove the infringing code or try to find prior art.

In particular, the core of Mono was standardized by ECMA and is licensed under RAND+Z terms (you can obtain written statements from Microsoft (RAND-Z) and ECMA (RAND as thats their only requirement) to this effect).

Regarding my blog: you can just sign up to the group (pick no delivery) and all posts get posted without any moderation involved. Its a standard Google group but I posted and approved it a few days ago.

As for the size, it is a valid point. 4-5 megs of disk space is what you get by just picking libmono.so, mscorlib.dll plus the DLR and Iron\*. It will never get down to 200k, but if you are planning on shipping Iron\* and JS that also adds to the baseline 200k JIT.

Reducing Mono's libmono.so and mscorlib.dll is a matter of removing unneeded features and running the tuner and picking out the pieces that are not necessary for a small deployment. Lots of people have done it and this is why Mono ships on tiny embedded devices.

But hey, if you have a happy working relationship with Adobe to get Tamarin fixed and integrated, more power to both of you. Obviously the features that Mono offers are not worth the switch.

Miguel.

Posted by Miguel de Icaza on August 09, 2007 at 07:48 AM PDT #

Miguel: one thing I still need to clarify. You wrote "if you are planning on shipping Iron\* and JS that also adds to the baseline 200k JIT".

We must include JS, much of which is self-hosted, even now in Flash 9. But again, the inclusion of Python and Ruby support via IronMonkey is for the future. One possibility is download on demand. This is winning for a number of reasons: smaller default Firefox, memory-safe add-on code, separation of concerns and better APIs reflecting that separation.

Until web content expects Python and Ruby script tag support, you won't find browsers taking on much footprint or other risk to add these. MS may, for all I know, in IE8, but who knows? The play in Silverlight to support multiple languages (all except C# equally slow? :-P) may be a differentiator against the Web.

I'm looking forward to multiple language support in web browsers, but it's not the primary goal, and we cannot sacrifice required footprint, compatibility, performance, or focus for it in the near term.

/be

Posted by Brendan Eich on August 09, 2007 at 08:02 AM PDT #

Preston: are your benchmarks measuring raw JS, or JS+DOM, or DOM performance. Mozilla is not fastest in all cases, but current Ajax workloads mainly stress the DOM.

We have to be careful here, for two reasons:

1. The DOM hides costs of dynamic binding in JS (argument and result marshalling, name lookup, even memory management) that can be improved through "better JS".

2. The lead users have moved beyond known Ajax workloads and are pushing JS to be as fast as possible for things like db apps, 3D graphics, even crypto and light DSP chores.

So we are working to meet future as well as latent present needs.

For some benchmarks that include Rhino and Tamarin, please see

http://ejohn.org/apps/js-speed/results/

You can click on the charts to make them bigger.

/be

Posted by Brendan Eich on August 09, 2007 at 08:06 AM PDT #

Brendan,

Apples to apples comparison should be restricted to the VM itself. You've mentioned repeatedly the JRE in your size arguments. Comparing Tamarin with no runtime libraries to the JRE isn't a valid comparison in my mind. Java VM's alone are relatively (hotspot) or very (monty) small. Even with the package declaration Tamarin was ~4-5 times slower than Hotspot for this case, which is a lot. I think that is a valid Apples to Apples comparison. That's only one simple benchmark, however, I'd speculate that the overall benefit of Hotspot is actually more that 5x on the average at the present time. Nevertheless, compared to interpreters, Tamarin does deliver high performance. Now the mere fact that Hotspot delivers higher performance doesn't mean it's suitable for Mozilla to use at the current time (for whatever reasons), and I shouldn't have made that comment.

Nobody's mentioned it in this thread, but from a purely technical perspective Silverlight delivers very impressive VM and graphics performance (in many cases far superior to Flash) in a relatively small footprint. I applaud Microsoft's technical efforts in this area, as it helps keep the bar high for everybody, in terms of striving to deliver a high quality user experience for the end user. I can honestly say the same thing about Macromedia/Adobe and Flash as it clearly has set the bar in many respects that a Java, Silverlight, or AJAX-based web content delivery platform will have to meet in terms of overall user experience.

Posted by Chris Oliver on August 09, 2007 at 11:45 AM PDT #

Why not use Lua as the VM? It is a very fast and light VM.

Posted by LuaQuestion on August 10, 2007 at 06:25 AM PDT #

Hi folks,

Nice to see heating discussions here. I've been experimenting with Tamarin when building Flash9 support for haXe (http://haxe.org). Its speed is not astonishing but is good enough for what's its meant for. I don't think it's been built to outperform Java, although the bytecode design is pretty similar to JVM.

I agree with Brendan that Java-in-the-browser is dead. My company is making web based games and the only choice we have so far is Flash. But I don't think that Tamarin-in-the-browser (meaning : in FF3) will also reach significant adoption. The main reason being that MS will not be likely to include it in IE, especially since now they have SilverLight.

In the end, we "simple users" can see one more time that corporate wars (Adobe VS Microsoft VS Sun) ends up with different technologies, maybe without anyone being able to get a significant share at least in the next 2 years.

That's why it would have maybe been more wise for Mozilla to adopt a "corporate-neutral" technology (open source of course) that would have been a bit more war-proof :)

My two cents...

Posted by Nicolas Cannasse on August 10, 2007 at 07:06 AM PDT #

I'm confused - Tamarin's benchmarks (excluding the very slow regexp tests) are an average only 20% faster than classic Spidermonkey:
http://ejohn.org/apps/js-speed/results/

I thought Tamarin was supposed to be 10x faster than Spidermonkey?

Posted by Confused on August 10, 2007 at 07:10 AM PDT #

NekoVM is incredibly fast and small and has support for dynamic languages. Any thoughts regarding using it instead of Tamarin to run Javascript?

Posted by Confused on August 10, 2007 at 07:15 AM PDT #

[Trackback] Clientside VM: I missed this conversation from the end of July, but it has picked up many interesting comments about in-browser virtual machines since then. Chris Oliver of Sun, who had the "F3" project which was later rebranded "JavaFX", ran some benc...

Posted by JD on EP on August 10, 2007 at 07:20 AM PDT #

[Confused: you posted the same text at ajaxian.com. Here's a copy with one grammatical fix of my reply there. /be]

Confused: see

http://www.playercore.com/pub/Tamarin/Avmplus_vs._javascript.htm

Tamarin was designed to do better with type annotations than without. This is why SpiderMonkey actually wins some of those benchmarks against Tamarin on untyped code. Our work with SpiderMonkey and Tamarin now in the ActionMonkey project aims to make it fast for untyped code as well.

Also, don't be alarmed by different benchmark results. See also

http://strongtalk.org/benchmarking.html

for some healthy attitude about the perils of micro-benchmarks.

/be

Posted by Brendan Eich on August 10, 2007 at 10:03 AM PDT #

Hey Nicolas,

You didn't say how "corporate neutral" VM tech would help solve a "how to get distribution to 90+% of desktops, with browser integration as a <script> engine" problem.

I have a project that might work, but it's built on Tamarin: ScreamingMonkey. Maybe you missed its announcement.

http://weblogs.mozillazine.org/roadmap/archives/2007/07/new_projects.html

BTW, love your work. We sometimes miss OCaml in our http://ecmascript.org/ reference implementation work (the main language is SML-NJ as you probably know). But that's also a done deal, for good reasons sorted out at http://lambda-the-ultimate.org/.

LuaQuestion: competitive browsers have to do JS first and fast, not Lua. See above. Sure, with enough work you could argue that \*any\* well-designed VM could be pressed into service. We're looking to do less work, and have multiple go-to-market distribution points, by joining forces on Tamarin.

/be

Posted by Brendan Eich on August 10, 2007 at 10:09 AM PDT #

Spidermonkey is consistantly last place in memory usage in the Computer Language Shootout.

http://shootout.alioth.debian.org/gp4/benchmark.php?test=partialsums&lang=all&sort=kb

It often uses 100x more memory than an equivalent Lua program. Is there something in the Javascript language that makes it so memory inefficient?

The benchmark above does not even allocate memory. It’s just creating a bunch of temporaries in a tight loop. Why isn’t that memory being collected more frequently? And will Tamarin do any better?

(sorry for same comment on Ajaxian - this forum seems more active)

Posted by CrossPoster on August 11, 2007 at 12:21 AM PDT #

Regarding Tamarin being faster with type annotations - Lua is untyped, yet when its code is interpreted it is many times faster than javascript. I wonder if its poorer performance is due to a garbage collection issue?

BTW, in all these Tamarin benchmarks above - is it using a JIT or is it interpreted?

Posted by CrossPoster on August 11, 2007 at 12:35 AM PDT #

CrossPoster: the SpiderMonkey tests there are using the "js shell", a tests-only REPL that does not schedule garbage collection at all! Do not consider those a fair comparison.

I'll try to get that fixed, but it's not a priority. In the mean time, no one should jump to unjustified conclusions from the shootout's results. They can show worst cases or upper bounds on performance. They don't have much to do with real workload and browser embeddings such as SpiderMonkey's, where GC is actually scheduled somewhat better (still not as well as I would like, but Firefox manages to compete for now).

I'll blog with more solid performance data when I have some useful results to report. Anyone who likes can follow the Mozilla bugsystem and John Resig's benchmark results page (which we'll keep updated).

/be

Posted by Brendan Eich on August 11, 2007 at 05:47 AM PDT #

CrossPoster: Tamarin JITs all methods that might be called more than once.

So you're seeing JITted performance for benchmarks that put the critical path code into a function, but: as noted early in this blog, without a package {} around the function, and type annotations, the JIT currently de-optimizes significantly. The empty package wrapping enables early binding, and the type annotations avoid "boxing" of runtime-type-tagged values in and out of the function. There may be other effects; perhaps Ed will comment again.

Hence the significant speedup for the "Typed" versions of the tests. We will make "Untyped" code fast too, for all the "Web JS" out there.

/be

Posted by Brendan Eich on August 11, 2007 at 05:51 AM PDT #

[Chris Oliver: I just tried it, and it seems that specifying the return type has no impact on performance in this case.]

This was the same for me until I recompiled avmplus in non-debug mode. Now on my T60p the test runs in 10 seconds. That is still 10x slower than the Java test but as Brendan has pointed out, Tamarin is relatively young, very small, and ubiquitously supported via Flash Player 9's 90+ percent adoption.

For another interesting benchmark, check out bubblemark.com
If you install the new FP9 beta and run the Flex tests you will see significant performance benefits, especially on multi-core/cpu machines. I get 170fps on the default Flex bubblemark test.

-James (Adobe)

Posted by James Ward on August 13, 2007 at 07:07 PM PDT #

Brendan has made Mozilla's reasons very clear their choice. I'm not trying to argue with that. Besides they already committed on their choice.

I think the arguments for using Java are very compelling with respect to speed, the possibility of other scripting languages, and maturity. If Sun wants Java on the browser why not donate the JVM in the same way Adobe donated Tamarin. Or possibly donate the human resources to perform the integration. Has there been any discussion about trying to do the same with the JVM?

Posted by Charlie Hubbard on August 14, 2007 at 02:20 AM PDT #

Brandon,
Got similar results (2-3x longer times in Firefox versus IE) for two rather different applications.

One is near pure JS computation (http://bannister.us/examples/sudoku/), an implementation of a simple Sodoku solver (why - http://bannister.us/weblog/2007/03/09/sudoku/).

The second (and third) are for heavy JS+DOM code that take large JSON structures and generated heavily marked up HTML (via DOM operations mainly - no public example).

Both went through several rounds of measurement and optimization (favoring Firefox over IE generally). No pretense that either of these are comprehensive as benchmarks. Both measure about the same in terms of relative elapsed time between Firefox and IE.

I think my original questions still remain - mainly how does Rhino benchmark on a small footprint JVM, and what is the size of a small footprint JVM. The micro-benchmark page you referred to is interesting, if perhaps bit too micro.

Posted by Preston L. Bannister on August 16, 2007 at 10:28 AM PDT #

Icaza is trolling in java land.. well well..

Posted by troll on August 16, 2007 at 03:15 PM PDT #

Preston: JScript is generally faster when it comes to numbers and math.

I did some tests last year ago (tests were sadly lost in a harddrivecrash), and generally repetitive but simple math took about four times longer in SpiderMonkey compared to JScript, with both JavaScriptCore and linear_b taking about two times longer than JScript. (The speed difference was consistent between ie5m/ff1.5m and ie6w/ff1.5w)

So it's not surprising to me that a sudoku solver would turn out to be faster on iew.

As for DOM+JSON+JS, in my experience it's always iew that needs the optimisation, so I'd find it interesting if you could profile it and find out which parts were slower in which browser. I know a few areas where moz is generally slower, but those are mostly avoidable...

Posted by liorean on August 30, 2007 at 12:59 AM PDT #

Charlie, I would rather see a JavaFX VM donated, rather than a Java VM, since JavaFX Script is, like ECMAScript (Action-, Java-, J-, etc.), an interpreted scripting language, while Java is a full-fledged OO, compiling programming language. Why execute an entire Java applet in the browser, when you can load a JavaFX applet instead (without the excessive loadtime that is commonly asscociated with Java applets, something that is actively denied by Java proponents with such snide remarks as "sorry, you should get new, post-1999 hardware")?

Posted by Rayne Van-Dunem on September 02, 2007 at 07:44 AM PDT #

Preston, I took at look at the Sudoku solver you point to in Gecko.

About 40% of the total time is spent in security checks. These are known to take too much time, and we're working on making them faster and doing them less often in situations like this where everything is same-origin to start with.

In fact, 50% of the time is spent computing the global "this" object (this time includes the security checks above), which is something Brendan is working on eliminating entirely for Gecko 1.9.

That would put us at close to parity with IE if I understand your numbers right.

Of course the goal is to do better, not just do parity. ;)

Posted by Boris Zbarsky on October 23, 2007 at 09:17 AM PDT #

Nice! i Like it ; )

Posted by vB-Hacker.com daki seo yarışması on November 04, 2007 at 07:44 PM PST #

thank you Chris

Posted by batman on September 05, 2008 at 11:47 AM PDT #

http://www.smsmatbaa.com

Posted by matbaa on June 22, 2009 at 03:05 AM PDT #

Charlie, I would rather see a JavaFX VM donated, rather than a Java VM, since JavaFX Script is, like ECMAScript (Action-, Java-, J-, etc.), an interpreted scripting language, while Java is a full-fledged OO, compiling programming language. Why execute an entire Java applet in the browser, when you can load a JavaFX applet instead (without the excessive loadtime that is commonly asscociated with Java applets, something that is actively denied by Java proponents with such snide remarks as "sorry, you should get new, post-1999 hardware")? thanks very much.

Posted by Batterie Dell E6400 on November 15, 2010 at 09:19 PM PST #

important and awake! great job bro thanks.

Posted by Egitim on December 11, 2010 at 05:49 AM PST #

Simple and Nice example !

Posted by شات on December 15, 2010 at 03:44 AM PST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

user12610627

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today