Although the intent of the previous article, entitled Comparing JVMs on ARM/Linux, was to introduce and highlight the availability of the HotSpot server compiler (referred to as c2) for Java SE-Embedded ARM v7, it seems, based on feedback, that everyone was more interested in the OpenJDK comparisons to Java SE-E. But there were two main concerns:
With those considerations in mind, we'll institute the following changes to this version of the benchmarking:
Certain OpenJDK implementations were eliminated from this round of testing for the simple reason that their performance was not competitive. The Java SE 7u2 c2 compiler was also removed because although quite respectable, it did not perform as well as the c1 compilers. Recall that c2 works optimally in long-lived situations. Many of these benchmarks completed in a relatively short period of time. To get a feel for where c2 shines, take a look at the first chart in this blog.
The first chart that follows includes performance of all benchmark runs on all platforms. Later on we'll look more at individual tests. In all runs, smaller means faster. The DaCapo aficionado may notice that only 10 of the 14 DaCapo tests for this version were executed. The reason for this is that these 10 tests represent the only ones successfully completed by all 4 JVMs. Only the Java SE-E 6u30 could successfully run all of the tests. Both OpenJDK instances not only failed to complete certain tests, but also experienced VM aborts too.
One of the first observations that can be made between Java SE-E 6 and 7 is that, for all intents and purposes, they are on par with regards to performance. While it is a fact that successive Java SE releases add additional optimizations, it is also true that Java SE 7 introduces additional complexity to the Java platform thus balancing out any potential performance gains at this point. We are still early into Java SE 7. We would expect further performance enhancements for Java SE-E 7 in future updates.
In comparing Java SE-E to OpenJDK performance, among both OpenJDK VMs, Cacao results are respectable in 4 of the 10 tests. The charts that follow show the individual results of those four tests. Both Java SE-E versions do win every test and outperform Cacao in the range of 9% to 55%.
For the remaining 6 tests, Java SE-E significantly outperforms Cacao in the range of 114% to 311%
So it looks like OpenJDK results are mixed for this round of benchmarks. In some cases, performance looks to have improved. But in a majority of instances, OpenJDK still lags behind Java SE-Embedded considerably.
Time to put on my asbestos suit. Let the flames begin...