Java SE Embedded Performance Versus Android 2.2
By bobvandette on Nov 22, 2010
Having recently released our latest version of Java SE Embedded 1.6.0_21 on the web, I thought I'd take a breather and have a little fun and do something completely different. You most likely have seen the barrage of TV commercials advertising the droid phones which run the Android Operating System from Google. Android is a Java-like platform that runs a Java Virtual Machine called Dalvik. Until recently, this Virtual Machine (VM) ran interpreted only which means that its Java execution speed was substantially slower than most Java runtimes available on the web. Google's latest version of Android (version 2.2 the Froyo Release), now contains a Just-In-Time compiler (or JIT for short). As a result, the speed at which Android 2.2 can now execute Java bytecodes has increase. There are claims on the web that Android 2.2 can now process Java bytecodes from 3 to 6 times faster than the previous release. Now that Android has at least gotten in the ballpark of other mature VMs, I thought it would be fun to see how well it stacks up against our own Java SE Embedded Hotspot Virtual Machine.
In order to have an apples-to-apples comparison, I had to overcome a few obstacles. First, Dalvik doesn't directly execute Java class files, it processes dex files. During the process of creating Android applications, Java class files are converted to an Android specific dex format with a "dx" utility provided in the Android SDK. It's these "dex" files that Android runs. Second, Android and Java SE don't share the same Graphical Interface APIs so I can't run any GUI based benchmarks. Finally, Java SE doesn't run on Android's version of Linux since it doesn't run the standard glibc "C" library.
In order to overcome these obstacles, I configured my test hardware platforms to run both Android and a standard Linux distribution. I selected some benchmarks that didn't require GUI APIS and I converted each benchmark to "dex" format for the runs on Android. I was then able to run the Java class file version of the benchmarks on standard Linux and the "dex" version of the benchmark was run on Android. I booted up Android, ran the benchmarks, then rebooted the same system and then ran Java SE Embedded.
I performed these benchmark runs on two different ARM based devices - a Beagleboard and an Nvidia Tegra2 device. The Beagleboard is a single core ARM device and the Tegra2 is a dual-core ARM Cortex-A9 based device.
Here are some notes on how I ran these benchmarks on Android 2.2:
The dex conversion command I used was:
% dx --dex --output=benchmark.dex benchmark.jar
% mv benchmark.jar benchmark.jar.save
% mv benchmark.dex benchmark.jar
To copy the converted benchmark to Android, I used their adb tool:
% adb push benchmark.jar /sdcard/benchmark.jar
To run the benchmark on Android, I used:
% adb shell dalvikvm -cp /sdcard/benchmark.jar Mainclassname
Note: substitute the real benchmark name for benchmark.jar and the main application name for Mainclassname.
DEVICES AND SOFTWARE DISTRIBUTIONS
Here are the specifications and links to the boards and operating system platforms that I installed and ran on these two devices.
BEAGLEBOARD (Rev C3) Specifications
Texas Instruments OMAP3530 ARM Cortex-A8™ Processor 600Mhz, 256MB RAM
Hardware specification URL:
Android 2.2 distribution for BeagleBoard was downloaded from here:
The Angstrom Linux Distribution for BeagleBoard was downloaded from here:
The main Angstrom beagle board project page is here:
NVIDIA TEGRA-2 250 Developer Kit Specifications
Dual-core ARM® Cortex-A9 MPCore™ processor, 1.0 GHz, 1024MB RAM
The Android 2.2 and Linux Distributions for Tegra-2 can be downloaded from the Nvidia developer site at :
Here are the benchmarks that I ran on these two devices. I had to select benchmarks that did not require any Java graphical APIs and could run on the APIs that are implemented in Android.
PENDRAGON CAFFEINEMARK (tm) ver 3.0 * (http://www.benchmarkhq.ru/cm30)
Description: The CaffeineMark is a series of tests that measure the speed of Java programs running in various hardware and software configurations.
* The CaffeineMark benchmark was run without independent verification by Pendragon Software. Pendragon Software makes no representations or warranties as to the result of the test. CaffeineMark is a trademark of Pendragon Software.
SCIMARK 2.0 (http://math.nist.gov/scimark2)
Description: SciMark 2.0 is a Java benchmark for scientific and numerical computing. It measures several computational kernels and reports a composite score in approximate Mflops (Millions of floating point operations per second).
KBENCH (Sun/Oracle Internal Benchmark)
Description: kBench is a collection of small Java programs used to determine the relative performance of Java Virtual Machines.
BEAGLEBOARD BENCHMARK RESULTS
TEGRA2 BENCHMARK RESULTS