Monday Nov 16, 2015

Installing Trusted Certificates into a Java Keystore

As software environments continue to ratchet up security measures, the odds of having to deal with digital certificates in more than a superficial manner only increases over time.  Furthermore, platforms are not only mandating the use of certificates, they are to a greater extent shunning the self-signed variety and instead insisting upon certs that originate from a trusted authority.  When managing certificates in the Java world, the utility you're most likely to encounter is keytool, an integral part of the Java Development Kit.  Although keytool can be effectively used to generate self-signed private/public key pairs, it's a little lacking when it comes to incorporating in certs generated by trusted authorities.  [Based upon feedback and recommendations received after the original posting of this article, a Postscript section has been appended which discusses alternative solutions to the original discussed below]

In particular, we'll focus on overcoming the following two shortcomings:

  1. An existing private key and certificate generated by a trusted Certificate Authority (CA) cannot be imported by keytool, at least not in the format traditionally provided by CAs.
  2. Not only must the unique private key be imported into the keystore, in some instances the root CA certificate and any intermediate certificates (referred to as a certificate chain) must be included, and more importantly in the correct order.  The keytool utility doesn't help much in the way of ensuring a valid order.

The following example uses a real SSL certificate chain from a real certificate authority and was done in the context of the Java Advanced Management Console 2.1 application.  Java AMC is a Java EE application and requires Oracle's WebLogic application server to function.  In this instance we'll be updating a keystore associated with WebLogic, but in reality this Java keystore should be no different from any other Java keystore, so these steps should apply elsewhere just fine.

We'll leave the minutiae of applying for a trusted certificate as an exercise for the reader; one of the first steps towards getting a certificate involves creating and submitting a CSR (Certificate Signing Request) to a Certificate Authority.  A byproduct of this process is that a private key file is generated based on the information in the CSR.  This file, when opened, looks something like this:

encryptedgobbledygook line 1
encryptedgobbledygook line n

This private key file is in PEM (Privacy Enhanced Mail) format, and will take a .pem suffix.  For our example, the file is stored as:

  • private-key.pem

For this article, we used Comodo (one of many alternatives) as our Certificate Authority.  When the entire process with Comodo was complete, the following documents were ultimately received from them:

  • amc-server_jtconnors_com.crt - This is the Comodo-generated host-specific SSL certificate for a machine with a Fully Qualified Domain Name (FQDN) of
  • AddTrustExternalCARoot.crt - Comodo Root Certificate
  • COMODORSAAddTrustCA.crt - Comodo Intermediate Certificate 1
  • COMODORSADomainValidationSecureServerCA.crt - Comodo Intermediate Certificate 2

With these files in place we can now begin the importing process.  We will take advantage of enhancements added to keytool with the Java 6 release: namely that keytool can merge and import keystores that are in PKCS12 format.

With this new information what remains is to figure out how to convert our private key and certificate chain into a PKCS12 file.  For this functionality we resort to the capabilities found in the ubiquitous OpenSSL toolkit, available on virtually all popular compute platforms.  In the example that follows, we'll be running on a Windows system and will utilize the Cygwin environment to run the required OpenSSL commands.

Step 1. (From Cygwin) Concatenate the certificates comprising the CA-supplied root certificate chain to one file.  Include only the root certificate and intermediate certificate(s) and exclude the host-specific SSL certificate.

$ cat AddTrustExternalCARoot.crt COMODORSAAddTrustCA.crt COMODORSADomainValidationSecureServerCA.crt > BUNDLE.crt

Step 2. (From Cygwin) Create a PKCS12 keystore.  This command incorporates both the certificate chain along with the SSL private key and certificates.  Your passwords may vary.

$ openssl pkcs12 -export -chain -in amc-server_jtconnors_com.crt -inkey private-key.pem -out keystore.p12 -name amc-server -CAfile BUNDLE.crt
Enter Export Password:
Verifying - Enter Export Password:

Step 3. (From Windows CMD) Using keytool, import the PKCS12 keystore into the resulting JKS keystore called keystore.jks. Again, you may select different passwords.

> "c:\Program Files\Java\jdk1.8.0_66\bin\keytool.exe" -importkeystore -destkeystore keystore.jks -srckeystore keystore.p12 -alias amc-server
Enter destination keystore password:
Re-enter new password:
Enter source keystore password:

Step 4. With the keystore successfully created we can now take the optional step of further verifying it.  Certain Java-based application frameworks, like Oracle's WebLogic for example, can be finicky about the completeness and order of the certificate chain.  To help validate the keystore, we can use the ValidateCertChain program which comes bundled with WebLogic 12c and can be found in the distribution's weblogic.jar file.  Here's a sample invocation of the program on our recently created keystore.jks file:

> java -cp %MW_HOME%\wlserver\server\lib\weblogic.jar utils.ValidateCertChain -jks amc-server keystore.jks
Cert[0]:,OU=PositiveSSL,OU=Domain Control Validated
Cert[1]: CN=COMODO RSA Domain Validation Secure Server CA,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB
Cert[2]: CN=COMODO RSA Certification Authority,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB
Cert[3]: CN=AddTrust External CA Root,OU=AddTrust External TTP Network,O=AddTrust AB,C=SE
Certificate chain appears valid

Here's hoping these examples save you some time, and more importantly, a little grief.


After having received recommendations from the engineers far better versed in AMC and WebLogic, the following section was added to briefly discuss alternatives to the above mentioned OpenSSL solution.

Alternative A:  For people that have complete control of the certificate generating process, they can, as previously hinted, use keytool exclusively for this process. For the first phase of the process they can use keytool in the following general manner (arguments to keytool will vary) to generate a keypair and certificate request:

keytool -keystore keystore.jks -genkeypair -keyalg rsa
keytool -keystore keystore.jks -certreq

In this scenario, OpenSSL would not be required since the keypair is already stored in the keystore.  From here you can import the certificates following a form similar to this:

keytool -import -keystore keystore.jks -alias root -file AddTrustExternalCARoot.crt
keytool -import -keystore keystore.jks -alias intermediate1 -file COMODORSAAddTrustCA.crt
keytool -import -keystore keystore.jks -alias intermediate2 -file COMODORSADomainValidationSecureServerCA.crt
keytool -import -keystore keystore.jks -alias mykey -file amc-server_jtconnors_com.crt

In the last command, "-alias mykey" is essential and must match the key pair in the keystone.  As a shortcut, you could also concatenate all PEM-encoded certificates into a big file and then call:

keytool -import -keystore keystore.jks -alias mykey -file thebigfile

Alternative B: Along with the ValidateChain program WebLogic 12c's weblogic.jar file also includes a utility called ImportPrivateKey which can also used to import a certificate chain into a Java keystore.  This utility command is described in Creating a Keystore Using ImportPrivateKey subsection of WebLogic documention on Configuring Keystores.  The command follows a form like this:

java -cp %WL_HOME%\server\lib\weblogic.jar utils.ImportPrivateKey -keystore newkeystore -storepass **keystorepassword**  -alias amctrust -certfile certificate.pem  -keyfile privatekey.pem [-keyfilepass **privatekeypassword**]

For further edification please consult the WebLogic docs.

Tuesday Sep 08, 2015

Using Java Flight Recorder with Compact Profiles

Like it's big brother Java SE, the Java SE-Embedded 8 platform provides support for Java Flight Recorder, an invaluable profiling and event collection framework built into the Java Runtime Environment.  However, Flight Recorder is only available with the Java SE Embedded 8 Full JRE, not with the smaller Compact Profiles.  So the question becomes, is there anything that can be done to use Java Flight Recorder with Compact Profiles?

At the current time, the smaller Compact1 and Compact2 profiles cannot realistically support Java Flight Recorder without substantial changes.  Hence we'll avoid discussing their prospects for inclusion here.  What we will focus on is Compact3 as its specification includes among others, the APIs, making for a more reasonable match.  As it turns out, all that is required to enable Flight Recorder use with the Compact3 profile is to copy over a few files over from the Full JRE.  The instructions that follow should aid in creating a Flight Recorder enabled instance:

1. Download a Java SE-Embedded EJRE for your platform.  You can do so here.  For this example, we'll do this on a Linux host and choose the ARMv6/v7 Hard Float option (suitable for the venerable Raspberry Pi).

2. Extract the EJRE.

$ tar xvf ejdk-8u51-linux-armv6-vfp-hflt.tar.gz

3. Use the EJRE's script to create a full JRE:

$ ./ejdk1.8.0_51/bin/ --dest full_jre -g -k
Building JRE using Options {
    ejdk-home: /home/pi/ejdk1.8.0_51
    dest: /home/pi/full_jre
    target: linux_armv6_vfp_hflt
    vm: all
    runtime: jre
    debug: true
    keep-debug-info: true
    no-compression: false
    dry-run: false
    verbose: false
    extension: []

Target JRE Size is 59,389 KB (on disk usage may be greater).
Embedded JRE created successfully

4. Create a Compact3 JRE:

$ ./ejdk1.8.0_51/bin/ --profile compact3 --dest compact3 -g -k
Building JRE using Options {
    ejdk-home: /home/pi/ejdk1.8.0_51
    dest: /home/pi/compact3
    target: linux_armv6_vfp_hflt
    vm: client
    runtime: compact3 profile
    debug: true
    keep-debug-info: true
    no-compression: false
    dry-run: false
    verbose: false
    extension: []

Target JRE Size is 24,336 KB (on disk usage may be greater).
Embedded JRE created successfully

5.  Check the size of the compact3 JRE.  We'll use this to see how much space was needed to add support for Flight Recorder.

$ du -sk compact3
24480   compact3

6.  Create the compact3/lib/jfr/ directory

$ mkdir compact3/lib/jfr

7.  Copy the following files over from the full JRE to the compact3 instance with these commands:

$ cd full_jre/
$ cp ./lib/jfr.jar ../compact3/lib
$ cp ./lib/arm/ ../compact3/lib/arm
$ cp ./lib/arm/ ../compact3/lib/arm
$ cp ./lib/jfr/default.jfc ../compact3/lib/jfr
$ cp ./lib/jfr/profile.jfc ../compact3/lib/jfr

8. That's all that is required.  You can compare the disk usage of the compact3/ directory before and after these modifications to get an idea of the additional space required to utilize Java Flight Recorder.

$ du -sk ../compact3
24824   ../compact3

So comparing the disk usage of step (5), we see that less than 400KB is added in order to enable usage of Java Flight Recorder. To better understand how you might remotely connect to a Flight Recorder enabled instance in a secure fashion, check out this article.

Friday Jul 24, 2015

New for Java 9: jshell

A sampling of indices gauging computer programming language popularity (e.g. PYPL and TIBOE) shows that Java, after 20 years, still enjoys a huge following.  In general, depending upon whom and/or when you ask, Java usually comes in first or second place in these surveys.  Although it might be hard to imagine recent computer science graduates without Java exposure, one trend is evident: top universities are gravitating towards teaching "simpler" languages in lieu of Java for their introductory programming classes.

The standard Java platform, compared to its counterparts like Python, currently lacks a Read-Eval-Print-Loop otherwise known as a REPL.  Instead of having to construct and compile complete syntactically correct programs before feedback can be achieved, REPLs allow much more interactivity, enabling the student/programmer to enter small snippets of code and receive immediate feedback.  According to the Java Enhancement Proposal which outlines the new jshell functionality, "The number one reason schools cite for moving away from Java as a teaching language is that other languages have a "REPL" and have far lower bars to an initial "Hello, world!" program."  With the introduction of jshell in the upcoming Java 9 release, this shortcoming will be eliminated.

Falling under the auspices of an OpenJDK project called Kulla, the code representing the REPL capability has yet to be incorporated into the core JDK 9 early access release.  As of the writing of this article (late July 2015), a separate build is required to get the jshell functionality.  In the (hopefully not-too-distant) future an early access release will bundle all these features together obviating the need for extra work.  In the interim, here's a brief video demonstrating some of it's features.

As we near the "feature complete" phase for JDK 9, we look forward to better integration and incorporation of yet more new important features into this upcoming release.

Thursday Apr 09, 2015

Update to JavaFX, Sockets and Threading: Lessons Learned

Recently, a reader commented on a dated article of mine, circa 2010, entitled JavaFX, Sockets and Threading: Lessons Learned.  In it he correctly stated that, as the original content is based on the deprecated JavaFX 1.3 framework (aka JavaFX Script), he cannot test or utilize that code with the modern JavaFX API.  What follows is an update to that blog entry with code and references appropriate for the JavaFX 8 (Java SE 8 with JavaFX) platform.


For a more thorough understanding of the method behind this madness, please consult the original article.  Briefly stated, socket programming, especially in Java, often times lends itself to utilizing threads.  To facilitate using sockets, both from the "client" and "server" side, an abstract class called was and is provided and, baring a few minor changes, remains quite similar to the original version. Just as in the Original JavaFX Script framework, The JavaFX UI is still not thread-safe and its scene graph must only be accessed through the JavaFX application thread.   What has changed between old and new is the class and method name required to perform work on the JavaFX application thread.

Just like before, we've identified two methods associated with socket manipulations that need to perform work on the main thread.  These abstract method calls are incorporated into the source and are specified in the interface file:

    public interface SocketListener {
        public void onMessage(String line);
        public void onClosedStatus(boolean isClosed);

Within, you'll see references to these method calls as follows:

     * The onClosedStatus() method has to be implemented by
     * a sublclass.  If used in conjunction with JavaFX,
     * use Platform.runLater() to force this method to run
     * on the main thread.


     * The onMessage() method has to be implemented by
     * a sublclass.  If used in conjunction with JavaFX,
     * use Platform.runLater() to force this method to run
     * on the main thread.

As implied by these comments, JavaFX-specifc classes that extend the GenericSocket class and implement the SocketListener interface are required.  Correspondingly two helper classes have been created: and  These extend the GenericSocket class and implement the SocketListener interface from the perspective of a JavaFX environment.  Both classes override the onMessage() method like this:

Likewise, the onClosedStatus() method is implemented as follows:

As these two code snippets exhibit, we enclose our execution requirements within a call to the Platform.runLater() method, ensuring that  it will be executed on the JavaFX application thread.  (For the Java 8 aficionados, you may notice that these two methods could be converted to lambda expressions. That exercise is left to the reader.)  What remains now is for any referencing JavaFX class to implement the SocketListener interface.

To demonstrate the usage of FxSocketClient and FxSocketServer classes within JavaFX 8, this article provides two NetBeans projects represented by the screenshot that follows.

The user interface for these programs was created with the assistance of a terrific tool called the JavaFX Scene Builder.  You can download the source code, including the UI (represented in FXML), here:

The javafx.concurrent Package

The intent of this article was to focus solely on the Platform.runLater() method, however, it is important to note that JavaFX 2.x also provides an additional means to create background tasks that safely interact with the JavaFX application thread.  Similar in capability to the venerable java.util.concurrent  package (and in fact extending some of those classes), the javafx.concurrent package also furnishes the ability to safely control the execution and track the progress of the application code running in background tasks. For an overview of their use, check out the following article published as part of the Oracle documentation: Concurrency in Java FX.

Tuesday Feb 17, 2015

Welcome Raspberry Pi 2!

Having surpassed 4 million units shipped and clearly establishing itself as a de facto reference platform, the folks at the Raspberry Pi Foundation are not resting on their laurels.  Earlier this month (February 2015), they introduced the Raspberry Pi 2 Model B:

Compared to its predecessor, the Raspberry Pi B+, this newer model is packaged with a Broadcom BCM2836 SoC at 900MHz.  It is not only superior in performance to the original single core BCM2835 processor on core-by-core basis, it now brings the added benefit of including 4 cores.  Add to that a doubling of RAM from 512MB to 1GB and it should be no mystery that this new platform can handle more serious workloads.  Later on down, a chart compares the SPECjvm2008 benchmark performance of the original Raspberry Pi Model B with the new Raspberry Pi 2 Model B.

In addition to the substantial performance bump, major kudos are to be given to the Raspberry Pi Foundation engineers for their focus on hardware and software compatibility:

  • The Pi2 maintains the same physical form factor as the Raspberry Pi B+, meaning that all of the previous accessories (USB power cable, serial console cable, microSD card ...) all work perfectly well with the new version.  That also includes any Raspberry Pi B+ enclosures you may have too.
  • The ARMv7-based BCM2836 processor is backwards-compatible with the original ARMv6 version.  In order to maintain binary compatibility, the latest version of the Raspbian distribution will boot either the new or old versions of the Raspberry Pi.  Note that in this mode, applications will not be able to take full advantage of the ARMv7 enhancements.  Alternate OS distributions may provide the extra performance boost at the expense of breaking backwards compatibility.
  • Perhaps best of all, the Raspberry Pi maintains its appeal to the masses.  At approximately £23/$35, no price increase comes along with this upgrade.

A Brief Look at Performance

The chart that follows examines Raspberry Pi Java performance via the SPECjvm2008 benchmark.  Both original and Raspberry Pi 2 systems ran the latest available Raspbian Image (2015-02-02) and the latest available Java 7 JDK (Java 7 update 72)1.  The four vertical bars (blue, red, green, purple) represent four separate runs of the benchmark:

  1. Blue: Raspberry Pi Model B.  Invoked with JVM argument -Xmx400m specifying a maximum heap of 400MB.
  2. Red: Raspberry Pi 2, single threaded.  Invoked with JVM argument -Xmx400m specifying a maximum heap of 400MB, and application argument -bt 1 instructing the SPECjvm2008 test harness to run the benchmark tests with a single thread.  This is meant to approximate an apples-to-apples comparison of single core performance between the two Raspberry Pi systems.
  3. Green: Raspberry Pi 2, multithreaded (default).  Invoked with JVM argument -Xmx600m specifying a maximum heap of 600MB2.
  4. Purple: Raspberry Pi 2, multithreaded (default), larger heap.  Invoked with JVM argument -Xmx900m specifying a maximum heap of 900MB3.

The larger the measurement (ops/min), the faster the result, the single core performance of the Raspberry Pi 2 is a big improvement over its predecessor.  As the SPECjvm2008 suite does include a healthy dose of mutithreaded code, enabling the test harness to take advantage of the 4 hardware threads available to the Pi 2 added another huge jump in performance.  And finally utilizing the extra RAM (in the form of a larger heap) provided for by the Raspberry Pi 2 yields only a very modest gain in throughput.  In particular, 2 of the 11 component tests (compress, xml) show an improvement with the extra heap.  For more detailed results, you can view this spreadsheet.

The added horsepower that the Raspberry Pi 2 brings solidifies the Raspberry Pi family as an important reference platform in the present as well as the future for companies like Oracle.  As the Internet of Things phenomena plays out and more processing is brought to the edge, the new Raspberry Pi is better suited to handle the increased workload that applications like Oracle Event Processing for Oracle Java Embedded and others require.


1 The latest Raspbian distro includes a recent version of Oracle's Java 8 JDK.  Unfortunately the somewhat dated SPECjvm2008 benchmark has a dependency that requires Java 1.7 or earlier, so for these benchmarks, we needed to use a Java 7 JDK.  The latest available, 7u72, was used for these tests.

2 With 4 cores, the SPECjvm2008 test suite would produce an OutOfMemoryError when run with the initial max heap size argument of -Xmx400m.  Increasing the max heap size to 600MB (-Xmx600m) enabled the tests to complete.  Evidently SPECjvm2008 requires more heap space as more hardware threads are configured in.

3 In order to utilize the full amount of RAM (1GB) available with the Raspberry Pi 2, a firmware upgrade may be required.  If necessary, you can invoke 'sudo rpi-update'  from a linux shell on the Raspberry Pi 2 to accomplish this task.

Wednesday Jan 14, 2015

Managing Java Flight Recorder Enabled JVMs with SSL

One advantage Java Flight Recorder enjoys is that its event collection and profiling framework is built right into the JDK, allowing instrumented JVM instances to operate with near-zero overhead.  At arguably negligible expense, would it then make sense to consider enabling Java Flight Recorder on production applications?  We say absolutely!

You can google around to search for how this framework can be configured to monitor remote JVMs, and nearly all of the reference material will help you get up to speed quickly.  However, you'll find that these examples favor simplicity over security.  They generally disable authentication and encryption leaving the remote application vulnerable to anyone who knows or can guess its host name and port number.  In the real world, if you wanted to monitor a production application remotely, you'd need to carefully consider the security ramifications.

Not finding much in the way of instruction here, we thought it might be helpful to document, by example, how a sample remote Java application can be more securely instrumented and managed with Java Flight Recorder and Java Mission Control.  What follows are the steps necessary to enable monitoring over a connection that is secured using Secure Sockets Layer (SSL).

For the remainder of this article, we'll refer to the instrumented JVM as the remote side of the solution.  To demonstrate the ubiquity of the Java SE platform, our remote JVM will actually reside on a Raspberry Pi.  As Java SE and Java SE Embedded runtime environments both now bundle the Java Flight Recorder feature set, it doesn't really matter what type of remote JVM instance we have.  The location where the Java Mission Control application runs will be referred to as the client side.  For this example we'll use an Ubuntu-based system.  In reality, the client could have run on any of the alternative supported Java clients like Windows or MacOS.

On the remote side (hostname pi1):

Remote side, step 0: Create a directory to house the signed digital certificates required for SSL communication.  Moreover, it should have minimal access.

pi1$ mkdir $HOME/.certs
pi1$ chmod 700 $HOME/.certs

Remote side, step 1: Create a self-signed cryptographic key pair with the JDK keytool(1) utility.  The are many alternatives to creating (and requesting) cryptographic keys.  The method we'll use for this example is the most straightforward, as keytool is part of the JDK, not to mention the least expensive too!

pi1$ keytool -genkey -alias jfrremote -keyalg RSA -validity 730 \
-keystore $HOME/.certs/jfrremoteKeyStore
Enter keystore password: changeit
Re-enter new password: changeit
What is your first and last name?
[Unknown]:  Joe Schmo
What is the name of your organizational unit?
  [Unknown]:  Acme Corp
What is the name of your organization?
  [Unknown]:  Skunkworks
What is the name of your City or Locality?
  [Unknown]:  New York
What is the name of your State or Province?
  [Unknown]:  NY
What is the two-letter country code for this unit?
  [Unknown]:  US
Is CN=Joe Schmo, OU=Acme Corp, O=Skunkworks, L=New York, ST=NY, C=US correct?
  [no]:  yes

Enter key password for <jfrremote>
        (RETURN if same as keystore password):

Remote side, step 2: Restrict the access of the newly created jfrremoteKeyStore file.

pi1$ chmod 600 $HOME/.certs/jfrremoteKeyStore

Remote side, step 3: Verify that the keystore contains our newly created key:

pi1$ keytool -keystore $HOME/.certs/jfrremoteKeyStore -list -v
Enter keystore password: changeit

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 1 entry

Alias name: jfrremote
Creation date: 13-Jan-2015
Entry type: PrivateKeyEntry
Certificate chain length: 1
Owner: CN=Joe Schmo, OU=Acme Corp, O=Skunkworks, L=New York, ST=NY, C=US
Issuer: CN=Joe Schmo, OU=Acme Corp, O=Skunkworks, L=New York, ST=NY, C=US
Serial number: 258d7624
Valid from: Tue Jan 13 17:32:51 UTC 2015 until: Thu Jan 12 17:32:51 UTC 2017
Certificate fingerprints:
         MD5:  2E:AD:5F:85:61:21:8D:1A:4B:ED:02:7C:67:26:8B:95
         SHA1: 82:12:D6:A0:4C:20:E4:7F:C5:C1:C7:BC:AD:C7:D1:E8:47:76:F2:A6
         SHA256: AF:E1:D8:7F:67:F3:DA:F1:22:58:42:B9:A5:50:37:6A:BA:49:76:BC:15:5F:11:9D:F0:1E:13:15:39:BB:9F:C4
         Signature algorithm name: SHA256withRSA
         Version: 3


#1: ObjectId: Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 5E EC 1E F8 D5 33 F4 E6   29 06 B6 65 39 85 68 05  ^....3..)..e9.h.
0010: F7 19 B3 AF                                        ....


Remote side, step 4: Export the certificate associated with the recently generated key pair. This will be used by the remote JVM instance and also has to be imported into the trust store of the client application (Java Mission Control or jmc).

pi1$ keytool -export -alias jfrremote -keystore \
$HOME/.certs/jfrremoteKeyStore -rfc -file jfrremote.cer
Enter keystore password: changeit
Certificate stored in file <jfrremote.cer>

Remote side, step 5: Securely transfer the certificate stored in the jfrremote.cer file over to the system where the Java Mission Control client (jmc) will be run

Switching over to the client side (hostname R840):

Client side, step 0: Create a directory to house the signed digital certificates required for SSL communication, with minimal access.

R840$ mkdir $HOME/.certs
R840$ chmod 700 $HOME/.certs

Client  side, step 1: Import the certificate, represented by the jfrremote.cer file, into the client's trust store.

R840$ keytool -importcert -keystore $HOME/.certs/jfrremoteTrustStore \
-alias jfrremote -file jfrremote.cer
Enter keystore password: changeit
Re-enter new password: changeit
Owner: CN=Joe Schmo, OU=Acme Corp, O=Skunkworks, L=New York, ST=NY, C=US
Issuer: CN=Joe Schmo, OU=Acme Corp, O=Skunkworks, L=New York, ST=NY, C=US
Serial number: 4fec928c
Valid from: Tue Jan 13 08:41:48 EST 2015 until: Thu Jan 12 08:41:48 EST 2017
Certificate fingerprints:
        MD5:  3D:81:45:16:49:13:85:38:E8:E9:90:50:4A:59:F5:5E
         SHA1: 6E:FA:63:D7:9A:58:26:A4:22:94:33:9F:AA:1A:6C:B6:E4:16:2C:DE
         SHA256: 58:EB:F0:C9:DD:F9:D4:F7:FD:95:4B:2B:61:4C:88:6D:57:E3:87:9F:71:F5:BD:25:67:FB:3C:C0:05:0B:C6:0F
         Signature algorithm name: SHA256withRSA
         Version: 3


#1: ObjectId: Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: D9 A6 A2 0E CE D2 F7 9D   FA 96 9C B9 9A 32 E2 3A  .............2.:
0010: 98 ED A7 5F                                        ..._

Trust this certificate? [no]:  yes
Certificate was added to keystore

Client side, step 2: Restrict the access of the newly created jfrremoteTrustStore file.

R840$ chmod 600 $HOME/.certs/jfrremoteTrustStore

Client side, step 3: Create a self-signed cryptographic key pair with the JDK keytool(1) utility.  This represents the certificate for the client-side Java Mission Control (jmc) application.

R840$ keytool -genkey -alias jmc -keyalg RSA -validity 730 \
-keystore $HOME/.certs/jmcKeyStore
Enter keystore password: changeit
Re-enter new password: changeit
What is your first and last name?
  [Unknown]:  Joe Schmo
What is the name of your organizational unit?
  [Unknown]:  Acme Corp
What is the name of your organization?
  [Unknown]:  Skunkworks
What is the name of your City or Locality?
  [Unknown]:  New York
What is the name of your State or Province?
  [Unknown]:  NY
What is the two-letter country code for this unit?
  [Unknown]:  US
Is CN=Joe Schmo, OU=Acme Corp, O=Skunkworks, L=New York, ST=NY, C=US correct?
  [no]:  yes

Enter key password for <jmc>
        (RETURN if same as keystore password):

Client side, step 4: Restrict the access of the newly created jmcKeyStore file.

R840$ chmod 600 $HOME/.certs/jmcKeyStore

Client side, step 5: Export the certificate associated with the recently generated key pair. This will be used by the Java Mission Control application and also has to be imported into the trust store of the remote JVM instance.

R840$ keytool -export -alias jmc -keystore $HOME/.certs/jmcKeyStore \
-rfc -file jmc.cer
Enter keystore password: changeit
Certificate stored in file <jmc.cer>

Client side, step 6: Securely transfer the certificate stored in the jmc.cer file over to the system where the remote JVM instance will be run

Returning to the remote side (hostname pi1):

Remote side, step 6: Import the certificate, represented by the jmc.cer file, into the remote JVM instance's trust store.

pi1$ keytool -import -alias jmc -file jmc.cer \
-keystore $HOME/.certs/jmcTrustStore
Enter keystore password: changeit
Re-enter new password: changeit
Owner: CN=Joe Schmo, OU=Acme Corp, O=Skunkworks, L=New York, ST=NY, C=US
Issuer: CN=Joe Schmo, OU=Acme Corp, O=Skunkworks, L=New York, ST=NY, C=US
Serial number: 860e0e4
Valid from: Tue Jan 13 20:15:33 UTC 2015 until: Thu Jan 12 20:15:33 UTC 2017
Certificate fingerprints:
        MD5:  7B:D7:F3:9D:71:52:F9:35:03:3A:68:BF:02:C2:52:51
         SHA1: 38:95:6D:2F:DE:FC:99:D6:63:55:00:A8:57:E2:31:FF:53:35:18:F7
         SHA256: 7D:87:87:01:E5:21:58:02:67:0E:7E:2F:14:77:86:12:9D:52:CD:11:A4:B1:C5:D3:32:D8:05:30:61:7B:F5:3E
         Signature algorithm name: SHA256withRSA
         Version: 3


#1: ObjectId: Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 0A E8 E3 5E 0A 3C 48 FF   D4 DB 10 A8 62 31 1E F9  ...^.<H.....b1..
0010: 55 D8 4C 7A                                        U.Lz

Trust this certificate? [no]:  yes
Certificate was added to keystore

Remote side, step 7: Restrict the access of the newly created jmcTrustStore file.

pi1$ chmod 600 $HOME/.certs/jmcTrustStore

Putting it All Together

With key stores and trust stores set up on both sides, we can now start up both components.  Our sample application is a very simple one called Download it onto your remote system and compile it with the javac program.  When ready, you can start up an SSL-enabled remote JVM instance of the Allocator program in continuous flight recorder mode with the following invocation:

pi1$ java \\ \ \$HOME/.certs/jfrremoteKeyStore \ \
-Djava.rmi.server.hostname=pi1 \
-XX:+UnlockCommercialFeatures \
-XX:+FlightRecorder \
-XX:FlightRecorderOptions=defaultrecording=true \

On the client side, Java Mission Control is started in the following way:

R840$ jmc -vmargs \$HOME/.certs/jfrremoteTrustStore \ \$HOME/.certs/jmcKeyStore\

Once Java Mission Control has started, we connect to the VM instance on host pi1 as shown in the screenshots that follow.  First off, we select "Connect..." from the File menu.

A "New Connection" window appears.  Select "Create a new connection" and click the "Next>" button.

In the next window that appears, "pi1" is selected as the host and "7091" as the port.  Clicking the "Finish" button continues the process.

 And here's a screenshot of the MBean Server window for the JVM running on host pi1.


To assist in this repetitive and potentially error-prone task, a series of shell scripts have been created which just might be of use.  Download this tarball, and check out the README files found in the two directories, one for the remote side of the equation, the other for the client.

For further edification, please check out these links:

Marcus Hirt's blog for all things related to Java Flight Recorder and Java Mission Control

If you run into SSL handshake problems, this link may prove helpful.
How to Anazyze Java SSL Errors:

Erik Costlow's article on Self-signed certificates for a known community:

Friday Oct 31, 2014

USB Device Access for Java SE and OSGi

One of the challenges in creating the content for the Java One 2014 Java SE Embedded Internet of Things Hands-on-Lab concerned interacting, via Java and OSGi, with a USB temperature sensor.   Unfortunately a USB communications API is not part of the Java SE standard (as of this post: Halloween 2014).  So the question is, how can Java/USB communication be established, and furthermore and how does this work within the OSGi framework?

In looking around at some of the available options, we chose the javahidapi as the basis for this connectivity.  As a Java/JNI wrapper around the C/C++ HID API for Linux, MacOS X and Windows, the appeal behind this API is that using it does not require the use of a custom driver for each device on each platform.

In order to operate within an OSGi framework (in this case Apache Felix 4.4), javahidapi's open source code has been slightly modified/enhanced. The end result is that an OSGi bundle is available that can be dropped into standard OSGi frameworks to support USB communication for HID devices.  It does contain a native component, and for the sake of simplicity, we've decided to include a separate jar file for each supported architecture.  For the OSGi enthusiast, here's what the generated MANIFEST.MF file looks like for the Linux/armhf (suitable for the Raspberry Pi) architecture:

Manifest-Version: 1.0
Bnd-LastModified: 1415889978962
Build-Jdk: 1.7.0_51
Built-By: jtconnor
Bundle-Activator: com.codeminders.hidapi.Activator
Bundle-ManifestVersion: 2
Bundle-Name: hidapi OSGi Bundle for Linux/armhf
Bundle-NativeCode: native/linux/armv6l/; osname=Linux; processor=armv6l
Bundle-SymbolicName: com.codeminders.hidapi-armhf
Bundle-Version: 1.0.0
Created-By: Apache Maven Bundle Plugin
Export-Package: com.codeminders.hidapi;uses:="org.osgi.framework";version="1.0.0"
Import-Package: org.osgi.framework;version="[1.6,2)"
Tool: Bnd-1.50.0

Here are some pre-built hidapi OSGi bundles for popular Linux platforms:

To get a feel for the changes made to the original source, here's what was done:

  1. A NetBeans project was created under the Maven->OSGi Bundle category
  2. The Java source code for javahidapi was placed in the project's src/main/java/com/codeminders/hidapi directory
  3. The architecture specific native library was placed in the project's src/main/resources/native/linux/architecture directory.  For example, the Linux/x86 version of the project places the file in the src/main/resources/native/linux/x86 directory.
  4. An class was added in the project's src/main/java/com/codeminders/hidapi directory.  In OSGi, the start() method in this class gets called when this bundle is activated.  It is specified in the bundle's MANIFEST.MF file.
  5. The original file was simplified and is currently only appropriate for Linux deployments.
  6. As this is a Maven-based project, the project's pom.xml file was edited (here's what the x86 version looks like), such that at build time it will generate a MANIFEST.MF file similar to the one referenced above.

And here are the associated NetBeans Projects which can be used to build the 4 bundles referenced above.

If you'd like to extend this template to include OSGi bundles for additional architectures, you can start with one of the projects above, clone it, and make the appropriate changes for your new environment.  If the native javahidapi component for your new platform is not available, you'll have to pull down the source for hidapi and build it for inclusion into your project.  If anyone is interested in going through this exercise, I'd be glad to post the fruits of their labor here.

Tuesday Aug 26, 2014

Java SE Embedded IoT Hands On Lab Returns for Java One 2014

After a one year hiatus, a Hands On Lab featuring Java SE for Embedded devices will return for The Java One 2014 Conference.  Entitled HOL2097 Java SE Embedded IoT Hands On Lab, students who attend this lab will:

  • Have their own individual embedded device to gain valuable hands on experience
  • Learn how to create Java 8 Compact Profiles using the eJDK jrecreate tool.  Students will use and deploy those Compact Profiles on their individual device.
  • Learn about profile-aware tools available both at the JDK and IDE (NetBeans) level that aid developers in creating Java applications suitable for specific Compact Profiles.
  • Learn about and utilize OSGi, an important Java-based modular service platform ideal for dynamically managing the lifecycle of software components, especially suited for gateway devices.
  • Attach a sensor to the gateway device and deploy a Java/OSGi bundle on the device to capture data from the sensor.
  • Build and deploy a Java/OSGi web service bundle on the gateway device that publishes data captured by the sensor.  Students will be able to remotely access this web service to retrieve the sensor data in standard JSON format.

For those attending the Java One 2014 Conference, seating for this lab is limited.  So reserve your spot early.  We could also conceivably host a smaller workshop at a customer location effectively duplicating this lab.  If you're interested, you can drop me a line at

Friday Jun 20, 2014

Java Serial Communications Revisited

Last touched upon in this article dated August 2011, it's high time to revisit the state of Java serial port communications options for the following reasons:

  • As the hype around Internet of Things continues to ratchet up, serial communications are a requirement for a certain class of IoT sensors and devices.
  • RxTx, the platform with arguably the most history, cannot be built with the recently released Java 8 JDK without some modifications.
  • For some time now the website hosting the RxTx project: has not been available.
  • Building support for additional specifications like RS-485 and parallel is available in RxTx but was never addressed in the previous serial communications article.
  • An alternative framework called jSSC is gaining in popularity and is worth further discussion.
  • Work in the OpenJDK Device I/O Project is progressing. Among the goals of this project is support for serial communications.


In my customer experiences, RxTx, despite its age, still gets mentioned most when considering serial communications with Java.  For whatever reasons, the RxTx project has gone off line, and access to the project source code is not readily available.  Fortunately, we have a copy, and have made a few enhancements such that:

  • The source can now be compiled with a JDK versions 6, 7 and 8.
  • The original article discussed only enough modifications to build the shared object required for traditional serial communications.  The,, and shared objects could not be built.  With some very slight modifications, these can successfully be built too.  I make absolutely no promises as to their usefulness, but they do compile. :)

The source code is based upon the original 2.1-7r2 version and in this instance is now called 2.1.7r2-Java8.  You can download the source here.  If you want to get a feel for the changes made, take a look at this file:  JAVA8_MODS.txt which can be found in the main directory of the source code.  To build RxTx on your native platform:

   $ tar xvf 2.1.7r2-Java8.tar.gz
   $ cd 2.1.7r2-Java8/
   $ ./configure
   $ make


The Java Simple Serial Connector, or jSSC for short, is another open source project that can be found here:  It is available for a host of processor/OS combinations and, in addition to source availability, comes bundled in binary form too.  Here's a list of supported platforms for its current 2.6.0 release:

Win32 Win64 Linux_x86 Linux_x86_64 Linux_ARM Solaris_x86 Solaris_x86_64 MacOSX_x86 MacOSX_x86_64 MacOSX_PPC MacOSX_PPC64

Like RxTx, it contains a native component but is packaged in a nice transparent fashion.  The one potential challenge here may be in trying to figure out how to support a new platform that isn't on this list.  I didn't have a whole lot of success finding out how to build the binary, and admittedly didn't spend an inordinate amount of time trying to figure it out either.  Nonetheless, this project is gaining in popularity and has a dedicated individual supporting the software.

OpenJDK Device I/O

Finally, a project is underway to treat serial communication and device I/O in general as a first class citizen for the Java SE standard. The wiki can be found here: It is based on the work done to provide device I/O to the Java ME 8 platform.  Further solidifying the universality theme of Java 8, the ultimate goal would be to have a consistent device I/O API across both Java SE and Java ME.  If you want to further understand what those APIs look like you can view them here:

In conclusion, support for serial communications in Java SE is -- albeit slowly -- progressing.  There are multiple open source projects and commercial alternatives too.  Ideally, it will be great to see a formal API supported by the Java SE Standard.

Monday Mar 10, 2014

Introducing the EJDK

In lock step with the introduction of Compact Profiles, Java 8 includes a new distribution mechanism for Java SE Embedded called the EJDK.  As the potential exists to confuse the EJDK with the standard JDK (Java Development Kit), it makes sense to dedicate a few words towards highlighting how these two packages differ in form and function.


The venerable Java Development Kit is the mainstay of Java developers.  It incorporates not only a standard Java Runtime Environment (JRE), but also includes critical tools required by those same developers.  For example, among many others, the JDK comes with a Java compiler (javac), a Java console application (jconsole), the Java debugger (jdb) and the Java archive utility (jar).  It also serves as the underpinnings for very popular Java Integrated Development Environments (IDEs) such as NetBeans, Eclipse, JDeveloper and IntelliJ to name a few.

Like Java, the Java Development Kit is constantly evolving, and Java 8 brings about its fair share of enhancements to the JDK.  For Java 8, javac can now be instructed (via the -profile command-line option) to insure that your source code is compatible with a specific compact profile.  Furthermore, the Java 8 JDK comes with a new useful tool called jdeps, providing a means to analyze your compiled class and jar files for dependencies.


The EJDK is new to Java 8, and although similar in namesake to the JDK, it serves quite a different purpose.  Prior to Java 8, supported Java SE-Embedded runtime platforms were provided as binaries by Oracle.  With the advent of Compact Profiles, the number of possible binary options per supported platform would simply be too unweildy.  Rather than furnishing binaries for each of the possible combinations, an EJDK will be supplied for each supported Java SE-Embedded platform.  It contains the tools needed to create the profile you wish to use.

The EJDK is designed to be run with either Windows or Linux/Unix platforms alongside a Java runtime environment.  It contains a wrapper called jrecreate ( for Unix/Linux and jrecreate.bat for Windows) whose function it is to create deployable compact profile instances. In the examples that follow, we'll show two sample invocations.

First off, let's briefly take a look at the contents of a typical EJDK.   For our first example, we've installed the EJDK on a linux/x86 system.   Listing the contents of the ejdk1.8.0/ directory, we see a subdirectory named linux_arm_vfp_hflt/.  This tells us what platform this instance of the EJDK supports.  For all our examples we'll use an EJDK that creates compact profiles suitable for Linux/Arm Hard Float platform, often times referred to as armhf.

$ ls ejdk1.8.0
bin  doc  lib  linux_arm_vfp_hflt

Looking one level deeper into the bin/ directory, we see the jrecreate.bat and files:

$ ls ejdk1.8.0/bin

As we're on a Linux system, let's use the script to create a compact profile:

$ ./ejdk1.8.0/bin/ --profile compact1 --dest compact1-minimal --vm minimal

Briefly reviewing this invocation, the --profile compact1 option instructs jrecreate to use the Compact1 profile.  The --profile option accepts [compact1 | compact2 | compact3]  as an argument. The --dest compact1-minimal option specifies the name of the destination directory containing the newly generated profile.  Note that the directory argument to --dest must not exist prior to invocation.  Finally, the --vm minimal option tells jrecreate to use the minimal (i.e. the smallest) virtual machine for this instance.  The --vm option accepts  [minimal | client | server | all] as an argument.  Running the complete command, we get the following output:

$ ./ejdk1.8.0/bin/ --profile compact1 --dest compact1-minimal --vm minimal
Building JRE using Options {
   ejdk-home: /home/java8/ejdk1.8.0
    dest: /home/java8/compact1-minimal
    target: linux_arm_vfp_hflt
    vm: minimal
    runtime: compact1 profile
    debug: false
    keep-debug-info: false
    no-compression: false
    dry-run: false
    verbose: false
    extension: []

Target JRE Size is 10,595 KB (on disk usage may be greater).
Embedded JRE created successfully

This creates a Compac1 profile distribution of about 10 ½ MB in the compact-1-minimal/ directory.  For our second example, we'll create a profile based on Compact2 and the client VM, this time from a Windows 7/64-bit system:

c:\demo>ejdk1.8.0\bin\jrecreate.bat --profile compact2 --dest compact2-client --vm client
Building JRE using Options {
    ejdk-home: c:\demo\ejdk1.8.0\bin\..
    dest: c:\demo\compact2-client
    target: linux_arm_vfp_hflt
    vm: client
    runtime: compact2 profile
    debug: false
    keep-debug-info: false
    no-compression: false
    dry-run: false
    verbose: false
    extension: []

Target JRE Size is 17,552 KB (on disk usage may be greater).
Embedded JRE created successfully

This Compact2 instance is created in the compact2-client/ directory and has an approximate footprint of 17 ½ MB.  Additional options to jrecreate are available for further customization.

Finally, lets migrate the generated profiles over to a real device.  As a host platform we'll use none other than the ubiquitous Raspberry Pi.  Here's a listing of the two profiles and their size (in 1K blocks) on the filesystem:

pi@pi0 ~/java8 $ ls
compact1-minimal  compact2-client

pi@pi0 ~/java8 $ du -sk compact*
10616   compact1-minimal
17660   compact2-client

And here's what each version outputs when java -version is run:

pi@pi0 ~/java8 $ ./compact1-minimal/bin/java -version
java version "1.8.0"
Java(TM) SE Embedded Runtime Environment (build 1.8.0-b127, profile compact1, headless)
Java HotSpot(TM) Embedded Minimal VM (build 25.0-b69, mixed mode)

pi@pi0 ~/java8 $ ./compact2-client/bin/java -version
java version "1.8.0"
Java(TM) SE Embedded Runtime Environment (build 1.8.0-b127, profile compact2, headless)
Java HotSpot(TM) Embedded Client VM (build 25.0-b69, mixed mode)

In conclusion, you are encouraged to experiment with the EJDK.  It will very quickly give you a feel for the compact profile configuration options available for your device.

Friday Dec 06, 2013

Java SE Embedded Pricing Explained

You're probably asking yourself, "Pricing?  Really?  In a techie blog?", and I would normally agree wholeheartedly with your assessment.  But in this one instance the topic might be worthy of a few words.  There is, as the expression goes, no such thing as a free lunch.  Whether you pay for software outright, or roll your own with open source projects, a cost must be paid.

Like clockwork, we regularly receive inquiries for Java embedded information that go something like this:

Dear Oracle,  We've downloaded and evaluated Java SE-Embedded and have found it to be a very appealing platform to run our embedded application.  We understand this is commercial software; before we decide to deploy our solution with your runtime, can you give us a feel for the royalties associated with shipping x number of units?

Seems pretty straightforward, right?  Well, yes, except that in the past Oracle required the potential customer to sign a non-disclosure agreement prior to receiving any embedded pricing information.  It didn't matter if the customer was interested in deploying ten units or ten thousand, they all had to go through this process.  Now certain aspects of pricing may still require confidential agreements, but why not make quantity 1 list prices available?   With the release of this document, that pricing information is now public.

The evidence is out there, both anecdotal and real, demonstrating that Oracle's Java SE-Embedded platform is unquestionably superior in quality and performance to the OpenJDK variants.  For the latest example, take a look at this blog entry.  So the question becomes, is it actually more affordable to pay for a commercial platform that is fully supported, faster and more reliable or to opt for a "free" platform and support it yourself.

So What Does Java SE-Embedded Cost?

The universal answer to such a question is: it depends.  That is to say it depends upon the capability of the embedded processor.  Before we lose you, let's show the list price for Java embedded licensing associated with three platforms and then explain how we arrived at the numbers.  As of the posting of this entry, 06 December, 2013, here they are:

  1. Per-unit cost for a Raspberry Pi: US $0.71
  2. Per-unit cost for system based on Intel Atom Z510P: US $2.68
  3. Per-unit cost for a Compulab Trim-Slice: US $5.36

How Does It Work?

These bullet points help describe the process, then we'll show how we arrived at our three sample platform prices.

  • Pricing is done on a per-core basis.
  • Processors are classified based on their capability and assigned a core factor.  The more capable the processor, the higher the core factor.
  • Per-core pricing is determined by multiplying the standard per-core Java embedded price by the core factor.
  • A 19% Software Update License & Support Fee is automatically added onto each system.

The core factor table that follows, found in the Oracle Java Embedded Global Price List, dated September 20, 2013, groups processors of similar capabilities into buckets called chip classes.  Each chip class is assigned a core factor.

Example 1

To compute the per-unit cost, use this formula:

Oracle Java Embedded per-core license fee  *  core factor  *  number of cores  *  support uplift

The standard per-core license fee is always $300.  The Raspberry Pi is a Class I device and therefore has a core factor of .002.  There is only one core in the Raspberry Pi, and the Software Update License & Support fee is always 19%.  So plugging in the numbers, we get:

$300  *  .002  *  1  *  1.19  =  $0.714

Example 2

The processor in this example, the Intel Atom Z510P, is a Class II device and has a core factor of .0075.  Using the same formula from Example 1, here's what we get:

$300  *  .0075  *  1  *  1.19  =  $2.6775

Example 3

The processor for the Trim-Slice is based on the ARM Cortex-A9, a Class II device.  Furthermore it is a dual-core system.  Using the same formula as the previous examples, we arrive at the following per-unit pricing:

$300  *  .0075  *  2  *  1.19  = $5.355


With your hardware specs handy, you should now have enough information to make a reasonable estimate of Oracle Java embedded licensing costs.  At minimum, it could be a help in your "buy vs. roll your own" decision making process.  And of course, if you have any questions, don't be afraid to ask.

Monday Oct 14, 2013

Oracle Java Now Part of Raspberry Pi Raspian Distribution

Having sold more than 1.75 million units in its brief lifespan, there is no denying the profound impact the Raspberry Pi has had on the embedded development community.  Amid all the hoopla surrounding the recently completed Oracle OpenWorld and Java One events, one announcement that flew under the radar dealt with the fact that the Oracle JDK is now part of the Raspberry Pi Raspian distribution.

This means that by default, when you download and install the latest Raspian distribution on your Raspberry Pi, the Oracle Java runtime environment and development tools are automatically part of the list of packages installed.  Here's a screenshot showing a login session to a Raspberry Pi running the 2013-09-25-wheezy-raspian distribution.  The JDK is installed as the oracle-java7-jdk package and is utilizing the Java 7u40 release.

For those who either cannot or do not wish to perform an entire Raspian upgrade, the Oracle JDK is available via the following command:

$ sudo apt-get update && sudo apt-get install oracle-java7-jdk

Great news.

Tuesday Sep 17, 2013

Comparing Linux/Arm JVMs Revisited

It's been about 18 months since we last compared Linux/Arm JVMs, and with the formal release of the much anticipated Java SE Embedded for Arm hard float binary, it marks a good time to revisit JVM performance.  The information and results that follow will highlight the following comparisons:

  1. Java SE-E Arm VFP (armel) vs. Arm Hard Float (armhf)
  2. Java SE-E armhf Client Compiler (c1) vs. armhf Server Compiler (c2)
  3. And last but certainly not least ... Java SE-E 7u40 armhf vs. Open JDK armhf

The Benchmark

For the sake of simplicity and consistency, we'll use a subset of the DaCapo benchmark suite.  It's an open source group of real world applications that put a good strain on a system both from a processor and memory workload perspective. We are aware of customers who use DaCapo to gauge performance, and due to its availability and ease of use, enables anyone interested to run their own set of tests in fairly short order.

The Hardware

It would have been grand to run all these benchmarks on one platform, most notably the beloved Raspberry Pi, but unfortunately it has its limitations:

  • There is no Java SE-E server compiler (c2) for the Raspberry Pi.  Why?  Because the Pi is based on an ARMv6 instruction set whereas the Java SE-E c2 compiler requires a minimum ARMv7 instruction set.
  • Demonstrating how rapidly advances are being made in the hardware arena, the Raspberry Pi, within the context of these tests, is quite a humble platform.  With 512MB RAM, it runs out of memory when running some of the large DaCapo component applications.
For these tests we'll primarily use a quad-core Cortex-A9 based system, and for one test we'll utilize a single core Marvell Armada system just to compare what effect the number of cores has on server compiler performance.  The devices in question are:
  1. Boundary Devices BD-SL-i.MX6, quad core 1GHz Cortex-A9 (Freescale i.MX6), 1GB RAM, Debian Wheezy distribution, 3.0.35 kernel (for both armel and armhf configurations)
  2. GlobalScale D2Plug, single core 800MHz ARMv6/v7 processor (Marvell PXA510), 1GB RAM, Debian Wheezy distribution, 3.5.3-cubox-di+ kernel for armhf

Java SE-E armel vs. armhf

The chart that follows compares the relative performance of the armel JavaSE-E 7u40 JRE with the armhf JavaSE-E 7u40 JRE for 8 of the DaCapo component applications.  These tests were conducted on the Boundary Devices BD-SL-i.MX6.  Both armel and armhf environments were based on the Debian Wheezy distribution running a 3.0.35 kernel.  For all charts, the smaller the result, the faster the run.

In all 8 tests, the armhf binary is faster, some only slightly, and in one case (eclipse) a few percentage points faster.  The big performance gain associated with the armhf standard deals with floating point operations, and in particular, the passing of arguments directly into floating point registers.  The performance gains realized by the newer armhf standard will be seen more in the native application realm than for Java SE-Embedded primarily because  the Java SE-E armel VM already uses FP registers for Java floating point methods.  There are still however certain floating point workloads that may show a modest performance increase (in the single digit percent range) with JavaSE-E armhf over Java SE-E armel.

Java SE-E Client Compiler (c1) vs. Server Compiler (c2)

In this section, we'll show tests results for two different platforms, the first a single core system, followed by the same tests on a quad-core system.  To further demonstrate how workload changes performance, we'll take advantage of the ability to run the DaCapo component applications in three different modes: small, default (medium) and large.  The first chart displays the aggregate time required to run the tests for the three modes, utilizing both the 7u40 client (c1) compiler and the server (c2) compiler.  As expected, c1 outperforms c2 by a wide margin for the tests that run only briefly.  As the total time to run the tests increases from small to large, the c2 compiler gets a chance to "warm up" and close the gap in performance.  But it never does catch up.  

Contrast the first chart with the one that follows where small, medium and large versions of the tests were run on a quad core system.  The c2 compiler is better able to utilize the additional compute resources supplied by this platform, the result being that initial gap in performance between c1 and c2 for the small version of the test is only 19%.  By the time we reach the large version, c2 outperforms c1 by 7%.  The moral of the story here is, given enough resources, the server compiler might be the better of the VMs for your workload if it is a long-lived process.

Java SE-E 7u40 armhf vs. Open JDK armhf

For this final section, we'll break out performance on an application-by-application basis for the following JRE/VMs:

  • Java SE Embedded 7u40 armhf Client Compiler (c1)
  • Java SE Embedded 7u40 armhf Server Compiler (c2)
  • OpenJDK 7 IcedTea 2.3.10 7u25-2.3.10-1~deb7u1 OpenJDK Zero VM (build 22.0-b10, mixed mode)
  • OpenJDK 7 IcedTea 2.3.10 7u25-2.3.10-1~deb7u1 JamVM (build 1.6.0-devel, inline-threaded interpreter with stack-caching)
  • OpenJDK 6 IcedTea6 1.12.6 6b27-1.12.6-1~deb7u1 OpenJDK Zero VM (build 20.0-b12, mixed mode)
  • OpenJDK 6 IcedTea6 1.12.6 6b27-1.12.6-1~deb7u1 JamVM (build 1.6.0-devel, inline-threaded interpreter with stack-caching)
  • OpenJDK IcedTea6 1.12.6 6b27-1.12.6-1~deb7u1 CACAO (build 1.6.0+r68fe50ac34ec, compiled mode)

The OpenJDK packages were pulled from the Debian Wheezy distribution.

It appears the bulk of performance work to OpenJDK/Arm still revolves around the OpenJDK 6 platform even though Java 7 was released over two years ago (and Java 8 is coming soon).  Regardless, Java SE still outperforms most OpenJDK tests by a wide margin, and perhaps more importantly appears to be the much more reliable platform considering the number of tests that failed with the OpenJDK variants.  As demonstrated in previous benchmark results, the older armel OpenJDK VMs appear to be more stable than the armhf versions tested here.  Considering the stated direction by the major linux distributions is to migrate towards the armhf binary standard, this is a bit eye opening.

As always, comments are welcome.

Monday Aug 12, 2013

Compact Profiles Demonstrated

Following up on an article introducing compact profiles, the video that follows demonstrates how this new feature in the upcoming Java 8 release can be utilized.  The video:

  • Describes the compact profile feature and the rationale for its creation.
  • Shows how to use the new jrecreate utility to generate compact profiles that can be readily deployed.
  • Demonstrates that even the smallest of profiles (less than 11MB) is robust enough to support very popular and important software frameworks like OSGi.

The software demonstrated is in early access.  For those interested in trying it out before the formal release of Java 8, there are two options:

  1. Members of the Oracle Partner Network (OPN) with a gold membership or higher can download the early access Java 8 binaries of Java SE-Embedded shown here.  For those not at this level, it may still be possible to get early access software, but it will require a qualification process beforehand.
  2. It's not as intimidating as it sounds, you can pull down the source code for OpenJDK 8, and build it yourself.  By default, compact profiles are not built, but this forum post shows you how.  The reference platform for this software is linux/x86.  Functionally, the generated compact profiles will contain the pared down modules for each compact profile, but you'll find the footprint for each to be much larger than the ones demonstrated in this video, as none of the Java SE-Embedded space optimizations are performed by default.

Not having any premium privileges on YouTube, the maximum allowed length of a video is 15 minutes.  There's actually lots more to talk about with compact profiles, including enhancements to java tools and utilities (javac, jar, jdeps, and the java command itself) that have incorporated intelligence for dealing with profiles.

Hmm.  Maybe there's an opportunity for a Compact Profiles Demonstrated Part II?

Wednesday Jul 31, 2013

An Introduction to Java 8 Compact Profiles

Java SE is a very impressive platform indeed, but with all that functionality comes a large and ever increasing footprint.  It stands to reason then that one of the more frequent requests from the community has been the desire to deploy only those components required for a particular application instead of the entire Java SE runtime environment.  Referred to as subsetting, the benefits of such a concept would seem to be many:

  • A smaller Java environment would require less compute resources, thus opening up a new domain of devices previously thought to be too humble for Java.
  • A smaller runtime environment could be better optimized for performance and start up time.
  • Elimination of unused code is always a good idea from a security perspective.
  • If the environment could be pared down significantly, there may be tremendous benefit to bundling runtimes with each individual Java application.
  • These bundled applications could be downloaded more quickly.

Despite these perceived advantages, the platform stewards (Sun, then Oracle) have been steadfast in their resistance to subsetting.  The rationale for such a stance is quite simple: there was sincere concern that the Java SE platform would fragment.  Agree or disagree, the Java SE standard has remained remarkably in tact over time.  If you need any further evidence of this assertion, compare the state of Java SE to that of Java ME, particularly in the mobile telephony arena.  Better still, look how quickly Android has spawned countless variants in its brief lifespan.

Nonetheless, a formal effort has been underway having the stated goal of providing a much more modular Java platform.  Called Project Jigsaw, when complete, Java SE will be composed of a set of finer-grained modules and will include tools to enable developers to identify and isolate only those modules needed for their application.  However, implementing this massive internal change and yet maintaining compatibility has proven to be a considerable challenge.  Consequently full implementation of the modular Java platform has been delayed until Java 9.

Understanding that Java 9 is quite a ways off, an interim solution will be available for Java 8, called Compact Profiles.  Rather than specifying a complete module system, Java 8 will define subset profiles of the Java SE platform specification that developers can use to deploy.  At the current time three compact profiles have been defined, and have been assigned the creative names compact1, compact2, and compact3. The table that follows lists the packages that comprise each of the profiles.  Each successive profile is a superset of its predecessor.  That is to say, the compact2 profile contains all of the packages in compact1 plus those listed under the compact2 column below.  Likewise, compact3 contains all of compact2 packages plus the ones listed in the compact3 column.

compact1                     compact2                    compact3
--------------------------   -----------------------     --------------------------                      java.rmi                    java.lang.instrument
java.lang                    java.rmi.activation
java.lang.annotation         java.rmi.registry 
java.lang.invoke             java.rmi.server             java.util.prefs
java.lang.ref                java.sql                    javax.annotation.processing
java.lang.reflect            javax.rmi.ssl               javax.lang.model
java.math                    javax.sql                   javax.lang.model.element                     javax.transaction           javax.lang.model.type
java.nio                     javax.transaction.xa        javax.lang.model.util
java.nio.channels            javax.xml         
java.nio.channels.spi        javax.xml.datatype
java.nio.charset             javax.xml.namespace
java.nio.charset.spi         javax.xml.parsers 
java.nio.file.spi                  javax.xml.transform           javax.xml.transform.dom     javax.xml.transform.sax     javax.naming           javax.xml.transform.stax
java.text            javax.naming.event
java.text.spi                javax.xml.validation        javax.naming.ldap
java.util                    javax.xml.xpath             javax.naming.spi
java.util.concurrent         org.w3c.dom                 javax.script
java.util.concurrent.atomic  org.w3c.dom.bootstrap
java.util.jar                    javax.sql.rowset
java.util.logging            org.xml.sax                 javax.sql.rowset.serial
java.util.regex              org.xml.sax.ext             javax.sql.rowset.spi
java.util.spi                org.xml.sax.helpers                                            javax.xml.crypto
javax.crypto                                             javax.xml.crypto.dom
javax.crypto.interfaces                                  javax.xml.crypto.dsig
javax.crypto.spec                                        javax.xml.crypto.dsig.dom                                                javax.xml.crypto.dsig.keyinfo                                            javax.xml.crypto.dsig.spec                                      org.ieft.jgss

You may ask what savings can be realized by using compact profiles?  As Java 8 is in pre-release stage, numbers will change over time, but let's take a look at a snapshot early access build of Java SE-Embedded 8 for ARMv5/Linux.  A reasonably configured compact1 profile comes in at less than 14MB.  Compact2 is about 18MB and compact3 is in the neighborhood of 21MB.  For reference, the latest Java 7u21 SE Embedded ARMv5/Linux environment requires 45MB.

So at less than one-third the original size of the already space-optimized Java SE-Embedded release, you have a very capable runtime environment.  If you need the additional functionality provided by the compact2 and compact3 profiles or even the full VM, you have the option of deploying your application with them instead.

In the next installment, we'll look at Compact Profiles in a bit more detail.


Jim Connors-Oracle


« November 2015