Tuesday Sep 08, 2015

Using Java Flight Recorder with Compact Profiles

Like it's big brother Java SE, the Java SE-Embedded 8 platform provides support for Java Flight Recorder, an invaluable profiling and event collection framework built into the Java Runtime Environment.  However, Flight Recorder is only available with the Java SE Embedded 8 Full JRE, not with the smaller Compact Profiles.  So the question becomes, is there anything that can be done to use Java Flight Recorder with Compact Profiles?

At the current time, the smaller Compact1 and Compact2 profiles cannot realistically support Java Flight Recorder without substantial changes.  Hence we'll avoid discussing their prospects for inclusion here.  What we will focus on is Compact3 as its specification includes among others, the javax.management APIs, making for a more reasonable match.  As it turns out, all that is required to enable Flight Recorder use with the Compact3 profile is to copy over a few files over from the Full JRE.  The instructions that follow should aid in creating a Flight Recorder enabled instance:

1. Download a Java SE-Embedded EJRE for your platform.  You can do so here.  For this example, we'll do this on a Linux host and choose the ARMv6/v7 Hard Float option (suitable for the venerable Raspberry Pi).

2. Extract the EJRE.

$ tar xvf ejdk-8u51-linux-armv6-vfp-hflt.tar.gz

3. Use the EJRE's jrecreate.sh script to create a full JRE:

$ ./ejdk1.8.0_51/bin/jrecreate.sh --dest full_jre -g -k
Building JRE using Options {
    ejdk-home: /home/pi/ejdk1.8.0_51
    dest: /home/pi/full_jre
    target: linux_armv6_vfp_hflt
    vm: all
    runtime: jre
    debug: true
    keep-debug-info: true
    no-compression: false
    dry-run: false
    verbose: false
    extension: []

Target JRE Size is 59,389 KB (on disk usage may be greater).
Embedded JRE created successfully

4. Create a Compact3 JRE:

$ ./ejdk1.8.0_51/bin/jrecreate.sh --profile compact3 --dest compact3 -g -k
Building JRE using Options {
    ejdk-home: /home/pi/ejdk1.8.0_51
    dest: /home/pi/compact3
    target: linux_armv6_vfp_hflt
    vm: client
    runtime: compact3 profile
    debug: true
    keep-debug-info: true
    no-compression: false
    dry-run: false
    verbose: false
    extension: []

Target JRE Size is 24,336 KB (on disk usage may be greater).
Embedded JRE created successfully

5.  Check the size of the compact3 JRE.  We'll use this to see how much space was needed to add support for Flight Recorder.

$ du -sk compact3
24480   compact3

6.  Create the compact3/lib/jfr/ directory

$ mkdir compact3/lib/jfr

7.  Copy the following files over from the full JRE to the compact3 instance with these commands:

$ cd full_jre/
$ cp ./lib/jfr.jar ../compact3/lib
$ cp ./lib/arm/libjfr.so ../compact3/lib/arm
$ cp ./lib/arm/libbci.so ../compact3/lib/arm
$ cp ./lib/jfr/default.jfc ../compact3/lib/jfr
$ cp ./lib/jfr/profile.jfc ../compact3/lib/jfr

8. That's all that is required.  You can compare the disk usage of the compact3/ directory before and after these modifications to get an idea of the additional space required to utilize Java Flight Recorder.

$ du -sk ../compact3
24824   ../compact3

So comparing the disk usage of step (5), we see that less than 400KB is added in order to enable usage of Java Flight Recorder. To better understand how you might remotely connect to a Flight Recorder enabled instance in a secure fashion, check out this article.

Tuesday Aug 26, 2014

Java SE Embedded IoT Hands On Lab Returns for Java One 2014

After a one year hiatus, a Hands On Lab featuring Java SE for Embedded devices will return for The Java One 2014 Conference.  Entitled HOL2097 Java SE Embedded IoT Hands On Lab, students who attend this lab will:

  • Have their own individual embedded device to gain valuable hands on experience
  • Learn how to create Java 8 Compact Profiles using the eJDK jrecreate tool.  Students will use and deploy those Compact Profiles on their individual device.
  • Learn about profile-aware tools available both at the JDK and IDE (NetBeans) level that aid developers in creating Java applications suitable for specific Compact Profiles.
  • Learn about and utilize OSGi, an important Java-based modular service platform ideal for dynamically managing the lifecycle of software components, especially suited for gateway devices.
  • Attach a sensor to the gateway device and deploy a Java/OSGi bundle on the device to capture data from the sensor.
  • Build and deploy a Java/OSGi web service bundle on the gateway device that publishes data captured by the sensor.  Students will be able to remotely access this web service to retrieve the sensor data in standard JSON format.

For those attending the Java One 2014 Conference, seating for this lab is limited.  So reserve your spot early.  We could also conceivably host a smaller workshop at a customer location effectively duplicating this lab.  If you're interested, you can drop me a line at james.connors@oracle.com.

Monday Mar 10, 2014

Introducing the EJDK

In lock step with the introduction of Compact Profiles, Java 8 includes a new distribution mechanism for Java SE Embedded called the EJDK.  As the potential exists to confuse the EJDK with the standard JDK (Java Development Kit), it makes sense to dedicate a few words towards highlighting how these two packages differ in form and function.


The venerable Java Development Kit is the mainstay of Java developers.  It incorporates not only a standard Java Runtime Environment (JRE), but also includes critical tools required by those same developers.  For example, among many others, the JDK comes with a Java compiler (javac), a Java console application (jconsole), the Java debugger (jdb) and the Java archive utility (jar).  It also serves as the underpinnings for very popular Java Integrated Development Environments (IDEs) such as NetBeans, Eclipse, JDeveloper and IntelliJ to name a few.

Like Java, the Java Development Kit is constantly evolving, and Java 8 brings about its fair share of enhancements to the JDK.  For Java 8, javac can now be instructed (via the -profile command-line option) to insure that your source code is compatible with a specific compact profile.  Furthermore, the Java 8 JDK comes with a new useful tool called jdeps, providing a means to analyze your compiled class and jar files for dependencies.


The EJDK is new to Java 8, and although similar in namesake to the JDK, it serves quite a different purpose.  Prior to Java 8, supported Java SE-Embedded runtime platforms were provided as binaries by Oracle.  With the advent of Compact Profiles, the number of possible binary options per supported platform would simply be too unweildy.  Rather than furnishing binaries for each of the possible combinations, an EJDK will be supplied for each supported Java SE-Embedded platform.  It contains the tools needed to create the profile you wish to use.

The EJDK is designed to be run with either Windows or Linux/Unix platforms alongside a Java runtime environment.  It contains a wrapper called jrecreate (jrecreate.sh for Unix/Linux and jrecreate.bat for Windows) whose function it is to create deployable compact profile instances. In the examples that follow, we'll show two sample invocations.

First off, let's briefly take a look at the contents of a typical EJDK.   For our first example, we've installed the EJDK on a linux/x86 system.   Listing the contents of the ejdk1.8.0/ directory, we see a subdirectory named linux_arm_vfp_hflt/.  This tells us what platform this instance of the EJDK supports.  For all our examples we'll use an EJDK that creates compact profiles suitable for Linux/Arm Hard Float platform, often times referred to as armhf.

$ ls ejdk1.8.0
bin  doc  lib  linux_arm_vfp_hflt

Looking one level deeper into the bin/ directory, we see the jrecreate.bat and jrecreate.sh files:

$ ls ejdk1.8.0/bin
jrecreate.bat  jrecreate.config.properties  jrecreate.sh

As we're on a Linux system, let's use the jrecreate.sh script to create a compact profile:

$ ./ejdk1.8.0/bin/jrecreate.sh --profile compact1 --dest compact1-minimal --vm minimal

Briefly reviewing this invocation, the --profile compact1 option instructs jrecreate to use the Compact1 profile.  The --profile option accepts [compact1 | compact2 | compact3]  as an argument. The --dest compact1-minimal option specifies the name of the destination directory containing the newly generated profile.  Note that the directory argument to --dest must not exist prior to invocation.  Finally, the --vm minimal option tells jrecreate to use the minimal (i.e. the smallest) virtual machine for this instance.  The --vm option accepts  [minimal | client | server | all] as an argument.  Running the complete jrecreate.sh command, we get the following output:

$ ./ejdk1.8.0/bin/jrecreate.sh --profile compact1 --dest compact1-minimal --vm minimal
Building JRE using Options {
   ejdk-home: /home/java8/ejdk1.8.0
    dest: /home/java8/compact1-minimal
    target: linux_arm_vfp_hflt
    vm: minimal
    runtime: compact1 profile
    debug: false
    keep-debug-info: false
    no-compression: false
    dry-run: false
    verbose: false
    extension: []

Target JRE Size is 10,595 KB (on disk usage may be greater).
Embedded JRE created successfully

This creates a Compac1 profile distribution of about 10 ½ MB in the compact-1-minimal/ directory.  For our second example, we'll create a profile based on Compact2 and the client VM, this time from a Windows 7/64-bit system:

c:\demo>ejdk1.8.0\bin\jrecreate.bat --profile compact2 --dest compact2-client --vm client
Building JRE using Options {
    ejdk-home: c:\demo\ejdk1.8.0\bin\..
    dest: c:\demo\compact2-client
    target: linux_arm_vfp_hflt
    vm: client
    runtime: compact2 profile
    debug: false
    keep-debug-info: false
    no-compression: false
    dry-run: false
    verbose: false
    extension: []

Target JRE Size is 17,552 KB (on disk usage may be greater).
Embedded JRE created successfully

This Compact2 instance is created in the compact2-client/ directory and has an approximate footprint of 17 ½ MB.  Additional options to jrecreate are available for further customization.

Finally, lets migrate the generated profiles over to a real device.  As a host platform we'll use none other than the ubiquitous Raspberry Pi.  Here's a listing of the two profiles and their size (in 1K blocks) on the filesystem:

pi@pi0 ~/java8 $ ls
compact1-minimal  compact2-client

pi@pi0 ~/java8 $ du -sk compact*
10616   compact1-minimal
17660   compact2-client

And here's what each version outputs when java -version is run:

pi@pi0 ~/java8 $ ./compact1-minimal/bin/java -version
java version "1.8.0"
Java(TM) SE Embedded Runtime Environment (build 1.8.0-b127, profile compact1, headless)
Java HotSpot(TM) Embedded Minimal VM (build 25.0-b69, mixed mode)

pi@pi0 ~/java8 $ ./compact2-client/bin/java -version
java version "1.8.0"
Java(TM) SE Embedded Runtime Environment (build 1.8.0-b127, profile compact2, headless)
Java HotSpot(TM) Embedded Client VM (build 25.0-b69, mixed mode)

In conclusion, you are encouraged to experiment with the EJDK.  It will very quickly give you a feel for the compact profile configuration options available for your device.

Monday Aug 12, 2013

Compact Profiles Demonstrated

Following up on an article introducing compact profiles, the video that follows demonstrates how this new feature in the upcoming Java 8 release can be utilized.  The video:

  • Describes the compact profile feature and the rationale for its creation.
  • Shows how to use the new jrecreate utility to generate compact profiles that can be readily deployed.
  • Demonstrates that even the smallest of profiles (less than 11MB) is robust enough to support very popular and important software frameworks like OSGi.

The software demonstrated is in early access.  For those interested in trying it out before the formal release of Java 8, there are two options:

  1. Members of the Oracle Partner Network (OPN) with a gold membership or higher can download the early access Java 8 binaries of Java SE-Embedded shown here.  For those not at this level, it may still be possible to get early access software, but it will require a qualification process beforehand.
  2. It's not as intimidating as it sounds, you can pull down the source code for OpenJDK 8, and build it yourself.  By default, compact profiles are not built, but this forum post shows you how.  The reference platform for this software is linux/x86.  Functionally, the generated compact profiles will contain the pared down modules for each compact profile, but you'll find the footprint for each to be much larger than the ones demonstrated in this video, as none of the Java SE-Embedded space optimizations are performed by default.

Not having any premium privileges on YouTube, the maximum allowed length of a video is 15 minutes.  There's actually lots more to talk about with compact profiles, including enhancements to java tools and utilities (javac, jar, jdeps, and the java command itself) that have incorporated intelligence for dealing with profiles.

Hmm.  Maybe there's an opportunity for a Compact Profiles Demonstrated Part II?

Wednesday Jul 31, 2013

An Introduction to Java 8 Compact Profiles

Java SE is a very impressive platform indeed, but with all that functionality comes a large and ever increasing footprint.  It stands to reason then that one of the more frequent requests from the community has been the desire to deploy only those components required for a particular application instead of the entire Java SE runtime environment.  Referred to as subsetting, the benefits of such a concept would seem to be many:

  • A smaller Java environment would require less compute resources, thus opening up a new domain of devices previously thought to be too humble for Java.
  • A smaller runtime environment could be better optimized for performance and start up time.
  • Elimination of unused code is always a good idea from a security perspective.
  • If the environment could be pared down significantly, there may be tremendous benefit to bundling runtimes with each individual Java application.
  • These bundled applications could be downloaded more quickly.

Despite these perceived advantages, the platform stewards (Sun, then Oracle) have been steadfast in their resistance to subsetting.  The rationale for such a stance is quite simple: there was sincere concern that the Java SE platform would fragment.  Agree or disagree, the Java SE standard has remained remarkably in tact over time.  If you need any further evidence of this assertion, compare the state of Java SE to that of Java ME, particularly in the mobile telephony arena.  Better still, look how quickly Android has spawned countless variants in its brief lifespan.

Nonetheless, a formal effort has been underway having the stated goal of providing a much more modular Java platform.  Called Project Jigsaw, when complete, Java SE will be composed of a set of finer-grained modules and will include tools to enable developers to identify and isolate only those modules needed for their application.  However, implementing this massive internal change and yet maintaining compatibility has proven to be a considerable challenge.  Consequently full implementation of the modular Java platform has been delayed until Java 9.

Understanding that Java 9 is quite a ways off, an interim solution will be available for Java 8, called Compact Profiles.  Rather than specifying a complete module system, Java 8 will define subset profiles of the Java SE platform specification that developers can use to deploy.  At the current time three compact profiles have been defined, and have been assigned the creative names compact1, compact2, and compact3. The table that follows lists the packages that comprise each of the profiles.  Each successive profile is a superset of its predecessor.  That is to say, the compact2 profile contains all of the packages in compact1 plus those listed under the compact2 column below.  Likewise, compact3 contains all of compact2 packages plus the ones listed in the compact3 column.

compact1                     compact2                    compact3
--------------------------   -----------------------     --------------------------
java.io                      java.rmi                    java.lang.instrument
java.lang                    java.rmi.activation         java.lang.management
java.lang.annotation         java.rmi.registry           java.security.acl
java.lang.invoke             java.rmi.server             java.util.prefs
java.lang.ref                java.sql                    javax.annotation.processing
java.lang.reflect            javax.rmi.ssl               javax.lang.model
java.math                    javax.sql                   javax.lang.model.element
java.net                     javax.transaction           javax.lang.model.type
java.nio                     javax.transaction.xa        javax.lang.model.util
java.nio.channels            javax.xml                   javax.management
java.nio.channels.spi        javax.xml.datatype          javax.management.loading
java.nio.charset             javax.xml.namespace         javax.management.modelbean
java.nio.charset.spi         javax.xml.parsers           javax.management.monitor
java.nio.file                javax.xml.stream            javax.management.openmbean
java.nio.file.attribute      javax.xml.stream.events     javax.management.relation
java.nio.file.spi            javax.xml.stream.util       javax.management.remote
java.security                javax.xml.transform         javax.management.remote.rmi
java.security.cert           javax.xml.transform.dom     javax.management.timer
java.security.interfaces     javax.xml.transform.sax     javax.naming
java.security.spec           javax.xml.transform.stax    javax.naming.directory
java.text                    javax.xml.transform.stream  javax.naming.event
java.text.spi                javax.xml.validation        javax.naming.ldap
java.util                    javax.xml.xpath             javax.naming.spi
java.util.concurrent         org.w3c.dom                 javax.script
java.util.concurrent.atomic  org.w3c.dom.bootstrap       javax.security.auth.kerberos
java.util.concurrent.locks   org.w3c.dom.events          javax.security.sasl
java.util.jar                org.w3c.dom.ls              javax.sql.rowset
java.util.logging            org.xml.sax                 javax.sql.rowset.serial
java.util.regex              org.xml.sax.ext             javax.sql.rowset.spi
java.util.spi                org.xml.sax.helpers         javax.tools
java.util.zip                                            javax.xml.crypto
javax.crypto                                             javax.xml.crypto.dom
javax.crypto.interfaces                                  javax.xml.crypto.dsig
javax.crypto.spec                                        javax.xml.crypto.dsig.dom
javax.net                                                javax.xml.crypto.dsig.keyinfo
javax.net.ssl                                            javax.xml.crypto.dsig.spec
javax.security.auth                                      org.ieft.jgss

You may ask what savings can be realized by using compact profiles?  As Java 8 is in pre-release stage, numbers will change over time, but let's take a look at a snapshot early access build of Java SE-Embedded 8 for ARMv5/Linux.  A reasonably configured compact1 profile comes in at less than 14MB.  Compact2 is about 18MB and compact3 is in the neighborhood of 21MB.  For reference, the latest Java 7u21 SE Embedded ARMv5/Linux environment requires 45MB.

So at less than one-third the original size of the already space-optimized Java SE-Embedded release, you have a very capable runtime environment.  If you need the additional functionality provided by the compact2 and compact3 profiles or even the full VM, you have the option of deploying your application with them instead.

In the next installment, we'll look at Compact Profiles in a bit more detail.


Jim Connors-Oracle


« November 2015