Thursday Nov 28, 2013

Java Mission Control - Production Time Profiling Out of the Box

By Guest blogger: Marcus Hirt

With the release of 7u40, there is a new kid on the JDK tooling block – Java Mission Control. This article will try to explain why it is worthwhile to take a closer look at this technology, as well as provide pointers on how to get started. 

History

Mission Control is a production time profiling and diagnostics tools suite that originated with the tooling available for the JRockit JVM. 

As part of the development effort of the JRockit JVM we started building tooling for analyzing the runtime performance of the JVM. We needed information about how real production systems were utilizing JRockit. Requests to get customers to lend us their latest top secret trading applications for evaluation in-house were usually, quite understandably, met with telling blank stares calling our sanity into question. It probably wouldn’t have done us much good anyways, since we wanted production data from systems under real loads. Thus we built a tool (the JRockit Runtime Analyzer, which later evolved into the JRockit Flight Recorder) with low enough overhead that we could convince customers to actually use it to collect production time data for us.

Eventually we accidentally solved some customer problems using the tools and customers started asking us if they could license them. The idea was born to spend some more resources on the tooling and make it a commercial tool to pay for its development. JRockit Mission Control was born.

Java Mission Control

After Oracle acquired Sun Microsystems, Oracle suddenly had two of the top three most commonly used general purpose JVMs available on the market. One was the open sourced reference JVM, with a lot of people knowing the source and licensees basing their own ports and versions of JVM off of it. The other JVM, whilst being a quick and pretty little thing, was proprietary with a rather small number of people knowing the code base. Instead of having to support two JVMs, Oracle wanted to pool the available resources into building a best of breed JVM. It was decided that the base would be HotSpot and that the most useful features in JRockit would be ported over – one of them being Mission Control.


In JDK 7u40 the functionality available in HotSpot had reached critical mass, and the very first version of Java Mission Control (unfortunately versioned 5.2.0) was released. It mainly contains equivalents of two of the JRockit Mission Control tools – the JMX Console and the Flight Recorder. There is no on-line heap analyzer yet. There is however a set of quite useful (experimental) plug-ins for JMC, extending Mission Control to do heap dump analysis, targeted analysis for various oracle products or simply extending existing functionality in more (or sometimes less – yes, I’m looking at you Twitter Trigger Action) useful ways .

Getting Started

Starting Java Mission Control is quite easy. Download a recent enough Java 7 JDK (7u40 or later), then simply launch %JDK_HOME%/bin/jmc. The alien thing that now starts is not, as I am sometimes asked, a native application. It’s Java, but it’s built upon Eclipse RCP technology. If you would rather run JMC inside of Eclipse, you can install JMC into your Eclipse from the JMC update site. A word of caution here – because of a bug in Eclipse/SWT, Mission Control performance is horrible in Eclipse 4.x. This bug is slated to be be fixed in Eclipse 4.4, but until then I strongly recommend either using the stand-alone version of JMC, or installing it into an Eclipse 3.8.2. 

The JMX Console

The console in Mission Control can be thought of as a JConsole on steroids. It allows you to monitor JMX data in various ways, to take action when attributes attain certain values and to persist the data and later look at what has been recorded. There are various experimental plug-ins for the console, such as a Coherence plug-in, a plug-in for running JConsole plug-ins and a plug-in for tweeting messages when a action triggers. 

To connect the console to a JVM, simply choose the JVM you want to connect to in the browser tree and select Start JMX Console. If the JVM was started locally or with JDP, then it will automatically appear in the JVM browser. If you have a remote JVM without JDP running, just enable the built in JMXRMI agent as you normally would to be able to connect with JMX clients such as JConsole.

The JMX console is typically used to monitor a small set of critical attributes, such as the CPU load and Java Heap usage, sampled at a relatively low frequency. The console can be configured to take action when undesirable values are reached for an attribute, and one of those actions can be to dump Flight Recorder data. The JMX console also contains special tabs for looking at thread information, such as deadlocked threads, per thread allocation information and per thread profiling information. That said, the JMX console is used for monitoring the runtime. When profiling capabilities or better diagnostic information is needed, the one stop shop is really the Flight Recorder.

The Java Flight Recorder

The Java Flight Recorder can be thought of as of the equivalent to the Data Flight Recorder for an airplane, but for the Java runtime. While it is running It records information about the JVM and its environment. When something “interesting” happens, the data in the Flight Recorder can be dumped, and the information analyzed off-line to gain an understanding of why everything suddenly went from good to “interesting”. Having the Flight Recorder running as an almost unnoticeable impact on the performance of the Java Application running in the JVM. The overhead is usually well below a per cent. This is achieved by having a high performance recording engine built directly into the runtime and collecting data already being tracked by the runtime or as part of an activity where the data is reachable. (As opposed to actively having to do additional work to get the data.) There are a lot of interesting things that can be said about the recording engine implementation, but this being an overview article, I’ll just move on to how to use it, not how it does what it does.

Creating Flight Recordings

The most important difference to how the Flight Recorder worked in JRockit is that in HotSpot two JVM start-up flags must be enabled on the JVM for which you want to do flight recordings:

-XX:+UnlockCommercialFeatures -XX:+FlightRecorder

And that was probably the most important line in this article. 

There are two different types of recordings, and you can have multiple recordings (of different types) running simultaneously:

1. Timed recordings 

These recording run for a pre-configured duration. They are automatically stopped when the time is up. If initiated from JMC, they will be automatically downloaded and opened in the user interface when done. 

2. Continuous recordings 

These recordings have no explicit end time and must be dumped by the end user. 

Now there are three different ways you can do actual recordings, once the parameters are in place:

1. From Mission Control. This is probably the easiest way. Just point and click. 

2. From jcmd. This is a very useful way to control the Flight Recorder from the command line. Quite useful when you can’t access the machine that is running the JVM of interest from Mission Control and you only have access to a shell. 

3. Using command line flags. This is handy when you want to always run with a continuous recording, or when you want to record the start up behaviour of the JVM right from the very start. 

If you want to know more about how to create Flight Recordings, this blog entry is probably a good place to start. 


Analyzing Flight Recordings

There is a lot of useful information in the flight recordings, and there are a lot of different things the information can be used for. For example:

Method profiling. The Flight Recorder will quite happily do method profiling in production systems with a very low overhead. As a matter of fact, it’s even enabled in the continuous template, so go ahead and use it. It will tell you where the hotspots are in your application. In other words, if you have a CPU bound problem, the method profiling information will tell you where to optimize to get things to go faster. 

GC profiling. The GC implementations emit useful events at GC related activity. Information that can be used to check on the live set, semi-refs, GC pauses (and their individual sub-phases) etc. Quite useful for GC tuning, finding out if you’re overusing finalizers and more. 

Allocation profiling. If you do notice a lot of garbage collections, but don’t notice anything strange about the individual GC phases, you may want to kick back a bit on the allocation. Allocation profiling will help you see where all that allocation activity is putting it’s toll on the memory system. 

WebLogic Server analysis. WebLogic is producing its own set of events for the Flight Recorder. They are quite useful in their own right, but can also be good for putting all the other information recorded in a context - “What was really happening during this transaction?”. This article on the Operative Set shows some of the capabilities. 

Latency Profiling. The Flight Recorder has a lot of different events for various thread stalling activities that can occur, such as blocking on monitor enter, parking, waiting etc. I can’t believe I haven’t written a blog post on this yet. Shame on me. This is usually the first place to look if you haven’t got a CPU bound problem, but still performance issues. 

OS information. CPU load, JVM CPU load, environment variables, running processes – there is a lot of operating system information. If you still can’t find what you’re looking for, Mission Control has a D-Trace plug-in for retrieving everything you ever wanted to know, but were too afraid to ask. Note that the overhead from using D-Trace, even with very few probes, is usually more than an order of magnitude worse than just using the Flight Recorder – use with caution. 

There is much more information available from the event providers built into the JVM, such as class loading and compiler events. One way to learn more details about what is available is to take a closer look at the metadata from a recording.

Summary

Since JDK 7u40 there is a new tools suite bundled with the JDK – Java Mission Control. The main focus of the suite is on production time profiling and diagnostics. This has the benefit that the data gathered is quite true to the dynamics of the application being profiled, as the observer effect is kept quite low. In other words, instead of profiling the profiler itself, most of the time is actually spent profiling the application and the runtime. Whilst the main focus of Mission Control is production systems, it can be quite useful in development too. It is also free for use in development, as per the standard Oracle Binary Code License (BCL).

This article provided a brief introduction to Java Mission Control.

Further Information

Java Mission Control Homepage

My Blog

@javamissionctrl;  @hirt


Tuesday Nov 26, 2013

Brian Goetz Offers a View Into Lambda Development

After delivering the Devoxx keynote, Java Chief Language Architect Brian Goetz talks to OTN.  He takes on the question of why lambda expressions took so long to implement and how lambdas will impact developers.  

Monday Nov 25, 2013

Lego, Robots and Java!

In this video, Java Evangelist Angela Caicedo talked about her new Duchess robot built with Lego Mindstorm. "We are moving into the Internet of Things and I am looking for cool devices that we can connect and use Java" said Angela   

The robot is connected to touch and color sensors, has a gyroscope to orient and prevent it from falling, 2 motors for the wheels and another for arms, and an infrared sensor for the remote control. 

Technical Article: JavaFX Integration Strategies

"You will rarely find isolated applications in the enterprise," says Adam Bien in the new OTN technical article entitled JavaFX Integration Strategies. Instead, an enterprise desktop application renders and manipulates the data of one or more back-end services exposed by an application server. This article focuses on the integration of Java EE services with JavaFX applications. It discusses synchronous and asynchronous communication with a JavaEE backend. LightFish and LightView were used as examples.

Author Adam Bien is a Java Champion, architect and Java developer. He is an Expert Group member for the Java EE 6 and 7, EJB 3.X, JAX-RS, and JPA 2.X JSRs.  He is the author of Real World Java EE Patterns—Rethinking Best Practices and Real World Java EE Night Hacks—Dissecting the Business Tier, and all around good guy.

Read JavaFX Integration Strategies article.

Want JDK8 to be Great?

Uncle DukeDo you want JDK8 to be great? Duke needs *your* help to find issues and find them now! JDK 8 B116 EA Build is now available for download & test.

As Java 8 gets closer to the endgame we all need to help test and report issues. So please, add it as a JVM to your CI build, try running it at home and have a play with Lambdas!

You can get more details and resources on all of this at http://adoptopenjdk.java.net

As Java Champion Martijn Verburg said, "If you want Lambdas nice and early then testing Java 8 now *really* helps the whole Java community get a great release on day one."

Friday Nov 22, 2013

Java Magazine: The Internet of Things for Java Developers

Futuristic, automated homes are becoming a reality, and home automation is a prime example of the Internet of Things (IoT). The current issue of Java Magazine looks at IoT and what it opportunities it presents for Java developers. We also bring you a special section on development best practices, community news, and more.

Highlights include:

Smartest House on the Street
The Internet of Things meets home automation with Java-based openHAB.

Internet of Things 101
Explore concepts relevant to the Internet of Things by creating an embedded application. 

Special Section: Development Best Practices 
Articles on Agile, performance tuning, concurrency, DevOps, and team dynamics.

Concurrency Utilities for Java EE
Learn how to execute task using application server concurrency services.

JCP Interview
Patrick Curran The JCP chair on charting the future with JCP.next

Java Magazine is a FREE, bi-monthly, online publication. It includes technical articles on the Java language and platform; Java innovations and innovators; JUG and JCP news; Java events; links to online Java communities; and videos and multimedia demos. Subscriptions are free, registration required.

Do you have feedback about Java Magazine? Send a tweet to @oraclejavamag.

Thursday Nov 21, 2013

Coding at Internet of Things (IoT) Hack Fest

Devoxx IoT Hack Fest offered the chance for attendees to code with Java Embedded for multiple devices. Attendees built embedded applications using devices and sensors in just a couple of days. A pre-configured panel, created by Globalcode founder Vinicius Senger, a long-time Java embedded developer and trainer, connects Raspberry Pi, Arduino, Beaglebone black, and Gemalto boards to a series of sensors such as distance, motion, alcohol and temperature sensors, as well as relays and camera. Those components are accessible remotely via a web server and REST calls. A new addition to IoT Hack Fest at Devoxx was the Leap Motion. ZeroTurnaround senior developer Geert Bevin lent nine Leap Motion controllers, gave a training session and was the onsite mentor for the Leap Motion.  

During two days, attendees learned and hacked the panel and the Leap Motion. For example, Hartmut Lang hacked a sensor distance and the blink(1) component with Java Embedded Suite, Jersey API. He turned on a ThingM blink(1) multi-colored USB light via the proximity sensor on the Java Embedded Panel. The proximity data is accessible through a Java sever running on a Raspberry PI with an Arduino bridge. REST calls give access to the current sensor data. This was then translated to Java calls that control the intensity of the blink(1) light. Everything was programmed in Netbeans and was running on a dedicated Raspberry PI.

A number of projects were about controlling the relays using Java SE Embedded, Pi4J and JavaFX. They connected their applications to the panel with the Raspberry Pi's and relays. Others worked with JavaFX and Raspberry Pi. They created a distance measurement using Fuzzy Logic, Arduino, Java Embedded Suite, RXTX, Things API and JavaFX to control lights. Thomas Kriechbaum worked with Java SE Embedded, Apache Camel, MQTT for Android, MQTT with Camel and Mosquito to control actuators and reading sensors. Geert Van Landeghem built an animation framework for LCD displays using Java SE Embedded, Pi4J/I2C and LCDPlate Adafruit

Senior software architect Claude Falguière created an interactive game to teach kids the principle of programmed operations. This is done by leading a robot through a maze. Changing the directions of the robot is done via hand gestures with the leap Motion controller. Each direction change is recorded as a card and shown as the list of operations. These can be manipulated and started again.

An ant-chasing game, written in Scratch used the Leap Motion plugin with a Java application for the gesture-interpreting gateway. Players moved their fingers around in the air and an ant chased its movements. 

Geert Bevin created an application controlling multi-colored led strip with hand gestures. The Leap Motion controller detects hand position in 3D-space and sends the X/Y/Z data straight to the led strip to control the intensity of the red/green/blue lights. The led strip was connected to a Raspberry PI with an Arduino bridge and could be piloted through REST calls running on an embedded Java server. In this video, Geert explains the project.  

More projects online at the IoT Community on Java.net

Wednesday Nov 20, 2013

More JavaOne Sessions Online

A new batch of JavaOne sessions are now online for you to view.  From Hadoop to M2M, and even Minecraft, these sessions give you the best technical information. My favorite from the latest batch: The Seven Deadly Sins of Java EE Projects by Markus Eisele.  

You can also download the JavaOne app from Parleys (for iPad or Android) to view the JavaOne sessions both online and offline.

We'll be rolling out new sessions until the end of the year, so check back often for the latest content. It's all free, brought to you by the Oracle. Check them out!

Monday Nov 18, 2013

Are You Ready for the Next Java SE Update?

Oracle has created two new resources, the 
Java RIA Security Checklist and the Java Security Resource Center to help you prepare for the next Java SE update, Java SE 7 update 51 (scheduled for January 2014). This release changes the deployment requirements for Applet & Web Start applications with two new requirements: 

  1. Use of the Permissions Manifest attribute
  2. Valid code signatures. 

These changes will not affect developers of back-end or client applications; the scope is limited only to Java Applets & Web Start Applications (RIAs). Read details in the blog  New security requirements for RIAs in 7u51.

Java RIA Security Checklist


The changes scheduled for Java 7 update 51 means the default security slider will require Code Signatures and the Permissions Manifest attribute. Java RIA Security Checklist
 provides best practices to help development teams track work necessary to accommodate user prompts.

Security Resource Center


Oracle has launched a new Java Security Resource Center to aggregate security-related information for the Java community based on your role: developer, system administrator, home user, or security professional.

Additional Resources

Note:
 To ensure that end users’ systems are secure when using Java-based content, Oracle strongly recommends that you always upgrade to the most recent release. You can remove old versions of Java either during upgrades or by using the Java Uninstall Tool on Java.com.

Thursday Nov 14, 2013

Devoxx HackFest In-Action Attracts Developers

by guest blogger Geert Bevin, Senior Developer, ZeroTurnaround. Geert Bevin presented the Leap Motion at IoT Hack Fest

The Raspberry PI and Leap Motion hands-on labs at Devoxx 2013 started with a packed room, we even had to fetch additional chairs for people standing in the back. Vinicius and Yara senger explained their super useful embedded devices panel where many sensors and boards can be accessed through easy REST URLs.



After my quick overview of the Leap Motion controller, the attendees could experiment with all available devices and get acquainted with it before starting a hack. Surprisingly though, only a couple of developers were interested to stick around for the real hack-fest that lasted the rest of the day.

 We set up everything upstairs in the Devoxx open-space and started hacking with the brave few that showed up. As the day advanced though, more and more people came by and were impressed to see the work in progress. On Tuesday evening all the tables were packed with hackers testing and building cool stuff. Ten of them were able to finish their project and some even presented it on video for the Oracle Technology Network YouTube channel.


I had a great time and it's wonderful to see how easily accessible physical embedded devices speak to the imagination of so many developers. 

Here's my hack that lets you control a strip of multi-colored lights that are connected to Arduino over a Raspberry Pi and are remotely controlled with the Leap Motion controller over the network.



About

Insider News from the Java Team at Oracle!

duke
javeone logo
Links


Search

Archives
« November 2013 »
SunMonTueWedThuFriSat
     
1
2
3
5
7
8
9
10
15
16
17
19
23
24
27
29
30
       
Today