Friday Jun 20, 2008

RESTful is Rastafarian thinking!

I am back from The Server Side Java Symposium (TSSJS) that occurred in Prague. I was invited to speak about JMX, Monitoring, Management and Troubleshooting in the JDK 6 platform. I spoke 1 hour and enjoyed the conference (only 2 days, I had to leave Prague before the end of the conference).

Some people went to me at the end of my talk with questions related to the ability to persist information that Visual VM or JConsole tools are exposing. My next post will cover this topic in details and should come with a Conf Dumper helping you achieve this use case.

TSSJS is a fantastic conference (it was the first time that I participated). Really it is. It is full of excellent talks from free and open minded speakers. It has been a delight to attend (more particularly) Ted Neward and John Davis talks.

I had already planed to leave Prague not before the promising Ted's REST and SOAP: Arch Enemies or BFF? presentation. And I have been well advised to do so...

Ted has simply presented us the Genesis of the Web Services... From the first second to the last one, it has been a flow full of excellent comparisons, well chosen anecdotes, funny paradoxes, well targeted criticisms, technical excellence, hope, joy, anger,...

The paroxysm has been reached when he has compared Restful with the way Rastafarians see human/business relationships. He described such relations as "cool point to point ones, no intermediary (notion that doesn't exist for Rastafari), nothing expected back, simply giving... exactly like an HTTP URL..." I can't re-transcript the demonstration but it was so true...

When he ended his talk by "Questions?", it has been, for me, like your grand father, seated nearby the fire camp, ending counting you his famous frightening story...

Ted had transformed a technical debates onto a Computing Tale...

Furthermore, I fully adhere with his analysis. He summarized the novelty of the REST approach by "the ability to identify values of your software thanks to URL". This is exactly what I have tried to describe in the 2 parts of the RESTful Access to JMX Instrumentation discussions (and more precisely by the notion of Well Known URL).

So to conclude, if one day you have the chance to go to the TSSJS, please do so, you will be not disappointed.

Really not.

Jean-Francois Denise

Monday Jun 16, 2008

OpenSolaris on VirtualBox, small tip to make network to work on XP host

This week I will do a presentation on JDK 6 Monitoring, Management and Troubleshooting. It takes place during The ServerSide Java Symposium in Prague.

In order to demonstrate JDK 6 command line tools (jps, jstat, jmap, jinfo, jstack, ...) that offer more features when running on Unix systems, I installed VirtualBox and run an image of OpenSolaris.

In 10 clicks (and a bit of patience to make OpenSolaris to install), I was able to run JDK 6 u6 on OpenSolaris from my little XP Laptop. Everything worked well except the network... very sad when you need to download 300+ Megs of tools and runtime...

What went bad? I simply chose all the default values from the installer. Nothing special. Doing so, you get OpenSolaris guest running behind a NAT. This is the best choice when you don't want your guest to be known from your network. So what was wrong?

Default DNS configuration... Simple to fix, difficult to find (when you are not installing OS every day).

To fix it:

  • You must be root. Switching to this user should be no problem. You installed the machine. So you know the root password.
  • Save the existing /etc/nsswitch.conf
  • Edit /etc/nsswitch.conf file
  • Replace the line ipnodes: files by ipnodes: files dns mdns
  • Replace the line hosts: files by hosts: files dns mdns

And you are done. No need to reboot. Just access to the WEB and it will work.

BTW, if you want to read the best paper ever written on JDK 6 troubleshooting, please read this one. AWESOME!
The entry point for JDK 6 troubleshooting is this URL : http://java.sun.com/javase/6/webnotes/trouble/

See you in Prague!

Jean-Fran├žois Denise

Tuesday Jun 03, 2008

RESTful Access to JMX Instrumentation, second part

This post is an answer to William Vambenepe who recently wrote an interesting post related to JSR 262 and the REST access to JMX I discussed a few months ago. William shows some interest in the REST adaptor for JMX and asks for the next steps. Here they are...


A bit of context

The REST approach is a possible solution for interaction with JMX agents mostly due to the fact that monitoring and management is increasingly present in our systems at a much higher level than it used to be.

The data exposed tends to be directly understandable. There is no need for complex processing on the client side to understand what is going on. Why is this possible? I think that it comes from the fact that the new trend in Monitoring and Management consoles (consoles such as JConsole or Visual VM), is to display "raw" data.

This raw data is interpreted by only viewing or displaying it graphically. It makes MBean designers think more and more about their MBean interfaces as human-readable APIs. My colleague Daniel Fuchs shows the limits of this approach in a recent blog entry. But I also remember Daniel saying a few years ago that people will soon start to design MBeans to comply with JConsole's capabilities. I think that he was absolutely right. This is what is emerging now.


So it seems an appropriate time to build an HTTP based access, along standardized lines, to allow web designers to benefit from this valuable management information by making it possible to embed it directly in Web pages.


My previous post put in place the foundations of the REST adaptor for JMX agents. This post covers the next features that should be added in the near future. I write “should be added” because I would like to hear your reactions to them before I add them to the current implementation. Please let me know whether or not they will meet any of your requirements, so that we can discuss how to improve them. The adaptor will be much more valid if it answers the needs of the community.


Security

HTTPS and Basic AUTH seem natural choices. Nothing special, simple and safe. They should cover most cases. Is Basic AUTH enough or do we also want to offer “mutual authentication”, to allow the server to authenticate clients based on client certificates? What about access control? We could easily add access control based on JMX permissions as is done today when you provide an access and password file when starting the JVM.


Snapshot of values

Currently we can't retrieve a complex MBean attribute atomically. Say that we want to retrieve all the values contained in the HeapMemoryUsage attribute of the Memory MBean. This attribute is of type CompositeData. The well-known URLs (that point to plain text data) associated to this attribute are the following:


http://curcuma:9999/rest/jmx/java.lang:type=Memory/HeapMemoryUsage/committed

http://curcuma:9999/rest/jmx/java.lang:type=Memory/HeapMemoryUsage/init

http://curcuma:9999/rest/jmx/java.lang:type=Memory/HeapMemoryUsage/max

http://curcuma:9999/rest/jmx/java.lang:type=Memory/HeapMemoryUsage/used


Currently, we can only retrieve committed, init, max and used individually.

If we introduce a new query parameter named capture, we could ask the adaptor to construct a frozen view of an attribute value (or more generally of a sub-tree). We could then retrieve the values located in the snapshot.


In practice:

GET http://curcuma:9999/rest/jmx/java.lang:type=Memory/HeapMemoryUsage?capture=mysnapshot1


Returned resource :

<ul>
<li>http://curcuma:9999/rest/snapshots/mysnapshot1/jmx/java.lang:type=Memory/NonHeapMemoryUsage/commited</li>
<li>http://curcuma:9999/rest/snapshots/mysnapshot1/jmx/java.lang:type=Memory/NonHeapMemoryUsage/init</li>
<li>http://curcuma:9999/rest/snapshots/mysnapshot1/jmx/java.lang:type=Memory/NonHeapMemoryUsage/max</li>
<li>http://curcuma:9999/rest/snapshots/mysnapshot1/jmx/java.lang:type=Memory/NonHeapMemoryUsage/used</li>
</ul>



The returned URLs reference the captured data. We now have the freedom to analyze the values outside the ever-changing real world. This pattern could also be applied higher up in the tree (at the MBean level, for example) to create a snapshot of multiple attributes.


Notifications

We could map from JMX Notifications to atom feeds. This could be a good approach. However, here I would like to keep it simple and carry on with the resource-centric approach and try not to break the model. I would like to make the number of required technologies as minimal as possible. Relying on HTTP(S) with a bit of HTML/XML should be the maximum we allow ourselves... ;-)


So, how can we proceed? JMX MBeans are the event sources. Clients generally subscribe to MBean notifications then pull notifications from the MBean server. Pulling seems very appropriate in our web context. Pushing has never really worked.

If you want reliable, zero-loss notifications then you should stop reading this post. You are going to be frightened by what I am about to say. In JMX we generally see notifications as an unreliable way to receive information. You can't count on a high level of reliability. Lost notifications are not common but they can occur. Once you know the context in which notifications are handfed, you can design an event system that copes with it. Making your MBean re-emit notifications until an acknowledgement comes back can be a decent solution.


If we agree with what has been said previously, we can attach a rolling buffer to each MBean resource that contains the last n emitted notifications (10 is a possible default value). This buffer can be pointed to using the URL scheme we have already put in place.

Here is an example URL for retrieving notifications:

http://curcuma:9999/rest/notifications/jmx/java.lang:type=Memory/



If events occur, the list of emitted notifications is returned in the GET response.

<ul>
    <li>
       Notification 1 
    </li>
    <li>
       Notification 2
    </li>
    ...
    <li>
       Notification N
    </li>
</ul>

Representation of notifications

A Notification is a complex type for which we can define a set of well known URLs. By nature, a notification is a snapshot of data, and its URL contains the information that makes it unique. The JMX API doesn't guarantee a unique sequence number. The emission time seems a better identifier.


<ul>
  <li>http://curcuma:9999/rest/notifications/jmx/java.lang:type=Memory/25_12_1970_12_00_00/message</li>
  <li>http://curcuma:9999/rest/notifications/jmx/java.lang:type=Memory/25_12_1970_12_00_00/number</li>
  <li>http://curcuma:9999/rest/notifications/jmx/java.lang:type=Memory/25_12_1970_12_00_00/type</li>
  <li>http://curcuma:9999/rest/notifications/jmx/java.lang:type=Memory/25_12_1970_12_00_00/userdata/<...possible path if complex type></li>
   ... user data URL continue if complex content.
</ul>

Custom notifications

If the emitted notification is custom-built, and some getters are exposed, then the added field is discovered and exposed. For example, the following notification defines a getPriority field:


public class PriorityNotification extends Notification {
  ...
  public int getPriority() {
    ...
  }
}

This will be returned as follows:

<ul>
 <li>http://curcuma:9999/rest/jmx/notifications/java.lang:type=Memory/25_12_1970_12_00_00/priority</li>
 <li>http://curcuma:9999/rest/jmx/notifications/java.lang:type=Memory/25_12_1970_12_00_00/message</li>
 <li>http://curcuma:9999/rest/jmx/notifications/java.lang:type=Memory/25_12_1970_12_00_00/number</li>
 <li>http://curcuma:9999/rest/jmx/notifications/java.lang:type=Memory/25_12_1970_12_00_00/type</li>
 <li>http://curcuma:9999/rest/jmx/notifications/java.lang:type=Memory/25_12_1970_12_00_00/userdata/<...possible path if complex type></li>
</ul>



Filtering based on emission time

You could argue that only keeping 10 notifications is insufficient and that all notifications should be stored. OK, but in this case, we would like to be able to filter out the old ones. That can be done by adding a since query parameter when getting notifications. For example:

http://curcuma:9999/rest/notifications/jmx/java.lang:type=Memory/?since=20_12_1970_00_00

We just need to agree on a date format.

Notification retrieval scope

This approach allows us to express a homogenous "retrieval scope”. By choosing a location in the tree (a URL where we can get notifications), we can retrieve notifications emitted by :

  • MBean servers (all MBeans, and potentially all attributes too, could add a JMX pattern to narrow the scope)

  • MBeans

  • MBean attributes (equivalent to AttributeChange).

  • Information inside an attribute value (namely, is it a real scope, and do we want it?)



Conclusion

It seems that things are still being kept simple here without compromising the JMX technology's philosophy. Or perhaps not, and you think it is already too complicated and I should stop mining this seam... just let me know and I will down tools! ;-)

What do we have on the plate for the next post? Setting attributes and invoking MBean operations. If you have some ideas, please let me know. They are tough ones!

Thanks.

Jean-François Denise

jean-francois.denise@sun.com



About

jeanfrancoisdenise

Search

Categories
Archives
« June 2008 »
SunMonTueWedThuFriSat
1
2
4
5
6
7
8
9
10
11
12
13
14
15
17
18
19
21
22
23
24
25
26
27
28
29
30
     
       
Today