Friday Jul 11, 2008

VBScript XML processing to extract JMX content

In a previous post I have presented the details of the VBScript script used to interact with a JMX application in which a Web Services Connector was deployed.

In order to deal with XML namespaces, I wrote a really horrible function based on the knowledge of the namespace prefix name computation. Something that I presented as a very bad way, caused by my lack of expertise in VBScript.

Since this previous post, I have written a bunch of scripts and dived into some VBScript librairies details. I have found the right way to process XML and deal with Namespaces. This is done thanks to XPATH usage. By setting a set of properties on the object returned by CreateObject("Microsoft.XMLDOM"), you can inject prefix/namespace associations. These associations are then usable when selecting XML nodes.

Retrieving the WS-Enumeration enumeration context of an MBean subscription

WS-Man session creation and subscription invocation is detailed in the previous post. The enumeration context must be extracted as follow:
set reply = ...

Dim objXMLDoc
set objXMLDoc = CreateObject("Microsoft.XMLDOM")

' Make the reply to be loaded synchronously
objXMLDoc.async = False

' Load the reply 
objXMLDoc.loadXML(reply)

' Make XPath the selection dialect
objXMLDoc.setProperty "SelectionLanguage", "XPath"

' Associate "wsen" prefix to WS-Enumeration namespace.
objXMLDoc.setProperty "SelectionNamespaces", "xmlns:wsen='http://schemas.xmlsoap.org/ws/2004/09/enumeration'"

' There is a single enumeration node, selectSingleNode returns the first one.
Dim node
set node = objXMLDoc.selectSingleNode("//wsen:EnumerationContext")

' node.text contains the enumeration context value
dim enumContext 
set enumContext = node.text

Retrieving the MBean Notifications pulled from the server

Notification pulling is detailed in the previous post. Extracting all the Messages contained in the list of notifications is done as follow:
set reply = ...
dim replyXml
set replyXml= CreateObject("Microsoft.XMLDOM")
replyXml.async = False
replyXml.loadXML(reply)

' Make XPath the selection dialect
replyXml.setProperty "SelectionLanguage", "XPath"

' Associate "jmx" prefix to JMX Web Services Connector namespace.
replyXml.setProperty "SelectionNamespaces", "xmlns:jmx='http://jsr262.dev.java.net/jmxconnector'"

' Select all the Notification Message nodes text values
set msgList = replyXml.selectNodes("//jmx:TargetedNotification/jmx:Message/text()")

' Display the Messages
for i = 0 To (msgList.length - 1)
    set msg = msgList(i)
    ' Display the notification message
    WScript.echo "Notification : " & msg
next

Conclusion

VBScript API to extract XML content is efficient and simple to use. It is very helpful when extracting values from complex data structures (such as CompositeData or MBeanInfo). Obviously its usage is much wider than JMX interoperability.

Hope this help.

Jean-Fran├žois

Thursday Mar 13, 2008

RESTful Access to JMX Instrumentation, Via URI-fication of MBean Attributes

As you perhaps know from reading my posts, I am involved in the definition of JSR 262, Web Services Connector for JMX Agents. JSR 262 offers an access to Web Services that is very far removed from Representational State Transfer (RESTful) Web Services. So, this blog is not about JSR 262, but instead it describes some of the investigations and development I have performed related to the Java API for RESTful Web Services (JAX-RS API) and the JMX API.

Having simple HTTP-based access to a Java Management Extensions (JMXTM) MBean is an idea that I like a lot and one for which I think that there are thousands of real use cases. For example, when developping a Web application, integrating a piece of information obtained from JMX instrumentation, such as the status of a service, should be as simple as loading an HTML page.

My first attempt was to define and implement a merge of JSR 262 and a RESTful approach... All in vain! The two technologies cannot co-habit, so you are forced to choose between them. You either have to go down the WS-\* path or you have to go down the RESTful path. There's no way to make one leverage the other to offer a tidy solution. In this case, keeping the two parts separate is much better than trying to bundle them together. In this investigation, I have limited the scope just to retrieving information from JMX instrumentation. Setting MBean attributes or invoking operations has not been addressed (yet).

June 2008. UPDATE TO THE ORIGINAL ENTRY. I added description for support of notifications and snapshot of values in this new entry.

If you want to play with this prototype for RESTful access to JMX instrumentation (based on Jersey, which you will need to download) before reading the blog entry, you can jump straight to Running the Adaptor. This RestFul JMX Adaptor is hosted by the OpenDMK java.net project.

But first, let's start with a bit of background. The next sections describe how the information from JMX instrumentation is accessed, and how this information can then be mapped into a format that is acceptable to the RESTful approach.

JMX Instrumentation Information Tree

So, how does JMX instrumentation present its information? First, we have an MBean server in which MBeans are registered. Each MBean contains a set of attributes that represent an actual piece of information. MBean attributes are not always exactly actual pieces of data, though, as this is dependent upon the Java class of the attribute. If the attribute is a primitive type, then yes, the attribute is indeed an actual piece of data. If it is a complex type (or a collection), then it is not. Complex types need to be traversed to discover the actual data.

So, we can represent the whole set of information exposed by the JMX instrumentation as a tree. The root of the tree is the MBean server, the MBeans are the root's child nodes and the attributes are the MBeans' child nodes. Some attributes are terminal nodes (primitive nodes), some other attributes (complex types) are intermediary nodes.

The following figure presents the JMX instrumentation information tree:


In this tree, we can see the data path as leaves.


Here, we can identify three elements:

  1. The leaf node, which is a data container.

  2. The intermediary node, which is a child container.

  3. The data path, which is composed of the set of parent/child relationships from the root to the leaf.

Traversing Complex Types

We just saw that you need to traverse complex types to access the actual data. However, how to deal with every sort of complex type that can be defined is outside the scope of these investigations. So, we have limited ourselves to Array, List, Set and the generic JMX types CompositeData (which is very similar to Map) and TabularData (which is a table of CompositeData).

The above list of types complies with MXBean usage, so we are able to go quite far, even with this limited set. These types are traversed in different ways, as follows:

  • To traverse a List or an Array, you need to iterate the structure.

  • To traverse a CompositeData, you need to find out each key and retrieve the associated value.

  • To traverse a TabularData, you need to iterate the structure and then traverse each CompositeData (as described above).

Mapping the Information Tree to RESTful Web Services

I am not making a revolutionary statement by saying that nodes are resources and data paths are URIs. In our information tree, we have two kinds of resources, container resources that are referenced by other resources, and the content resources. These resources can be identified by URIs, as explained in the next sections.

Resource URIs

The different resources in our information tree are identified by the following URIs:

  • MBean server: http://<host>:<port>/<root context>

  • MBean: http://<host>:<port>/<root context>/<MBean ObjectName>

  • MBean attribute: http://<host>:<port>/<root context>/<MBean ObjectName>/<attributeName>

  • Collection (list or array) element: http://<host>:<port>/<path to complex type>/<index>

  • Map element: http://<host>:<port>/<path to complex type>/<URL encoded(key.toString)>

  • CompositeData entry: http://<host>:<port>/<path to complex type>/<key>

  • TabularData element: http://<host>:<port>/<path to complex type>/<index>

Container Resources

MBean servers, MBeans, arrays, lists, sets, CompositeData entries and TabularData elements are all container resources. A simple XML definition is enough to represent these resource types. To avoid defining a new XML element, we can reuse the XHTML list element (<ul><li></li></ul>).

References to a child can either be relative to the current resource or absolute. I chose to make them absolute. Below are some examples of representations of different container resources.

  • MBean server containing two MBeans:

    <ul>
    <li>http://<host>:<port>/<root context>/java.lang:type=Memory</li>
    <li>http://<host>:<port>/<root context>/java.lang:type=Thread</li>
    </ul>
  • MBean with two attributes, NonHeapMemoryUsage and HeapMemoryUsage:

    <ul>
    <li>http://<host>:<port>/<root context>/java.lang:type=Memory/NonHeapMemoryUsage</li>
    <li>http://<host>:<port>/<root context>/java.lang:type=Memory/HeapMemoryUsage</li>
    </ul>
  • CompositeData, composed of commited and init keys:

    <ul>
    <li>http://<host>:<port>/<root context>/java.lang:type=Memory/NonHeapMemoryUsage/commited</li>
    <li>http://<host>:<port>/<root context>/java.lang:type=Memory/NonHeapMemoryUsage/init</li>
    </ul
Note: You will notice that I am not using the HTML link tag (<a href="..."></a>). This is because I am not defining HTML access (such as that provided by the JMX technology's HTML adaptor), but rather XML access.

Container resources like those above are not of particularly great interest to us. They help with navigation when you are having difficulties finding your way through the information tree, but when you know where you want to go, they are almost useless. When you know where you want to go you don't need to navigate at all, you just jump at light speed to the end of the path by providing the full URI.

However, in some cases where the information is dynamic, container resources can be really helpful.

Content Resources

In this prototype, actual data is represented by straightforward plain text. For example, Hello world, is plain text...

A Restful Adaptor for JMX Instrumentation

I have developed a prototypal implementation of the above mapping using the JAX-RS Reference Implementation (a.k.a the Jersey project). The adaptor attaches to the platform MBean server that is located inside any Java Virtual Machine (JVMTM) from Java Development Kit (JDK) version 5.0 or above. The adaptor offers URI-based navigation, and the returned pages are in either HTML or XML format, according to the receiving client's "Accept Headers" setting.

The current prototype covers all the types exposed by the JVM software's MXBean interfaces (used for monitoring memory, threads, classes, etc.). If your MBeans are also MXBeans, they you should be able to navigate to them without problems. I re-used the HTML <ul><li> tags to handle lists in XML. In HTML, the <a href="..."> tag is used in conjonction with the <li> tag in order to make clickable URLs.

The adaptor provides an API (JMXWSMBeanRestResource#setListXMLTags) that allows you to change these list tags and provide your own.

If your MBeans expose unsupported types, you can provide your own resources by calling JMXWSMBeanRestResource#registerResourceHandler. Look at the source code for JMXWSMBeanRestResource.CollectionResource to understand how the adaptor's JAX-RS resources are defined.

JAX-RS Resources

JAX-RS allows you to define “top level” resources that return sub-resources. I defined a top level resource named JMXWSMBeanRestResource, which serves as the JMX adaptor's entry point and handles the MBean server and the MBean resource types. It also delegates the navigation of attribute values to sub-resources. The current sub-resources are the following:

  • PlainResource

  • ArrayResource

  • CollectionResource

  • MapResource

  • CompositeResource

  • TabularResource

Each resource handles a data type and, in turn, delegates it to other sub-resources to reach the leaf (the plain text resource).

The JAX-RS resource / sub-resource pattern allows for very simple tree navigation. Each resource is annotated using JAX-RS annotations. The following ArrayResource shows what a sub-resource actually does:

public class ArrayResource { 

/\*\* Returns the list of URI that point to array items. \*\*/

@GET

@ProduceMime("text/xml")

public Response getXMLItemURIs() throws Exception {

return getURIs();

}

/\*\* Returns the Sub resource pointed by the array index. ResourceFinder is a reference to the JMXWSMBeanRestResource that manages Sub Resources. \*\*/

@Path("collection/{index}")

public Object getItem(@UriParam("index") String index) throws Exception {

int idx = Integer.valueOf(index).intValue();

return resourceFinder.findResource(Array.get(array, idx));

}

...

}

That's about it for the introduction. Now you can see the adaptor in action.

Running the adaptor

  1. You need first to do a CVS checkout of the Restful Adaptor sources from OpenDMK java.net project

    1. export CVSROOT=:pserver:guest@cvs.dev.java.net:/cvs

    2. cvs co opendmk/contributions/rest-adaptor

    3. Open the NetBeans project rest-adaptor/RestAdaptor.

  2. Follow the steps documented in Restful Adaptor readme file.

  3. Run the adaptor (this requires a minimum of JDK version 5.0, although JDK 6 is recommended).

  4. The adaptor will display the URL on which it is bound, as well as some other information, such as the composition of the URIs and the names of the MBeans.

  5. In your browser type the following URL : http://localhost:9999/rest/jmx.

  6. If your browser is set to accept XML headers by default over HTML headers (as is the case for Firefox), you will receive the resources in XML form, which is not really the best form for a browser. However, you can still navigate by copying and pasting the URIs until you reach a leaf. The best thing to do is to change your Firefox settings to accept HTML headers by default over XML headers, as follows:

    1. In your browser, type about:config

    2. Go to network.http.accept.default

    3. Prefix the value with text/html.

    You can now navigate from the root to the leaves.

But, as I said previously, navigation is really not the best way. What you really want to do is to jump to a value using a well-known URL. But where are these well known URLs? They are there already, waiting for you. If you want to see them all, go to the following URI:

http://localhost:9999/rest/jmx/all

This URI references a special resource that lists ALL the leaves of a JMX information tree for a given MBean server. All listed URLs identify an actual piece of data.


This list of URLs (that can be very long) can be filtered by using a URI parameter that contains the following JMX specification ObjectName pattern: http://localhost:9999/rest/jmx/all?pattern={your jmx ObjectName pattern}

Well Known URLs

In a system being managed by JMX instrumentation, the set of exposed MBeans is generally well defined. The set of MBean ObjectName instances is well known and documented so that clients can access the exposed information. We can take advantage of this and select a set of URLs that will never change for a given system. For example, the URIs that identify the commited heap memory of the JVM software from JDK version 5.0 or version 6 will never change, except for the host and port deployment information, as shown below :

http://<host>:<port>/rest/jmx/java.lang:type=Memory/NonHeapMemoryUsage/commited

This file contains the URIs you expect to find in a JVM implementation from JDK version 6, regardless of your context. For monitoring and management of the JDK software, the URLs that change are related to Array and TabularData (i.e. thread IDs, system properties, logger names, etc.) and to memory managers that are not identical as far as the hardware architecture is concerned.

You can easily discover these URLs without navigating through the tree, by starting at index 0 and incrementing by 1 until you reach a 404 error. For example, to retrieve all the thread IDs, you just have to increment the {index} :

http://<host>:<port>/rest/jmx/java.lang:type=Threading/AllThreadIds/{index 0-N}

The example of thread IDs shows up a major limitation of this approach when the information is very dynamic (as threads are). The array size and elements can change in such a way as makes the returned values invalid. In this case we would need a snapshot of the array.

June 2008. UPDATE TO THE ORIGINAL ENTRY. I added description for support of notifications and snapshot of values in this new entry.

Conclusion

This RESTful adaptor provides a possible way of implementing a RESTful access to JMX instrumentation. I have a strong feeling that it could be very helpful when aggregating loosely coupled pieces of information. The current adaptor does not address all data structures (although MXBeans are fully handled), but it is provided in Java source code and offers an API for you to add your resource handlers.

Possible usages:

  • Develop AJAX based dashboads to monitor Java VM and/or your applications.

  • Quickly construct HTML pages to offer human readable access to the internals of your systems.

What are the next steps:

  • Add a support for Notifications.

  • Add a support for POST/PUT.

  • Add a support for snapshot of values.

I would like to thank Paul Sandoz for is great help. REST philisophy is Paul's second nature. Expose him your problem and he will Restify it. Really smart!

I would be very happy to receive any feedback about this approach. Feel free to contact me and thank you for reading.

Regards.

Jean-François Denise

Saturday May 05, 2007

JMX Web Services Connector is out, in WS-Man dress.


JMX Web Services Connector (JSR 262) RI is out! You can download it here.

The novelety in comparaison with the previous Early Access is that the Connector protocol is now based on WS-Management (WS-Man). Connector WS-Man implementation is based on the WiseMan project.

Native WS-Man clients can now interact with MBeans. MBeans are no more the exclusivity of JMX clients. Clients such as Windows Remote Manager (WinRM) and openwsman, can retrieve MBean attributes, call MBean operations and subscribe to MBean notifications.

JSR 262 specification defines a mapping between JMX MBeans and WS-Man Managed Resources. More details can be found in the Early Draft 2 specification. Have a look to the spec and send us your feedbacks.

Next Thursday, at JavaOne, during Eamonn and myself Technical Session, we are presenting an interoperability demo that highlights these new JMX interoperation capabilities. Demo is composed of three scenario:
  • HP Open View Operations / JSR 262 (done by Denis Rachal from HP)
  • Microsoft WinRM Scripting for JSR 262
  • Microsoft Vista Event Viewer to subscribe to MBeans
See you there!

Jean-Fran├žois
About

jeanfrancoisdenise

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today