Friday Jul 11, 2008

VBScript XML processing to extract JMX content

In a previous post I have presented the details of the VBScript script used to interact with a JMX application in which a Web Services Connector was deployed.

In order to deal with XML namespaces, I wrote a really horrible function based on the knowledge of the namespace prefix name computation. Something that I presented as a very bad way, caused by my lack of expertise in VBScript.

Since this previous post, I have written a bunch of scripts and dived into some VBScript librairies details. I have found the right way to process XML and deal with Namespaces. This is done thanks to XPATH usage. By setting a set of properties on the object returned by CreateObject("Microsoft.XMLDOM"), you can inject prefix/namespace associations. These associations are then usable when selecting XML nodes.

Retrieving the WS-Enumeration enumeration context of an MBean subscription

WS-Man session creation and subscription invocation is detailed in the previous post. The enumeration context must be extracted as follow:
set reply = ...

Dim objXMLDoc
set objXMLDoc = CreateObject("Microsoft.XMLDOM")

' Make the reply to be loaded synchronously
objXMLDoc.async = False

' Load the reply 
objXMLDoc.loadXML(reply)

' Make XPath the selection dialect
objXMLDoc.setProperty "SelectionLanguage", "XPath"

' Associate "wsen" prefix to WS-Enumeration namespace.
objXMLDoc.setProperty "SelectionNamespaces", "xmlns:wsen='http://schemas.xmlsoap.org/ws/2004/09/enumeration'"

' There is a single enumeration node, selectSingleNode returns the first one.
Dim node
set node = objXMLDoc.selectSingleNode("//wsen:EnumerationContext")

' node.text contains the enumeration context value
dim enumContext 
set enumContext = node.text

Retrieving the MBean Notifications pulled from the server

Notification pulling is detailed in the previous post. Extracting all the Messages contained in the list of notifications is done as follow:
set reply = ...
dim replyXml
set replyXml= CreateObject("Microsoft.XMLDOM")
replyXml.async = False
replyXml.loadXML(reply)

' Make XPath the selection dialect
replyXml.setProperty "SelectionLanguage", "XPath"

' Associate "jmx" prefix to JMX Web Services Connector namespace.
replyXml.setProperty "SelectionNamespaces", "xmlns:jmx='http://jsr262.dev.java.net/jmxconnector'"

' Select all the Notification Message nodes text values
set msgList = replyXml.selectNodes("//jmx:TargetedNotification/jmx:Message/text()")

' Display the Messages
for i = 0 To (msgList.length - 1)
    set msg = msgList(i)
    ' Display the notification message
    WScript.echo "Notification : " & msg
next

Conclusion

VBScript API to extract XML content is efficient and simple to use. It is very helpful when extracting values from complex data structures (such as CompositeData or MBeanInfo). Obviously its usage is much wider than JMX interoperability.

Hope this help.

Jean-Fran├žois

Tuesday Jun 03, 2008

RESTful Access to JMX Instrumentation, second part

This post is an answer to William Vambenepe who recently wrote an interesting post related to JSR 262 and the REST access to JMX I discussed a few months ago. William shows some interest in the REST adaptor for JMX and asks for the next steps. Here they are...


A bit of context

The REST approach is a possible solution for interaction with JMX agents mostly due to the fact that monitoring and management is increasingly present in our systems at a much higher level than it used to be.

The data exposed tends to be directly understandable. There is no need for complex processing on the client side to understand what is going on. Why is this possible? I think that it comes from the fact that the new trend in Monitoring and Management consoles (consoles such as JConsole or Visual VM), is to display "raw" data.

This raw data is interpreted by only viewing or displaying it graphically. It makes MBean designers think more and more about their MBean interfaces as human-readable APIs. My colleague Daniel Fuchs shows the limits of this approach in a recent blog entry. But I also remember Daniel saying a few years ago that people will soon start to design MBeans to comply with JConsole's capabilities. I think that he was absolutely right. This is what is emerging now.


So it seems an appropriate time to build an HTTP based access, along standardized lines, to allow web designers to benefit from this valuable management information by making it possible to embed it directly in Web pages.


My previous post put in place the foundations of the REST adaptor for JMX agents. This post covers the next features that should be added in the near future. I write “should be added” because I would like to hear your reactions to them before I add them to the current implementation. Please let me know whether or not they will meet any of your requirements, so that we can discuss how to improve them. The adaptor will be much more valid if it answers the needs of the community.


Security

HTTPS and Basic AUTH seem natural choices. Nothing special, simple and safe. They should cover most cases. Is Basic AUTH enough or do we also want to offer “mutual authentication”, to allow the server to authenticate clients based on client certificates? What about access control? We could easily add access control based on JMX permissions as is done today when you provide an access and password file when starting the JVM.


Snapshot of values

Currently we can't retrieve a complex MBean attribute atomically. Say that we want to retrieve all the values contained in the HeapMemoryUsage attribute of the Memory MBean. This attribute is of type CompositeData. The well-known URLs (that point to plain text data) associated to this attribute are the following:


http://curcuma:9999/rest/jmx/java.lang:type=Memory/HeapMemoryUsage/committed

http://curcuma:9999/rest/jmx/java.lang:type=Memory/HeapMemoryUsage/init

http://curcuma:9999/rest/jmx/java.lang:type=Memory/HeapMemoryUsage/max

http://curcuma:9999/rest/jmx/java.lang:type=Memory/HeapMemoryUsage/used


Currently, we can only retrieve committed, init, max and used individually.

If we introduce a new query parameter named capture, we could ask the adaptor to construct a frozen view of an attribute value (or more generally of a sub-tree). We could then retrieve the values located in the snapshot.


In practice:

GET http://curcuma:9999/rest/jmx/java.lang:type=Memory/HeapMemoryUsage?capture=mysnapshot1


Returned resource :

<ul>
<li>http://curcuma:9999/rest/snapshots/mysnapshot1/jmx/java.lang:type=Memory/NonHeapMemoryUsage/commited</li>
<li>http://curcuma:9999/rest/snapshots/mysnapshot1/jmx/java.lang:type=Memory/NonHeapMemoryUsage/init</li>
<li>http://curcuma:9999/rest/snapshots/mysnapshot1/jmx/java.lang:type=Memory/NonHeapMemoryUsage/max</li>
<li>http://curcuma:9999/rest/snapshots/mysnapshot1/jmx/java.lang:type=Memory/NonHeapMemoryUsage/used</li>
</ul>



The returned URLs reference the captured data. We now have the freedom to analyze the values outside the ever-changing real world. This pattern could also be applied higher up in the tree (at the MBean level, for example) to create a snapshot of multiple attributes.


Notifications

We could map from JMX Notifications to atom feeds. This could be a good approach. However, here I would like to keep it simple and carry on with the resource-centric approach and try not to break the model. I would like to make the number of required technologies as minimal as possible. Relying on HTTP(S) with a bit of HTML/XML should be the maximum we allow ourselves... ;-)


So, how can we proceed? JMX MBeans are the event sources. Clients generally subscribe to MBean notifications then pull notifications from the MBean server. Pulling seems very appropriate in our web context. Pushing has never really worked.

If you want reliable, zero-loss notifications then you should stop reading this post. You are going to be frightened by what I am about to say. In JMX we generally see notifications as an unreliable way to receive information. You can't count on a high level of reliability. Lost notifications are not common but they can occur. Once you know the context in which notifications are handfed, you can design an event system that copes with it. Making your MBean re-emit notifications until an acknowledgement comes back can be a decent solution.


If we agree with what has been said previously, we can attach a rolling buffer to each MBean resource that contains the last n emitted notifications (10 is a possible default value). This buffer can be pointed to using the URL scheme we have already put in place.

Here is an example URL for retrieving notifications:

http://curcuma:9999/rest/notifications/jmx/java.lang:type=Memory/



If events occur, the list of emitted notifications is returned in the GET response.

<ul>
    <li>
       Notification 1 
    </li>
    <li>
       Notification 2
    </li>
    ...
    <li>
       Notification N
    </li>
</ul>

Representation of notifications

A Notification is a complex type for which we can define a set of well known URLs. By nature, a notification is a snapshot of data, and its URL contains the information that makes it unique. The JMX API doesn't guarantee a unique sequence number. The emission time seems a better identifier.


<ul>
  <li>http://curcuma:9999/rest/notifications/jmx/java.lang:type=Memory/25_12_1970_12_00_00/message</li>
  <li>http://curcuma:9999/rest/notifications/jmx/java.lang:type=Memory/25_12_1970_12_00_00/number</li>
  <li>http://curcuma:9999/rest/notifications/jmx/java.lang:type=Memory/25_12_1970_12_00_00/type</li>
  <li>http://curcuma:9999/rest/notifications/jmx/java.lang:type=Memory/25_12_1970_12_00_00/userdata/<...possible path if complex type></li>
   ... user data URL continue if complex content.
</ul>

Custom notifications

If the emitted notification is custom-built, and some getters are exposed, then the added field is discovered and exposed. For example, the following notification defines a getPriority field:


public class PriorityNotification extends Notification {
  ...
  public int getPriority() {
    ...
  }
}

This will be returned as follows:

<ul>
 <li>http://curcuma:9999/rest/jmx/notifications/java.lang:type=Memory/25_12_1970_12_00_00/priority</li>
 <li>http://curcuma:9999/rest/jmx/notifications/java.lang:type=Memory/25_12_1970_12_00_00/message</li>
 <li>http://curcuma:9999/rest/jmx/notifications/java.lang:type=Memory/25_12_1970_12_00_00/number</li>
 <li>http://curcuma:9999/rest/jmx/notifications/java.lang:type=Memory/25_12_1970_12_00_00/type</li>
 <li>http://curcuma:9999/rest/jmx/notifications/java.lang:type=Memory/25_12_1970_12_00_00/userdata/<...possible path if complex type></li>
</ul>



Filtering based on emission time

You could argue that only keeping 10 notifications is insufficient and that all notifications should be stored. OK, but in this case, we would like to be able to filter out the old ones. That can be done by adding a since query parameter when getting notifications. For example:

http://curcuma:9999/rest/notifications/jmx/java.lang:type=Memory/?since=20_12_1970_00_00

We just need to agree on a date format.

Notification retrieval scope

This approach allows us to express a homogenous "retrieval scope”. By choosing a location in the tree (a URL where we can get notifications), we can retrieve notifications emitted by :

  • MBean servers (all MBeans, and potentially all attributes too, could add a JMX pattern to narrow the scope)

  • MBeans

  • MBean attributes (equivalent to AttributeChange).

  • Information inside an attribute value (namely, is it a real scope, and do we want it?)



Conclusion

It seems that things are still being kept simple here without compromising the JMX technology's philosophy. Or perhaps not, and you think it is already too complicated and I should stop mining this seam... just let me know and I will down tools! ;-)

What do we have on the plate for the next post? Setting attributes and invoking MBean operations. If you have some ideas, please let me know. They are tough ones!

Thanks.

Jean-François Denise

jean-francois.denise@sun.com



Friday May 23, 2008

Use VBScript to cheat at poker with JMX!

This year at JavaOne Eamonn and I presented during our technocal Session, where we stand with respect to JMX. Eamonn covered JMX in general and JMX 2.0, while I covered the Web Services Connector for JMX. At the end of the talk, we performed a demo named “CSPoker, JMX Technology, Java VisualVM and WinRM at the JavaOne Tournament's Final Table”. We linked together a set of JMX related technologies to offer, in an original Online Poker Web Site use case, an interesting setup highlighting the power of JMX (here's a PDF file of the demo architecture) .

At the end of the talk, some people asked me some questions about the WS-Management access to JMX we had just demonstrated. More details on the WS-Management to JMX interoperability can be found in this article.

Here, I am providing some details on the actual script used during the demo. This script automates the Poker Engine monitoring and management tasks. It first registers to listen for JMX Notifications exposed as WS-Eventing notifications. Each time a “Poker table joined” notification is received, it checks that the IP address from which the player joined is not already known to the system. If an IP address is already known, it means that another user already joined from the same host.... which is very bad if you want to avoid the same player using multiple identities....

When such a “bad user” is detected, the script ejects all the players that joined from that machine.

Intialisation of the WS-Management access

dim wsmanObj
set wsmanObj = CreateObject("WSMAN.Automation")
dim objConnectionOptions
set objConnectionOptions = wsmanObj.CreateConnectionOptions
dim iFlags
iFlags = wsmanObj.SessionFlagUseNoAuthentication
dim session
set session = wsmanObj.CreateSession("http://localhost:8080/admin", iFlags)

At this point, you have a WS-Management proxy (the session object), that allows you to interact with a JMX Agent attached to the Web Services Connector.


Subscription to the MBean notifications

The MBean we want to subscribe to is named : cspoker:type=CSPokerControl

Because the WS-Management session doesn't offer a nice API for subscriptions, we need to construct the subscription invocation request using the “all purpose” invoke method :


Dim reply
reply = session.invoke("http://schemas.xmlsoap.org/ws/2004/08/eventing/Subscribe", _
"http://jsr262.dev.java.net/DynamicMBeanResource?ObjectName=cspoker:type=CSPokerControl", _
"<wse:Subscribe xmlns:wse='http://schemas.xmlsoap.org/ws/2004/08/eventing'>"_
& "<wse:Delivery Mode='http://schemas.dmtf.org/wbem/wsman/1/wsman/Pull'/></wse:Subscribe>")

At this point we have sent a WS-Eventing subscription request (in PULL mode, meaning that the client will send a request to pull notifications from the server) and received a response. The response contains the WS-Eventing context to be used to retrieve notifications.

Retrieving the Eventing context

To get the eventing context from the response, we need to make use of the VBScript DOM library : Microsoft.XMLDOM

Warning : Because namespaces are not handled well in this DOM library, we are trying to discover the Namespace prefix based on our knowledge of the way they are declared (ns<i>). I am not a VBScript expert and would be very interested to know if there is an equivalent to getElementsByTagNameNS using VBScript???

Function findElement(elem, tagElem)
' wscript.echo "Finding Element " & tagElem
 for i = 0 to 15
  set findElement = elem.getElementsByTagName("ns"& i &":" & tagElem)
  If findElement.Length > 0 Then Exit For
 next
End Function

Dim objXMLDoc
set objXMLDoc = CreateObject("Microsoft.XMLDOM")
objXMLDoc.async = False
objXMLDoc.loadXML(reply)

' Get the Eventing context to use in the next WS-Eventing Pull request
Dim nodeList
set nodeList = findElement(objXMLDoc.documentElement, "EnumerationContext")
dim context 
context = nodeList(0).text
wscript.echo "Subscription Enumeration Context " & context

At this point, the context variable contains the WS-Eventing context that identifies our subscription. This context is to be used to retrieve notifications.

Retrieving the Notifications

This is done by sending a Pull request to the server and providing it the context. The pull request blocks until some Notifications are emitted or the timeout (1 minute by default) is reached. As for subscribe requests, no Pull API is offered. We need to use the general purpose invoke operation. We have written a pull function that handles pull request timeouts properly. When the timeout occurs, instead of exiting the script, the error is trapped and the script continues. Because we expect multiple notifications, the Notification pulling is done in a loop.

Function pull(session, context)
 On Error Resume Next
Dim pullXml
pullXml = "<wsen:Pull xmlns:wsen='http://schemas.xmlsoap.org/ws/2004/09/enumeration'><wsen:EnumerationContext>" & context & "</wsen:EnumerationContext><wsen:MaxTime>PT1M0.000S</wsen:MaxTime><wsen:MaxElements>1000</wsen:MaxElements></wsen:Pull>"
 pull = session.invoke("http://schemas.xmlsoap.org/ws/2004/09/enumeration/Pull", "http://jsr262.dev.java.net/MBeanNotificationSubscriptionManager", pullXml)
  On Error Goto 0
End Function

' This piece of script is called in an infinite loop

Dim notifs

 notifs = pull(session, context)

At this point, the notifs variable contains (or does not contain, if the timeout occurred) the JMX notifications in an XML format.

Extracting the Notification content and tracking the cheat

We only do this extraction if the list of notifications is not null. The VBScript DOM library is again used to parse the notifications. Because a pull request can return multiple notifications, we must iterate on all TargetedNotification elements contained in the notification list. A TargetedNotification is an XML representation of a JMX Notification defined by the JSR 262 (JSR in which the Web Services Connector is defined).

In our case, the TargetedNotification/Message XML Element contains the Player name. The TargetedNotification/UserData XML Element contains the IP Address as a string (eg:<jmx:String>192.168.0.1</jmx:String>)

To keep the IP <==> Player relationship, we use a Map data structure and array. In VBScript, we use a dictionnary (scripting.dictionary Libray)

' Create a Map
dim dict
set dict = CreateObject("scripting.dictionary")
Dim ipArray(10)
Dim arrayIndex
arrayIndex = 0

if Not(isNull(notifs)) then

  set objXMLNotif = CreateObject("Microsoft.XMLDOM")
  objXMLNotif.async = False
  objXMLNotif.loadXML(notifs)
  set NodeListNotifs = findElement(objXMLNotif.documentElement, "TargetedNotification")
  WScript.echo "Pull returned " & NodeListNotifs.length & " notifications"

  ' Now that we get the list of notifications, extract the content.
  Dim i
  Dim EventType
  Dim msgObj
  ' Loop over the received Notifications
  For i = 0 To NodeListNotifs.length - 1
      Dim listMsg
      set listMsg = findElement(NodeListNotifs(i), "Message")
      Dim eventMessage 
      eventMessage = listMsg(0).text

      Dim listUserData
      set listUserData = findElement(NodeListNotifs(i2), "String")
      ' We have an IP address
      if(listUserData.length > 0) then
           Dim ipAddress
           ipAddress = listUserData(0).text
           wscript.echo eventMessage & " Joined From " & ipAddress
           if dict.exists(ipAddress) then
              Dim prev
              prev = dict.item(ipAddress)
              if Not StrComp(prev,eventMessage) Eqv 0 Then 
                 dim users
                  users = Array(eventMessage, prev)
                  For Each user In users
                    wscript.echo user & " already connected from same host [" & ipAddress &"], will eject him." '"Type return."
                    wscript.echo "Ejecting player " & user
                    ejectPlayer session, user
                    wscript.echo "Player " & user & " ejected"
                  Next
              end if
           else
             dict.add ipAddress, eventMessage
           end if
      else
         wscript.echo eventMessage
      end if
  Next

Ejecting the player

The function ejectPlayer calls the ejectPlayer MBean operation by making use of the general purpose invoke operation.

Function ejectPlayer(session, playerName)
 dim pokerURI
 pokerURI = "http://jsr262.dev.java.net/DynamicMBeanResource?ObjectName=cspoker:type=CSPokerControl"
 dim xmlInvoke
 xmlInvoke = "<jmx:ManagedResourceOperation name=" & Chr(34) & "ejectPlayer" & Chr(34) & " xmlns:jmx=" & Chr(34) & "http://jsr262.dev.java.net/jmxconnector" & Chr(34) & "><jmx:Input><jmx:Param><jmx:String>" & playerName & "</jmx:String></jmx:Param></jmx:Input></jmx:ManagedResourceOperation>"
 session.invoke "http://jsr262.dev.java.net/DynamicMBeanResource/Invoke", pokerURI, xmlInvoke
End Function


Hope that these VBScript extracts helped you understand better how you can use this path to interoperate with JMX.

Thanks.

Jean-François Denise

jfd@sun.com




        
    
About

jeanfrancoisdenise

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today