Monday Oct 13, 2008

JDBC Pool Monitoring using Management Rules

After my basic blog blog about creating and writing mbeans and self management rules, here is a post on how to monitor a certain attribute of a jdbc connection pool using Self Management Rule.
I would take for example, the connection validation failures. The Monitoring framework of glassfish provides statistics on various attributes including an attribute specially for the number of connection validation failures. Here are the steps :

1. First of all, enable the monitoring for JDBC Connection Pool to HIGH from the admin console or from CLI.

2. Create JDBC Connection Pool/JDBC Resource for this exercise : conValPool and jdbc/conval respectively.

3. We would be monitoring the attribute for connection validation failure on this pool conValPool.

4. The objectName for this conValPool for monitoring would be "com.sun.appserv:name=conValPool,type=jdbc–connection–pool,
I got this objectName from listing all mbeans and querying the names for conValPool. Note that this mbean is specifically for monitoring this pool since the category is monitor.

5. When i open jconsole and login to it, i see that com.sun.appserv  section, there is a head called conValPool. Under this, we see a server element if we browse down the tree. On the right pane, we can see all the attributes and i would take a note of the attribute with name numconnfailedvalidation–count.
I would use this attribute in my custom mbean event.

6. Create an mbean called ValidationMonitor for getting the value of numconnfailedvalidation–count at any point of time. It would have code that gets the value as follows

MBeanServer server = ManagementFactory.getPlatformMBeanServer();
numConnFailVal = (Long) server.getAttribute(new
    category=monitor,server=server”) , "numconnfailedvalidation–count";)

I use the getAttribute method on the MBeanServer with the above mentioned ObjectName and the attribute name. The numConnFailVal is the attribute that will be returned whenever this mbean is invoked.

7. Create an mbean called ValidationAction for the action part of the management rule. This action would be triggered every time some threshold is crossed as part of the rule that we are going to create shortly. The action as of now is going to be a simple System.out.println  to the server.log as follows

public class ValidationAction implements NotificationListener,
    com.sun.example.action.ValidationActionMBean {
    public synchronized void handleNotification(Notification notification, Object handback) {
         try {
System.out.println("Validation  Failure Threshold Notification: " +
          } catch(Exception ex) {

Note that the action mbean implements the NotificationListener interface.

8. Coming to the part of creating a management rule to hook up this action and event, we log into the admin console and deploy the custom mbeans created above.
Then under the Management Rules link under Configuration, create a new management rule called ValFailRule. Specify the event type as monitor and other values as follows

observed MBean value : user:impl–class–name=com.sun.example.mbeans.ValidationMonitor,

observedAttribute value : numConnFailVal
Monitor type to Counter
Granularity period to 1000 milliseconds so that every second the rule would check for the numConnFailVal value.
Number type to long since the return value from our mbean is of type long.
Init Threshold value to 2 so after 2 failed validations, the rule is going to write something to the
server log.
Offset to 1
Select the validationaction from the list under Action section for the action mbean name.

9. The management rule is successfully setup and once you have a test running that would change the validation failure values, the rule would play its role. Whenever the numConnFailVal value goes above 2, the following can be seen in the server.log

Validation  Failure Threshold[

43f6a09f5ae1be9f0e51;|Logging selfmanagement event :

SEVERE messages are because the log level was set to SEVERE while the rule was created.

This is just an example of how a JDBC pool monitoring attribute can be observed via mbeans and action taken above a threshold using a management rule. Any other meaningful action could be taken instead of the above.

Hope this is useful. :)

Tuesday Nov 06, 2007

Management Rules - observedmbean property

My previous blog on selfmanagement and explain how to create management rules with an example.

Assumptions : 

Same example as in the above link (memory rule) taken. 

Custom Mbeans  Memory (impl-class-name:com.sun.example.mbeans.Memory) and MemoryActionBean (impl-class-name:com.sun.example.action.MemoryAction) are deployed.

Problem : 

In GUI, when a new management rule is created following the steps and with the values :

Observed Mbean : com.sun.examples.mbeans.Memory

Observed Attribute : FreeMemory 

and other values as specified in the example,

the server.log shows an error message that the rule is not configured because of some errors. The log looks like

Error creating and configuring the rule : MemRule due to
IlegalArguments specified for its event type : monitor. Create the
rule with correct arugments for the configured event.|#]

Solution :

Modify the domain.xml under install_directory/domains/domain1/config/ directory - edit the management rule MemRule :

Change the line

property name="observedobject" value="com.sun.example.mbeans.Memory"


property name="observedmbean" value="Memory"

Restart the server after this change. And you will see that the error message goes away. Note here that "observedobject" is changed to "observedmbean" and the new value is the name of the custom mbean deployed with the impl-class-name : com.sun.example.mbeans.Memory.

Explanation :

In GUI, the Observed Mbean takes values like

This value is translated into "observedobject" property in domain.xml. Hence if you specify "com.sun.example.mbeans.Memory", in domain.xml you will see this value against the property "observedobject". 

Very few know that there is another property called "observedmbean" that accepts the mbean name directly. But in GUI, there is no explicit way of specifying this observedmbean property. One has to enter the whole object name for the input "Observed Mbean" in GUI, which will get translated into "observedobject" in domain.xml. Alternatively, domain.xml can be modified with observedmbean property.

Whereas in CLI, you could just specify the parameter "observedmbean"
with value "Memory" in the command line, without even specifying the "observedobject" property.

Ex: asadmin create-management-rule --port 4848 --user admin_user
--action  MemoryActionBean --eventtype monitor --eventloglevel WARNING --eventproperties
<blah-blah>:observedmbean=Memory:<blah-blah> rule_name

This is an advantage CLI has over GUI.  


Thursday Jan 19, 2006

GlassFish : Self Management Rules

Self Management is an initiative taken in GlassFish to reduce the complexity of the management tasks. Mundane management tasks are automated by means of self-configuring. System Administrators who are responsible for monitoring the application server for unpredictable conditions no longer have to detect them manually. This is an easy-to-use feature that comes in handy at the unpredictable run-time conditions. This feature enables to recover from failures by detecting and preventing them, thus improving availability (self-healing). Even security threats are detected and self-protective actions are taken in response. The framework has been developed using standard technologies.

Self Management comprises of a set of management rules. You could create different rules based on the various event types. As mentioned at, the different type of events are as follows.
  • Lifecycle events
  • Monitor events
  • Log events
  • Trace events
  • Timer events
  • Notification events
This blog is more about how to create management rules easily. I will show examples for some of the different type of rules. The action we take in response to the occurrence of an event, could be anything from sending an email notification to displaying an alert or killing some processes depending on what is appropriate for the particular event.

Let us take an example of displaying a pop-up window as an alert. Action mBeans implement the NotificationListener interface. The handleNotification() method for our action will look like the following.

    public synchronized void handleNotification(Notification notification,Object handback) {
        try {
            JFrame. setDefaultLookAndFeelDecorated(true);
            JFrame myFrame = new JFrame("MyAction");
            myFrame. setSize(250, 180);
            myFrame. setLocation(300,200);
            myFrame. getContentPane().add( new Label("Action has been invoked",Label.CENTER), BorderLayout.CENTER);
            myFrame. setVisible(true);
            myFrame. show();
        } catch (Exception ex) { // Log the exception }

Let us name this class as CustomAction which is in the com.sun.myrule.action package. It has a corresponding CustomActionMBean. Follow these steps to deploy the action on the application server.

1. Compile the files and make a jar out of these two files.

2. After starting the application server, open the admin console by typing http://localhost:4848 on a browser.

Figure1: Admin Console

3. On the left pane, click on "Custom MBeans". On the right pane, all the deployed mbeans are listed. To deploy our action mbean, click on Deploy. Select the jar file that you just created against the "File to upload" column and click Next.

Figure2: Custom MBeans

Figure3: Creating Custom MBeans

4. Enter a name for the mbean. This will later be used while creating the rule. Enter name as "CustomAction".

5. Enter the class name along with the package name. In this case, it is com.sun.myrule.action.CustomAction.

Figure4: Creating Custom MBeans: Step 2

6. Click Finish.

The action mbean is successfully deployed on the application server.

We will use the same action mbean for all the rules. This action mbean will be called each time an event is triggered. For more information on actions, refer to actions section in

Now let us proceed and create rules.

Simple rules for creating a rule:

1. Identify the event type you need. For example : lifecycle
2. Identify the mandatory and optional properties for that event type. For example : lifecycle event requires a property called "name".
3. Identify the action mbean you would like to use for your rule.
4. You could use the Application server Command line interface (CLI) or the Graphical user interface (GUI) to create rules.
5. Make sure the server is started before proceeding with rule creation.

Example 1 : Assuming the administrator wants to be notified everytime the application server shuts down or is terminated because of a failure. Continuous monitoring is necessary for this kind of a situation. Therefore, a rule could be created mentioning a notification to be sent to the administrator when this event is triggered. Management rule bundles together the event and the action.

Let us examine how this rule could be created.

After starting the application server with the command "asadmin start-domain", use either CLI or GUI to create the rule.

CLI mode :

/bin/asadmin create-management-rule
--host localhost --port 4848 --user admin
--action CustomAction
--eventtype lifecycle
--eventloglevel WARNING --eventproperties name=shutdown

GUI mode :

  • Start admin console by typing http://hostname:port on a browser, after starting the application server.
  • Go to the Management Rules tab under the Configurations tab in the left pane. This displays the list of rules deployed on the server on the right pane.

  • Figure5 : Management Rules

  • Click on New
  • Type the name of the rule as "my_lifecycle_rule".
  • Check the Status checkbox to enable the rule.
  • Enter description
  • Choose event type as lifecycle.
  • Click on Next if you are done with this page.

  • Figure6 : New Management Rule

  • In the next page, select "shutdown" against Event and enter a description if desired.
  • Select "CustomAction" listed in the action list box and click Finish.

Figure7 : New Management Rule : Step 2

The rule is successfully created and is deployed on the application server. The action will be triggered everytime the application server shuts down. Similarly, rule could be configured to take some action on ready and termination.

Example 2 : The administrator wants to be beeped once in a while regarding some status. A rule could be created to do this as follows.

CLI mode :

/bin/asadmin create-management-rule
--host localhost --port 4848 --user admin
--action CustomAction
--eventtype timer
--eventloglevel INFO
--eventproperties pattern=MM\\\\/dd\\\\/yyyy\\\\/HH\\\\:mm\\\\:ss:

Note here that "/" and ":" are escape sequences. Period, mentioned in milliseconds, is the period after which the notification is to be sent after the date mentioned in datestring.

GUI mode :
  • Start admin console by typing http://hostname:port on a browser, after starting the application server.
  • Go to the Management Rules tab under the Configurations tab in the left pane. This displays the list of rules deployed on the server on the right pane.
  • Type the name of the rule as "my_timer_rule".
  • Check the Status checkbox to enable the rule.
  • Enter description
  • Choose event type as timer.
  • Click on Next if you are done with this page.
  • Enter the values for datestring, pattern and other fields.
  • Select "CustomAction" listed in the action list box and click Finish.
Another example of a rule to monitor the runtime memory of the JVM can be found here .




« April 2014