Custom Coherence Monitoring and Reporting

In this post I'll present an example about how to perform custom management measurements in Coherence and then build custom reports based on that measurements.


Example Scenario: In this example I am performing some sort of coherence benchmark. Several operations (insert, update and continuous query) are performed against the coherence cache. I am measuring the time results for this operations and then I am providing the custom Mbeans with the results. Example topology: 1 coherence storage enabled nodes, 1 proxy node, 3 coherence extend clients and 1 management node.

Based on the custom MBeans I am building custom reports with Coherence JMX Reporter.


The necessary steps for achieving this are:

I. Custom Configuration of Coherence Mbeans

1. Create and configure Coherence Management Node

2. Create a Custom Mbean

a. create a new package in your project called "mbeans"

b. create a generic mbean interface class

c. implement the PerformanceMBean interface

d. inherit the Performance class for each operation that you want to measure

d. add Mbeans classes to the Coherence Management node classpath

e. create a custom Mbean XML configuration file

f. specify the name and location of a custom MBean configuration file to the coherence management node

3. Connect to the JMX custom Mbeans to update them with the latest results

4. Use JConsole to watch your custom Mbeans

II. Create a custom report

1. Localise and extract your custom reports configuration files

2. Configure the management node to load your custom report configuration files

3. Create and configure a new configuration file for each report that you want to be generated

4. Add your new report configuration file to the general Coherence JMX Reporter configuration file


All this steps are detailed bellow.

I. Custom Configuration of Coherence Mbeans

1. Create and configure Coherence Management Node

Coherence can be configured to load platform and standard MBeans.

To manage a Coherence cluster using JMX, ensure that you have the necessary JMX 1.0 or later classes (javax.management.*) in the classpath of at least one Coherence cluster node, known as an MBeanServer host. The cluster nodes that are not MBeanServer hosts will be managed by the MBeanServer host(s) by using the Coherence Invocation service.


Management node start file:


JAVA_OPTS="-Xms$MEMORY -Xmx$MEMORY -Dtangosol.coherence.management.report.configuration=$TEST_HOME/reportsIntel/report-group.xml -Dtangosol.coherence.override=$TEST_HOME/config/tangosol-management-coherence-override-dev.xml -Dtangosol.coherence.management.remote=true; -Dtangosol.coherence.management.report.autostart=true -Dtangosol.coherence.mbeans=mBeans.xml -Dtangosol.coherence.management=all

"


$JAVAEXEC -server -showversion $JAVA_OPTS -cp "$COHERENCE_HOME/lib/coherence.jar:$TEST_HOME/build/classes:$TEST_HOME/config:$TEST_HOME/lib/jmxri.jar:$TEST_HOME/lib/jmxtools.jar" com.tangosol.net.management.MBeanConnector -rmi


com.tangosol.net.management.MBeanConnector -rmi = Start a Coherence cluster node that enables JMX agents such as a web browser or JConsole to connect to the in-process JMX MBeanServer and manage Coherence MBeans. -rmi is used when connecting via Jconsole.


2. Create a Custom Mbean

For more information about standard Mbeans please access the following documentation:

http://download.oracle.com/javase/tutorial/jmx/mbeans/standard.html

In the following example you will see how I've created a custom MBean for measuring insert, update and continuous query operations:

a. create a new package in your project "mbeans" for instance:

b. create a generic mbean interface class called PerformanceMBean that describes what are the measurables for a certain operation:

/**

*

*/

package com.oracle.coherence.test.mbeans;


/**

* @author tucosmin

*

*/

public interface PerformanceMBean

{

public int getTotalNoOfThisOperation();


public void setTotalNoOfThisOperation(int totalNoOfThisOperation);


public int getIntermTotalNoOfThisOperation();


public void setIntermTotalNoOfThisOperation(int intermTotalNoOfThisOperation);


public int getIntermTotalNoOfOperations();


public void setIntermTotalNoOfOperations(int intermTotalNoOfOperations);


public double getIntermAverageOperationDuration();


public void setIntermAverageOperationDuration(

double intermAverageOperationDuration);


public double getIntermMinOperationDuration();


public void setIntermMinOperationDuration(double intermMinOperationDuration);


public double getIntermMaxOperationDuration();


public void setIntermMaxOperationDuration(double intermMaxOperationDuration);


public int getTotalNoOfOperations();


public void setTotalNoOfOperations(int totalNoOfOperations);


public double getAverageOperationDuration();


public void setAverageOperationDuration(double averageOperationDuration);


public double getMinOperationDuration();


public void setMinOperationDuration(double minOperationDuration);


public double getMaxOperationDuration();


public void setMaxOperationDuration(double maxOperationDuration);


public String getClient();


public void setClient(String client);


public String getNumberOfOperationsPerTimeUnit();


public void setNumberOfOperationsPerTimeUnit(

String numberOfOperationsPerTimeUnit);

}


c. implement the PerformanceMBean interface:

    /**

*

*/

package com.oracle.coherence.test.mbeans;


/**

* @author tucosmin

*

*/

public class Performance implements PerformanceMBean

{

public int totalNoOfThisOperation;

public int totalNoOfOperations;

public double averageOperationDuration;

public double minOperationDuration;

public double maxOperationDuration;

public int intermTotalNoOfThisOperation;

public int intermTotalNoOfOperations;

public double intermAverageOperationDuration;

public double intermMinOperationDuration;

public double intermMaxOperationDuration;

public String numberOfOperationsPerTimeUnit;

public String client;


public int getTotalNoOfThisOperation()

{

return totalNoOfThisOperation;

}


public void setTotalNoOfThisOperation(int totalNoOfThisOperation)

{

this.totalNoOfThisOperation = totalNoOfThisOperation;

}


public int getIntermTotalNoOfThisOperation()

{

return intermTotalNoOfThisOperation;

}


public void setIntermTotalNoOfThisOperation(int intermTotalNoOfThisOperation)

{

this.intermTotalNoOfThisOperation = intermTotalNoOfThisOperation;

}


public int getIntermTotalNoOfOperations()

{

return intermTotalNoOfOperations;

}


public void setIntermTotalNoOfOperations(int intermTotalNoOfOperations)

{

this.intermTotalNoOfOperations = intermTotalNoOfOperations;

}


public double getIntermAverageOperationDuration()

{

return intermAverageOperationDuration;

}


public void setIntermAverageOperationDuration(

double intermAverageOperationDuration)

{

this.intermAverageOperationDuration = intermAverageOperationDuration;

}


public double getIntermMinOperationDuration()

{

return intermMinOperationDuration;

}


public void setIntermMinOperationDuration(double intermMinOperationDuration)

{

this.intermMinOperationDuration = intermMinOperationDuration;

}


public double getIntermMaxOperationDuration()

{

return intermMaxOperationDuration;

}


public void setIntermMaxOperationDuration(double intermMaxOperationDuration)

{

this.intermMaxOperationDuration = intermMaxOperationDuration;

}


public int getTotalNoOfOperations()

{

return totalNoOfOperations;

}


public void setTotalNoOfOperations(int totalNoOfOperations)

{

this.totalNoOfOperations = totalNoOfOperations;

}


public double getAverageOperationDuration()

{

return averageOperationDuration;

}


public void setAverageOperationDuration(double averageOperationDuration)

{

this.averageOperationDuration = averageOperationDuration;

}


public double getMinOperationDuration()

{

return minOperationDuration;

}


public void setMinOperationDuration(double minOperationDuration)

{

this.minOperationDuration = minOperationDuration;

}


public double getMaxOperationDuration()

{

return maxOperationDuration;

}


public void setMaxOperationDuration(double maxOperationDuration)

{

this.maxOperationDuration = maxOperationDuration;

}


public String getClient()

{

return client;

}


public void setClient(String client)

{

this.client = client;

}


public String getNumberOfOperationsPerTimeUnit()

{

return numberOfOperationsPerTimeUnit;

}


public void setNumberOfOperationsPerTimeUnit(String numberOfOperationsPerTimeUnit)

{

this.numberOfOperationsPerTimeUnit = numberOfOperationsPerTimeUnit;

}

}


d. inherit the Performance class for each operation that you want to measure:

/**

*

*/

package com.oracle.coherence.test.mbeans;


/**

* @author tucosmin

*

*/

public class UpdatePerformance extends Performance

{

public UpdatePerformance()

{

super();

// TODO Auto-generated constructor stub

}

}


/**

*

*/

package com.oracle.coherence.test.mbeans;


/**

* @author tucosmin

*

*/

public class WritePerformance extends Performance

{


public WritePerformance()

{

super();

// TODO Auto-generated constructor stub

}


}

d. add Mbeans classes to the Coherence Management node classpath:

-cp $TEST_HOME/build/classes



e. create a custom Mbean XML file

    <mbeans>

<mbean id="100">

<mbean-class>com.oracle.coherence.test.mbeans.WritePerformance

</mbean-class>

<mbean-name>type=Performance,name=WritePerformance</mbean-name>

<enabled>true</enabled>

</mbean>

<mbean id="101">

<mbean-class>com.oracle.coherence.test.mbeans.ContinuousQueryInsertPerformance

</mbean-class>

<mbean-name>type=Performance,name=ContinuousQueryInsertPerformance

</mbean-name>

<enabled>true</enabled>

</mbean>

<mbean id="102">

<mbean-class>com.oracle.coherence.test.mbeans.ContinuousQueryUpdatePerformance

</mbean-class>

<mbean-name>type=Performance,name=ContinuousQueryUpdatePerformance

</mbean-name>

<enabled>true</enabled>

</mbean>

<mbean id="103">

<mbean-class>com.oracle.coherence.test.mbeans.UpdatePerformance

</mbean-class>

<mbean-name>type=Performance,name=UpdatePerformance

</mbean-name>

<enabled>true</enabled>

</mbean>

</mbeans>

f. specify the name and location of a custom MBean configuration file to the coherence management node:

-Dtangosol.coherence.mbeans=mBeans.xml

3. Connect to the JMX custom Mbeans to update them with the latest results

      ---

      com.oracle.coherence.jmxUrl=service:jmx:rmi://tucosmin-ro:3000/jndi/rmi://tucosmin-ro:9000/server

---

public void sendOperationPerformanceToJmx(String jmxPath,

ReportOutput reportOutput)

{

if (reportOutput != null)

{

// Create a JMXServiceURL

JMXServiceURL serviceUrl;

try

{


serviceUrl = new JMXServiceURL(DEFAULT_JMX_SERVER_URL);

JMXConnector connector = JMXConnectorFactory.newJMXConnector(

serviceUrl, null);


connector.connect();


// Create an MBeanServerConnection

MBeanServerConnection connection = connector

.getMBeanServerConnection();


ObjectName name = new ObjectName(jmxPath);

Set<ObjectName> nodes = connection.queryNames(name, null);

for (Object o : nodes)

{


Double average = 0.0;

int count = 0;

if (reportOutput.getOpperation().equalsIgnoreCase(

Constants.WRITE_TIME))

{

average = getWriteAverage().getAverage();

count = getWriteAverage().getCount();

}

else if (reportOutput.getOpperation().equalsIgnoreCase(

Constants.CONTINUOUS_QUERY_INSERT_TIME))

{

average = getContinuousQueryInsertAverage()

.getAverage();

count = getContinuousQueryInsertAverage().getCount();

}

else if (reportOutput.getOpperation().equalsIgnoreCase(

Constants.CONTINUOUS_QUERY_UPDATE_TIME))

{

average = getContinuousQueryUpdateAverage()

.getAverage();

count = getContinuousQueryUpdateAverage().getCount();

}

else if (reportOutput.getOpperation().equalsIgnoreCase(

Constants.UPDATE_TIME))

{

average = getUpdateAverage().getAverage();

count = getUpdateAverage().getCount();

}

connection.setAttribute(new ObjectName(o.toString()),

new Attribute(Constants.AVERAGE_OPERATION_DURATION,

average));


int totalNoOfThisOp = Integer.parseInt(connection

.getAttribute(new ObjectName(o.toString()),

Constants.TOTAL_NO_OF_THIS_OPERATION)

.toString());

totalNoOfThisOp += reportOutput.getTotalNoOfOperations();

connection.setAttribute(new ObjectName(o.toString()),

new Attribute(Constants.TOTAL_NO_OF_THIS_OPERATION,

count));


connection.setAttribute(new ObjectName(o.toString()),

new Attribute(Constants.TOTAL_NO_OF_OPERATIONS,

getTotalNoOfOperations()));


Double minOperationDuration = new Double(connection

.getAttribute(new ObjectName(o.toString()),

Constants.MIN_OPERATION_DURATION)

.toString());


Double maxOperationDuration = new Double(connection

.getAttribute(new ObjectName(o.toString()),

Constants.MAX_OPERATION_DURATION)

.toString());

if (minOperationDuration == 0)

{

connection.setAttribute(new ObjectName(o.toString()),

new Attribute(Constants.MIN_OPERATION_DURATION,

reportOutput.minOperationDuration));

}

else

{

if (minOperationDuration > reportOutput.minOperationDuration

&& reportOutput.minOperationDuration > 0)

{

connection.setAttribute(

new ObjectName(o.toString()),

new Attribute(

Constants.MIN_OPERATION_DURATION,

reportOutput.minOperationDuration));


}

else

{

if (minOperationDuration > 0)

{

connection.setAttribute(new ObjectName(o

.toString()), new Attribute(

Constants.MIN_OPERATION_DURATION,

minOperationDuration));

}


}

}


if (maxOperationDuration < reportOutput.maxOperationDuration)

{


connection.setAttribute(new ObjectName(o.toString()),

new Attribute(Constants.MAX_OPERATION_DURATION,

reportOutput.maxOperationDuration));

}

else

{

connection.setAttribute(new ObjectName(o.toString()),

new Attribute(Constants.MAX_OPERATION_DURATION,

maxOperationDuration));

}


connection.setAttribute(new ObjectName(o.toString()),

new Attribute(

Constants.INTERM_MAX_OPERATION_DURATION,

reportOutput.getMaxOperationDuration()));


connection

.setAttribute(

new ObjectName(o.toString()),

new Attribute(

Constants.INTERM_TOTAL_NO_OF_THIS_OPERATION,

reportOutput

.getTotalNoOfOperations()));


connection

.setAttribute(

new ObjectName(o.toString()),

new Attribute(

Constants.INTERM_AVERAGE_OPERATION_DURATION,

reportOutput

.getAverageOperationDuration()));


if (reportOutput.getMinOperationDuration() > 0)

{

connection

.setAttribute(

new ObjectName(o.toString()),

new Attribute(

Constants.INTERM_MIN_OPERATION_DURATION,

reportOutputsendOperationPerformanceToJmx

.getMinOperationDuration()));

}


connection.setAttribute(new ObjectName(o.toString()),

new Attribute(

Constants.INTERM_TOTAL_NO_OF_OPERATIONS,

getResults().size()));


connection.setAttribute(new ObjectName(o.toString()),

new Attribute(Constants.CLIENT, "Client "

+ Constants.COHERENCE_EXTEND_CLIENT_ID));


}


}

catch (Exception e)

{

// TODO Auto-generated catch block

e.printStackTrace();

}

}

}

4. Use JConsole to watch your custom Mbeans


img.jpg

II. Create a custom report

The Coherence reporting feature provides a capable query definition that allows for any information residing in the Coherence JMX data source to be logged to a text file. After a custom report has been created, it can be included in a report batch and executed on a specified time interval by the ReportControl Mbean.

Coherence provides a JMX reporting capability (the Reporter). The Reporter provides out-of-the-box reports that help administrators and developers manage capacity and trouble shoot problems. Custom reports can also be created.

1. Localize and extract your custom reports configuration files:

  • Go to your COHERENCE_HOME/lib location, go to the archive coherence.jar and extract from there into your project folder the reports folder. In that folder you should find a report-group.xml file together with other configuration files

2. Configure the management node to load your custom report configuration files

  • In your management node start file specify from where the management node could access the reports configuration files: -Dtangosol.coherence.management.report.configuration=$TEST_HOME/reportsIntel/report-group.xml Without this specification the reports configuration files are loaded from the default location: coherence.jar\reports. You can modify the configuration from that location as well but the best practice is to copy them in your project location.

3. Create and configure a new configuration file for each report that you want to be generated

  • Create an xml file for each report that you want to be generated. As an example you can see the following xml report configuration file for the insert operation:

<!DOCTYPE report-config SYSTEM "report-config.dtd">


<report-config>

<report>

<file-name>{date}-total-write-performance.txt</file-name>

<delim>{tab}</delim>


<query>

<pattern>Coherence:type=Performance,name=WritePerformance,*</pattern>

</query>


<row>

<column id="BatchCounter">

<type>global</type>

<name>{batch-counter}</name>

<header>Batch Counter</header>

</column>


<column id ="ReportTime">

<type>global</type>

<name>{report-time}</name>

<header>Report Time</header>

</column>


<column id ="MinOperationDuration">

<name>MinOperationDuration</name>

<header>Total(until now) Minimum Write Operation Duration</header>

</column>


<column id ="MaxOperationDuration">

<name>MaxOperationDuration</name>

<header>Total(until now) Maximum Write Operation Duration</header>

</column>


<column id ="AverageOperationDuration">

<name>AverageOperationDuration</name>

<header>Total(until now) Write Average Operation Duration</header>

</column>


<column id ="TotalNoOfOperations">

<name>TotalNoOfOperations</name>

<header>Total Number of All Operations Until Now</header>

</column>


<column id ="TotalNoOfThisOperation">

<name>TotalNoOfThisOperation</name>

<header>Total Number Of Write Operations Until Now</header>

</column>


<column id ="NumberOfOperationsPerTimeUnit">

<name>NumberOfOperationsPerTimeUnit</name>

<header>Number Of Write Operations per TimeUnit</header>

</column>


<column id ="Client">

<name>Client</name>

<header>Client</header>

</column>


</row>

</report>

</report-config>



The element <file-name> defines the reports file name. As you can see it includes the the keyword date which means that every report name will contain the creation date.

The next element defines the Mbean that will be used in your custom report:

<query>

<pattern>Coherence:type=Performance,name=WritePerformance,*</pattern>

</query>

Then use your Mbean attributes and report default attributes to build your custom report:

  • Report default attributes:

<column id ="ReportTime">

<type>global</type>

<name>{report-time}</name>

<header>Report Time</header>

</column>

  • Mbean attributes:

<column id ="MaxOperationDuration">

<name>MaxOperationDuration</name>

<header>Total(until now) Maximum Write Operation Duration</header>

</column>

4. Add your new report configuration file to the general Coherence JMX Reporter configuration file:

  • Add your custom report xml configuration file to the report-group.xml Coherence JMX Reporter configuration file:


<!DOCTYPE report-group SYSTEM "report-group.dtd">


<report-group>

<frequency>30s</frequency>

<output-directory>/media/Software/reports/</output-directory>

<report-list>

<report-config>

<location>/media/Software/report-running-write-performance.xml</location>

</report-config>

<report-config>

<location>/media/Software/report-running-update-performance.xml</location>

</report-config>

<report-config>

<location>/media/Software/report-running-continuous-query-insert-performance.xml</location>

</report-config>

<report-config>

<location>/media/Software/report-running-continuous-query-update-performance.xml</location>

</report-config>

<report-config>

<location>/media/Software/report-total-write-performance.xml</location>

</report-config>

<report-config>

<location>/media/Software/report-total-update-performance.xml</location>

</report-config>

<report-config>

<location>/media/Software/report-total-continuous-query-insert-performance.xml</location>

</report-config>

<report-config>

<location>/media/Software/report-total-continuous-query-update-performance.xml</location>

</report-config>

</report-list>

</report-group>


The following can be also specified:

  • reporting frequency: <frequency>30s</frequency>

  • The reports output directory: <output-directory>/media/Software/reports/</output-directory>



Report example (the reports are generated as txt documents by they are comma separated, so they are csv actually):

ty.jpg

Comments:

Post a Comment:
  • HTML Syntax: NOT allowed
About

bocadmin_ww

Search

Archives
« prill 2014
DieHënMarMërEnjPreSht
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today