Thursday May 21, 2009

Java EE Olio - source code released

The Java EE source code of the Apache incubator project Olio was released two weeks ago. To learn more about the Java EE version of Olio, please check out Kim Lichong's overview blog. Shanti's blog provides an overiew of Olio.


It took a long time for us to get here and there were several detours along the way. This project started off as an extension of the Java EE Blueprints Live project. Thanks to Inderjeet Singh, Mark Basler, Sean Brydon, and Greg Murray (some of them are shown in the picture in the Olio snapshot) for their contributions. Significant portions of the Live code are still in Olio (I am sure there are a few unused artifacts there as well, which we have missed to take out).

We modified the base Live code so that we could analyze the performance and scalability of a Web 2.0 application and added a benchmark driver and harness. The feature set available in Live was used for the 1.0 version of  PHP and Rails versions. We went on to modify the Java EE version to incorporate distributed caching using Memcached and some experiments using a distributed file system. At this stage, the plans were changed to release a stripped down version of the code that we had for feature parity with PHP 1.0 (which did not use Memcached). Due to some other priorities, it took us a lot longer to refactor the required changes and get to where we are. So I am extremely glad that we are finally able to release it.

Current Status

We have a working version of the code available -

Workload Driver:
Guide  for initial setup and configuration:

Help Needed

We are just getting started and there are lots more to be done. Here is a set of tasks that are already identified (I am sure there will be more coming in) -

  1. Incorporate Memcached or some other distributed caching mechanism.
  2. Improve the file system interaction. There are parts of the code that assume a local file system. We started on cleaning up this code by defining a file system abstraction layer. But more work is needed to complete the clean up. Once the clean up is complete, look at using distributed file systems (MogileFS, Hadoop).
  3. Clean up the JSPs to replace JMaki with appropriate JS libraries.
  4. Implement 'true' REST based services.
This is an open invitation to all of you to join us and move this project along. Please subscribe to our mailing lists via the Olio home page as a user (, or contribute as a developer ( .

Thursday Apr 16, 2009

Measuring HTTP Listener Service Time With BTrace

In the last entry, I discussed who BTrace can be used to monitor the HTTP thread pool. Here, I show how a similar script can be used to measure the service time of the HTTP listener.[Read More]

Friday Apr 10, 2009

Monitoring GlassFish Using BTrace

BTrace, the dynamic tracing utility in Java can be used to monitor a GlassFish server instance. Here I list a script that can be used to collect the HTTP thread pool statistics.[Read More]

Wednesday Apr 01, 2009

GlassFish V3 profiling

I have been using JProfiler and BTrace for the performance analysis of GlassFish V3. Things were working well until I picked up build 42 last week. With this build, I was unable to attach the profiler or BTrace. A work around is discussed here.[Read More]

Tuesday Mar 31, 2009

Monitoring Webapps .. part 2

VisualVM GlassFish plugin allows you to monitor the response times of web application components. In some cases, however, the need is to track the throughput and response times of all requests processed by a http-listener. A very simple (graphically challenged) tool to address this requirement is provided.[Read More]

Thursday Mar 26, 2009

Monitoring Web Applications in GlassFish

GlassFish monitoring framework can be used to collect Servlet service time statistics. This is a valuable debugging tool for investigating high latency problems. The data can help identify whether the problem is caused within the application or the application server.[Read More]

Monday Mar 23, 2009

If I Twitter does it matter?

I finally got a twitter account, but was confronted by the question - If I twitter and I don't have any followers, am I really tweeting? To me, this was akin to the century old question, if a tree falls in a forest and no one is around to hear it, does it make a sound?[Read More]

Thursday Mar 19, 2009

Youth Volleyball League - Spring 2009 season

United States Youth Volleyball League (USYVL) is a non-profit organization that teaches volleyball to children aged 7 to 14. The Spring 2009 season runs from April 1 to June 6 and registration is currently open. If you have a child who is interested in learning/playing volleyball, I invite you to check it out.[Read More]

Monday Mar 16, 2009

GlassFish Tuning - HTTP Connection Queue & Keep Alive

In my last entry I discussed how to monitor and tune the HTTP thread pool in GlassFish. In this installment, I discuss the monitoring and tuning of the http listener's connection queue and the keep alive settings.

[Read More]

Thursday Mar 12, 2009

Too much thinking ...

A good post by Jonah Lehrer, Free Throws, on how too much thinking while playing can in fact worsen your performance. This explains why my slap shots suck - I am thinking too much. It is the same pattern over and over again. Someone tells me the finer points of how to hit it correctly, I replay what they tell me in my mind during the shot and I get off a shot that is weak and off target. From now on, no more thinking ... just line up and whack at the puck. 

Friday Feb 13, 2009

GlassFish Tuning - HTTP Thread Pool

In this entry I discuss how to monitor and tune the HTTP thread pool in GlassFish V2. Includes a short description of what values to monitor and how to tune these values for optimal performance. This a follow up of Kim Lichong's Optimize GlassFish Performance in a Production Environment white paper.
[Read More]

Thursday Oct 16, 2008

Profiling GlassFish V2 using Sun Studio

In one of my earlier blogs, I had mentioned that Collector/Analyzer within Sun Studio is my primary performance analysis tool. It would have been nice if we could collect GlassFish V2 profiles using Collector by following a few easy steps. Unfortunately, this is not the case. You will have to go through a couple of convoluted steps: 

  1. Start the domain and then capture the entire 'java .... PELaunch start' command.
  2. Create a password file containing the following information, each in a separate line - <admin user>, <admin password>, <master password>
  3. Start the collect command as follows: collect -j on -y WINCH -d result_dir java_command < password_file

The 'WINCH' signal is used to specify when to start and stop the profile collection. In most scenarios, you want the collection during the steady state of the benchmark run. So you would typically wait for the ramp up time to complete, before starting profile collection using the kill -WINCH <pid> command. Since this is a sampling collector, it  is advisable to collect the profile for a longer duration (I typically use 20 minutes). You can stop collection using the kill -WINCH <pid> command.

I have written a script that makes this process much easier. It is given below.


if [ $# -lt 5 ]
  echo AS_HOME: Root directory of GlassFish
  echo SUNSTUDIO_HOME Root directory of Sun Studio installation
  echo RESULT_DIR: Directory to store experiments
  echo DOMAIN: Domain name
  echo PASSWORDFILE: Password file. A Simple file with the following information, each on a separate line - Admin user, Admin password, Master password. A sample password file has been created at /tmp/password
  echo admin > /tmp/password
  echo adminadmin >> /tmp/password
  echo changeit >> /tmp/password
  echo Example: $0 /export/glassfishv2/glassfish /opt/SUNWspro /results domain1 /tmp/password



if [ $os = "SunOS" ]
elif [ $os = "Linux" ]
   echo "Unsupported OS"

mkdir -p $RESULT_DIR

AS_JAVA=`grep AS_JAVA $AS_HOME/config/asenv.conf | grep -v \\# | $AWK -F= '{print $2}' | $AWK -F\\" '{print $2}'`

# Check if the domain is running.
pid=`$AS_JAVA/bin/jps | grep PELaunch | grep -v grep | $AWK '{print $1}'`

if [ "pid$pid" != "pid" ]
   echo An instance of the app server is running. Please shut it down first.

echo This may take a minute. Please be patient....
echo Starting $DOMAIN to access java command
$AS_HOME/bin/asadmin start-domain $DOMAIN  > /dev/null
echo ....
sleep 10
pid=`$AS_JAVA/bin/jps | grep PELaunch | grep -v grep | $AWK '{print $1}'`
cp=`$AS_JAVA/bin/jinfo $pid 2> /dev/null | grep java.class.path | $AWK -F= '{print $2}'`
props=`$AS_JAVA/bin/jinfo -flags $pid  2> /dev/null| grep '\\-D'`
echo Stopping $DOMAIN
$AS_HOME/bin/asadmin stop-domain $DOMAIN > /dev/null
echo ....
sleep 10
echo Starting domain in collect mode
$COLLECT -j on -d $RESULT_DIR -y WINCH $AS_JAVA/bin/java -cp $cp $props com.sun.enterprise.server.PELaunch start < $PASS &
echo ....
sleep 10
pid=`$AS_JAVA/bin/jps | grep PELaunch | grep -v grep | $AWK '{print $1}'`
if [ "pid$pid" = "pid" ]
  echo Error is starting GF is collect mode, refer to domains log file - $AS_HOME/domains/$DOMAIN/logs/server.log
  echo "AS Process Started, pid=$pid. Execute the following command to start and stop profile collection"
  echo "kill -WINCH $pid"

Running the script without any parameters prints out the usage.

Usage /home/binu/projects/perf-book/benchmarks/ AS_HOME SUNSTUDIO_HOME RESULT_DIR DOMAIN PASSWORDFILE
AS_HOME: Root directory of GlassFish
SUNSTUDIO_HOME Root directory of Sun Studio installation
RESULT_DIR: Directory to store experiments
DOMAIN: Domain name
PASSWORDFILE: Password file. A Simple file with the following information, each on a separate line - Admin user, Admin password, Master password. A sample password file has been created at /tmp/password
Example: /home/binu/benchmarks/ /glassfish /SUNWspro/bin/collect /results domain1 /tmp/password

The experiments are stored in the folder, RESULT_DIR/ (where x is a number). The profiles can be viewed using 'analyzer' as follows -


 While anaylzing results, it is a good practice to set the data presentation format to expert mode. This can be set by selecting View --> Set Data Presentation. In the pop up window, select the Formats tab and click on Expert in the View Mode. More information about performance analyzer can be found here.


GlassFish V2
Sun Sudio 12

Monday Dec 04, 2006

DOM Performance Tip - adoptNode

DOM Level 3 introduced adoptNode which allows you to adopt a node from another document to this document. This effectively allows moving a subtree from one document to another (unlike importNode() which create a copy of the source node instead of moving it). adoptNode is much more performance efficient than importNode especially for deep Nodes since it avoids the creation and copying overheads. My experiments using XMLTest (with UBL Invoice document) showed a performance improvement of about 25%. Note that this large performance improvement is for a microbenchmark, improvement for your application may vary.

Wednesday Nov 01, 2006

Varargs and debug print

I  had not used the Varargs feature available in Java 5 until recently. As expected, it was pretty easy to use. From the performance point of view, I was looking mainly at Object allocation and other performance costs for a specific problem in one of our applications.

We had debug print messages like the one below sprinkled in several parts of the application. Yes, I know, this problem would not even be present if we had used logging correctly. Well, now that we are stuck with it, I was curious to know how the performance, especially Object allocation statistics would change if I used varargs.

javax.xml.datatype.DatatypeFactory factory;
javax.xml.datatype.XMLGregorianCalendar date;
debugPrint ("In getData: Factory = " + factory + " date = " + date);

public void debugPrint (String msg) {
   if (debug)
      System.out.println (msg);

Even with the debug flag set to false, this causes the creation of a StringBuilder, several Strings as well as char[] objects (My microbenchmark showed 13 char[] objects/1904 bytes, 4 Strings/98 bytes and 1 StringBuilder/16 bytes), as well as the additional cost of generating String representation of the factory and date objects. Even though these are relatively inexpensive operations, if used repeatedly, these can add up and cause poor performance.

I rewrote the debugPrint method and its usage as follows -

debugPrint ("Factory = ", factory, " date = ", date);

public void debugPrint (Object ... msg) {
   if (!debug || msg==null)
   StringBuilder strb = new StringBuilder();
   for (int i=0; i<msg.length; i++)
      strb.append (msg[i].toString());
  System.out.println (strb.toString());

As far as Object allocations go, this created just one Object[] (size=32 bytes). Additionally, no other work was carried out since debug flag was set to false.  So overall, this is a pretty good solution.

Note: Varargs does support use of primitives through autoboxing in which case some additional objects may be created.

Monday May 15, 2006

Meet the Sun Java EE Platform Performance Experts BOF at JavaOne 2006

"Meet the Sun Java Technology Performance Experts" BOF (Wednesday 5/17, 8:30 pm Moscone Center Gateway 102/103) has been a staple at JavaOne for the last few years. This is an interactive session where members from the Java Performance and SE Engineering teams field questions from the audience on everything relating to Java SE Performance.

This year we added a second BOF - "Meet the Sun Java EE Platform Performance Experts" that will address all questions related to Java EE, XML and Web Services performance. This is an opportunity for all developers to ask performance related questions to a panel of experts from the Java EE Performance and Engineering teams. This is an audience driven interactive session and I invite all of you to come and participate and make it a lively discussion session. The BOF is scheduled on Wednesday, 5/17 starting at 9:30 pm in City conference room in the Argent Hotel.

A list of all the activities the performance team is involved in at this JavaOne is given below. Be sure to come to our sessions and check out our pod (POD# 721) on the exhibition floor.
  • BOF-0539: Meet the Sun Java Technology Performance Experts, Wednesday 5/17, 8:30 pm Moscone Center Gateway 102/103
  • BOF-0559: Meet the Sun Java EE Platform Performance Experts, Wednesday 9:30 pm, Argent Hotel City Conference Room
  • BOF-2953: Implementing High-Performance Web Services With Next Generation Java Technology APIs, Wednesday 8:30 pm, Moscone Center Hall E 135
  • BOF-0623: Java Technology-Based Performance Multithreaded Hardware Systems, Thursday 7:30 pm, Moscone Center Gateway 102/103
  • TS-1624: Writing Performant EJBs in the Java EE Platform 5 (EJB 3.0) Using Annotations, Friday 12 -1 pm, Moscone Hall E 134
  • TS-6264: Secure XML Processing Using Chip Multithreaded Processors, Friday 3:45 pm, Moscone North Meeting Room 121/122
  • LAB-1205: Java Technology-Based Application Performance Analysis, Tuesday 11 am
  • LAB-4335: Developing Interoperable Next Generation Web Services Using Glassfish, Netbeans and WSIT
  • POD-721: Java Platform, Standard Edition: Performance & Diagnostics



« February 2015