Thursday May 21, 2009

Java EE Olio - source code released

The Java EE source code of the Apache incubator project Olio was released two weeks ago. To learn more about the Java EE version of Olio, please check out Kim Lichong's overview blog. Shanti's blog provides an overiew of Olio.


It took a long time for us to get here and there were several detours along the way. This project started off as an extension of the Java EE Blueprints Live project. Thanks to Inderjeet Singh, Mark Basler, Sean Brydon, and Greg Murray (some of them are shown in the picture in the Olio snapshot) for their contributions. Significant portions of the Live code are still in Olio (I am sure there are a few unused artifacts there as well, which we have missed to take out).

We modified the base Live code so that we could analyze the performance and scalability of a Web 2.0 application and added a benchmark driver and harness. The feature set available in Live was used for the 1.0 version of  PHP and Rails versions. We went on to modify the Java EE version to incorporate distributed caching using Memcached and some experiments using a distributed file system. At this stage, the plans were changed to release a stripped down version of the code that we had for feature parity with PHP 1.0 (which did not use Memcached). Due to some other priorities, it took us a lot longer to refactor the required changes and get to where we are. So I am extremely glad that we are finally able to release it.

Current Status

We have a working version of the code available -

Workload Driver:
Guide  for initial setup and configuration:

Help Needed

We are just getting started and there are lots more to be done. Here is a set of tasks that are already identified (I am sure there will be more coming in) -

  1. Incorporate Memcached or some other distributed caching mechanism.
  2. Improve the file system interaction. There are parts of the code that assume a local file system. We started on cleaning up this code by defining a file system abstraction layer. But more work is needed to complete the clean up. Once the clean up is complete, look at using distributed file systems (MogileFS, Hadoop).
  3. Clean up the JSPs to replace JMaki with appropriate JS libraries.
  4. Implement 'true' REST based services.
This is an open invitation to all of you to join us and move this project along. Please subscribe to our mailing lists via the Olio home page as a user (, or contribute as a developer ( .

Thursday Apr 16, 2009

Measuring HTTP Listener Service Time With BTrace

In the last entry, I discussed who BTrace can be used to monitor the HTTP thread pool. Here, I show how a similar script can be used to measure the service time of the HTTP listener.[Read More]

Friday Apr 10, 2009

Monitoring GlassFish Using BTrace

BTrace, the dynamic tracing utility in Java can be used to monitor a GlassFish server instance. Here I list a script that can be used to collect the HTTP thread pool statistics.[Read More]

Wednesday Apr 01, 2009

GlassFish V3 profiling

I have been using JProfiler and BTrace for the performance analysis of GlassFish V3. Things were working well until I picked up build 42 last week. With this build, I was unable to attach the profiler or BTrace. A work around is discussed here.[Read More]

Tuesday Mar 31, 2009

Monitoring Webapps .. part 2

VisualVM GlassFish plugin allows you to monitor the response times of web application components. In some cases, however, the need is to track the throughput and response times of all requests processed by a http-listener. A very simple (graphically challenged) tool to address this requirement is provided.[Read More]

Thursday Mar 26, 2009

Monitoring Web Applications in GlassFish

GlassFish monitoring framework can be used to collect Servlet service time statistics. This is a valuable debugging tool for investigating high latency problems. The data can help identify whether the problem is caused within the application or the application server.[Read More]

Monday Mar 16, 2009

GlassFish Tuning - HTTP Connection Queue & Keep Alive

In my last entry I discussed how to monitor and tune the HTTP thread pool in GlassFish. In this installment, I discuss the monitoring and tuning of the http listener's connection queue and the keep alive settings.

[Read More]

Friday Feb 13, 2009

GlassFish Tuning - HTTP Thread Pool

In this entry I discuss how to monitor and tune the HTTP thread pool in GlassFish V2. Includes a short description of what values to monitor and how to tune these values for optimal performance. This a follow up of Kim Lichong's Optimize GlassFish Performance in a Production Environment white paper.
[Read More]

Thursday Oct 16, 2008

Profiling GlassFish V2 using Sun Studio

In one of my earlier blogs, I had mentioned that Collector/Analyzer within Sun Studio is my primary performance analysis tool. It would have been nice if we could collect GlassFish V2 profiles using Collector by following a few easy steps. Unfortunately, this is not the case. You will have to go through a couple of convoluted steps: 

  1. Start the domain and then capture the entire 'java .... PELaunch start' command.
  2. Create a password file containing the following information, each in a separate line - <admin user>, <admin password>, <master password>
  3. Start the collect command as follows: collect -j on -y WINCH -d result_dir java_command < password_file

The 'WINCH' signal is used to specify when to start and stop the profile collection. In most scenarios, you want the collection during the steady state of the benchmark run. So you would typically wait for the ramp up time to complete, before starting profile collection using the kill -WINCH <pid> command. Since this is a sampling collector, it  is advisable to collect the profile for a longer duration (I typically use 20 minutes). You can stop collection using the kill -WINCH <pid> command.

I have written a script that makes this process much easier. It is given below.


if [ $# -lt 5 ]
  echo AS_HOME: Root directory of GlassFish
  echo SUNSTUDIO_HOME Root directory of Sun Studio installation
  echo RESULT_DIR: Directory to store experiments
  echo DOMAIN: Domain name
  echo PASSWORDFILE: Password file. A Simple file with the following information, each on a separate line - Admin user, Admin password, Master password. A sample password file has been created at /tmp/password
  echo admin > /tmp/password
  echo adminadmin >> /tmp/password
  echo changeit >> /tmp/password
  echo Example: $0 /export/glassfishv2/glassfish /opt/SUNWspro /results domain1 /tmp/password



if [ $os = "SunOS" ]
elif [ $os = "Linux" ]
   echo "Unsupported OS"

mkdir -p $RESULT_DIR

AS_JAVA=`grep AS_JAVA $AS_HOME/config/asenv.conf | grep -v \\# | $AWK -F= '{print $2}' | $AWK -F\\" '{print $2}'`

# Check if the domain is running.
pid=`$AS_JAVA/bin/jps | grep PELaunch | grep -v grep | $AWK '{print $1}'`

if [ "pid$pid" != "pid" ]
   echo An instance of the app server is running. Please shut it down first.

echo This may take a minute. Please be patient....
echo Starting $DOMAIN to access java command
$AS_HOME/bin/asadmin start-domain $DOMAIN  > /dev/null
echo ....
sleep 10
pid=`$AS_JAVA/bin/jps | grep PELaunch | grep -v grep | $AWK '{print $1}'`
cp=`$AS_JAVA/bin/jinfo $pid 2> /dev/null | grep java.class.path | $AWK -F= '{print $2}'`
props=`$AS_JAVA/bin/jinfo -flags $pid  2> /dev/null| grep '\\-D'`
echo Stopping $DOMAIN
$AS_HOME/bin/asadmin stop-domain $DOMAIN > /dev/null
echo ....
sleep 10
echo Starting domain in collect mode
$COLLECT -j on -d $RESULT_DIR -y WINCH $AS_JAVA/bin/java -cp $cp $props com.sun.enterprise.server.PELaunch start < $PASS &
echo ....
sleep 10
pid=`$AS_JAVA/bin/jps | grep PELaunch | grep -v grep | $AWK '{print $1}'`
if [ "pid$pid" = "pid" ]
  echo Error is starting GF is collect mode, refer to domains log file - $AS_HOME/domains/$DOMAIN/logs/server.log
  echo "AS Process Started, pid=$pid. Execute the following command to start and stop profile collection"
  echo "kill -WINCH $pid"

Running the script without any parameters prints out the usage.

Usage /home/binu/projects/perf-book/benchmarks/ AS_HOME SUNSTUDIO_HOME RESULT_DIR DOMAIN PASSWORDFILE
AS_HOME: Root directory of GlassFish
SUNSTUDIO_HOME Root directory of Sun Studio installation
RESULT_DIR: Directory to store experiments
DOMAIN: Domain name
PASSWORDFILE: Password file. A Simple file with the following information, each on a separate line - Admin user, Admin password, Master password. A sample password file has been created at /tmp/password
Example: /home/binu/benchmarks/ /glassfish /SUNWspro/bin/collect /results domain1 /tmp/password

The experiments are stored in the folder, RESULT_DIR/ (where x is a number). The profiles can be viewed using 'analyzer' as follows -


 While anaylzing results, it is a good practice to set the data presentation format to expert mode. This can be set by selecting View --> Set Data Presentation. In the pop up window, select the Formats tab and click on Expert in the View Mode. More information about performance analyzer can be found here.


GlassFish V2
Sun Sudio 12




« June 2016