Tuesday Aug 07, 2007

Regperf - The followup


Tom Daly and I have been looking at open source solutions in the web application framework area.  It complemented our shared interest of open source databases.  We noticed there was a large amount of interest around Ruby on Rails.   It seemed like an obvious place to start researching where the right place, if such a place existed for frameworks like RoR, was and what type of applications were it's sweet spot.  On the way we learned about Grails also, a similar purpose project and we decided to include Grails in our investigation.


Regperf is a trivial application to represent many small web applications that individuals and corporations build to do a simple task, in this case keep details about software registrations.  The genre of applications that Regperf belongs to is usually used for very simple, one or more table, applications to Create/Read/Update/Delete simple data for a particular purpose. In it's current state it is a single table application that can be simply generated for each of the three platforms we chose to compare.  Java EE 5.0, Ruby on Rails (which includes JRuby on Rails) and Grails.  We had intended getting this data up on our blogs earlier but I got busy on other stuff.  Recently there has been some activity around our JavaOne presentation  Some of the blogs I have seen: Joab Jackson and Grails Team  So I thought I better spend the time.

The Ruby Performace Tests:

Before we start we should state that Regperf is not nor can be, in it's current state, considered a benchmark.  The purpose was to simply test the characteristics of each of the frameworks compared to each other to determine where the framework was best suited and to make recommendations as to where each might be used. It was later we decided that Regperf using FABAN might be useful to others and decided to extend the application from a single table to a multi table application (Regperf 2.0)

Regperf 1.0

We generated the trivial application and tested it using FABAN  on each of the platforms.  Running the RoR application standalone and with in Netbeans 6 (JRuby).  We ran the Java EE 5.0 version on Glassfish and the Grails version was also run on the Glassfish application Server. Our findings were published in the Java One presentation.

Configuring Regperf 1.0 for Ruby:

Download the following files:


FABAN driver

Sample Database 

Start Postgres

Create a regperf user (createuser regperf)

Create a database called regperf ($ createdb regperf)

import the database ($pg_restore -d regperf regperfdb.tar)

To start the application:

cd regperf


In your web browser point to localhost:3000/subscription  The application should be running (email as much detail as possible if you are having problems and need some assistance include postgres server log, screen shots etc) Now all we need is the test suite (Faban Driver) which needs to be expanded in $FABAN_HOME/samples directory for pre-installed FABAN.  Now you can conduct your own tests.   Instructions that need to be followed are:

Setup Faban and the Faban Regperf driver Download and install Faban. Note Faban has a driver and also a harness for automatically queuing and running multiple benchmarks but in these instructions we are going to run just the driver component and again assuming localhost set the FABAN_HOME environment variable and add $FABAN_HOME/bin to your PATH mkdir $FABAN_HOME/output (this is where the tests results will be stored) Download the Faban Regperf driver for Rails (as indicated above) and unjar it into $FABAN_HOME/samples.  Edit the $FABAN_HOME/samples/regperf-grails/config/run.xml, change the value for outputDir to match your $FABAN_HOME/output directory.  Modify the number of simulated users by changing the value for <scale>1<scale> you will get 10 users for each scale factor e.g. scale 1 will give 10 simulated users.   Note the number of loaded subscribers should match the number of rows you load into the database (this is set initially to 100) refer to the Faban documentation for other values to change in the run.xml

Running the performance using the Rails runtime (webbrick)

cd $FABAN_HOME/samples/regperf-rails/sbin,  Assuming that the database and the rails application is still running execute ./loader.sh 10 start the rmi registry (this runs in the background) ./registry.sh &

Start and execute the benchmark

./master.sh Starts up all of the simulated users and writes results of the run into $FABAN_HOME/output/xx directory. The results can be conveniently viewed with a web browser.

We would love to hear what you find with either our app or your own.  Let us know if you have any problems and stay tuned for Regperf 2.0


Monday Jul 16, 2007

In response to the Grails Blog entry.

I recently saw this blog entry commenting about mistakes in our presentaion and questioning our findings.  Tom clearly stated in the presentation that the slide set used was not the final agreed set, but rather an earlier draft, that was prepared prior to us sitting down with the trademark people and copy editors.  We will have all of our code and presentation support documents posted to our blogs within a couple of days.  We look forward to having comments or suggestions on the correct slide set with the right graphs and results.

While I understand the time it took to get corrections done and get our info up onto the blogs was unfortunate, I can assure you we are in the final stages of getting the right slides to replace the ones on the website.  If you still want to contact us please feel free to do so our email addresses are on the presentation.

Monday Apr 16, 2007

The genesis of RegPerf

In this section of my blog you will find information regarding RegPerf, the small project Tom Daly and I have been creating for our JavaOne '07 presentation (TS-9535).

RegPerf is a small web based database application based on PostgreSQL that we have generated in Rails, Grails and Java EE 5, to demonstrate the characteristics of each framework and the more show the strengths and weaknesses of each framework. While using each of the three frameworks plus FABAN we found this tool maybe useful to a wider audiance and our intention is to continue this project after JavaOne to assist people to choose the right tool for the right job.

Stay Tuned for more information including performance results and source code. 




« July 2016