Freitag Mai 29, 2009

FileBench ported to Mac OS X

I recently ported the benchmarking utility FileBench to Mac OSX. The chief attraction of FileBench is, that one can define the I/O operations put on the filesystem in something called the "Workload Definition Language". So - it's not a fixed benchmark, burden the system with the very same load pattern over and over again but an amazingly flexible Software which can simulate various workloads like file servers, databases, web servers etc.

FileBench was created by Richard McDougall who is a Solaris and Systems programming guru and one of the authors of the famous Solaris Internals book. More information about FileBench could be found at the official touch base here. FileBench is now included in OpenSolaris. Take a look here for the sources. There is also a SourceForge-Version landing page for sources which build on Linux.

Here's an example, how this definition language looks like:

set $dir=/tmp
set $nfiles=1000
set $meandirwidth=1000000
set $filesize=16k
set $nthreads=16
set $meaniosize=16k
 
define fileset name=bigfileset,path=$dir,size=$filesize,entries=$nfiles,dirwidth=$meandirwidth,prealloc=80
 
define process name=filereader,instances=1
{
  thread name=filereaderthread,memsize=10m,instances=$nthreads
  {
    flowop deletefile name=deletefile1,filesetname=bigfileset
    flowop createfile name=createfile2,filesetname=bigfileset,fd=1
    flowop appendfilerand name=appendfilerand2,iosize=$meaniosize,fd=1
    flowop fsync name=fsyncfile2,fd=1
    flowop closefile name=closefile2,fd=1
    flowop openfile name=openfile3,filesetname=bigfileset,fd=1
    flowop readwholefile name=readfile3,fd=1
    flowop appendfilerand name=appendfilerand3,iosize=$meaniosize,fd=1
    flowop fsync name=fsyncfile3,fd=1
    flowop closefile name=closefile3,fd=1
    flowop openfile name=openfile4,filesetname=bigfileset,fd=1
    flowop readwholefile name=readfile4,fd=1
    flowop closefile name=closefile4,fd=1
  }
}

On the FileBench homepage and in the distributions "workloads" directory one can find many more examples how to write these definitions.

Here is an console-log of loading and running the varmail benchmark:

$ /opt/filebench/bin/go_filebench

FileBench Version 1.4.4
filebench> load varmail
1364: 10.634: Varmail Version 2.1 personality successfully loaded
1364: 10.634: Usage: set $dir=<dir>
1364: 10.634: set $filesize=<size> defaults to 16384
1364: 10.634: set $nfiles=<value> defaults to 1000
1364: 10.634: set $nthreads=<value> defaults to 16
1364: 10.634: set $meaniosize=<value> defaults to 16384
1364: 10.634: set $readiosize=<size> defaults to 1048576
1364: 10.634: set $meandirwidth=<size> defaults to 1000000
1364: 10.634: (sets mean dir width and dir depth is calculated as log (width, nfiles)
1364: 10.634: dirdepth therefore defaults to dir depth of 1 as in postmark
1364: 10.634: set $meandir lower to increase depth beyond 1 if desired)
1364: 10.634:
1364: 10.634: run runtime (e.g. run 60)
filebench> set $dir=/tmp
filebench> run 10
1364: 26.250: Creating/pre-allocating files and filesets
1364: 26.252: Fileset bigfileset: 1000 files, 0 leafdirs avg dir = 0, avg depth = 0.0, mbytes=4602678819172646912
1364: 26.310: Removed any existing fileset bigfileset in 1 seconds
1364: 26.314: making tree for filset /tmp/bigfileset
1364: 26.314: Creating fileset bigfileset...
1364: 27.009: Preallocated 807 of 1000 of fileset bigfileset in 1 seconds
1364: 27.009: waiting for fileset pre-allocation to finish
1364: 27.009: Starting 1 filereader instances
1364: 27.009: Starting 16 filereaderthread threads
1364: 28.010: Running...
1364: 38.011: Run took 10 seconds...
1364: 38.012: Per-Operation Breakdown
closefile4 95ops/s 0.0mb/s 0.0ms/op 0us/op-cpu
readfile4 95ops/s 1.3mb/s 16.8ms/op 0us/op-cpu
openfile4 95ops/s 0.0mb/s 0.0ms/op 0us/op-cpu
closefile3 95ops/s 0.0mb/s 0.1ms/op 0us/op-cpu
fsyncfile3 96ops/s 0.0mb/s 62.1ms/op 0us/op-cpu
appendfilerand3 96ops/s 0.7mb/s 4.1ms/op 0us/op-cpu
readfile3 96ops/s 1.3mb/s 17.1ms/op 0us/op-cpu
openfile3 96ops/s 0.0mb/s 0.0ms/op 0us/op-cpu
closefile2 96ops/s 0.0mb/s 0.1ms/op 0us/op-cpu
fsyncfile2 97ops/s 0.0mb/s 59.7ms/op 0us/op-cpu
appendfilerand2 97ops/s 0.7mb/s 0.1ms/op 0us/op-cpu
createfile2 97ops/s 0.0mb/s 0.8ms/op 0us/op-cpu
deletefile1 97ops/s 0.0mb/s 3.2ms/op 0us/op-cpu

1364: 38.012:
IO Summary: 12515 ops, 1251.4 ops/s, (192/194 r/w) 4.2mb/s, 3587919346114us cpu/op, 40.9ms latency
1364: 38.012: Shutting down processes
filebench> stats dump "stats.varmail"
1364: 69.662: Stats dump to file 'stats.varmail'
1364: 69.662: in statsdump stats.varmail
filebench> quit
$ more stats.varmail
Flowop totals:
closefile4 95ops/s 0.0mb/s 0.0ms/op 0us/op-cpu
readfile4 95ops/s 1.3mb/s 16.8ms/op 0us/op-cpu
openfile4 95ops/s 0.0mb/s 0.0ms/op 0us/op-cpu
closefile3 95ops/s 0.0mb/s 0.1ms/op 0us/op-cpu
fsyncfile3 96ops/s 0.0mb/s 62.1ms/op 0us/op-cpu
appendfilerand3 96ops/s 0.7mb/s 4.1ms/op 0us/op-cpu
readfile3 96ops/s 1.3mb/s 17.1ms/op 0us/op-cpu
openfile3 96ops/s 0.0mb/s 0.0ms/op 0us/op-cpu
closefile2 96ops/s 0.0mb/s 0.1ms/op 0us/op-cpu
fsyncfile2 97ops/s 0.0mb/s 59.7ms/op 0us/op-cpu
appendfilerand2 97ops/s 0.7mb/s 0.1ms/op 0us/op-cpu
createfile2 97ops/s 0.0mb/s 0.8ms/op 0us/op-cpu
deletefile1 97ops/s 0.0mb/s 3.2ms/op 0us/op-cpu

IO Summary: 12515 ops 1251.4 ops/s, 192/194 r/w 4.2mb/s, 3587919346114uscpu/op

I've created a Diskimage including an Apple Installer package which installs the software into /opt/filebench. Just download the image file and you could install the program in an Apple fashion:


As a next step the process of creating the Mac-version from the OpenSolaris sources should be automated like this:


Here's the first version of the script, the skeleton and a set of patches to automate this.

Freitag Feb 20, 2009

Running Openbravo on Glassfish v3

Last week i tried to run Openbravo 2.40 on Glassfish v3 (prelude version). Here i like to share my findings and how to get this setup running for those who follow. I neither explain how to install the necessary database nor the tomcat installation. The database was in my case PostgreSQL. I did the installation on a MacBook Pro/Intel running Mac OS X 10.5.6.

I had to get the "official" installation image for Openbravo on my platform (Mac OSX) which asks for the location of a pre-installed tomcat and login credentials for either a Oracle or PostgreSQL database. You have to run the standard installation process to get all schemas and data loaded into your preferred DB.

This can take up quite some time, but after this installation Openbravo could be found in tomcat's webapps directory: $TOMCAT_HOME/webapps/openbravo/openbravo.war. Trying to deploy this WAR-archive right away to Glassfish fails for reasons I'm going to explain below. Since there is one slight modification to Openbravo's deployment-descriptor (WEB-INF/web.xml) needed, we're unpack openbravo.war into it's own directory and deploy the application out of this directory.

The problem with the existing deployment-descriptor is, that it's written for Servlet 2.3 webcontainers like Glassfish v2 or tomcat. Glassfish v3 adheres to the Servlet 2.4 specification which is a bit picky about url-pattern. In principle this means you have to code this:

<url-pattern>
       /ad_callouts/SL_ProductionPlan_Conversion.html
</url-pattern>

this way:

<url-pattern>/ad_callouts/SL_ProductionPlan_Conversion.html</url-pattern>

This is tracked, accepted and fixed as issue 7493 in Openbravo. Once this change makes it into Openbravo, this whole document becomes obsolete. But till then you have to apply this patch to WEB-INF/web.xml to rewrite the deployment descriptor.

After this modification, just deploy Openbravo using Glassfish's admin GUI using this trail: Applications/Web Applications/Deploy ...:

After a few seconds the new deployed application should show up in the list of your web-applications:


Now you should be able to login using Openbravo's default account Openbravo/openbravo :

which in turn take you to Openbravo's main dashboard:



About

schmidtm

Search

Archives
« April 2014
MoDiMiDoFrSaSo
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Heute