Thursday Mar 08, 2012

Creating Vdbench503 Swat charts using Swat302

A few weeks ago I was reminded of an incompatibility between Vdbench503and Swat302 that I introduced quite a while back. The file format when using Swat302 'File' 'Import Vdbench data' for Vdbench503 data has changed. Since at this time no new version of Swat is available outside of Oracle that of course is a problem.

To help you convert the new 503 'swat_mon.txt' file contents so that it can be used by Swat302, run the  AWK program below.


# Conversion of Vdbench503 swat_mon.txt file to a format that can be read
# by Swat302 using 'Import Vdbench data'.
# Note that this AWK program reads file swat_mon.txt (or a copy of it) and
# then replaces swat_mon.txt again.
# I suggest therefore that you first make a copy of swat_mon.txt and use that
# as input, because if you would accidentally run TWICE using swat_mon.txt as
# both input AND output you will have destroyed the original content of the file.
# To run:
# - cd /vdbench503
# - awk -f convert.awk output/swat_mon.txt.copy > output/swat_mon.txt
# Henk.



  if ($1 == ":vdbench503_vdbench_data_for_swat303")
    print ":vdbench_data";
  else if (NF != 14)
    printf("%d %d %d %d %d %d %d %d %d %d %d %d %d %d\n",
            $1, $2, $3, $4, $5/1000, $6, $7, $8, $9/1000, 0, 0, 0, 0, 0);

Thursday May 26, 2011

Installing Java so that you can run Swat and Vdbench

Just received an email:

"Vdbench and Swat require JAVA to run.  Is there any way around this?  They would not let me load Java for fear it would change how the system was running."

There is no need to install Java in system directories and/or replace an existing version of Java. You can install Java in, and then run it from, your own private directories.


Monday Dec 20, 2010

Swat 3.02 has been moved to My Oracle Support (MOS)

As of February 15 the link to MOS in this blog entry has changed!

Swat 3.02 has been moved to My Oracle Support (MOS) and is available for download for properly licensed Oracle customers from here. Swat is now only distributed as a zip file but contains everything needed for both Unix or Windows systems.

You will need to have a valid MOS logon ID. You can also just go to MOS and request patch 10350687.


November 2013: An update: when I made Swat 302 available I put a five year expiration date on, fully expecting a new version to be available by then. And then Sun became Oracle, and a decision was made to no longer offer external support for Swat or make new versions available.

This now will cause Swat302 to expire on 'Thu Feb 13 14:45:12 MST 2014', warning messages will show up three months before that.

To work around this, download,  place it in the swat302 directory, and unzip. You should now have a new /swat302/classes/Swt/ directory. This will eliminate the expiration check.

Thursday Dec 16, 2010

Follow up on Swat download from failed: "User Not Authorized"

Though my discussions with Oracle legal have not completed yet it looks as if the limitation that only customers that have 'some' Oracle contract can download Swat will stay. I understand that the contract itself will be some minimal contract (you don't have to spend $100 million to get it), so loads of customers therefore should be able to get access to Swat. There is a Swat 3.03 coming; the legal status is pending.


Tuesday Sep 21, 2010

Swat download from failed: "User Not Authorized"

Swat has been generally available for free now for several years, and that will continue to be so.
The transition from Sun to Oracle however is causing some problems.
Masses of websites are being moved, lots of changes are made to policies related to Solaris and other licenses.
Since Swat alas resides on one of the websites whose policies have changed it fell through the cracks.
I have been working with legal to get this resolved, but legal is now 100% busy with Oracle's annual Oracle Open World (OOW) conference and, sad to say, Larry Ellison has a higher priority than poor lowly me :-)
Until then, if your company is a licensed Oracle Sun customer you may have to find a valid contract number to add to your signon. If you don't have a license you'll have to hold off until I get this resolved.
I'll keep working on it from my side here.


Friday Jun 18, 2010

Vdbench and Swat: how to identify what is what when using 'ps -ef'

For obvious reasons I frequently have multiple Swat or Vdbench processes running, and sometimes get confused as to what is what. 'ps' output is not very helpful. I can maybe compare the heap size values, but I don't always remember them:

hvxxxx 21027 21008   0 09:02:01 pts/12      0:06 java -Xmx512m -Xms128m -cp ./:./classes:./swat.jar:./javachart.jar:./swing-layo
hvxxxx 21060 21041   0 09:02:04 pts/6       0:03 java -Xmx1024m -Xms512m -cp ./:./classes:./swat.jar:./javachart.jar:./swing-lay

A primitive little trick now is making my life easier by using the -D java parameter. It shows me that I am running one background data collector and one local real time monitor (swat -c and swat -l)

hvxxxx 21102 21083   0 09:04:49 pts/12      0:07 java -Dreq=-c -Xmx512m -Xms128m -cp ./:./classes:./swat.jar:./javachart.jar:./s
hvxxxx 21240 21221   0 09:06:01 pts/6       0:34 java -Dreq=-l -Xmx1024m -Xms512m -cp ./:./classes:./swat.jar:./javachart.jar:./

Update to the swat script, you can make a similar change to the Vdbench script

if ("$1" == "-t" || "$1" == "-p" || "$1" == "-l") then
  $java -Dreq=$1 -Xmx1024m -Xms512m -cp $cp Swt.swat $\*
  $java -Dreq=$1 -Xmx512m  -Xms128m -cp $cp Swt.swat $\*


Wednesday Jun 09, 2010

Swat trace and prtvtoc

For Swat to be able to create a proper Replay parameter file it needs to know how large the luns are. For that the prtvtoc command is run during trace creation for each lun found in /dev/rdsk/. That can get awfully annoyingly slow when there are a lot of luns. To make this much faster replace the 'prtvtoc' lines in with:

# Generate prtvtoc data (Is needed to create Replay parameter file)
# (devfsadm -C will clean up old garbage in /dev/rdsk)
printf "Running prtvtoc command"
ls /dev/rdsk/\* > /tmp/tnfe1
nawk '{disk = substr($1,1,length($1)-2); if (disk != last) print $1; last = disk}' \\
      /tmp/tnfe1 > /tmp/tnfe2
rm /tmp/tnf_prtvtoc.txt 2> /dev/null
while read disk;do
   echo Running prtvtoc $disk
   echo $disk       >> /tmp/tnf_prtvtoc.txt
   prtvtoc -h $disk >> /tmp/tnf_prtvtoc.txt 2>/dev/null
done  < /tmp/tnfe2
cp /tmp/tnf_prtvtoc.txt tnf_prtvtoc.txt


Tuesday Apr 13, 2010

Swat Trace Facility (STF) memory problems during the Analyze phase.


STF during the Analyze phase keeps thirty seconds worth of I/O detail in memory. That is done so that when a trace probe of an I/O completion is found STF can still find the I/O start probe if that occurred less than thirty seconds earlier. Believe me, in some very difficult customer problem scenarios I have seen I/O that have taken that long.

If you run 5000 iops, keeping thirty seconds worth of detailed information in memory is not a real problem. Even 10k or 20k will work fine.

And now we have solid state devices, running 100k, 200k iops and up. Just do the math, and you know you will run into memory problems.

There is an undocumented option in STF to lower this 30-second value. In STF, go to the Settings tab, click on ‘batch_prm’, enter ‘-a5’, click ‘Save’, and run the Analyze phase again. Of course, -a2 works fine too. Note that any I/O that takes longer than the new ‘age’ value you specified will not be recognized by STF, but usually two seconds should just be fine.


Thursday Feb 11, 2010

Fixes for Swat 3.02

There are two little annoying bugs in swat 3.02:

- When using the 'swat=' parameter in a Vdbench parameter file, Swat runs into a date parsing error. (java.text.ParseException: Unparseable date)

- The Workload Visualizer in Swat Trace Facility (STF) does not work.

Download, and copy it as 'swat.jar' in your Swat installation directory.




Sunday Aug 16, 2009

Swat302 buffer overflow: java.nio.BufferOverflowException

When a system has lots of devices Swat may abort with:

   at java.nio.LongBuffer.put(
   at Utils.Bin.write_record( 

You can find a fix here:

- place tar file in your swat install directory, then 'cd /swatxxx' and 'tar -xvf binfix.tar'

You'll now have a new file: classes/Utils/Bin.class.

If that still fails then you have a REAL large amount of device. Contact me at  if needed.


Tuesday Jul 14, 2009

Vdbench: Sun StorageTek Vdbench, a storage I/O workload generator.

This is a copy of the blog entry I just created on Sun's BestPerf blog:

Vdbench is written in Java (and a little C) and runs on Solaris Sparc and X86, Windows, AIX, Linux, zLinux, HP/UX, and OS/X.

I wrote the SPC1 and SPC2 workload generator using the Vdbench base code for the Storage Performance Council:

Vdbench is a disk and tape I/O workload generator, allowing detailed control over numerous workload parameters like:


· For raw disk (and tape) and large disk files:

o Read vs. write

o Random vs. sequential or skip-sequential

o I/O rate

o Data transfer size

o Cache hit rates

o I/O queue depth control

o Unlimited amount of concurrent devices and workloads

o Compression (tape)

· For file systems:

o Number of directory and files

o File sizes

o Read vs. write

o Data transfer size

o Directory create/delete, file create/delete,

o Unlimited amount of concurrent file systems and workloads

Single host or Multi-host:

All work is centrally controlled, running either on a single host or on multiple hosts concurrently.


Centralized reporting, reporting and reporting using the simple idea that you can't understand performance of a workload unless you can see the detail. If you just look at run totals you'll miss the fact that for some reason the storage configuration was idle for several seconds or even minutes!

  • Second by second detail of by Vdbench accumulated performance statistics for total workload and for each individual logical device used by Vdbench.
  • For Solaris Sparc and X86: second by second detail of Kstat statistics for total workload and for each physical lun or NFS mounted device used.
  • All Vdbench reports are HTML files. Just point your browser to the summary.html file in your Vdbench output directory and all the reports link together.
  • Swat (an other of my tools) allows you to display performance charts of the data created by Vdbench: Just start SPM, then 'File' 'Import Vdbench data'.
  • Vdbench will (optionally) automatically call Swat to create JPG files of your performance charts.
  • Vdbench has a GUI that will allow you to compare the results of two different Vdbench workload executions. It shows the differences between the two runs in different grades of green, yellow and red. Green is good, red is bad.

Data Validation:

Data Validation is a highly sophisticated methodology to assure data integrity by always writing unique data contents to each block and then doing a compare after the next read or before the next write. The history tables containing information about what is written where is maintained in memory and optionally in journal files. Journaling allows data to be written to disk in one execution of Vdbench with Data Validation and then continued in a future Vdbench execution to make sure that after a system shutdown all data is still there. Great for testing mirrors: write some data using journaling, break the mirror, and have Vdbench validate the contents of the mirror.

I/O Replay

A disk I/O workload traced using Swat (an other of my tools) can be replayed using Vdbench on any test system to any type of storage. This allows you to trace a production I/O workload, bring the trace data to your lab, and then replay your I/O workload on whatever storage you want. Want to see how the storage performs when the I/O rate doubles? Vdbench Replay will show you. With this you can test your production workload without the hassle of having to get your data base software and licenses, your application software, or even your production data on your test system.

For more detailed information about Vdbench go to where you can download the documentation or the latest GA version of Vdbench.

You can find continuing updates about Swat and Vdbench on my blog:

Henk Vandenbergh

PS: If you're wondering where the name Vdbench came from :  Henk Vandenbergh benchmarking.

Storage performance and workload analysis using Swat.

This is a copy of the blog entry I just created on Sun's BestPerf blog:

Swat (Sun StorageTek Workload Analysis Tool) is a host-based, storage-centric Java application that thoroughly captures, summarizes, and analyzes storage workloads for both Solaris and Windows environments.

This tool was written to help Sun’s engineering, sales and service organizations and Sun’s customers understand storage I/O workloads.

 Sample screenshot:

Swat can be used for among many other reasons:

  • Problem analysis
  • Configuration sizing (just buying x GB of storage just won't do anymore)
  • Trend analysis: is my workload growing, and can I identify/resolve problems before they happen?

Swat is storage agnostic, so it does not matter what type or brand of storage you are trying to report on. Swat reports the host's view of the storage performance and workload, using the same Kstat (Solaris) data that iostat uses.

Swat consists of several different major functions:

· Swat Performance Monitor (SPM)

· Swat Trace Facility (STF)

· Swat Trace Monitor (STM)

· Swat Real Time Monitor

· Swat Local Real Time Monitor

· Swat Reporter

Swat Performance Monitor (SPM):

Works on Solaris and Windows. An attempt has been made in the current Swat 3.02 to also collect data on AIX and Linux. Swat 3.02 also reports Network Adapter statistics on Solaris, Windows, and Linux. A Swat Data Collector (agent) runs on some or all of your servers/hosts, collecting I/O performance statistics every 5, 10, or 15 minutes and writes the data to a disk file, one new file every day, automatically switched at midnight.

The data then can be analyzed using the Swat Reporter.

Swat Trace Facility (STF):

For Solaris and Windows. STF collects detailed I/O trace information. This data then goes through a data Extraction and Analysis phase that generates hundreds or thousands of second-by-second statistics counters. That data then can be analyzed using the Swat Reporter. You create this trace for between 30 and 60 minutes for instance at a time when you know you will have a performance problem.

A disk I/O workload traced using Swat can be replayed on any test system to any type of storage using Vdbench (an other of my tools, available at This allows you to trace a production I/O workload, bring the trace data to your lab, and then replay that I/O workload on whatever storage you want. Want to see how the storage performs when the I/O rate doubles or triples? Vdbench Replay will show you. With this you can test your production workload without the hassle of having to get your data base software and licenses, your application software and licenses, or even your production data.

Note: STF is currently limited to the collection of about 20,000 IOPS. Some development effort is required to handle the current increase in IOPS made possible by Solid State Devices (SSDs).

Note: STF, while collecting the trace data is the only Swat function that requires root access. This functionality is all handled by one single KSH script which can be run independently. (Script uses TNF and ADB).

Swat Trace Monitor (STM):

With STF you need to know when the performance problem will occur so that you can schedule the trace data to be collected. Not every performance problem however is predictable. STM will run an in-memory trace and then monitors the overall storage performance. Once a certain threshold is reach, for instance response time greater than 100 milliseconds, the in-memory trace buffer is dumped to disk and the trace then continues collecting trace data for an amount of seconds before terminating.

Swat Real Time Monitor:

When a Data Collector is active on your current or any network-connected host, Swat Real Time Monitor will open a Java socket connection with that host, allowing you to actively monitor the current storage performance either from your local or any of your remote hosts.

Swat Local Real Time Monitor:

Local Real Time Monitor is the quickest way to start using Swat. Just enter './swat -l' and Swat will start a private Data Collector for your local system and then will show you exactly what is happening to your current storage workload. No more fiddling trying to get some useful data out of a pile of iostat output.

Swat Reporter:

The Swat Reporter ties everything together. All data collected by the above Swat functions can be displayed using this powerful GUI reporting and charting function. You can generate hundreds of different performance charts or tabulated reports giving you intimate understanding of your storage workload and performance. Swat will even create JPG files for you that then can be included in documents and/or presentations. There is even a batch utility (Swat Batch Reporter) that will automate the JPG generation for you. If you want, Swat will even create a script for this batch utility for you.

Some of the many available charts:

  • Response time per controller or device
  • I/O rate per controller or device
  • Read percentage
  • Data transfer size
  • Queue depth
  • Random vs. sequential (STF only)
  • CPU usage
  • Device skew
  • Etc. etc.

Swat has been written in Java. This means, that once your data has been collected on its originating system, the data can be displayed and analyzed using the Swat Reporter on ANY Java enabled system, including any type of laptop.

For more detailed information go to (long URL)where you can download the latest release, Swat 3.02.

You can find continuing updates about Swat and Vdbench on my blog:

Henk Vandenbergh

Thursday Jul 09, 2009

Swat Analyze function and Java heap problems, continued

If you read my earlier blog about java heap problems with Analyze, here is an other change that you can make, but it has some risks.

Swat Analyze by default keeps the last 180 seconds worth of i/o detail in memory. This is done so that Swat can identify any i/o that takes up to 180 seconds (180 sounds high, but I have seen them). If you know for sure that you do not have any i/o taking longer than for instance 30 seconds, you can change this 180 second value.

When using the GUI, select the 'Settings' tab, click on the 'batch_prm' line, add '-a30' (don't add the quotes), click 'Save', and restart Analyze.

If you can't use the GUI, go to file 'options.sUSERID.ini' and manually change the value after 'batch_prm' to '-a30' and restart Analyze. 

Note though that then any i/o lasting longer than 30 seconds will NOT be recognized by Swat.

Swat Analyze function and Java heap problems (OutOfMemoryError)

The default Java heap size given to Swat Analyze is -Xmx1024m (this is hardcoded, and is not to be confused with the heap size specified in the swat and swat.bat scripts).

For long traces and/or when you have lots of luns, 1024m may not always be enough. Could I increase the default? Technically, yes. However, for users that do not have much memory or swap space that could mean that they then can not even run the Swat Analyze at all. 1024m therefore is a decent starting value.

When your Analyze fails with 'java.lang.OutOfMemoryError: Java heap space', ignore the suggestion in  message 'Rerun as 'java -Xmx512m...', since that though technically is correct, is not complete enough. Changing the swat/swat.bat scripts also will not solve this.

When using the GUI, select the 'Settings' tab, click on the 'java_prm' line, increase the -Xmx value (the highest I have been able to do is -Xmx2560m), click 'Save', and restart Analyze.

If you can't use the GUI, go to file 'options.sUSERID.ini' and manually change the value after 'java_prm' and restart Analyze. 

Monday Jul 06, 2009

External mailing addresses for Swat and Vdbench

I just received an email from sourceforge where vdbench resides, but my reply email bounced, 'unknown user'. Don't know why, but here is a better way to communicate with me:

Vdbench:, and Swat: I'll do my best to respond ASAP. Of course, Sun employees know where to find me at my internal email address, and they will get priority.



Blog for Henk Vandenbergh, author of Vdbench, and Sun StorageTek Workload Analysis Tool (Swat). This blog is used to keep you up to date about anything revolving around Swat and Vdbench.


« July 2016