Filesystem Benchmarks: vxbench
By dom on Jan 23, 2006
For a long time I've used a simple I/O load generator from Veritas called vxbench for doing just that - generating I/O loads against systems that have been configured up either in the lab or on customer sites. vxbench is a tool available on AIX, HP-UX, Linux and Solaris for benchmarking I/O loads on raw disk or file systems. It can produce various I/O workloads such as sequential and random reads/writes asynchronous I/Os, and memory mapped (mmap) operations. It has many options specific to the VERITAS File System (VxFS).
It also has characteristics that I need in a simple load generator - specifically it can generate multithreaded workloads which are essential and it has a simple command-line interface which makes it easy to incorporate in a scripting harness. It can also do strided reads/writes and sleep - important for database-like operations.
vxbench arrives on the CD in the package VRTSspt - the Veritas Software Support Tools and most sites have it to hand. However I've always shied away from publishing any work done with it because I 've never quite pinned down its status as a piece of software in terms of copyright or license. Recently however I've been driven to take a closer look. Two papers appeared recently which I'm afraid I can cite but not give you a URL for:
- Study of Linux I/O Performance Characteristics for Volume Managers on an Interl Xeon Server (2004) Xianneng Shen, Jim Nagler, Randy Taylor, Clark McDonald. Proc CMG 2004
- I/O Performance Characteristics for Volume Managers on LInux 2.6 Servers (2005) Dan Yee, Xianneng Shen. Proc CMG 2005
As corporate history has moved on, the first paper is copyrighted by the VERITAS Software Corporation and the second by the Symantec Corporation. I never realised that CMG does not own the content of its own proceedings but there you are. The second paper is a continuation of the first and uses the same methodology and tools. Yes; vxbench.
At first sight this is a little annoying - as another recent paper (which I won't point you at just at the moment because I want to talk about it in more detail in a later post) pointed out, if you can't reproduce a benchmark from its report, its not really very scientific and I'm sure thats not what the authors of these papers intended. This need for the rigor imposed by writing reproduceability into benchmarking papers is one reason why people working in the field often resort to the "usual suspects" when looking for load generators - iozone, postmark, bonnie++. They all have their weaknesses but are at least available on the net.
So I set about tracking down vxbench. The header in the source code was not encouraging; "This software contains confidential information and trade secrets of VERITAS Software. Use, disclosure or reproduction is prohibited without prior express written permission of VERITAS Software". Well, I won't be sharing any more of the contents of vxbench.c with you, thats for sure. Onward!
to....a Veritas support document pointed out to me by the README that comes with the package. Apparently you can download the package from the VERITAS ftp site (without the need to purchase media and/or a license). The support document was no more encouraging than the source header; "These tools are designed to be used under the direction of a VERITAS Technical Support Engineer only." Does this mean you shouldn't use them in other circumstances? (For "other circumstances" read "benchmarking against competing vendors of storage software") Well, it seems you can. Document 261451 leaves out the sentence that follows the one the I've quoted, but in the README.VRTSspt it continues on; "Any other use of these tools is at your own risk." So you can amuse yourself with vxbench and publish the results but if you fry your disks and panic your system you have only yourself to blame.
Vxbench is a useful tool. Its availability is important - the implementors of Linux LVM (and VxVM!) will no doubt want to study these papers and work to improve their products. I'm glad Symantec continue to make it available to the storage software community.