Tuesday Dec 18, 2007

Complete Sourceforge build for FileBench

Awhile back, we gave FileBench a much needed facelift. After its botox injection, we updated the sourceforge site's source code and integrated FileBench into Opensolaris.

Now i'm happy to report that we have updated the building process on the sourceforge site so that anyone can do a complete build with the updated source (as in the freshest FileBench source in the world). This complete build not only builds the open source parts (go_filebench, filebench(1), workloads, .prof files, scripts), but also includes the closed source binaries (such as davenet and statit). So yes xanadu is back! For those curious, the reason davenet and statit are closed binaries is because i don't happen to have the source code for them.

This update also includes pre-built packages for x86 and sparc (Solaris 10/OpenSolaris only). As i'm not familiar with creating packages for OSX, \*BSD, or linux, if someone from that knowledge set wants to help out and automate the process to build packages for non-OpenSolaris platforms, we'd be much obliged.

"file"s merged into "fileset"s

Drew just putback 6601818 Turn FileBench "files" into filesets with 1 entry, which was a nice cleanup that merged the implementation of files into the filesets's implementation. In FileBench News (sometimes comes out quarterly, sometimes bi-monthly), you can see Drew's implications of the changes.

This is a very nice simplification of the code and something that has been on the "todo" list for over two years. This was a major change, so FileBench has been updated to version 1.1.0 (from the previous 1.0.1). You can find these changes in OpenSolaris build snv_81 and immediately on sourceforge.

More goodness in the works...

FileBench Source and Bug/RFE Info

You can now easily browse the source code for FileBench in OpenSolaris using OpenGrok. I find OpenGrok much friendlier to use than what sourceforge offers.

For a basic breakdown of the source, the \*.c's, \*.h's, \*.l, and \*.y that construct the C binary 'go_filebench' can be found here. You can also browse the workloads, the main perl script filebench(1), the .prof files, and the scripts to compare results and to flush file system caches.

You can now also query our bug database for FileBench bugs found and perhaps more interestingly RFEs requested on OpenSolaris.

Thursday Oct 04, 2007

FileBench : a New Era in FS Performance

I'm happy to report that FileBench has gone over a significant overhaul and we're happy to release that updated version. Bits will be posted to sourceforge tonight. I'm also happy to report that FileBench is now included in OpenSolaris. You can find it in our new "/usr/benchmarks" path.

Ok, great, just what the industry needed - FileBench is just another simple benchmark. right? Nope.

First let me give you, dear reader, a taste of what we get internally and externally here at the ZFS team:

"I ran bonnie, dd, and tar'd up myfav_linuxkernel_tarball.tar.  Your file system sucks."

Though sometimes i'm happy to note we get:

"I ran bonnie, dd, and tar'd up myfav_linuxkernel_tarball.tar.  Your file system rulz!."

What is FileBench?

It is nice to hear that your file system does in fact "rule", but the problem with the above is that bonnie, dd, and tar are (obviously) not a comprehensive set of applications that can completely measure a file system. IOzone is quite nice but it only tests basic I/O patterns. And there are many other file system benchmarks (mkfile, fsstress, fsrandom, mongo, iometer, etc). The problem with all of these benchmarks is that they only measure a specific workload (or a small set of workloads). None of them actually let you measure what a real application does. Yes part of what Oracle does is random aligned reads/writes (which many of the pre-mentioned benchmarks can measure), but the key is how the random read/writes interact with each other \*and\* how they interact with the other parts of what Oracle does (the log writer as a major example). None of the pre-mentioned benchmarks can do that.

Enter FileBench.

So how does FileBench differ? FileBench is a framework of file system workloads for measuring and comparing file system performance. The key is in the workloads. FileBench has a simple .f language that allows you to describe and build workloads to simulate applications. You can create workloads to replace all the pre-mentioned benchmarks. But more importantly, you can create workloads to simulate complex applications such as a database. For instance, i didn't have to buy an Oracle license nor figure out how to install it on my system to find out if my changes to the vdev cache for ZFS helped database performance or not. I just used FileBench and its 'oltp.f' workload.

How do i Start using FileBench?

The best place to start is via the quick start guide. You can find out lots more at our wiki. Lots of good information and a great place to contribute. For trouble shooting see the gotchas section.

How do i Contribute to FileBench?

If you'd like to write your own workloads, check out the very nice documentation Drew Wilson wrote up. This is actually where we (where we is the FileBench community, not just us Sun people) would love the most contribution. We would really like to have people verify the workloads we have today and build the workloads for tomorrow. This is a great opportunity for industry types and academics. We very much plan to incorporate new workloads into FileBench.

If you would like to help with the actual code of the filebench binaries, find us at perf-discuss@opensolaris.org.

FileBench is done, right?

Um, nope. There's actually lots of interesting work up ahead for FileBench. Besides building new workloads, the two other major focuses that need help are: multi-client support and plug-ins. The first is pretty obvious - we need to have support for multiple clients to benchmark NFS and CIFS. And we need that to work on multiple platforms (OpenSolaris, linux, \*BSD, OSX, etc). The second is where experts in specific communities can help out. Currently, FileBench goes through whatever client/initiator implementation you have on your machine. But if you wanted to just do a comparison of server/target implementations, then you need a plug-in built into FileBench that both systems utilize (even if they are different operating systems). We started prototyping a plug-in for NFSv3. We've also thought about NFSv4, CIFS, and iSCSI. A community member suggested XAM. This is a very interesting space to explore.

So What does this all Mean?

If you need to do file system benchmarking, try out FileBench. Let us know what you like and what needs some love.

If you're thinking about developing a new file system benchmark, consider creating a new workload for FileBench instead. If that works out for you, please share your work. If for some reason it doesn't, please let us know why.

We really believe in the architecture of FileBench and really want it to succeed industry wide (file system and hence storage). We know it works quite well on OpenSolaris and would love other developers to make sure it works just as well on their platforms (linux, \*BSD, OSX, etc.).

Happy Benchmarking and long live RMC!




« June 2016