FileBench : a New Era in FS Performance

I'm happy to report that FileBench has gone over a significant overhaul and we're happy to release that updated version. Bits will be posted to sourceforge tonight. I'm also happy to report that FileBench is now included in OpenSolaris. You can find it in our new "/usr/benchmarks" path.

Ok, great, just what the industry needed - FileBench is just another simple benchmark. right? Nope.

First let me give you, dear reader, a taste of what we get internally and externally here at the ZFS team:

"I ran bonnie, dd, and tar'd up myfav_linuxkernel_tarball.tar.  Your file system sucks."

Though sometimes i'm happy to note we get:

"I ran bonnie, dd, and tar'd up myfav_linuxkernel_tarball.tar.  Your file system rulz!."

What is FileBench?

It is nice to hear that your file system does in fact "rule", but the problem with the above is that bonnie, dd, and tar are (obviously) not a comprehensive set of applications that can completely measure a file system. IOzone is quite nice but it only tests basic I/O patterns. And there are many other file system benchmarks (mkfile, fsstress, fsrandom, mongo, iometer, etc). The problem with all of these benchmarks is that they only measure a specific workload (or a small set of workloads). None of them actually let you measure what a real application does. Yes part of what Oracle does is random aligned reads/writes (which many of the pre-mentioned benchmarks can measure), but the key is how the random read/writes interact with each other \*and\* how they interact with the other parts of what Oracle does (the log writer as a major example). None of the pre-mentioned benchmarks can do that.

Enter FileBench.

So how does FileBench differ? FileBench is a framework of file system workloads for measuring and comparing file system performance. The key is in the workloads. FileBench has a simple .f language that allows you to describe and build workloads to simulate applications. You can create workloads to replace all the pre-mentioned benchmarks. But more importantly, you can create workloads to simulate complex applications such as a database. For instance, i didn't have to buy an Oracle license nor figure out how to install it on my system to find out if my changes to the vdev cache for ZFS helped database performance or not. I just used FileBench and its 'oltp.f' workload.

How do i Start using FileBench?

The best place to start is via the quick start guide. You can find out lots more at our wiki. Lots of good information and a great place to contribute. For trouble shooting see the gotchas section.

How do i Contribute to FileBench?

If you'd like to write your own workloads, check out the very nice documentation Drew Wilson wrote up. This is actually where we (where we is the FileBench community, not just us Sun people) would love the most contribution. We would really like to have people verify the workloads we have today and build the workloads for tomorrow. This is a great opportunity for industry types and academics. We very much plan to incorporate new workloads into FileBench.

If you would like to help with the actual code of the filebench binaries, find us at perf-discuss@opensolaris.org.

FileBench is done, right?

Um, nope. There's actually lots of interesting work up ahead for FileBench. Besides building new workloads, the two other major focuses that need help are: multi-client support and plug-ins. The first is pretty obvious - we need to have support for multiple clients to benchmark NFS and CIFS. And we need that to work on multiple platforms (OpenSolaris, linux, \*BSD, OSX, etc). The second is where experts in specific communities can help out. Currently, FileBench goes through whatever client/initiator implementation you have on your machine. But if you wanted to just do a comparison of server/target implementations, then you need a plug-in built into FileBench that both systems utilize (even if they are different operating systems). We started prototyping a plug-in for NFSv3. We've also thought about NFSv4, CIFS, and iSCSI. A community member suggested XAM. This is a very interesting space to explore.

So What does this all Mean?

If you need to do file system benchmarking, try out FileBench. Let us know what you like and what needs some love.

If you're thinking about developing a new file system benchmark, consider creating a new workload for FileBench instead. If that works out for you, please share your work. If for some reason it doesn't, please let us know why.

We really believe in the architecture of FileBench and really want it to succeed industry wide (file system and hence storage). We know it works quite well on OpenSolaris and would love other developers to make sure it works just as well on their platforms (linux, \*BSD, OSX, etc.).

Happy Benchmarking and long live RMC!

Comments:

Well done! I'm really looking forward to seeing how the different workloads exercise different hardware.

BTW, also missing is support for raw devices. Though RMC says that should be easy.

Posted by Richard Elling on October 04, 2007 at 03:07 PM PDT #

Aaaah, raw device support is coming very soon. Drew Wilson has a code review out for adding that support to "files" (but not "filesets" - obviously).

Posted by eric kustarz on October 05, 2007 at 02:21 AM PDT #

Sweeeeet! I'm a bit bummed that some of the auxiliary tools like davenet and statit were removed, but I'm excited to play with this new release.

Posted by benr on October 05, 2007 at 05:09 AM PDT #

Yeah those didn't make it as they aren't in OpenSolaris quite yet and i wasn't able to track down the source.

If we find them and they are not too bloated, then its quite conceivable that they make it into OpenSolaris as well (and then FileBench could leverage them).

Posted by eric kustarz on October 05, 2007 at 05:55 AM PDT #

Excellent news! Started playing with filebench only last week, so it's really good to see it's had an overhaul.

Do you know what build it will appear in yet?

Posted by Dick Davies on October 07, 2007 at 05:56 AM PDT #

It will be in snv_76 and i'll try to get it into s10u5.

Posted by eric kustarz on October 07, 2007 at 11:39 AM PDT #

Congrats Eric! There's lots of interest in FileBench here too!

On the raw device comment: one of the future items we always wanted to do was eliminate files, and make them a subset of filesets. RAW could be emulated by allowing filesets to be defined by a string of pathnames, using some additional syntax...

Posted by Richard McDougall on October 10, 2007 at 05:44 AM PDT #

Wow, the filebench creator came down and read my blog :)

Funny you should mention that, drew wilson actually has the code review out for eliminating most of the "file" code to make them implementation wise a subset of "filesets". We're deciding to keep the defintion of "files" around as it makes more sense to define workloads based on a single file than requiring users to specify a "fileset" with 1 file. We're of course open to comments

And yes... RAW device support be there as well.

Posted by eric kustarz on October 10, 2007 at 05:51 AM PDT #

Post a Comment:
Comments are closed for this entry.
About

erickustarz

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today