Thursday Apr 22, 2010

Vdbench file system testing WITHOUT using 'format=yes'

Just found a bug in Vdbench 5.02 around file system testing.

When not having Vdbench create all the files using the 'format=yes' parameter, but instead coding fileselect=sequential,fileio=sequential,operation=write in the File system Workload Definition (FWD) to simulate the format, Vdbench only creates the first 'threads=n' files and then overwrites them again and again. If you are in desperate need for a fix, let me know.

 Henk.

Monday Jan 26, 2009

Example for multi-host file system testing

\*
\* This is a multi-host file system test example.
\* Note that an FSD can be used only from ONE host. This to make sure that
\* one host is not deleting a file that an other host is using.
\*
\* Note: with Vdbench 5.00 not specifying specific host names in the FWD
\* will cause a nullpointer Exception.
\*
host=default,user=userx
host=(hosta,one)
host=(hostb,two)

fsd=fsd1,anchor=/home/userx/junk/dir1,depth=2,width=2,files=2,size=128k
fsd=fsd2,anchor=/home/userx/junk/dir2,depth=2,width=2,files=2,size=128k

fwd=fwd1,host=one,fsd=fsd1,operation=read,xfersize=4k,fileio=sequential,fileselect=random,threads=2
fwd=fwd2,host=two,fsd=fsd2,operation=read,xfersize=4k,fileio=sequential,fileselect=random,threads=2

rd=rd1,fwd=fwd\*,fwdrate=max,format=yes,elapsed=10,interval=1

File system testing with large amounts of files.

When you start playing with thousands of files, you may get the following message:
"Waiting for slave synchronization: "

In the next version the message text changes to:
"Waiting for slave synchronization: Building and validating file structure(s) "

When Vdbench internally builds the file and directory structure there are two things that can take a while:
- Allocating about 60 bytes of memory for each file in the Java Heap, and Java doing a lot of Garbage Collection to find all that.
- If the directory structure already exists Vdbench needs to keep going to the directory to find out: "does the file exist", and 'How long is it".

This all can take quite a while if you're dealing with huge amounts of files.
About

Blog for Henk Vandenbergh, author of Vdbench, and Sun StorageTek Workload Analysis Tool (Swat). This blog is used to keep you up to date about anything revolving around Swat and Vdbench.

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today