Thursday Apr 22, 2010

Vdbench and SSD alignment, continued.

Of course, it took only a few minutes before someone asked 'how can I run this against files, not volumes'. Here is the response:

Just change the lun to a file name (use the same file name each  time) and add a size.
Vdbench then will first create the file for you.
The problem will be that you need to make sure Vdbench will not read from file system or file server cache, so the file size must be at least 5 times the system's cache size,
Unless of course you mount stuff directio, but then you still have the file server cache to deal with.
Just take your time and create a large file (I am using 100g). Vdbench will automatically create it for you.

BTW: the elapsed time the elapsed time needs to be long enough to make sure you get away from cache. I set it here to 60 seconds, which should be a good start..

Henk.

hd=default,jvms=1
sd=default,th=32
sd=default,size=100g
sd=sd_0000,lun=/dir/filename,offset=0000
sd=sd_0512,lun=/dir/filename,offset=0512
sd=sd_1024,lun=/dir/filename,offset=1024
sd=sd_1536,lun=/dir/filename,offset=1536
sd=sd_2048,lun=/dir/filename,offset=2048
sd=sd_2560,lun=/dir/filename,offset=2560
sd=sd_3072,lun=/dir/filename,offset=3072
sd=sd_3584,lun=/dir/filename,offset=3584
sd=sd_4096,lun=/dir/filename,offset=4096
wd=wd1,sd=sd_1,xf=4k,rdpct=100
rd=default,iorate=max,elapsed=60,interval=1,dist=d,wd=wd1
rd=rd_0000,sd=sd_0000
rd=rd_0512,sd=sd_0512
rd=rd_1024,sd=sd_1024
rd=rd_1536,sd=sd_1536
rd=rd_2048,sd=sd_2048
rd=rd_2560,sd=sd_2560
rd=rd_3072,sd=sd_3072
rd=rd_3584,sd=sd_3584
rd=rd_4096,sd=sd_4096


Vdbench and SSD alignment

These last months I have heard a lot about issues related to solid state devices not properly being aligned to the expected data transfer sizes. Each OS has its own way of creating volumes and partitions so trying to figure out if everything is neatly aligned is not an easy job. Add to that the possibility of the OS thinking everything is in order but alignment somewhere down the line not being accurate in one of the many possible layers of software when we have virtual volumes.

Without really being interested in the 'how to figure it all out and how to fix alignment issues' I created a small Vdbench parameter file that will allow you to at least figure out whether things are properly aligned or not. It revolves around the use of the Vdbench 'offset=' parameter that allows you to artificially change the alignment from Vdbench's point of view.

If your SSDs are on a storage subsystem that has a large cache, make sure that your volume is much larger than that cache. You rally need to make sure you are getting your data from the SSD, not from cache.

Henk:

hd=default,jvms=1
sd=default,th=32
sd=sd_0000,lun=/dev/rdsk/c7t0d0s4,offset=0000
sd=sd_0512,lun=/dev/rdsk/c7t0d0s4,offset=0512
sd=sd_1024,lun=/dev/rdsk/c7t0d0s4,offset=1024
sd=sd_1536,lun=/dev/rdsk/c7t0d0s4,offset=1536
sd=sd_2048,lun=/dev/rdsk/c7t0d0s4,offset=2048
sd=sd_2560,lun=/dev/rdsk/c7t0d0s4,offset=2560
sd=sd_3072,lun=/dev/rdsk/c7t0d0s4,offset=3072
sd=sd_3584,lun=/dev/rdsk/c7t0d0s4,offset=3584
sd=sd_4096,lun=/dev/rdsk/c7t0d0s4,offset=4096
wd=wd1,sd=sd_1,xf=4k,rdpct=100
rd=default,iorate=max,elapsed=60,interval=1,dist=d,wd=wd1
rd=rd_0000,sd=sd_0000
rd=rd_0512,sd=sd_0512
rd=rd_1024,sd=sd_1024
rd=rd_1536,sd=sd_1536
rd=rd_2048,sd=sd_2048
rd=rd_2560,sd=sd_2560
rd=rd_3072,sd=sd_3072
rd=rd_3584,sd=sd_3584
rd=rd_4096,sd=sd_4096


These are the 'avg' lines:

offset=0000   avg_2-3   19223.00    75.09    4096 100.00    1.580    2.803    0.231     1.1   0.9
offset=0512   avg_2-3    3655.50    14.28    4096 100.00    8.772    9.473    0.067     0.3   0.2
offset=1024   avg_2-3    3634.00    14.20    4096 100.00    8.784    9.390    0.064     0.3   0.2
offset=1536   avg_2-3    3633.00    14.19    4096 100.00    8.799    9.472    0.062     0.3   0.2
offset=2048   avg_2-3    3614.50    14.12    4096 100.00    8.831    9.440    0.066     0.3   0.2
offset=2560   avg_2-3    3604.00    14.08    4096 100.00    8.852    9.477    0.067     0.2   0.2
offset=3072   avg_2-3    3602.50    14.07    4096 100.00    8.853    9.430    0.059     0.3   0.2
offset=3584   avg_2-3    3597.50    14.05    4096 100.00    8.888    9.468    0.069     0.2   0.2
offset=4096   avg_2-3   20050.50    78.32    4096 100.00    1.584    2.811    0.231     1.0   0.9

As you can see, the runs with offset=0 and offset=4096 offer more than 5 times the throughput than the others. This tells me that this volume is properly aligned.
If for instance the run results would show that offset=512 has the best results the volume is on a 512 byte offset.
To then run properly 4k aligned tests with Vdbench, add to all your runs:
sd=default,offset=512
and Vdbench, after generating each lba, will always add 512.




About

Blog for Henk Vandenbergh, author of Vdbench, and Sun StorageTek Workload Analysis Tool (Swat). This blog is used to keep you up to date about anything revolving around Swat and Vdbench.

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today