UFS vs ZFS: commandline editing

My previous post on UFS vs ZFS showed too much of a difference. I looked at all my tests and they all showed this same difference. Still I wanted to make sure...


So I redid the test and .... gone were does 85MB ZFS writes. Doing these tests I rely on command line editing and command line history. Try to use the same command line as before. So I walked through my history and did not spot anything. On the first pass that is.


Than I "just" spotted it. Once more the devil is in the details. The command line used was


#zfs set recordsize=8k pg


And that # character was not the prompt but a comment marker... Well this explains a lot. The record size that in fact was used was the deault setting, 128K. Each 8KB write was in fact a 128KByte write.


So I redid the ZFS based tests and here are the results for the 'all on one volume of nine disks.'


First the Writes/sec:


writeio.png


And the corresponding IO load:




writekb.png


Finally here is the TPM throughput graph for the 'all on nine' tests.




tpm2.png


Once more ZFS writes out more data. Looking at the TPM numbers the ZFS tests pushes a little less transactions through the system. Therefore the extra writing must be related to ZFS "overhead." The "good" thing is ZFS uses less but much larger IOs.


Although the maximum throughput is a little better for UFS, the ZFS behaviour is more constant. Overall the average looks very similar. No clear winner. But than it was not a contest, just a comparison. And surely I would like to understand those too noticable high-lows in throughput. Of course the writes are related to PostgreSQL its checkpoint configuration. This configuration was "tuned" for a certain set of tests were focus was on CPU consumption. In order to do this the checkpoints were moved to once per 900 seconds. Also PostgreSQL attempt to finish all checkpoint related IO in half of the checkpoint time. Another test is called for that causes a more constant and less extreme IO load.


And btw, the UFS tests were done without directio mounting: the ramp up is almost as fast as with ZFS.

Comments:

I don't understand the 85% and 15% numbers. If they're percentiles as in "85% of the time the i/o rate was above x writes/s" then it doesn't make sense that the 15% number would be above the 85% number for the same filesystem. Is it possible they're mislabeled and the red and blue are one filesystem and the green and purple the other filesystem?

Posted by Gerg on June 28, 2008 at 08:37 PM PDT #

Gerg, the PostgreSQL parameter "checkpoint_completion_target" was used for this setting. Its range is from 0.0 to 1.0. It depticts a ratio. Checkpoint A needs to finish before checkpoint B starts is the driving rule. (B is of course also the first checkpoint after A). When the completion target is 0.85 PostgreSQL tries hard to finish checkpoint A in 85% of the time that it has available. In my case where I have plenty of checkpoint files etc I "know" the checkpoint_timeout is the only checkpoint driving factor. I think this pattern shows quite nicely in the graphs. 15% (ratio=0.15) means a high peak of IO when checkpoint A starts that should finish real soon. In my case the time available was 300 seconds. 15% of this is very close to the observed interval of about 50 seconds.
I also experimented with 5%, but the amount of IO was too big for the number of disks I had available. It took approx 12% before the peak load could be completed.

Posted by Paul van den Bogaard on June 29, 2008 at 08:03 PM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

paulvandenbogaard

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today