Speed Up Your Server For Free: Using ZFS To Your Advantage For Access Logs

Rationale

Access that can cause some performance focused users some discomfort. The one main thing usually making logs a performance hog is the fact that entries must be ordered somehow. I our products, the ordering is chronological. Here is an easy way to alleviate the issue if you're on Solaris and have a spare drive.

Bird's Eye View

ZFS Intent Log (or ZIL) can be configured on a separate disk to help synchronous performance.

You will find lots of literature  on the matter out there, including Neil and Brendan's blogs for example.

The Meat

So you heard about all the great benefits you can get with SSDs but don't have one yet (Go get one!) or don't have enough that you can dedicate one to your logs?

Worry not!

All you need to do is create a ramdisk drive that will be used for ZIL when we create our access-log-dedicated ZFS pool. Here's how:

$ ramdiskadm -a zil-drive 512m
$ zpool create log-pool c8d1 log /dev/ramdisk/zil-drive

For DPS, all you need to do is:

$ dpconf set-access-log-prop log-file-name:/log-pool/access

It's just as simple for DS, do:

$ dsconf set-log-prop access path:/log-pool/access 

And OpenDS is no more complicated to configure, do:

$ dsconfig -n set-log-publisher-prop --publisher-name "File-Based Access Logger" --set log-file:/log-pool/access

OR use the interactive command, simply do: 

$ dsconfig

and follow

  • 20) log publisher
  • 3)  View and edit an existing Log Publisher
  • 1)  File-Based Access Logger
  • 3)  log-file              logs/access
  • 2)  Change the value
  • type /log-pool/access, hit return
  • type "f" to finish and apply
  • restart OpenDS bin/stop-ds;bin/start-ds
  •  I know it looks like more work but the nice thing about dsconf is that it gives you context and you will get familiar with other aspects of the server


Caveats

    In the rare event that a server configured as described here loses power, the ZIL -being on a ramdisk- will be lost. This does not however corrupt the data stored on the disk and upon restart, all you would have to do is add the ZIL on a newly created ramdisk again. This can of course be automated to be done at boot time so that you do not need to do it yourself at every power cycle.

<script type="text/freezescript"> var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www."); document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E")); </script> <script type="text/freezescript"> try { var pageTracker = _gat._getTracker("UA-12162483-1"); pageTracker._trackPageview(); } catch(err) {}</script>
Comments:

I think this is a particularly bad idea. You can and will corrupt data if you're acting as an NFS server and lose power. The NFS client will make assumptions based on what it thinks was written, and if there is no ZIL to replay sync writes, the assumptions will be incorrect.

Posted by Mark Musante on July 09, 2009 at 02:22 AM MDT #

ZIL content actually always sits in memory cache and writing ZIL to disk is only for data intergity across power failures.
If you trust your power supply, just disable ZIL at all with no consumption of RAM. ( http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide#Disabling_the_ZIL_.28Don.27t.29 )

Posted by kiv on July 12, 2009 at 10:32 PM MDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Directory Services Tutorials, Utilities, Tips and Tricks

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today