Joining the ZFS Revolution

For a long time now I've been meaning to migrate my home file storage over to a ZFS server but the project kept getting postponed due to other priorities. Finally it's alive!

For the last ten years or so my home fileserver has been through the general purpose debian box in the garage. It has three disks, one for the system and home directories, a larger one which gets exported over NFS and the largest one which backs up the other two (nightly rsync). It has been an adequate solution, in as far as I've never lost data. But whenever a disk dies I always have several days of downtime and have to scramble to restore from backups and maybe reinstall.

There are many articles about this topic that make for good reading if you're considering the same. My goals were:

1. Data reliability, above all.
Initially I had visions of maximizing space, mainly for the geek value of having many terabytes of home storage. But in the end, I don't really need that much. The NFS export drive on my debian box was currently only 500GB and that was used not only by the shared data (pictures, mostly, and documents) but also for MythTV storage. Since I wasn't planning on moving the MythTV data to the ZFS pool, even 500GB would be plenty adequate for some time.

2. Low power consumption.
Since this is another server that'll need to run 24/7, I wanted to keep an eye on the power it uses.

3. But useful for general computing.
Since this will be the only permanent (24/7) OpenSolaris box on my home network, I also wanted to be able to use it for general purpose development work and testing whenever needed. So despite the goal of low power consumption, I didn't want to go all out with the lowest possible power setup, needed a compromise.

Here's the final setup:

CPU: AMD Phenom II X4 (quad core) 925. Reasonable power consumption and the quad cores give me something fun to play with.

Memory: 8GB ECC memory. Since I'm going primarily for data reliability, might as well go with ECC.

ZFS pool: 3 x 1TB drives. These are in a mirror setup, so total storage is just 1TB. That's still about three times as much as I really need right now. With three drives, even if two fail before I get to replace them I should be ok. I got each of the three drives from a different manufacturer, hopefully that'll make them fail at different times.

        NAME        STATE     READ WRITE CKSUM
        represa     ONLINE       0     0     0
          mirror    ONLINE       0     0     0
            c8d0    ONLINE       0     0     0
            c8d1    ONLINE       0     0     0
            c9d0    ONLINE       0     0     0

System disk: I expected to just use one older drive I had on the shelf, but after installing it I found it was running hot. Maybe it is ok but decided to do a two-way mirror of the rpool as well, maybe it'll save me some time down the road. I don't need much space here so found the cheapest drive I could get ($40) to add to the rpool. At that price, might as well mirror!

        NAME         STATE     READ WRITE CKSUM
        rpool        ONLINE       0     0     0
          mirror     ONLINE       0     0     0
            c9d1s0   ONLINE       0     0     0
            c10d1s0  ONLINE       0     0     0

Total power consumption for the box hovers around 78-80W most of the time.



looking at the above output of "zpool status represa" it seems, you're still using IDE disks there. Right? I would expect to see SATA there, which would then be listed as "cxtydz". Or did you simply not set "AHCI" in the BIOS for these disks?



Posted by Matthias Pfützner on March 29, 2010 at 01:31 AM PDT #

Indeed, good observation.

While experimenting with AHCI mode, I couldn't get the machine to boot from disk. Rather than spend all day on that I gave up and set it to IDE mode which worked fine. I'm hoping to go back and research more on that front when I have some time...

Posted by Jyri Virkki on March 29, 2010 at 03:13 AM PDT #


Yes, I can see that problem... I bought a Jetway NC96-510-LF board, and the original BIOS didn't really allow AHCI for some of the SATA ports. Jetway and myself are currently fixing that, I have a beta version of a newer BIOS, but that version still has some problems... Although currently OpenSolaris sees all SATA ports as AHCI, the boot-disk selection screen in the BIOS does not... So, some internal BIOS settings still need to be fixed in that BIOS...

So, it might be, that changing the bootdisk from IDE to SATA might pose problems to some of the OpenSolaris configs. Although a reconfigure boot should fix it... And in some BIOSes, you might also be forced to CHANGE the list of bootdisks, that the BIOS sees, so, when I change the bootdisk from IDE to SATA it disappears from the list of devices, that the BIOS can boot from.

Still, that's why I did choose a board, that has BOTH, physical SATA and IDE ports. I do use: SATA for DATA, and IDE for bootdisk...


Posted by Matthias Pfützner on March 29, 2010 at 03:43 AM PDT #

Yes, similar story. At least on this BIOS it's all or none for IDE/AHCI. Switching to AHCI made all the drives disappear from the BIOS, so it didn't know how to boot anymore...

Posted by Jyri Virkki on March 29, 2010 at 10:00 AM PDT #

Post a Comment:
Comments are closed for this entry.



Top Tags
« June 2016