A ZFS Home Server
By dap on May 24, 2010
This entry will be a departure from my recent focus on the 7000 series to explain how I replaced my recently-dead Sun Ultra 20 with a home-built ZFS workstation/NAS server. I hope others considering building a home server can benefit from this experience.
Of course, if you're considering building a home server, it pays to think through your goals, constraints, and priorities, and then develop a plan.
Goals: I use my desktop as a general purpose home computer, as a development
workstation for working from home and on other hobby projects, and as a
NAS file server. This last piece is key because I do a lot of photography work in Lightroom and Photoshop. I want my data stored redundantly on ZFS with snapshots and such, but I want to work from my Macbook. I used pieces of this ZFS CIFS workgroup sharing guide
to set this up, and it's worked out pretty well. More recently I've
also been storing videos on the desktop and streaming them
to my Tivo using pytivo and streambaby.
Priorities: The whole point is to store my data, so data integrity is priority #1 (hence ZFS
and a mirrored pool). After that, I want something unobtrusive (quiet) and
inexpensive to run (low-power). Finally, it should be reasonably fast and cheap, but I'm willing to sacrifice sacrifice speed and up-front cost for quiet and low-power.
Before sinking money into components I wanted to be sure they'd work well with Solaris and ZFS, so I looked around at what other people have been doing and found Constantin Gonzalez's "reference config." Several people have successfully built ZFS home NAS systems with similar components so I started with that design and beefed it up slightly for my workstation use:
- CPU: AMD Athlon II x3 405e. 3 cores, 2.3GHz, 45 watts, supports ECC. I had a very tough time finding the "e" line of AMD processors in stock anywhere, but finally got one from buy.com via Amazon.
- Motherboard: Asus M4A785T-M CSM. It's uATX, has onboard video, audio, NIC, and 6 SATA ports, and supports DDR3 and ECC (which uses less power than DDR2).
- RAM: 2x Super Talent 2GB DDR3-1333MHz with ECC. ZFS loves memory, and NAS loves cache.
- Case: Sonata Elite. Quiet and lots of internal bays.
- Power supply: Cooler Master Silent Pro 600W modular power supply. Quiet, energy-efficient, and plenty of power. Actually, this unit supplies much more power than I need.
I was able to salvage several components from the Ultra 20, including the graphics card, boot disk, mirrored data disks, and a DVD-RW drive. Total cost of the new components was $576.
Putting the system together went pretty smoothly. A few comments about the components:
- The CoolerMaster power supply comes with silicone pieces that wrap around both ends, ostensibly to dampen the vibe transferred to the chassis. I only ran with this on so I can't say how effective it was, but it made the unit more difficult to install.
- The case has similar silicone grommets on the screws that lock the disks in place.
- The rails for installing 5.25" external drives are actually sitting on the inside of each bay's cover. I didn't realize this at first and bought (incompatible) rails separately.
- Neither the case nor the motherboard has an onboard speaker, so you can't hear the POST beeps. This can be a problem if there's a problem with the electrical connections when you're first bringing it up. I had an extra case from another system around, so I wired that up, but I've since purchased a standalone internal speaker. (I did not find one of these at any of several local RadioShacks or computer stores, contrary to common advice.)
- I never got the case's front audio connector working properly. Whether I connected the AC97 or HDA connectors (both supplied by the case, and both supported by the motherboard), the front audio jack never worked. I don't care because the rear audio is sufficient for me and works well.
The trickiest part of this whole project was migrating the boot disk from the Ultra 20 to this new system without reinstalling. First, identifying the correct boot device in the BIOS was a matter of trial and error. For future reference, it's worth writing the serial numbers for your boot and data drives down somewhere so you can tell what's what if you need to use them in another system. Don't save this in files on the drives themselves, of course.
When I first booted from the correct disk, the system reset immediately after picking the correct GRUB entry. I booted into kmdb and discovered that the system was actually panicking early because it couldn't mount the root filesystem. With no dump device, the system had been resetting before the panic message ever showed up on the screen. With kmdb, I was able to actually see the panic message. Following the suggestion from someone else who had hit this same problem, I burned an OpenSolaris LiveCD, booted it, and imported and exported the root pool. After this, the system successfully booted from the pool. This experience would be made much better with 6513775/6779374.
Since several others have used similar setups for home NAS, my only real performance concern was whether the low-power processor would be enough to handle desktop needs. So far, it seems about the same as my old system. When using the onboard video, things are choppy moving windows around, but Flash videos play fine. The performance is plenty for streaming video to my Tivo and copying photos from my Macbook.
The new system is pretty quiet. While I never measured the Ultra 20's power consumption, the new system runs at just 85W idle and up to 112W with disks and CPU pegged. That's with the extra video card, which sucks about 30W. That's about $7.50/month in power to keep it running. Using the onboard video (and removing the extra card), the system idles at just 56W. Not bad at all.