DIY: Home NAS Box with OpenSolaris and ZFS (English version)
By Zhong Tom Wang on Dec 12, 2008
OpenSolaris and ZFS make it possible to build a NAS box with general PC hardware and free open source software, which is different from other NAS boxes in a way that data redundancy is achieved without RAID hardware, and at a very acceptable cost. As a super DIY fan, I'm not going to miss this chance to DIY an OpenSolaris and ZFS based NAS box, not only for fun but also for serious use.
Some principles were established first:
- Use low-power 64-bit microprocessor to run Solaris and ZFS smooth and fast;
- Use four hot-swappable 3.5" SATA HDD to build a RAIDZ1 pool with over 2TB capacity;
- Use Gigabit Ethernet interface to take advantage of the network bandwidth.
Other factors needs to be considered include total cost, total power consumption and noise level. After all, this is for personal and home use.
Many x86 microprocessors in the market are 64-bit capable, unfortunately most of them are not designed for low power application, hence their TDP is fairly high (over 35W), which means active cooling is mandatory and system TDP would be higher than 60W, not so good for 7x24 use. Things changed with the emergence of Intel Atom processor, with the advanced 45 nm process, the TDP of the single-core Atom 230 and dual-core Atom 330 are merely 4W and 8W respectively, no need to be cooled by a fan at all. And the exciting news is that Atom processors are 64-bit capable.
But every Atom 230 based motherboard in the market lacks an on-board Gigabit Ethernet controller, a obvious result of cutting cost. My final choice is Intel D945GCLF2 mini-ITX motherboard, with 1.6G dual-core Atom 330 CPU on board (supports Hyper Threading), 945GC express chipset, Realtek Gigabit Ethernet controller and a standard 32-bit PCI slot. You will be impressed by the passive cooling of the CPU and active cooling for the north bridge at the first glance of the board. Compared with the 8W TDP of the CPU, the 20W TDP of the 945GC north bridge is really high, another "reasonable" design of cutting cost. Fortunately 20W is not so bad and the noise of the cooling fan is almost inaudible. The storage device interfaces of the board include one IDE and two SATA II ports, definately not enough for a 4-HDD NAS, so we need an extra PCI SATA adapter to provide all the necessary SATA ports.
Before choosing the SATA card, the chassis needs to be determined. To me, the chassis should be small and pretty, support mini-ITX motherboard and have 4 hot-swappable 3.5" HDD bays. It turns out to be really difficult to find a good chassis that meets all the requirements, not quite surprising though. The model I choose is ES34069 (from the Taiwan chassis manufacturer Chenbro) and it's a perfect product in my point of view. However it took me quite some time to find a distributor in mainland China, and they have to import the chassis for me, at a fairly high cost of RMB1850 (over USD260). The delivery was also awful, it took over 50 days, but what can I say, I just wish I could order it from US.
The perfect chassis arrived, and the motherboard was bought from www.taobao.com (the equivalent of eBay in China). With some simple measurement after successful installation of the motherboard, it turns out any PCI card higher than 40 mm is not going to fit into the chassis. Be aware that a PCI SATA card lower than 40mm does not exist. The solution to this problem is a PCI riser card, which is a small PCB with standard PCI gold fingers and a standard PCI slot with every signal routing to the gold fingers directly, so that a PCI card can be installed in parallel with the motherboard via the riser card. Choosing a proper PCI riser card was not an easy job either. The height of the riser card has to be carefully chosen: it will conflict with the audio interface of the motherboard if it's too low, and it will not fit into the chassis if it's too high. At last, a 35mm-high PCI riser card and a Sil3124-based PCI 4-port SATA II card were bought from www.taobao.com, although the riser card is a non-branded one and seems not well tailored. I could not wait to assemble all the stuff, and the space usage is just perfect (see the pictures), but after the power-up, the machine did not boot at all!
Something must be wrong with either the PCI riser card or the PCI SATA card. Removing the SATA card and leaving the riser card on the motherboard, the machine still did not boot. Removing the riser card and plugguing the SATA card on the motherboard directly, the machine booted OK! Seems the riser card problem, but where exactly?
I started to check each signal on the riser card with the reference of the PCI specification. Only one signal looks suspicious, it should be tri-stated according to the specification but connecting to the ground plane directly in the riser card. By cutting off the connection between the signal and the ground plane, problem was solved! The big lesson here is: better get a branded product!
Next step was installing Solaris. At first I considered using DOM (Disk-on-Module) with IDE interface or a CF-to-IDE as boot device, but the performance price ratio turned out to be low -- a fast 4GB DOM or CF card cost more than RMB 200 (over USD 30). With much larger capacity and faster access, 2.5" HDD seems to be a better solution. So I bought a 160G 2.5" SATA HDD for no more than RMB300 (aournd USD 40).
Now which Solaris release to use? Of course the fresh released OpenSolaris 2008.11! Installation was quite smooth. The Realtek Gigabit Ethernet controller and the Sil3124 PCI SATA card were easily recognized and started to work. With the power of the dual 1.6G cores, the graphic desktop environment runs very smoothly, and please remember, only 8W power consumption! The wattmeter shows 45W average power consumption of the system, the temperature of the CPU and north bridge are all under 40 degree Celsius, green and cool!
The hardware part is almost complete (except the four 3.5" SATA disk drives) and the cost is around RMB 3200 (USD 450). Looking at similar products from HP, Thecus, QNAP and Buffalo, with average price over $700, they all run specially tailored embedded Linux, which means you can not run your favorite applications or play with the system at all. In contrast, using OpenSolaris as the firmware of the NAS box brings higher performance price ratio and much more fun. Of course, lacking of a good human-machine interface is the Achilles heel, but I believe with the open source of FISHworks, things will change a lot!
Now it's time to choose the 3.5" SATA disk drives. 7200RPM disks are just out of the problem because of the power consumption and noise. I found a very good 5400RPM candidate from Western Digital, the Caivar Green series 1TB disk drive only consumes less than 5W when idle and the performance is good enough for the NAS box. I bought four for RMB3000 (USD 430) and created a RAIDZ1 pool on top of them. Since the real capacity of each disk drive is 931GB, the total available capacity is around 2.7TB. That's RMB 2.3 (USD 0.33) per GB, pretty good deal! I created several ZFS file systems for storing different contents: books, musics, pictures, movies, etc.The rest steps were setting up Solaris NFS and CIFS services, really simple tasks and you can find lots of tutorials on internet. After that the NAS box was finally completed.
Through NFS and CIFS over the Gigabit network interface, the average write speed is around 32MB/sec and read speed is around 40MB/sec, not bad. The total power consumption of the system ranges from 50W to 55W, pretty green for 7x24 use, right?