DIY: Home NAS Box with OpenSolaris and ZFS (English version)

OpenSolaris and ZFS make it possible to build a NAS box with general PC hardware and free open source software, which is different from other NAS boxes in a way that data redundancy is achieved without RAID hardware, and at a very acceptable cost. As a super DIY fan, I'm not going to miss this chance to DIY an OpenSolaris and ZFS based NAS box, not only for fun but also for serious use.

Some principles were established first:

  • Use low-power 64-bit microprocessor to run Solaris and ZFS smooth and fast;
  • Use four hot-swappable 3.5" SATA HDD to build a RAIDZ1 pool with over 2TB capacity;
  • Use Gigabit Ethernet interface to take advantage of the network bandwidth.

Other factors needs to be considered include total cost, total power consumption and noise level. After all, this is for personal and home use.

Many x86 microprocessors in the market are 64-bit capable, unfortunately most of them are not designed for low power application, hence their TDP is fairly high (over 35W), which means active cooling is mandatory and system TDP would be higher than 60W, not so good for 7x24 use. Things changed with the emergence of Intel Atom processor, with the advanced 45 nm process, the TDP of the single-core Atom 230 and dual-core Atom 330 are merely 4W and 8W respectively, no need to be cooled by a fan at all. And the exciting news is that Atom processors are 64-bit capable.

But every Atom 230 based motherboard in the market lacks an on-board Gigabit Ethernet controller, a obvious result of cutting cost. My final choice is Intel D945GCLF2 mini-ITX motherboard, with 1.6G dual-core Atom 330 CPU on board (supports Hyper Threading), 945GC express chipset, Realtek Gigabit Ethernet controller and a standard 32-bit PCI slot. You will be impressed by the passive cooling of the CPU and active cooling for the north bridge at the first glance of the board. Compared with the 8W TDP of the CPU, the 20W TDP of the 945GC north bridge is really high, another "reasonable" design of cutting cost. Fortunately 20W is not so bad and the noise of the cooling fan is almost inaudible. The storage device interfaces of the board include one IDE and two SATA II ports, definately not enough for a 4-HDD NAS, so we need an extra PCI SATA adapter to provide all the necessary SATA ports.

Intel D945GCLF2

Before choosing the SATA card, the chassis needs to be determined. To me, the chassis should be small and pretty, support mini-ITX motherboard and have 4 hot-swappable 3.5" HDD bays. It turns out to be really difficult to find a good chassis that meets all the requirements, not quite surprising though. The model I choose is ES34069 (from the Taiwan chassis manufacturer Chenbro) and it's a perfect product in my point of view. However it took me quite some time to find a distributor in mainland China, and they have to import the chassis for me, at a fairly high cost of RMB1850 (over USD260). The delivery was also awful, it took over 50 days, but what can I say, I just wish I could order it from US.

Chenbro ES34069 Chassis

The perfect chassis arrived, and the motherboard was bought from www.taobao.com (the equivalent of eBay in China). With some simple measurement after successful installation of the motherboard, it turns out any PCI card higher than 40 mm is not going to fit into the chassis. Be aware that a PCI SATA card lower than 40mm does not exist. The solution to this problem is a PCI riser card, which is a small PCB with standard PCI gold fingers and a standard PCI slot with every signal routing to the gold fingers directly, so that a PCI card can be installed in parallel with the motherboard via the riser card. Choosing a proper PCI riser card was not an easy job either. The height of the riser card has to be carefully chosen: it will conflict with the audio interface of the motherboard if it's too low, and it will not fit into the chassis if it's too high. At last, a 35mm-high PCI riser card and a Sil3124-based PCI 4-port SATA II card were bought from www.taobao.com, although the riser card is a non-branded one and seems not well tailored. I could not wait to assemble all the stuff, and the space usage is just perfect (see the pictures), but after the power-up, the machine did not boot at all!

Something must be wrong with either the PCI riser card or the PCI SATA card. Removing the SATA card and leaving the riser card on the motherboard, the machine still did not boot. Removing the riser card and plugguing the SATA card on the motherboard directly, the machine booted OK! Seems the riser card problem, but where exactly?

I started to check each signal on the riser card with the reference of the PCI specification. Only one signal looks suspicious, it should be tri-stated according to the specification but connecting to the ground plane directly in the riser card. By cutting off the connection between the signal and the ground plane, problem was solved! The big lesson here is: better get a branded product!

PCI riser card and PCI SATA card

Next step was installing Solaris. At first I considered using DOM (Disk-on-Module) with IDE interface or a CF-to-IDE as boot device, but the performance price ratio turned out to be low -- a fast 4GB DOM or CF card cost more than RMB 200 (over USD 30). With much larger capacity and faster access, 2.5" HDD seems to be a better solution. So I bought a 160G 2.5" SATA HDD for no more than RMB300 (aournd USD 40).

Now which Solaris release to use? Of course the fresh released OpenSolaris 2008.11! Installation was quite smooth. The Realtek Gigabit Ethernet controller and the Sil3124 PCI SATA card were easily recognized and started to work. With the power of the dual 1.6G cores, the graphic desktop environment runs very smoothly, and please remember, only 8W power consumption! The wattmeter shows 45W average power consumption of the system, the temperature of the CPU and north bridge are all under 40 degree Celsius, green and cool!

The hardware part is almost complete (except the four 3.5" SATA disk drives) and the cost is around RMB 3200 (USD 450). Looking at similar products from HP, Thecus, QNAP and Buffalo, with average price over $700, they all run specially tailored embedded Linux, which means you can not run your favorite applications or play with the system at all. In contrast, using OpenSolaris as the firmware of the NAS box brings higher performance price ratio and much more fun. Of course, lacking of a good human-machine interface is the Achilles heel, but I believe with the open source of FISHworks, things will change a lot!

Now it's time to choose the 3.5" SATA disk drives. 7200RPM disks are just out of the problem because of the power consumption and noise. I found a very good 5400RPM candidate from Western Digital, the Caivar Green series 1TB disk drive only consumes less than 5W when idle and the performance is good enough for the NAS box. I bought four for RMB3000 (USD 430) and created a RAIDZ1 pool on top of them. Since the real capacity of each disk drive is 931GB, the total available capacity is around 2.7TB. That's RMB 2.3 (USD 0.33) per GB, pretty good deal! I created several ZFS file systems for storing different contents: books, musics, pictures, movies, etc.The rest steps were setting up Solaris NFS and CIFS services, really simple tasks and you can find lots of tutorials on internet. After that the NAS box was finally completed.

Complete view

Through NFS and CIFS over the Gigabit network interface, the average write speed is around 32MB/sec and read speed is around 40MB/sec, not bad. The total power consumption of the system ranges from 50W to 55W, pretty green for 7x24 use, right?

Comments:

[Trackback] Explica como montarse un GreenNas (1), con un micro Atom de 64bits, cuatro discos duros SATA hot-swappable 3.5, una caja MiniITX, todo esto con dispositivos de muy bajo consumo, se tiene un NAS con un consumo de tan solo 55W y una capacidad de 2,7 Tera...

Posted by meneame.net on December 12, 2008 at 10:38 PM CST #

Its a great blog, Really cheap and good way to build a NAS

Posted by tom on December 19, 2008 at 04:31 PM CST #

Can you post a consolidated parts list with links to where you purchased your products? Might try this myself.

Posted by kDawg on December 21, 2008 at 01:39 PM CST #

Cool project. Can you tell if that SATA card would fit with a PCI riser card that had the socket facing the other way (towards the top of the case)? e.g. http://www.logicsupply.com/products/80h094340

Posted by mark on December 23, 2008 at 03:30 AM CST #

kDawg,

I bought all the stuff from www.taobao.com but unfortunately it's a Chinese-only website. I believe you can find most parts at lots of online shopping sites by searching the part name I listed except the PCI riser card -- it's not a standard PCI riser card for 1U server, but I posted some pictures and you will have to do some search just as I did :-)

mark,

The PCI riser card listed in logicsupply.com won't fit into ES34069 if used with 4-port Sil3124 PCI SATA card. And the height of the riser card will make it conflict with the audio interface of the D945GCLF2 motherboard.

Regards,
Zhong

Posted by Zhong on December 27, 2008 at 12:57 PM CST #

"But every Atom 230 based motherboard in the market lacks an on-board Gigabit Ethernet controller..."

Not true, you can get Atom 230 boards with gigabit ethernet for example the Jetway JNC92-230-LF

http://linitx.com/viewproduct.php?prodid=12253

Posted by ScaredyCat on January 02, 2009 at 05:01 PM CST #

Great article.

I was looking at the Norco DS-520, but just today found the Chenbro which seems cheaper for my needs. I just started pricing up a possible system and then found your article :-)

My biggest concern is energy consuption and I was wondering if you had any thoughts on using the following to reduce power:

- Via C7 CPU
- 2.5" hard disks instead of 3.5"

Thanks Adam

adam DOT retter AT google mail DOT com

Posted by Adam Retter on January 07, 2009 at 03:02 PM CST #

Thanks for posting this! I built a very similar system a couple of months ago, with the same case and a Jetway NC62K motherboard. But I've not been able to get either of its gigabit Ethernet interfaces working well with OpenSolaris, and am thinking of changing the motherboard.

I'd think the PCI adapter would be \*much\* slower than directly-connected SATA drives. But do you think this has any significant effect on your system's performance?

Posted by Andrew McRae on January 17, 2009 at 01:43 PM CST #

You said :

By cutting off the connection between the signal and the ground plane, problem was solved

But i don't know what i must cut ????

Can you send me a picture or schema.

Thanks

Alex

Posted by Alex on January 22, 2009 at 07:18 PM CST #

Can you try booting your DIY NAS with this image and share if all the hardware works as expected?

http://eonstorage.blogspot.com/

Posted by Sean on January 27, 2009 at 09:50 PM CST #

can you perhaps tell me how you bechmarked it and get the result "write speed is around 32MB/sec and read speed is around 40MB/sec, not bad"

becouse i have a "bigger cpu", but my performance isn't as good as yours..

have a nice day, and it would be cool to tell us more about your system...

HN

Posted by Hanna_ on February 03, 2009 at 12:59 AM CST #

Does anyone know what driver Solaris uses for the Sil3124? Does it use PCI-IDE for does it have a SATA driver. I just made a new server with the same board hoping that the ICH7 would support hot-swap, but Intel only uses the cheap ICH on this board and it doesn't support AHCI or hot swap. In my old server which was running Linux I had a Promise TX4 SATA card that had a good Linux driver that supported hot swap but Solaris has 0 support for the card, it can't see any of the disks, AT ALL. Before I waste my money on a Sil3124 based card I'd like to know how well it is supported in Solaris. I'm also open to other suggestions for cheap SATA cards that people know work with Solaris. Solaris hardware compatibility info is very hard to find and most of the time non-existent.

Thanks.

Posted by Joe Mulloy on February 05, 2009 at 04:34 PM CST #

I've just finished a zfs NAS box with an old dothan powered P4C800E-deluxe. There's no driver available for on board promise sata controller so I just disabled it and added a sil3124 card. It worked flawlessly with 2008.11. The driver is called 'sil3124' and it was installed automatically during OS install.I stuffed in 4 old 250G drives, each drive has a read/write speed of ~55MB/sec and upto ~69MB/sec in raidz1 configuration. The onboard ether is an intel 82547EI chip which is notoriously bad with the e1000g driver due to lack of LSO support. I can only get ~2.5MB/sec with cifs service. I also tried the snv_107b since I was informed that the LSO issue has been fixed there. But I only got slightly increased performance of about 5MB/sec. These results are of course over a well tested gigabit network where I had hosted samba service in ubuntu upto 60MB/sec. So I decided to try other ethernet cards. I digged out a dual port IBM NetXtreme(broadcom 5704 chip) from my old junk and put it inside. Wow, ~30MB/sec read and 20~25MB/sec write with cifs. I couldn't be happier since this old dothan rig was almost forgotten and now it serves me happily and likely reliably.

It indeed is a nice $50(sil3124) warm-up excercise. My next and serious zfs nas box will likely be either a 64bit MoDt(penryn?) or 64bit atom plus nice chipset such as Geforce 9400M with hmmm maybe 4x2TB hdds。 LOL

Posted by biophys on February 12, 2009 at 12:54 AM CST #

Thanks for a nice informative article! I'm looking to build pretty much the same thing. (I can't find Sil3124 boards though... and Solaris HCL seems a little short)

How is the noise level with this system? My current FreeBSD+Adaptec 1TB NAS in a big case is fairly noisy. This system is small enough I'd like to tuck it away somewhere.

Posted by Howie on February 26, 2009 at 09:54 AM CST #

Hi, Howie There are quite a few sil3124 cards on ebay for less than $50 each. Just be sure to get the one with 4 internal ports not the one with 2 externals 2 internals. Another good choice is supermicro's sat2-mv8 which has 8 ports in case you are building a bigger server.

As for the noise, it seems fairly easily to solve by replacing the NB stock HSF with a heatpipe chipset cooler. People have reported great success with thermalright hr-05 and antazone as-n1000/as-n2000. See here,
http://www.silentpcreview.com/forums/viewtopic.php?t=50620
and here
http://www.winfuture-forum.de/index.php?showtopic=158802
Good luck and pls don't forget to share some of your experience back.

Posted by biophys on March 01, 2009 at 04:27 AM CST #

I just built a system with the same case, motherboard, and data disks. I used an EIDE WD 320G 2.5" for the system disk. I used this SATA card:

http://www.newegg.com/Product/Product.aspx?Item=N82E16816124003

With the Chenbro 80h094340 riser card: http://www.logicsupply.com/products/80h094340

I used the Syba card with 2 internal and 2 external ports (I figured I could use the eSata ports for expansion if I cut a small window in the back of the case). Disks 1 and 2 are connected to the onboard SATA ports; I see ~94MB/s raw read throughput using dd. The Syba card is connected to drives 3 and 4, and I see ~69MB/s read performance with these disks (same speed for the 2.5" EIDE disk).

There is one problem, however - the case is 4-5 millimeters to short for this motherboard/riser/PCI card combo. I had to cut away a small window in the metal frame of the case, and break out the plastic ribs on the other side. This mod gives enough room for the cables sticking out of the top of the card, but is totally invisible from the outside.

Posted by Greg on March 13, 2009 at 05:34 PM CST #

Riser cards: having tried several riser cards (and read a lot about them) when VIA came out with their first mini-itx boards told me you cannot expect every riser card to work with every motherboard: success depends on several factors (chips or so - I do not remember the details anymore, sorry).

Posted by guest on May 04, 2009 at 10:52 AM CST #

I had the same problem with the riser card (I don't think there's too many companies that make 2U right facing single pci slot risers). The bad pin in question ended up being A40 according to the specification found on this site.

http://pinouts.ru/Slots/PCI_pinout.shtml

Happy hunting. Obviously I am not responsible if you mess up your riser card =)

Posted by Devin on June 08, 2009 at 12:31 AM CST #

I have recently built a ZFS Solaris 10 based NAS...in the end I settled for a HP ML115G5 with 5x1Tb disks in RAIDZ1... i am getting 65MB/sec over CIFS ....which makes it a great backup box... specs are a quad core amd with 8gb eec ram....works great.

Posted by Ciaran Montgomery on November 25, 2009 at 01:54 PM CST #

Looking to build a very similar system. You mentioned the use of a 35mm PCI riser card. Can you tell me if you meant 35mm from top to bottom or without the connector? Thanks in advance.

Posted by Mr_W on January 20, 2010 at 05:28 AM CST #

Hi all,

This is a pretty old article. Not sure if someone would read this. I just finished building my nas with almost the same specs as above. Strangely the BIOS can recognise only 2 out of 4 hdds(1 & 3). Hdds 2 & 4 are invisible. Anyone has any clue? I installed windows home server in it.

Posted by Fuad on May 22, 2010 at 02:59 AM CST #

It's great to be able to compare notes! Thanks for the info. As an FYI..I ran into a major performance issue when using the SIL3124 based SYBA SD-SATA2-2E2I board. It ships with the RAID bios. It recognized the drives in opensolaris but shown wierd stats using 'iostat -cxn 3'. Looked like the adapter was pegged at 20MB/s. After much headache and reading. I changed the bios to the SataLink (IDE) bios from Silcon Image. (http://www.siliconimage.com/docs/3124_6409.zip) Now I can boot from the drives and iostat shows 100MB/s using 'time dd if=/dev/zero of=4G bs=1M count=4096'. Much improvement over the 20MB/s. The 100MB/s seems to be the limit of my cheap SATA drives. None the less, i'm pleased now. ;)

Posted by Alan V on June 05, 2010 at 10:07 PM CST #

PLEASE SEND NSA20-0022G43A DRIVE (VFD) MANUAL IN ENGLISH ON GIVEN MY E-MAIL

Posted by TARKESH PATEL on August 26, 2010 at 06:37 AM CST #

So, this is really strange - I have almost EXACTLY the same machine. Same board, RAID card, make of drives (WD Green), same case... spooky! I'm curious as to how much RAM you have though. I was considering upgrading my machine - it's running OpenSuSE using MDRAID, but performance is lousy - the good old RAID5 write hole biting me in the bum. I was toying with switching to FreeBSD with ZFS (I don't get on well with Solaris), but plenty of people refer to ZFS needing plenty of RAM to work well. I think I'll upgrade the RAM and possibly the disks too, and all will be well!

Posted by David on February 03, 2011 at 07:24 PM CST #

Hi Zhong,

I am searching for a case to house my D525 Atom ITX motherboard which can hold at least 2x3.5" disks + an OS drive (3.5" is better, 2.5" if needs be) -- but it must be very quiet or silent.

The PSU tends to be the noiser part of an Atom system, so how did find the noise level of the PSU?

Thanks,

Alex

Posted by Alex Ellis on March 27, 2011 at 09:43 PM CST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Zhong Tom Wang

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today