Sunday Nov 22, 2009

My Home Media Server using OpenSolaris, ZFS, and free software

A while ago, I wrote several blog entries about what I did to set up a media server at home.  I'm writing this blog entry to wrap things up with some details about how much it all cost, and the software I'm running on the computer now.

Background

I decided to design a "media server" for home that would be the main data storage for our family's music, photos, recorded TV shows and movies, and personal documents and backups of our home directories on the computers we commonly use at home.  I had a few objectives for the media server:
  1. All of this data would be in a single computer that we could grab and stick in the car in case of emergency.  Friends of ours lost their house, but they had the foresight of having all of their personal data (over a terabytes' worth) on a single file server, so when they evacuated the house, they didn't lose any personal data.  Seemed like a great idea to me.
  2. The media server would store data reliably; i.e., I assumed that a disk would fail, and I wanted the media server to be able to continue working in case of failure.
  3. Reduce noise in the house as much as possible.  That meant putting the server in the garage where it could make as much fan noise as it needed to, but we wouldn't hear a thing in the home office or where we watch TV.
  4. Keep the cost down to a reasonable amount.  This was not the primary factor, but it was important enough for me to pay attention to when shopping for the components.

What I Purchased

Here's the build of materials for the computer I put together (I'm sure prices on these will have gone down since I bought them in October 2008).  Total price of the system: $742.  If you take out the cost of the disks, the rest of the computer cost $345.
  • Motherboard (newegg.com, $62.99) : ECS Elitegroup A780VM-M2 Micro ATX AMD Motherboard.  Supported the cheap AMD processor I wanted, and had plenty of on-board SATA ports (6).  Remember, I want all the disk to be in a single enclosure for easy, snatch-and-grab transportation.
  • Memory (newegg.com, $69.99): GSkill 4GB (2 x 2GB) DDR2 SDRAM (PC2 6400).  No reason to get greedy, and it keeps power draw and price down for the system.
  • CPU (newegg.com, $59.50): AMD Athlon 64 X2 5000 Brisbane 2.6GHz Socket AM2 65W Dual-Core Processor.  Nice price for a processor that does everything I need in a media server that can also run some apps.  More on the apps in a bit.
  • Disks (newegg.com, $396.80): 2 Seagate Barracuda 1.5TB 7200 RPM SATA internal disks.  Prices have gone down considerably for the storage, but this was a fair enough price last year.
  • Case (Fry's Electronics, $130): Antec Sonata III Mid Tower ATX Case.  Got good reviews, came with a 500W power supply which is plenty for what I was putting in it, and has plenty of ports and internal drive bays for the storage.

How I Set Up The Media Server

Here is the first blog entry I wrote that describes my thought process about using ZFS on OpenSolaris to store our home's data.

Here is the second blog entry I wrote, giving all the details about what I needed to do to format the disks and set up the ZFS pools and filesystems.  This was based on the OpenSolaris 2008.11 release; I plan to upgrade to a more recent release (probably an OpenSolaris build after the June 2009 release; I'd love to try the new ZFS deduplication feature).  Everything has been running fine, with the exception of the TimeSlider feature for doing automated ZFS snapshots.  That was fixed in the June 2009 release.

What I'm Running On The Media Server

The media server is doing a few things for us at home; here are the services it's providing:
  • iTunes music storage.  We have a Mac mini that we use for iTunes; all of our music is in iTunes, and the Mac mini NFS-mounts a filesystem from the media server.  I tried both CIFS (Samba) and NFS.  I preferred NFS, but it seemed to have troubles with the Mac as an NFS client.  After reading James Gosling's blog entry on what he observed with Macs and Solaris and NFS, I made a tweak on the Mac mini and everything has been working smoothly ever since.  We make this library shareable to our other Macs on the home network.
  • iPhoto storage.  The same Mac mini also stores all of our photos.
  • TiVo media backup and playback.  There is a great open source Java application called Galleon that uses the TiVo Home Media Engine (HME) API; the API lets you write your own Java apps that show up as part of the TiVo's on-screen menu system, and it lets you talk to the TiVo to grab shows off the TiVo's disk and put shows on there, among other functions.  Primarily, we use it as a way to keep backups of shows that we don't want to lose in case the TiVo's disk crashes and loses data.  This has happened several times, mostly due to unforeseen loss of power.
  • Personal finances (via Intuit's Quicken software).  To do this, I run the free, open source VirtualBox software.  I used to use VMware, but VMware isn't free and it doesn't run on any version of Solaris.  VirtualBox runs on every operating system I use, it's free, and it's improving much faster than VMware is.  So, I run Windows XP as a guest OS inside VirtualBox, and use that guest OS to run Quicken.  When I'm feeling like the guest OS is getting slow or am worried that it's collected viruses, I just blow away that guest image and go back to an earlier snapshot.

My Wish List

Are any of you doing something similar with your home setups?  If so, maybe you have done some of the things I'd like to set up but haven't gotten done yet.  Here's my wish list of apps and features I'd like to add, and I'd like to do it all natively in OpenSolaris (in other words, none of these services would need to run in a guest OS like Windows or Linux under VirtualBox):
  • DLNA server running natively in OpenSolaris.  DLNA will give me the ability to share and stream my media (photos, music, video) to the PlayStation 3.  I've played with several solutions; more on this in a moment.
  • Live streaming of TiVo content from the media server back through the TiVo.  Galleon lets me transfer shows to and from the TiVo, but not play them live.  This is a feature that the ReplayTV product had (I still miss that product, and am sorry they went out of business), and their Java app was superior in several ways to Galleon.  But, it looks like somebody has written software to let you play TiVo shows from your media server straight through the TiVo.  I just haven't gotten it running on the media server yet.

Powered by ScribeFire.



Thursday Dec 18, 2008

My Home Media Server on OpenSolaris + ZFS: Part 1

A couple of people gave me some good pointers after my last blog entry, in which I was saying that doing an ssh or vncviewer into my new OpenSolaris installation was taking a while.  They pointed out that it may be reverse DNS lookups, not something about the OpenSolaris box itself, and that reminded me that recently I had changed something else about my home network setup: I had two hubs/routers active.  So, that mystery's solved; things are looking much better now and doing an ssh into the OpenSolaris box doesn't take so long.  Moving on...

My mission: use the current release of OpenSolaris (2008.11) as the basis for the main fileserver for home.  Right now, we've got several computers that have external disks attached to them for extra storage (music, photos, movies, etc.) and I want to centralize that for a couple of reasons:

  1. Friends of mine lost their house to a wildfire; fortunately, they had stored all of their critical data on a single computer with lots of disk so when they had to evacuate the house they grabbed one box and didn't lose any critical data.  Laugh all you want; when The Big One comes, I want to be ready.
  2. I'd like to simplify the administration of our home machines.  This is home, for goodness' sake; I don't want to be hiring a system administrator to keep our stuff in order.
I'll document here what I'm doing to build the home server.  My intent is to use OpenSolaris, use ZFS to manage the disks and files, and to do it with a cheap computer that I build myself from off-the-shelf parts (as opposed to, say, buying a Dell computer).  Besides, I want to put a bunch of disks in that computer and it's hard for me to find a cheap computer from Dell or HP with a bunch of internal drive bays, but you can pretty easily buy a reasonable computer enclosure with plenty of internal drive bays.  Building yourself can save money, and I'm all about saving some money on this.

But first, I'm going to try this out on a computer that is known to work with OpenSolaris.  I'll get the setup running there to make sure that ZFS + OpenSolaris really is as easy and reliable as I think it is.  Once I'm convinced that works, I'll switch to my cheapo computer and see if OpenSolaris runs on that.

So here goes...

Step 1: Data Protection

I have four disks: two 1TB drives, two 1.5 TB drives.  I'll split the larger drives into two partitions: 500GB for the operating system and the remaining TB for the big bucket-o-storage.  (let's call it my media pool: ZFS pool used primarily for storing audio, video, and photos)

So my first decision to make is: how should I have ZFS protect my data against disk failure?  After all, I'm buying consumer-grade disk drives but the server will be on 24/7.  The disks will fail.  I don't want to lose my data just because I don't want to pay extra for more reliable disks.  I want the software (ZFS) to take care of the problem for me.  I start by looking at the ZFS Best Practices Guide to see what my options are.

I'm considering three options for ZFS protecting my data:
  1. mirror
  2. raidz (shorthand for "raidz1", meaning 1 error can happen and I don't lose data)
  3. raidz2 (meaning 2 errors can happen and I don't lose data)
The guide points me to this blog entry by Roch Bourbonnais which tells me that the tradeoff I need to make is space versus performance: mirroring gives me maximum performance but cuts my disk storage in half: 4 TB of storage over 4 disks would take 50% overhead to protect the data by mirroring, leaving me a 2TB storage pool for storing movies, photos, music, and the like.  I don't need super-high performance but I want as much usable space as I can get out of my disks, so I choose raidz which should give me about 3TB of usable space; I lose 1TB to data protection, which sounds fine to me.  Later, I may buy a fifth disk and use raidz2 to give me even more robust data protection, but I'm not going to do that right now.

Now that I've decided how to protect my data, I just need to create the appropriate partitions on the larger disks, and I'll be ready to install the OS.  I'll document that in my next blog entry.



Powered by ScribeFire.

About

The views expressed on this blog are my own and do not necessarily reflect the views of Oracle. What more do you need to know, really?

Search

Archives
« July 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
  
       
Today