Sunday Nov 22, 2009

My Home Media Server using OpenSolaris, ZFS, and free software

A while ago, I wrote several blog entries about what I did to set up a media server at home.  I'm writing this blog entry to wrap things up with some details about how much it all cost, and the software I'm running on the computer now.

Background

I decided to design a "media server" for home that would be the main data storage for our family's music, photos, recorded TV shows and movies, and personal documents and backups of our home directories on the computers we commonly use at home.  I had a few objectives for the media server:
  1. All of this data would be in a single computer that we could grab and stick in the car in case of emergency.  Friends of ours lost their house, but they had the foresight of having all of their personal data (over a terabytes' worth) on a single file server, so when they evacuated the house, they didn't lose any personal data.  Seemed like a great idea to me.
  2. The media server would store data reliably; i.e., I assumed that a disk would fail, and I wanted the media server to be able to continue working in case of failure.
  3. Reduce noise in the house as much as possible.  That meant putting the server in the garage where it could make as much fan noise as it needed to, but we wouldn't hear a thing in the home office or where we watch TV.
  4. Keep the cost down to a reasonable amount.  This was not the primary factor, but it was important enough for me to pay attention to when shopping for the components.

What I Purchased

Here's the build of materials for the computer I put together (I'm sure prices on these will have gone down since I bought them in October 2008).  Total price of the system: $742.  If you take out the cost of the disks, the rest of the computer cost $345.
  • Motherboard (newegg.com, $62.99) : ECS Elitegroup A780VM-M2 Micro ATX AMD Motherboard.  Supported the cheap AMD processor I wanted, and had plenty of on-board SATA ports (6).  Remember, I want all the disk to be in a single enclosure for easy, snatch-and-grab transportation.
  • Memory (newegg.com, $69.99): GSkill 4GB (2 x 2GB) DDR2 SDRAM (PC2 6400).  No reason to get greedy, and it keeps power draw and price down for the system.
  • CPU (newegg.com, $59.50): AMD Athlon 64 X2 5000 Brisbane 2.6GHz Socket AM2 65W Dual-Core Processor.  Nice price for a processor that does everything I need in a media server that can also run some apps.  More on the apps in a bit.
  • Disks (newegg.com, $396.80): 2 Seagate Barracuda 1.5TB 7200 RPM SATA internal disks.  Prices have gone down considerably for the storage, but this was a fair enough price last year.
  • Case (Fry's Electronics, $130): Antec Sonata III Mid Tower ATX Case.  Got good reviews, came with a 500W power supply which is plenty for what I was putting in it, and has plenty of ports and internal drive bays for the storage.

How I Set Up The Media Server

Here is the first blog entry I wrote that describes my thought process about using ZFS on OpenSolaris to store our home's data.

Here is the second blog entry I wrote, giving all the details about what I needed to do to format the disks and set up the ZFS pools and filesystems.  This was based on the OpenSolaris 2008.11 release; I plan to upgrade to a more recent release (probably an OpenSolaris build after the June 2009 release; I'd love to try the new ZFS deduplication feature).  Everything has been running fine, with the exception of the TimeSlider feature for doing automated ZFS snapshots.  That was fixed in the June 2009 release.

What I'm Running On The Media Server

The media server is doing a few things for us at home; here are the services it's providing:
  • iTunes music storage.  We have a Mac mini that we use for iTunes; all of our music is in iTunes, and the Mac mini NFS-mounts a filesystem from the media server.  I tried both CIFS (Samba) and NFS.  I preferred NFS, but it seemed to have troubles with the Mac as an NFS client.  After reading James Gosling's blog entry on what he observed with Macs and Solaris and NFS, I made a tweak on the Mac mini and everything has been working smoothly ever since.  We make this library shareable to our other Macs on the home network.
  • iPhoto storage.  The same Mac mini also stores all of our photos.
  • TiVo media backup and playback.  There is a great open source Java application called Galleon that uses the TiVo Home Media Engine (HME) API; the API lets you write your own Java apps that show up as part of the TiVo's on-screen menu system, and it lets you talk to the TiVo to grab shows off the TiVo's disk and put shows on there, among other functions.  Primarily, we use it as a way to keep backups of shows that we don't want to lose in case the TiVo's disk crashes and loses data.  This has happened several times, mostly due to unforeseen loss of power.
  • Personal finances (via Intuit's Quicken software).  To do this, I run the free, open source VirtualBox software.  I used to use VMware, but VMware isn't free and it doesn't run on any version of Solaris.  VirtualBox runs on every operating system I use, it's free, and it's improving much faster than VMware is.  So, I run Windows XP as a guest OS inside VirtualBox, and use that guest OS to run Quicken.  When I'm feeling like the guest OS is getting slow or am worried that it's collected viruses, I just blow away that guest image and go back to an earlier snapshot.

My Wish List

Are any of you doing something similar with your home setups?  If so, maybe you have done some of the things I'd like to set up but haven't gotten done yet.  Here's my wish list of apps and features I'd like to add, and I'd like to do it all natively in OpenSolaris (in other words, none of these services would need to run in a guest OS like Windows or Linux under VirtualBox):
  • DLNA server running natively in OpenSolaris.  DLNA will give me the ability to share and stream my media (photos, music, video) to the PlayStation 3.  I've played with several solutions; more on this in a moment.
  • Live streaming of TiVo content from the media server back through the TiVo.  Galleon lets me transfer shows to and from the TiVo, but not play them live.  This is a feature that the ReplayTV product had (I still miss that product, and am sorry they went out of business), and their Java app was superior in several ways to Galleon.  But, it looks like somebody has written software to let you play TiVo shows from your media server straight through the TiVo.  I just haven't gotten it running on the media server yet.

Powered by ScribeFire.



Monday Apr 27, 2009

How to push a simple Drupal web site from test to production

As I'm learning how to use Drupal for creating and deploying web sites, I'm keeping track of it in blog entries.

The way I work: I do my development on a virtualized environment: I run OpenSolaris as a guest OS under VirtualBox; that way, I can easily blow away my development/test environment or send it to other machines running VirtualBox.  Then, I install the WebStack (PHP, Apache web server, MySQL), then I install Drupal using the instructions on the drupal.org web site.

Once I get my test Drupal site working, I follow these instructions to deploy to an actual production server.  It makes the develop-test-deploy cycle pretty easy, and I can develop and test pretty much anywhere.



Powered by ScribeFire.

Thursday Apr 09, 2009

VirtualBox 2.2 supports software appliances

Looks like the VirtualBox people keep chugging along (although if you're moving at 150mph, is it right to call it "chugging"?).  Yesterday I saw they've released version 2.2, which supports the Open Virtualization Format (OVF).  This is what I like about it: when I want to share my software configuration with somebody, I make a vbox image and give it to somebody, but then they have to know the vbox VM configuration I used.  That means they have to go into the vbox UI and manually set up the same settings I did.  It's not difficult, but it's error-prone and it's tedious.

No longer: now I just tell VirtualBox to create an appliance out of my vbox image and it creates two files: an OVF image and the OVF description of that image.  When my co-worker wants to use my appliance, she tells vbox to import that appliance (the OVF description) and it does the right thing.  No configuration, nothing: it's just ready to go.  Nice and easy.

Jignesh Shah tried it out yesterday and created a relatively small-footprint OVF appliance of PostgreSQL 8.3.

I think this is going to be the way to distribute software in the near future.  And if not the way, then a valid way.  Virtualized images solve a few problems that I can think of:
  1. You don't have to worry about which operating system the customer has deployed on their desktop or server; as long as they're running a hypervisor, you can deliver your software to them easily, nicely pre-packaged in a virtualized image "appliance";
  2. it eliminates the install step for trying software: you've already packaged up your app in an appliance, no installation needed for the customer just makes things simpler and faster for them to get rolling;
  3. The transition to cloud computing becomes easier; if you use a virtualized image on your desktop, you can use the same image on a cloud like Sun's cloud computing offering, Amazon EC2, or other clouds that I'm sure will come online over the next few years.  This gives customers the flexibility to run apps where they want, and to migrate to/from clouds.
JumpBox is one example of a company that provides open source applications in virtualized appliance format, but also lets you try their appliances right now, for free, on a cloud: it's JumpBox.  Nice idea.

TurnKey Linux seeems to be something similar; I haven't looked much into it yet so don't know if they offer the cloud preview feature that JumpBox provides, but they do have the download-an-app-in-a-virtualized-image feature.


Powered by ScribeFire.

Sunday Mar 15, 2009

VirtualBox bug fixed: clonehd works in version 2.1.4

Not too long ago, I blogged about a workaround to a bug in VirtualBox.  The problem is fixed in VirtualBox version 2.1.4, available for download here.

Here's where the bug fix comes in handy: suppose you use VirtualBox to create a virtual machine that you want to use again and again; maybe it's a test environment of Windows, Linux, OpenSolaris...whatever you like.  An easy way to make copies of that test environment is to type "VBoxManage clonevdi WindowsXP.vdi copyOfWinXP.vdi" (or whatever you call your VirtualBox hard disk images).  But the "clonevdi" command wasn't making copies correctly, so there was a workaround.  It's not a big deal, but having the bug fixed makes it just that much easier to make perfect copies of the environment you worked so hard to create.

I discovered at DrupalCon that a lot of people are using VirtualBox.  I have been a faithful VMware user for well over five years and have liked it, but I've also been using VirtualBox for about six months now and I find it good enough for my personal needs that I've switched from VMware to VirtualBox.  One nice little benefit: it can be a host on all the operating systems I use (Ubuntu Linux, Mac OS X, OpenSolaris; I tend not to use Windows as a host OS, only as a guest, because I'm too concerned about viruses infecting my Windows environment and I'd like to be able to just blow it away and start from scratch easily; VirtualBox lets me do just that).


About

The views expressed on this blog are my own and do not necessarily reflect the views of Oracle. What more do you need to know, really?

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today