Sunday Nov 22, 2009

My Home Media Server using OpenSolaris, ZFS, and free software

A while ago, I wrote several blog entries about what I did to set up a media server at home.  I'm writing this blog entry to wrap things up with some details about how much it all cost, and the software I'm running on the computer now.

Background

I decided to design a "media server" for home that would be the main data storage for our family's music, photos, recorded TV shows and movies, and personal documents and backups of our home directories on the computers we commonly use at home.  I had a few objectives for the media server:
  1. All of this data would be in a single computer that we could grab and stick in the car in case of emergency.  Friends of ours lost their house, but they had the foresight of having all of their personal data (over a terabytes' worth) on a single file server, so when they evacuated the house, they didn't lose any personal data.  Seemed like a great idea to me.
  2. The media server would store data reliably; i.e., I assumed that a disk would fail, and I wanted the media server to be able to continue working in case of failure.
  3. Reduce noise in the house as much as possible.  That meant putting the server in the garage where it could make as much fan noise as it needed to, but we wouldn't hear a thing in the home office or where we watch TV.
  4. Keep the cost down to a reasonable amount.  This was not the primary factor, but it was important enough for me to pay attention to when shopping for the components.

What I Purchased

Here's the build of materials for the computer I put together (I'm sure prices on these will have gone down since I bought them in October 2008).  Total price of the system: $742.  If you take out the cost of the disks, the rest of the computer cost $345.
  • Motherboard (newegg.com, $62.99) : ECS Elitegroup A780VM-M2 Micro ATX AMD Motherboard.  Supported the cheap AMD processor I wanted, and had plenty of on-board SATA ports (6).  Remember, I want all the disk to be in a single enclosure for easy, snatch-and-grab transportation.
  • Memory (newegg.com, $69.99): GSkill 4GB (2 x 2GB) DDR2 SDRAM (PC2 6400).  No reason to get greedy, and it keeps power draw and price down for the system.
  • CPU (newegg.com, $59.50): AMD Athlon 64 X2 5000 Brisbane 2.6GHz Socket AM2 65W Dual-Core Processor.  Nice price for a processor that does everything I need in a media server that can also run some apps.  More on the apps in a bit.
  • Disks (newegg.com, $396.80): 2 Seagate Barracuda 1.5TB 7200 RPM SATA internal disks.  Prices have gone down considerably for the storage, but this was a fair enough price last year.
  • Case (Fry's Electronics, $130): Antec Sonata III Mid Tower ATX Case.  Got good reviews, came with a 500W power supply which is plenty for what I was putting in it, and has plenty of ports and internal drive bays for the storage.

How I Set Up The Media Server

Here is the first blog entry I wrote that describes my thought process about using ZFS on OpenSolaris to store our home's data.

Here is the second blog entry I wrote, giving all the details about what I needed to do to format the disks and set up the ZFS pools and filesystems.  This was based on the OpenSolaris 2008.11 release; I plan to upgrade to a more recent release (probably an OpenSolaris build after the June 2009 release; I'd love to try the new ZFS deduplication feature).  Everything has been running fine, with the exception of the TimeSlider feature for doing automated ZFS snapshots.  That was fixed in the June 2009 release.

What I'm Running On The Media Server

The media server is doing a few things for us at home; here are the services it's providing:
  • iTunes music storage.  We have a Mac mini that we use for iTunes; all of our music is in iTunes, and the Mac mini NFS-mounts a filesystem from the media server.  I tried both CIFS (Samba) and NFS.  I preferred NFS, but it seemed to have troubles with the Mac as an NFS client.  After reading James Gosling's blog entry on what he observed with Macs and Solaris and NFS, I made a tweak on the Mac mini and everything has been working smoothly ever since.  We make this library shareable to our other Macs on the home network.
  • iPhoto storage.  The same Mac mini also stores all of our photos.
  • TiVo media backup and playback.  There is a great open source Java application called Galleon that uses the TiVo Home Media Engine (HME) API; the API lets you write your own Java apps that show up as part of the TiVo's on-screen menu system, and it lets you talk to the TiVo to grab shows off the TiVo's disk and put shows on there, among other functions.  Primarily, we use it as a way to keep backups of shows that we don't want to lose in case the TiVo's disk crashes and loses data.  This has happened several times, mostly due to unforeseen loss of power.
  • Personal finances (via Intuit's Quicken software).  To do this, I run the free, open source VirtualBox software.  I used to use VMware, but VMware isn't free and it doesn't run on any version of Solaris.  VirtualBox runs on every operating system I use, it's free, and it's improving much faster than VMware is.  So, I run Windows XP as a guest OS inside VirtualBox, and use that guest OS to run Quicken.  When I'm feeling like the guest OS is getting slow or am worried that it's collected viruses, I just blow away that guest image and go back to an earlier snapshot.

My Wish List

Are any of you doing something similar with your home setups?  If so, maybe you have done some of the things I'd like to set up but haven't gotten done yet.  Here's my wish list of apps and features I'd like to add, and I'd like to do it all natively in OpenSolaris (in other words, none of these services would need to run in a guest OS like Windows or Linux under VirtualBox):
  • DLNA server running natively in OpenSolaris.  DLNA will give me the ability to share and stream my media (photos, music, video) to the PlayStation 3.  I've played with several solutions; more on this in a moment.
  • Live streaming of TiVo content from the media server back through the TiVo.  Galleon lets me transfer shows to and from the TiVo, but not play them live.  This is a feature that the ReplayTV product had (I still miss that product, and am sorry they went out of business), and their Java app was superior in several ways to Galleon.  But, it looks like somebody has written software to let you play TiVo shows from your media server straight through the TiVo.  I just haven't gotten it running on the media server yet.

Powered by ScribeFire.



Using OpenSolaris as a TiVo backup device

OpenSolaris logo TiVo logo

Sometimes my TiVo gets confused after a power outage, and it takes forever to reboot (I think it's trying to recover its files which takes a while when you have over 1TB of storage connected to the TiVo).  Sometimes, I lose some of my shows, and that makes me sad.

So I decided to find some software I could get running on OpenSolaris so I could use my home media server as a backup device for my TiVo shows.

I did it, using the Galleon open source Java application; here's how.

Now, if somebody smarter than me would just port the TiVo software to OpenSolaris, we'd have the benefits of ZFS on a TiVo box.  That would be fantastic.

Powered by ScribeFire.

Thursday May 21, 2009

Compiling ffmpeg on OpenSolaris

Here's something I don't understand: I'm trying to compile the open source program "ffmpeg", useful for transcoding from one kind of movie file and video encoding format to another, plus a whole big bag-o-tricks.

So I download the source code:
svn checkout svn://svn.ffmpeg.org/ffmpeg/trunk ffmpeg
Then I go into the ffmpeg source directory and type "./configure".  It fails, reporting:

check_ld
check_cc
BEGIN /tmp/ffconf.XXRJaOkG.c
   1   #include <signal.h>
   2   static void sighandler(int sig){
   3       raise(SIGTERM);
   4   }
   5   int main(void){
   6       signal(SIGILL, sighandler);
   7       signal(SIGFPE, sighandler);
   8       signal(SIGSEGV, sighandler);
   9   #ifdef SIGBUS
  10       signal(SIGBUS, sighandler);
  11   #endif
  12       {     volatile int i=0;
  13       __asm__ volatile (
  14           "xorl %%ebp, %%ebp"
  15       ::: "%ebp");
  16       return i; }
  17   }
END /tmp/ffconf.XXRJaOkG.c
gcc -D_ISOC99_SOURCE -D_POSIX_C_SOURCE=200112 -std=c99
-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D__EXTENSIONS__
-fomit-frame-pointer -c -o /tmp/ffconf.XXYJaalG.o
/tmp/ffconf.XXRJaOkG.c
gcc -o /tmp/ffconf.XXTJaWkG /tmp/ffconf.XXYJaalG.o
./configure: line 663: 16530: Terminated

I can't tell why it fails even making a Makefile, but after scanning the web for a while, I find a suggestion that says "use bash as your shell instead of whatever shell you were using."  I had been using tcsh, then tried ksh, then tried sh, all with the same error result.

So then I try "bash configure".  What do you know?  It created a Makefile just fine.

Now why is that?  Can somebody help me out here?  Because I don't know why the choice of a shell would make the configure script succeed or fail.  And that seems bad for the other shells available on OpenSolaris.

The app seems to compile, mostly.  Next: see how much of the app got compiled and how much of it runs.

Why am I doing this?  Because it's part of "pyTiVo", which I'd like to get running on my OpenSolaris home media server, so that I can back up the TiVo's content onto this nice, ZFS-enabled server.  I'll update my progress here.


Powered by ScribeFire.

About

The views expressed on this blog are my own and do not necessarily reflect the views of Oracle. What more do you need to know, really?

Search

Archives
« July 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
  
       
Today