Monday Jan 04, 2010

OpenSolaris, ZFS, USB 2.0 + 30 Watts = Home Fileserver

Background

A little project I'd had in the back of my mind for some time was to replace the family fileserver over the Holidays. I'd been using a Linksys NSLU2 for several years, but we were about to outgrow our 500GB capacity, and response time was never all that great. I'd installed Unslung early on, which gave me SMB, NFS, FTP, etc., but the poor little Intel XScale chip inside struggled just to drive one server, let alone several simultaneous access methods.

The NSLU2 was most recently running with two 500GB USB 2.0 drives attached. The only redundancy provided is the ability to backup from the disk to an identical disk once a day - not very robust. I'd been thinking about it ever since I saw Zhong's really good writeup of creating an OpenSolaris fileserver about a year ago. The factors driving my decision were as follows:

  1. Green: be as energy-efficient as possible; reuse existing pieces if possible
  2. Robust and Redundant: Provide more storage and more robust storage (RAID Z or better)
  3. Compatible: Provide the SMB/CIFS (for family) and NFS (for me) access we were accustomed to at home
  4. Familiar: Leverage technologies I know:

Well, when I saw the rollout for Shuttle's X27D dual-core Atom-based SFF, I knew that was what I wanted. I waited, I planned. Last month the stars aligned, the prices came down, and I acted.

Old Setup 


Make/Model
Best Current Price
Specs
Server
Linksys NSLU2
US$43 refurbished; US$86 new
USB2 disk I/F; 100Mbps net I/F; running Unslung 6.8
Storage
2 x Simpletech SimpleDrive External Disk drives
US$88 refurbished; US$51 used (EBay)

3.5" form; USB2 disk I/F; 524GB raw storage each

This setup provides:

  • 490GB formatted storage
  • Nightly automatic backup onto clone disk
  • SMB/CIFS access via LAN
  • Boots in around 15 minutes
  • NFS access via LAN (only via reflashing with Unslung)
  • ~4.5MB/s write (NFS), ~3MB/s write (SMB)
  • ~5MB/s read (NFS), ~4MB/s read (SMB), ~150KB/s (scp, sftp or rsync) - note: any encrypted transfers will be CPU-limited by that poor XScale processor)

New Setup

I decided to reuse my existing external disk drives, as well as an existing laptop drive. This meant I only needed to purchase the X27D itself, plus memory. No DVD/CD - I use the network and bootable USB devices instead. To allow for larger, more robust, and more robust storage, I was able to find a used Simpletech drive (same access times, same total raw storage capacity) on EBay.


Make/Model
Price Paid
Specs
Server
Shuttle X27D SFF
US$139 refurbished
Builtin dual-core Atom 330 CPU @ 1.6GHz; GigE networking


US$33 new
2GB DDR2 memory


Existing disk
2.5" 120GB SATA disk
Storage
3 x Simpletech SimpleDrive External Disk drives
Two existing USB disks plus a third from EBay - US$51

3.5" form; USB2 disk I/F; 524GB raw storage each

This setup provides:

  • 957GB formatted storage via 3-disk RAID-Z pool
  • SMB/CIFS, SSH, NFS protocols supported
  • Boots in around 75 seconds
  • Server consumes 26W idle, 32W under load; disks consume power only when accessed, and spin down after 20 minutes
  • ~14MB/s write (NFS)
  • ~18MB/s read (NFS)
  • Same disk latency (due to disk spin-up) as original solution
  • More onboard processing 'headroom' for future services

Configuring OpenSolaris and ZFS

I downloaded the USB Boot Image of Build 129 of OpenSolaris from here, placing the image on a 1GB USB Flash drive using the usbcopy command. After assembling the server, I set the BIOS to boot from USB first, then booted from the 'Live USB'. Installation just worked, and everything seems to be properly supported (haven't tried audio, as this is a headless system).

I'd rsync'ed the contents of the old fileserver to another server the day before (slooowwwww from an NSLU2 - when one of your systems is CPU-limited, make sure you rsync with the lightest-weight encryption possible [I used arcfour]). Setting up the powerful, performant, redundant ZFS filesystem required only a few steps:

# zpool create fs raidz c4t0d0 c6t0d0 c7t0d0   # RAIDZ pool (2+1)
# zfs set sharenfs=on fs # NFS share
# zfs set sharesmb=on fs # Samba share

Still to do:

  • Investigate adding cache to the zpool via part of the internal SATA disk (or possibly eventually an SSD, when prices come down)
  • More rigorous and apples-to-apples performance comparisions
  • Optimizing the OpenSolaris settings and services


Saturday Oct 17, 2009

Drupal-on-a-Stick: now in easy-to-take download form!

At Bay Area DrupalCamp today, Sun is handing out USB Flash sticks that boot any USB-bootable X86/X64 box into OpenSolaris with a running Drupal 6 instance. For those of you not at BADCamp this weekend, we've also made the USB image available for download here. It's a little less than 1GB, and fits on 1GB flash drives.

To burn the image to your own USB flash drive:

  • From a Windows box: see here
  • From an OpenSolaris box: install and use /usr/bin/usbcopy from SUNWdistro-const package

For those who'd rather try this from a "Live DVD", the .iso image is here.

Update 19 October: thanks to alert readers who spotted my ill-formed DVD link above. It's fixed now.druplicon

Thursday Jul 23, 2009

Joomla! and recent OpenSolaris distros - workaround

Though documented elsewhere, it's a good idea to propagate for the benefit of the next person.

Problem: Installation of Joomla! 1.5.x on recent versions of OpenSolaris (2009.06 and later) using MySQL fail with errors after install

Symptom: After otherwise successful install of config on MySQL on OpenSolaris, the home page comes up in the browser with numerous MySQL errors, and the MySQL server restarts:

   Error loading Modules:MySQL server has gone away SQL=SELECT id, title, module, position, 
   content, showtitle, control, params FROM jos_modules AS m LEFT JOIN jos_modules_menu 
   AS mm ON mm.moduleid = m.id WHERE m.published = 1 AND m.access =  0 AND m.client_id = 0 
   AND ( mm.menuid = 1 OR mm.menuid = 0 ) ORDER BY position, ordering

Diagnosis: known issue with MySQL 32-bit server as delivered from pkg.opensolaris.org

Workaround: use the 64-bit version of the MySQL server (/usr/mysql/<version>/bin/64/mysqld_safe, et al)




        
    

Open, Robust and Performant Preservation/Archival - First Steps

As part of the Sun Open Archive for Digital Repositories effort, Sun has been working closely with Fedora Commons to develop and characterize performance of Fedora on Sun (OpenSolaris, ZFS, Sun Storage 7000 appliances, Sun Storage JBOD storage).

The first phase of performance testing and configuration validation was completed recently, with encouraging performance observed from Fedora, running on OpenSolaris 2008.11 and using RAID0 ZFS Pools. Much, much more testing is needed (and is underway at this writing), but much of what we've seen so far bests any Fedora performance testing done to date.

There's quite a compelling case to be made with the combination of Sun hardware and open source OS and application stack, so we're pretty excited about this work.

Check out the attached documents:

  • Poster presented by Sun and Fedora Commons at Open Repositories '09
  • FInal version of this first performance testing document released this month - ingest and access performance of Fedora on OpenSolaris, ZFS and JBODs

Tuesday Jul 21, 2009

When your 'pkg install' is slow on Virtualbox

I like VirtualBox. It does what I need -- often times it does many different things via multiple clients on the same server.

Recently, I've been creating OpenSolaris VirtualBox clients, and, again, they do what I need. Except when it comes to accessing OpenSolaris repositories. With default NAT networking, an OpenSolaris client tends to have slower networking, and at times when I'm installing several packages from pkg.opensolaris.org,  I'll get:

 # pkg install amp
          :
          :
 pkg: Maximum number of network retries exceeded during download. Details follow:
          :
          :

Frustrating, to be sure, but there is a way to put up with all the waiting:

  # export PKG_CLIENT_TIMEOUT=300   # Seconds (default=30)
  # export PKG_TIMEOUT_MAX=10       # Attempts (default=4)

Now, although your pkg commands won't be any faster without tweaking the VirtualBox networking stack, they will eventually finish. This works outside of VirtualBox as well.


Monday Jun 15, 2009

Archival and Preservation: Open Source and Open Archive

One of the more exciting and interesting projects I've been involved with this year centers around Sun's Open Archive systems for Digital Repositories - the use of Open Source software and Sun off-the-shelf hardware to implement world-class data preservation/archival systems. For those of you who remember Sun's Honeycomb archival hardware, this new architecture represents a truly open alternative approach - based on Fedora Commons' offering, OpenSolaris, Sun's X64 servers, JBOD disk arrays and (soon) Unified Storage Systems.

One of the first efforts Sun ISV Engineering contributed to involved sizing and performance testing of Fedora atop OpenSolaris, ZFS and JBODs. The results are both encouraging and impressive, and provide myriad follow-on testing opportunities. I've attached the first draft of test report here.


Tuesday Jun 02, 2009

Community One West 2009

Pulling into San Francisco this weekend with a beefed-up laptop (all the OpenSolaris virtualization goodness requires lots-o-memory - I'm up to 4GB now on my Tecra M2) and a mysterious case of poison ivy (I blame my cat), I set my sights on the Community One West and Java One conferences. Perhaps the last time the 'Sun Microsystems' name is attached to these conferences, and as such a mixture of excitement, trepidation and sadness.

First impression: just as many developer attendings as ever, as just as much energy as ever. Community One is a one-day developer frenzy before JavaOne itself starts tomorrow. Open Source developers. OpenSolaris mavens. Java afficienados. PHP and Perl and Python and Apache and MySQL and Postgres. Something for everyone.

Some notes from the talks I attended Monday:

  • Keynote address (Dave Douglas and John Fowler)
    • OpenSolaris 2009.06 is out!
    • Crossbow 'virtual networking' in Solaris/OpenSolaris - the entire networking stack was done from the bottom up
    • The coming Computing Cloud offering from Sun - impressive!
    • Open Storage (which includes the Open Archive effort I blogged about a couple weeks back)

  • Becoming a ZFS Ninja
    • It's amazing that after four years how many folks still don't know about ZFS
    • It's still easy as pie, performant and bulletproof (my observation, not his)

  • DTrace for Applications

    • Delivered by my ISV-E peer Angelo Rajadurai
    • Great demonstration of using basic and PHP DTrace probes to instrument Drupal (watch this space for his scripts later this week)

  • Source Juicer for Open Solaris
    • Has come a long way in just a few months, to provide packages into OpenSolaris's /pending and /contrib packages repos
    • Coolest new feature: repositories and Source Juicer mechanism itself now have 'install' one-click package install buttons


Tuesday May 26, 2009

Freshly Squeezed Packages!

I work with Open Source Communities. Part of that involves the quest to make Community offerings more accessible, easier to install, and easier to integrate with our platforms.

OpenSolaris comes with the notion of software repositories (much like its Linux cousins) which facilitate easy install of Sun, Community and other software. While popular F/OSS software is available straight from the Communities themselves, often times having it also available (even if we don't add anything appreciable to it) in an OpenSolaris Repository can make for an easier install experience.

That's what I hope to do very soon with Drupal, Joomla!, and several other F/OSS offerings. To that end, the OpenSolaris Community recently announced the availability of Source Juicer, a web-based, automated mechanism to build and deploy IPS packages into the /pending, and ultimately, the /contrib Repositories.

At this moment, and thanks to helping with the Beta testing,  I've gotten the following packages into the temporary /pending repository (eventually, Source Juicer will send things directly into the 'real' /pending, but this gets us on our way to /contrib):

  • Drupal 5
  • Drupal 6
  • Acquia Drupal

So, I say to you, and to the OpenSolaris Community, and the Drupal Community - do you run OpenSolaris? Wanna try these out? If so:

  • Make sure you're running OpenSolaris 2008.11 or a later /dev snapshot
  • Point 'pkg' or Package Manager at the temporary /pending repository as follows:
% pfexec pkg authority -O http://jucr.opensolaris.org/pending  JuicerPending
  • Install the package of your choice (in this case, the Apache Web Server requirement will pull in the Apache Web Server if not already installed0:
% pfexec pkg install drupal6
  • In this case, Drupal 6.x will be installed in /var/apache2/2.2/htdocs/drupal6
Try these and other packages that will appear in /pending. This will help us vet the packages, and get them promoted to /contrib sooner.

Wednesday Dec 05, 2007

OpenSolaris on a new Shuttle SFF

A new week, a new OpenSolaris install experience...

Picked up a new replacement 'everything' server last week, a Shuttle SN68SG2. The intent is to upgrade my main work server, and take advantage of some latest technologies:

  • Multi-core lower-power AMD CPU
  • OpenSolaris xVM technology for virtualization right on top of hardware hypervisor
  • Gigabit Ethernet (I tend to schlep a lot of big .iso images from box to box here)

So I tricked this box out (under $500!) as follows:

  • AMD Athlon 64 X2 4800+ (2.5GHz, dual-core, 65W) CPU
  • 2 x 2GB DDR2 400 memory
  • 250GB SATA disk

Installing the latest Nevada (nv78), went fairly well, except for the NIC. The rundown:

  • CPU/memory: works (frkit isn't happy with dual-core, but that's a known issue)
  • SATA: works!
  • Audio: not yet tested [update: it works]
  • USB 2.0: works
  • Firewire: not yet tested
  • Onboard graphics: works great with bundled nVidia driver, driving 1680x1050 Benq monitor
  • Onboard NIC (nVidia 630a chipset): 'nge' driver did not recognize this chipset, but Murayama-san's 'nfo' 2.X driver does (once again, Arigato Gozaimashita, Murayama-san!)
Next step: playing with xVM (which this CPU allows via AMD-V). Stay tuned.


About

Main Sequence: 1) an astronomy term denoting the lifecycle of a majority of known stars. 2) err @ Sun/Oracle: long-time (since 1988) Sun/Oracle veteran, still shining in an ever-changing high-tech universe

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today