Monday Jan 04, 2010

OpenSolaris, ZFS, USB 2.0 + 30 Watts = Home Fileserver

Background

A little project I'd had in the back of my mind for some time was to replace the family fileserver over the Holidays. I'd been using a Linksys NSLU2 for several years, but we were about to outgrow our 500GB capacity, and response time was never all that great. I'd installed Unslung early on, which gave me SMB, NFS, FTP, etc., but the poor little Intel XScale chip inside struggled just to drive one server, let alone several simultaneous access methods.

The NSLU2 was most recently running with two 500GB USB 2.0 drives attached. The only redundancy provided is the ability to backup from the disk to an identical disk once a day - not very robust. I'd been thinking about it ever since I saw Zhong's really good writeup of creating an OpenSolaris fileserver about a year ago. The factors driving my decision were as follows:

  1. Green: be as energy-efficient as possible; reuse existing pieces if possible
  2. Robust and Redundant: Provide more storage and more robust storage (RAID Z or better)
  3. Compatible: Provide the SMB/CIFS (for family) and NFS (for me) access we were accustomed to at home
  4. Familiar: Leverage technologies I know:

Well, when I saw the rollout for Shuttle's X27D dual-core Atom-based SFF, I knew that was what I wanted. I waited, I planned. Last month the stars aligned, the prices came down, and I acted.

Old Setup 


Make/Model
Best Current Price
Specs
Server
Linksys NSLU2
US$43 refurbished; US$86 new
USB2 disk I/F; 100Mbps net I/F; running Unslung 6.8
Storage
2 x Simpletech SimpleDrive External Disk drives
US$88 refurbished; US$51 used (EBay)

3.5" form; USB2 disk I/F; 524GB raw storage each

This setup provides:

  • 490GB formatted storage
  • Nightly automatic backup onto clone disk
  • SMB/CIFS access via LAN
  • Boots in around 15 minutes
  • NFS access via LAN (only via reflashing with Unslung)
  • ~4.5MB/s write (NFS), ~3MB/s write (SMB)
  • ~5MB/s read (NFS), ~4MB/s read (SMB), ~150KB/s (scp, sftp or rsync) - note: any encrypted transfers will be CPU-limited by that poor XScale processor)

New Setup

I decided to reuse my existing external disk drives, as well as an existing laptop drive. This meant I only needed to purchase the X27D itself, plus memory. No DVD/CD - I use the network and bootable USB devices instead. To allow for larger, more robust, and more robust storage, I was able to find a used Simpletech drive (same access times, same total raw storage capacity) on EBay.


Make/Model
Price Paid
Specs
Server
Shuttle X27D SFF
US$139 refurbished
Builtin dual-core Atom 330 CPU @ 1.6GHz; GigE networking


US$33 new
2GB DDR2 memory


Existing disk
2.5" 120GB SATA disk
Storage
3 x Simpletech SimpleDrive External Disk drives
Two existing USB disks plus a third from EBay - US$51

3.5" form; USB2 disk I/F; 524GB raw storage each

This setup provides:

  • 957GB formatted storage via 3-disk RAID-Z pool
  • SMB/CIFS, SSH, NFS protocols supported
  • Boots in around 75 seconds
  • Server consumes 26W idle, 32W under load; disks consume power only when accessed, and spin down after 20 minutes
  • ~14MB/s write (NFS)
  • ~18MB/s read (NFS)
  • Same disk latency (due to disk spin-up) as original solution
  • More onboard processing 'headroom' for future services

Configuring OpenSolaris and ZFS

I downloaded the USB Boot Image of Build 129 of OpenSolaris from here, placing the image on a 1GB USB Flash drive using the usbcopy command. After assembling the server, I set the BIOS to boot from USB first, then booted from the 'Live USB'. Installation just worked, and everything seems to be properly supported (haven't tried audio, as this is a headless system).

I'd rsync'ed the contents of the old fileserver to another server the day before (slooowwwww from an NSLU2 - when one of your systems is CPU-limited, make sure you rsync with the lightest-weight encryption possible [I used arcfour]). Setting up the powerful, performant, redundant ZFS filesystem required only a few steps:

# zpool create fs raidz c4t0d0 c6t0d0 c7t0d0   # RAIDZ pool (2+1)
# zfs set sharenfs=on fs # NFS share
# zfs set sharesmb=on fs # Samba share

Still to do:

  • Investigate adding cache to the zpool via part of the internal SATA disk (or possibly eventually an SSD, when prices come down)
  • More rigorous and apples-to-apples performance comparisions
  • Optimizing the OpenSolaris settings and services


Saturday Oct 17, 2009

Drupal-on-a-Stick: now in easy-to-take download form!

At Bay Area DrupalCamp today, Sun is handing out USB Flash sticks that boot any USB-bootable X86/X64 box into OpenSolaris with a running Drupal 6 instance. For those of you not at BADCamp this weekend, we've also made the USB image available for download here. It's a little less than 1GB, and fits on 1GB flash drives.

To burn the image to your own USB flash drive:

  • From a Windows box: see here
  • From an OpenSolaris box: install and use /usr/bin/usbcopy from SUNWdistro-const package

For those who'd rather try this from a "Live DVD", the .iso image is here.

Update 19 October: thanks to alert readers who spotted my ill-formed DVD link above. It's fixed now.druplicon

Thursday Jul 23, 2009

Joomla! and recent OpenSolaris distros - workaround

Though documented elsewhere, it's a good idea to propagate for the benefit of the next person.

Problem: Installation of Joomla! 1.5.x on recent versions of OpenSolaris (2009.06 and later) using MySQL fail with errors after install

Symptom: After otherwise successful install of config on MySQL on OpenSolaris, the home page comes up in the browser with numerous MySQL errors, and the MySQL server restarts:

   Error loading Modules:MySQL server has gone away SQL=SELECT id, title, module, position, 
   content, showtitle, control, params FROM jos_modules AS m LEFT JOIN jos_modules_menu 
   AS mm ON mm.moduleid = m.id WHERE m.published = 1 AND m.access =  0 AND m.client_id = 0 
   AND ( mm.menuid = 1 OR mm.menuid = 0 ) ORDER BY position, ordering

Diagnosis: known issue with MySQL 32-bit server as delivered from pkg.opensolaris.org

Workaround: use the 64-bit version of the MySQL server (/usr/mysql/<version>/bin/64/mysqld_safe, et al)




        
    

Open, Robust and Performant Preservation/Archival - First Steps

As part of the Sun Open Archive for Digital Repositories effort, Sun has been working closely with Fedora Commons to develop and characterize performance of Fedora on Sun (OpenSolaris, ZFS, Sun Storage 7000 appliances, Sun Storage JBOD storage).

The first phase of performance testing and configuration validation was completed recently, with encouraging performance observed from Fedora, running on OpenSolaris 2008.11 and using RAID0 ZFS Pools. Much, much more testing is needed (and is underway at this writing), but much of what we've seen so far bests any Fedora performance testing done to date.

There's quite a compelling case to be made with the combination of Sun hardware and open source OS and application stack, so we're pretty excited about this work.

Check out the attached documents:

  • Poster presented by Sun and Fedora Commons at Open Repositories '09
  • FInal version of this first performance testing document released this month - ingest and access performance of Fedora on OpenSolaris, ZFS and JBODs

Monday Jun 15, 2009

Archival and Preservation: Open Source and Open Archive

One of the more exciting and interesting projects I've been involved with this year centers around Sun's Open Archive systems for Digital Repositories - the use of Open Source software and Sun off-the-shelf hardware to implement world-class data preservation/archival systems. For those of you who remember Sun's Honeycomb archival hardware, this new architecture represents a truly open alternative approach - based on Fedora Commons' offering, OpenSolaris, Sun's X64 servers, JBOD disk arrays and (soon) Unified Storage Systems.

One of the first efforts Sun ISV Engineering contributed to involved sizing and performance testing of Fedora atop OpenSolaris, ZFS and JBODs. The results are both encouraging and impressive, and provide myriad follow-on testing opportunities. I've attached the first draft of test report here.


Tuesday May 26, 2009

Digital Repositories and Archives - One Solution

Last week, at OpenRepositories09, Sun announced a new, open approach to digital archival and preservation. It leverages:

  • Commodity Sun x64-based servers
  • Sun OpenStorage, including the new 7000-series Unified Storage Systems
  • OpenSolaris, including ZFS
  • Fedora Commons, Drupal and Islandora open source offerings

The result is a totally-open hardware and software stack - scalable, robust, and made for long-term archival (\*and\* retrieval) of massive amounts of data.

In the days to come, we'll be publishing the results of initial functional and performance testing with portions of this stack, and the results are quite exciting. For now, take a look at the "glossy" and the "one-pager" Sun was circulating at OR09 last week. Good useful stuff - intuitively obvious solutions to substantial needs.

Wednesday May 20, 2009

Repositories!!

Open Repositories 2009

Greetings from Open Repositories 2009, the premier Open Source Archival and Repositories conference. Sun has a strong presence here, with lots of good interactions, and an exciting rollout of our Open Archive solution....

Oh. Wait. You say you don't know what a Repository is? You say it sounds like an embarrassing, unmentionable over-the-counter medication. Well, soothe thyself, gentle reader, it's nothing like that. Think of a repository as an online Library. A huge collection of digital data or documents. As human beings continue to create petabytes (1PB = 1 million GB) of data each and every day, it not only has to live somewhere, but almost always requires long-term storage. Forever. For the ages. A Real Long Time.

The trick, you see, is not just in storing the stuff. We can do that. We can stream data to disk and tape and optical media and eventually holographic cubes (oh, wait, I'm getting ahead of myself). The real Interesting Thing is not just storing it, but being able to intelligently access this data days, months or decades later. Intelligently. Quickly.

That's why we're here, along with developers, archivists and software architects. Sun is working closely with the Fedora (Commons) community, as well as University of Prince Edward Island. Look for more postings this week on this in greater detail. Until then, the gracious hospitality of downtown Atlanta awaits...



Wednesday Mar 11, 2009

DrupalCon DC Rewind

Amazingly, it was my very first DrupalCon, but that made it even more memorable! What an amazing, dynamic, welcoming, sharp, productive Community the Drupal bunch are. It was really a pleasure to attend, participate, soak it up and interact.

Others, including my colleagues Scott Mattoon and George Drapeau have done a good job at covering what happened, so I'll just mention what struck me in D.C. last week.

  • The Drupal Community is still very interested in, and welcoming of, what Sun can bring to the table. I suppose that means we're being Good Community Citizens. The most common question at the Sun Table in the Exhibit Hall was 'so, what is Sun's role in the Community'. Because of that, Scott covered all this goodness in the Sponsor Interview of Sun conducted by DrupalCon organizers this year.
  • Angelo gave a compelling BOF, mostly on DTrace -- too few people know about DTrace, ZFS, etc., and when they find out, they're excited!
  • When I arrived Tuesday to set up, a fellow was squatting at our table. I politely asked him if we could have the table, and at that time I recognized him as Victor Kane, author of the book I'd just purchased (and is just available), Leveraging Drupal - Getting Your Site Done Right. I do have to recommend this book as one of the best 'hands on'/practicum books for Drupal, and that honestly has nothing to do with the fact that I now have my very own autographed copy!
  • washingtonpost.com has two live Drupal sites on Postgres/Solaris/SPARC, and possibly two more on the way!

My biggest lasting impression: Sun remains a welcomed and productive member of the Drupal Community because we Listen to it, rather than Sell into it. Though we must constantly resist the corporate urge to treat Community as Potential Customer, there are significant benefits to be had.

P.S. 12 March: One big lasting thing: The Powers That Be in the Drupal Community are very serious about their assertion that the Drupal Core shouldn't be touched (and for good reason). The reason I personally will always remember this henceforth comes from the enduring line of the week:

"Every time someone hacks the core of Drupal, God kills a kitten."



Drupal In A Flash

druplicon

Flash at Sun

No, really, this isn't a Marketing or Sales FluffPiece! I promise!

Today, Sun announces SSD drives for our servers. One of the proof points we used in support of this was the Drupal CMS. While Drupal 6 doesn't always stress the I/O subsystems themselves (or maybe because of it!), we went ahead to test some pseudo-real-world workloads against Drupal/PHP/Apache/MySQL/Solaris/ZFS. The results are posted today here in the Drupal-on-Sun Wiki site.

What did we find?

  1. As predicted, more performance benefits were observed with Read-only or Read-Write loads than Write-only (we populated a Drupal MySQL database from script)
  2. Up to 50% performance improvement was seen in complex Drupal workloads when using SSDs compared to traditional hard disks
  3. A most cost-effective ZFS Hybrid Storage Pool (utilizing both technologies) provides more modest performance gains for less money

Hybrid Storage PoolExpected, yes, but still good when it proves out!

Intelligent site and database architectures will be able to take advantage of this technology in ways we've not yet envisioned, especially in a tiered approach that localized most-accessed data in the faster tiers (main memory, Flash/SSD, hard disk).

Wednesday Mar 04, 2009

Acquia @ DrupalCon DC

Sitting in the Acquia update presentation this afternoon. What are they up to?

  • Announced a new CEO today (Tom Erickson) 
  • 40 customer and 50 partners so far
  • The Acquia Drupal Stack Installer ("DAMP)" (Note to Acquia: when is this available for OpenSolaris?)
  • New Acquia Themes
  • Acquia Search (based on Apache Solr)
About

Main Sequence: 1) an astronomy term denoting the lifecycle of a majority of known stars. 2) err @ Sun/Oracle: long-time (since 1988) Sun/Oracle veteran, still shining in an ever-changing high-tech universe

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today