Demonstrating ZFS

Last week I held a presentation about ZFS. Simplicity is one of the features of ZFS, but there are some things beyond ZFS, and the parts all together are everything but trivial. This text is a step-by-step instruction on how such a presentation can be done.

Preparatory steps

Get some USB flash drives of the same size. I used eight 1 GB USB flash drives with LED lights (so the activity on the storage pool was visible) and two 4-port USB hubs. Don't worry about the uselessness of owning eight memory sticks, at the end of the presentation you'll give them away to the people.

Install an OpenSolaris distribution. I used the SXDE 1/08 distribution on a 1.7 GHz Pentium M laptop with 1 GB of main memory and two USB ports. During the presentation you'll use a video file. Download a Chaplin movie from Install mplayer from and tell the file browser to use it as the default application for the video file. In Preferences → Removable Drives and Media disable Mount removable drives when hot-plugged.

Creating a zpool

Type su in a terminal window and become root. Connect one of the hubs to the laptop and the second hub to the first (on a laptop with two USB ports the free port can be used for a mouse). Connect two sticks to the first hub and four sticks to the second.

Find out the device names of the flash drives, rmformat lists all removable devices when started without parameters. The names are like /dev/rdsk/c2t0d0p0 or /dev/rdsk/c18t0d0p0, where c = controller, t = target, d = disk, and p0 means "whole disk". On all six sticks, delete the data at the beginning of the space. Write a shell script for that, use

   dd if=/dev/zero of=/dev/rdsk/DeviceName count=2048

to fill the first 2048 sectors with zeroes (delete this data on the remaining two sticks too). Create a zpool that contains three mirrors and call it usbpool, the command may look like

   zpool create -f usbpool mirror c2t0d0p0 c4t0d0p0 \\
      mirror c18t0d0p0 c19t0d0p0 mirror c20t0d0p0 c21t0d0p0

Run the zpool list command to see that the pool is actually there, zpool status shows a tree structure of the pool. The pool is automatically mounted and is accessible through the /usbpool directory. Add rights, so that you can write to it as a normal user: chmod a+rw /usbpool.

Simulating a disk failure

Open the file browser, copy the Chaplin movie to /usbpool and open it from there. Depending on your laptop, the video may appear only on the laptop, so that the mplayer window remains blank on the projector output, try to use a different video output driver (mplayer -vo help prints a list of drivers). While playing the movie copy something else to the pool, you can copy the /opt/csw directory, which has 232 MB after installing mplayer. To simulate a disk failure, disconnect one stick from the first hub and one from the second. If every mirror is completely on one hub, no data is lost. In order to have some activity on the pool, delete a directory within /usbpool/csw. Now the zpool status command says, that two sticks have been removed. The video keeps playing. Connect the sticks back to the hubs, the ports do matter, which stick you connect to a port does not. Delete one more directory within the pool. The changes are automatically distributed across all sticks. The zpool status command now says, that all sticks are online. Stop the video and close the file browser.

Export, import, add, replace, destroy

Unmount the pool with the zpool export usbpool command. Disconnect all sticks and put them back in some random order. This step shows, that you could take the data with you and reinstall the pool somewhere else. The zpool import usbpool command brings everything back. You can now open the video again. Connect a new stick to the first hub and add it to the pool.

   zpool add -f usbpool c5t0d0p0

With the zpool list command you can see that the size of the pool has grown. Disconnect one of the mirror sticks and replace it with a new one. The zpool status command says, that the state of one of the sticks is FAULTED.

   zpool replace usbpool c21t0d0p0

Now the state of all sticks should be ONLINE. Cancel all activity and destroy the pool.

   zpool destroy usbpool

Before giving the sticks away, you should first create an FAT32 file system on each of them. Use the fdisk utility and manually create an FAT32 primary partition on one of the sticks. Save the partition table to a file.

   fdisk -W ptable c2t0d0p0

The partition table is now stored in a file called ptable. Write a shell script, that creates the file system on all connected sticks at once.


   # Create partition tables, ptable is a file created with fdisk -W ...,
   # ptable describes the partition table to be written to the sticks,
   # do not use this method, unless the sticks have the same capacity/geometry.

   # WARNING: make sure there are only USB sticks in this range
   for i in `seq 2 21`; do
      fdisk -F ptable c"$i"t0d0p0

   # Create FAT32 file systems, label=SUNTECHDEMO

   for i in `seq 2 21`; do
      echo "Creating FAT32 on $i . . ."
      echo y | mkfs -F pcfs -o fat=32,b=SUNTECHDEMO /dev/rdsk/c"$i"t0d0p0:c

Coincidentally, there were eight people in my presentation, so I was able to give a USB stick to each of them. If there are more people you might want to use random numbers. A Python interpreter is preinstalled in the SXDE 1/08 distribution, so you could draw the numbers there, for instance. If one of the numbers repeats, it would be a bad idea to give the stick to a person sitting next to a person that already has a stick, because for the former the probability of getting a stick is greater than for the people who are not sitting next to an already drawn number.

Try out RAID-Z.

More material

  • There is a video (in German) on YouTube, where a similar presentation is done (update: there is also an English version).
  • Kommentare:

    Hi Igor,

    Good entry :-)

    Just one word about ZFS demos on USB sticks. There is a write limit (between 100,000 and 1 million times) on flash memory cells. After that many writes, the cell becomes burn out. As ZFS updates checksums continuously, there is a possibility that some cells on the stick get burn out, when a zpool is allocated and heavily used. For example, the cells where the uberblock is written might get permanently damaged first.
    It is unlikely that this could happen during a demo, or that it has any impact (ZFS should recover from it, the uberblock is duplicated 4 times). It's just one thing to keep in mind (USB sticks are not hard disks).

    Gesendet von Marc Kossa am April 21, 2008 at 09:32 AM MESZ #

    Hi Marc,

    thank you. Sure, during a demo, one should emphasize, that the sticks just represent hard drives, and it's not the goal of the demo, to show how to build a memory-stick storage pool (:

    Gesendet von Igor Geier am April 23, 2008 at 01:59 PM MESZ #

    Senden Sie einen Kommentar:
    • HTML Syntax: Ausgeschaltet

    Sun Campus Ambassador,
    University of Frankfurt (Germany)


    « März 2015