Tuesday Feb 03, 2009

other filesystems

On my laptop I have a number of non-Solaris filesystems that have useful things on them. I have an ext3 filesystem from my Linux/Ubuntu install, a fat32 filesystem to transfer things between Windows and other OSes and a NTFS for Vista. In the past when I've added these things to vfstab it created problems with OpenSolaris booting. Now I know I should really debug that problem, but I'm lazy...

In the past I hacked my .bashrc to handle the mounting, but this is a terrible solution. I have since come up with a slightly less terrible solution that has the added benefit of me digging into SMF. So I created a simple SMF service that would mount these filesystems at boot.

So the first part is to craft a config. I put the manifest in

And also create a script that will be run as part of this service. I placed the script in
All that was left was to import the manifest:
pfexec svccfg import /var/svc/manifest/system/filesystem/pete-fs.xml

So now that means my non-Solaris filesystems are mounted at boot no problems at all. I do still need to dig into the vfstab problem though, but that's likely to be on a different day..

Wednesday Dec 10, 2008

Get it whilst it's hot!

OpenSolaris 2008.11 is available right now. There has been tremendous work by the OpenSolaris Team to get this ready and usable. I use OpenSolaris everyday on my laptop. I definitely come across a few growing pains (see my blog), but there is an amazing rate of improvement. This makes Solaris look and feel like a modern development Operating System.

There are a definite bunch of changes and improvement to the core OS. COMSTAR is now shipping as part of the core OpenSolaris. There are several highlights from the storage community:

  • COMSTAR and Fiber Channel target support
  • ZIL + L2ARC support for ZFS (support for separate read and write caches especially to utilise FLASH/SSD)
  • NPIV (virtualisation of FC ports)
  • MMS (Media Management System)
  • Multipathing for Tape
  • Storage FMA
  • NFS over RDMA
  • CIFS significant performance improvement over 2008.05

We also created a few meta-packages for storage:

  • NAS
  • All storage packages

I strongly encourage everyone to go and check it out. The CD can be run as a live CD so you don't need to futz around with your disk to try it.

Thursday Dec 04, 2008

Life With Solaris -- Use at your own risk

I came across Life With Solaris just the other day. It has a lot of the video applications that you might have grown to expect from an OS. Note that using these may violate laws or several other things depending on your location, so you need to use at your own risk.

It was pretty simple to add this to my OpenSolaris package repositories:

pfexec pkg set-authority -O http://pkg.lifewithsolaris.jp:10000/ video

and now I can access all their goodness. With that done I could do:

pfexec pkg install LWSvlc
pfexec pkg install LWSlibdvdcss

Now I can play DVDs in OpenSolaris. This is more significant than it sounds. A while back my dvd player broke in my laptop. Sony did a good job replacing it, but it was replaced with a Matshita drive (UJ-852S). And it seems that this has been impervious to my attempts to play DVDs on anything except windows. It turns out OpenSolaris is the winner in getting DVDs to play!

Wednesday Dec 03, 2008

Keep Austin Weird!

A few weeks ago I was in Austin for SC08. There were a number of things going on. Prior to the main event Sun organised a HPC consortium at the Airport Hilton. We managed to get a trip out to TACC, although I had to film a bunch of the presentations which meant I didn't get to view the TACC setup.

There was also a SAMQFS BOF session associated with the HPC consortium. It was really interesting to see how enthusiastic the customer base is for SAMQ. Harriet gave an overview of SAMQ 5.0 and I gave a presentation of COMSTAR and iSCSI/iSER which provides essentially a storage transport layer for SAMQ 5.0. I also had to film the event. As it turns out the tape ran out in the middle of me speaking :-/ (which I didn't notice), so I won't have to subject you to yet more video of myself.

We also had a student event that we organised at the Karma Lounge. We had a bunch of students, live music, ipods, macbooks and the occasional beverage. Much fun was had by all. Deirdre did an awesome job organising the event.

It was then on to the main event, SC08. There was a lot going on here. I spent some time walking the vendor hall, it was massive. However, I unfortunately had work to do. Dominic and I were manning the Open Storage Bar. This sounds very cool, but it really is a pub with no beer. We'd set up a demo for SAMQ 5.0 using VMWare. However, there was a lot of talk about the Unified Storage appliance with demos too. Several other folks had the VMWare image running to demo the analytics. So I presented that and talked a lot about the Hybrid Storage Pool concept and flash in general.

SC08 kept going for the rest of the week, but I had to hit the road to PASIG in Baltimore, I'll put a post up soon about what was going on there.

Tuesday Nov 25, 2008

ZFS send/recv

After my moments of stupidity documented below I decided I'd fix the problem. I've coded up a script that will use zfs send and recv to replicate my home directory. At the moment it's manual, but I'll look into hooking into an automatic system:


NOW=`date +%Y%m%d%H%M%S`

set -- `getopt i $\*`
if [ $? != 0 ]
echo ${USAGE}
exit 2

for i in $\*
case $i in
-i) echo Initialising Snapshot backup;

ALIVE=`ping ${HOST} 2>&1 | grep -c alive`

if [ ${ALIVE} ]
SNAPS=`zfs list -t snapshot |grep "${FS}@" | wc -l | tr -d " "`
echo Determine incremental
if [ "${INIT}" == "0" ]
ISNAP=`zfs list -t snapshot | grep rpool/export/home@ |tail -1 |cut -d " " -f 1`

echo "Creating zfs snapshot ${FS}@${NOW} ..."
zfs snapshot ${FS}@${NOW} > /dev/null 2> /tmp/error.$$
if [ $? -ne 0 ]
echo "Failed ($?)!"
cat /tmp/error.$$
rm /tmp/error.$$
exit 1
echo "Successfully created snapshot: ${FS}@${NOW} !"
rm /tmp/error.$$
if [ ${SNAPS} -ge ${MAX_SNAP} ]
LAST=`zfs list -t snapshot | grep "${FS}@" | head -1 | awk '{ print $1}'`
echo "Destroying last snapshot: ${LAST}..."
zfs destroy ${LAST} > /dev/null 2> /tmp/error.$$
if [ $? -ne 0 ]
echo "Failed ($?)!"
cat /tmp/error.$$
rm /tmp/error.$$
exit 1
echo "Successfully destroyed snapshot: ${LAST} !"
rm /tmp/error.$$

if [ "${INIT}" == "1" ]
echo "Initialising the zfs replication of ${FS}..."
zfs send ${FS}@${NOW} | ssh ${USER}@${HOST} pfexec /sbin/zfs recv ${RS}@${NOW}
echo "Initiate the zfs send of ${FS}@${NOW} to ${RS}@${NOW}..."
zfs send -i ${ISNAP} ${FS}@${NOW} | ssh ${USER}@${HOST} pfexec /sbin/zfs recv -F ${RS}@${NOW}

if [ $? -ne 0 ]
echo "Failed ($?)!"
exit 1
echo "Successfully synced snapshot: ${FS}@${NOW} and ${RS}@${NOW} !"
echo "Cannot connect to ${HOST}"

It's pretty simple really, just relies on the underlying aspects of ZFS to make it all ok.

Grr.. it looks like html destroys the format... I'll fix that later.

Monday Nov 24, 2008

Phew! Not a total doofus!

Turns out a judicious import from a LiveCD does the right thing. I just needed to pass the correct device string (s0, not p2).

It may turn out to prove that I am stupid in many other ways, but at least in this case it was recoverable ;-).

Why am I so stupid?

I always tend to prefer stuff on the bleeding edge warts and all. So I was in Baltimore for PASIG during the week and I suggested to a few folks they should consider using Celeste and a basis for a data store, seems fine right? It worked out of the box for me on OpenSolaris, they had a few problems so I rebooted into Ubuntu to install and try it there. Again no problems.

So whilst in Ubuntu I decided to use zfs-fuse to pull in my mail filters for Thunderbird. Again should be simple enough. I'd had problems before with this causing problems to the underlying zpool, but I thought this would be fine since I just wanted read access. Big mistake.

Ok, so what happens? When I try to reboot the ZFS filesystem isn't recognised by GRUB. Ok that's fine, I've seen it before. Simple solution, boot with the LiveCD import the pool done, right? Nope, unfortunately no LiveCD and I'm also not patient (hence the title of this entry). So what do I do next? Try and fail to create a OpenSolaris Live USB keyring. Then I get the idea that I should update ZFS fuse and see if there is anything I can do.

hg update, scons, sudo scons install. Good to go. Then I notice the ZFS version on the pool is only 10 and ZFS fuse goes all the way to 14. Maybe that's the problem.. This is where the good fairy should have been crying "Don't do it, Don't do it". Unfortunately there was no one around to pry my fingers away from the keyboard and I upgraded the filesystem.

Now ZFS doesn't see the pool at all... Now that I'm back in the office I'll try to do some surgery with zdb, but I'm not hopeful..

Friday Nov 14, 2008

Why do things keep breaking?

So I've been having a few problems with my laptop lately. It boots fine, but occasionally it will just hang in the middle of doing things. Also occasionally on boot I get random pixelated weirdness. This led me to start to suspect something was wrong with the onboard nVidia adapter. Fortunately I have a Sony SZ laptop that I can switch to use the Intel graphics device instead. So I powered off, flipped the switch and booted. This seems to have fixed that problem, but it turned out this was only the start.

I rock up to the office on Monday morning. I have an external monitor so I don't develop eyestrain, etc. Now I'm faced with the Herculean task of making Xorg recognise and do the appropriate things with an external monitor. To add to this difficulty I also have to do a bunch of presentations with the laptop so I need to have a configuration that will work nicely on the road too (with all sorts of random projectors, etc).

After fooling around with this stuff for a while I discovered that it's necessary to the 'Virtual' desktop size so that Xrandr can do it's magic. This wasn't actually a trivial task. Since virtual console support in Solaris is still working it's way to the surface the only way to test new Xorg.conf files (that I could figure out) was essentially to reboot if X didn't come back up it made the process fairly slow.

What I ended up with was the following config. It all seems to be working fine now. I've spent most of the last month with this config. The only issue is that by having such a large Virtual setting (Virtual 3200 1200) that compiz won't work for this setup. The next step is to find enough time to convince Sony that the nVidia chipset is irrecoverably broken...

Tuesday Oct 14, 2008

what to do?

I've had a few random thoughts about what to poke at and do. The current idea revolves around USB keyring devices. I was recently at SDC and was passing out 2GB USB key rings with OpenSolaris bits on them. It included a bunch of videos etc. So that was great to promote Open Storage, etc, but there had to be something more interesting for a geek to do with 2GB. So I've started to investigate creating a bootable OpenSolaris image on a small partition of this device. The idea is to have this be a Solaris recovery device ala knoppix, but also to have a pcfs/fat32 partition that is recognised by every operating system under the sun. So I'm poking around boot sectors, partition labeling, etc. I'll post when I have something working with the details.

Wednesday Oct 08, 2008

Emacs for OpenSolaris

So I'm now using OpenSolaris fulltime on my laptop (WoHoo!). I came across the fact that it doesn't have emacs there. So I did the obvious thing:

pkg search -r emacs
. Unfortunately this showed that there isn't currently an emacs package in OpenSolaris. No worries, I have a compiler and will travel. So I went and downloaded the source.

I discovered there were a few steps that I needed to follow in order to be able to build this properly:

  1. pfexec pkg install SUNWgnome-common-devel

  2. pfexec pkg install SUNWxorg-headers

Now that is done we can get to work:

  1. tar xf emacs-22.3.tar.gz

  2. cd emacs-22.3

  3. ./configure --with-gtk

  4. make -j 10

  5. pfexec make install

So now that is done and we have a working emacs installed. There was one last thing I noticed. The Meta key is not mapped to Alt, so I had to add

keycode 64 = Meta_L
to my .xmodmap and also the following to my .bashrc:

if [ -f ${HOME}/.xmodmap ]
xmodmap ${HOME}/.xmodmap

all is now right with the world.


Peter Buckingham


« April 2014