Friday May 15, 2009

KCA2009 - first media contact

KCA received its first media hit with a query from the editor of An interview (via email) with me! New and exciting :-)

I was nervous about how the answers I gave to her questions might get transformed into an article - I've never dealt with a media enquiry before - so I was relieved to see that the article that popped up just before lunchtime today was true to the answers I'd provided: Sun shines more light on Open Source Kernels.

On the registration front, people are starting to register for the conference (cue sigh of relief) and there's still plenty of time (well, another 2 weeks) to get your seat at the early bird price. So don't be shy, decide that you really do want to meet, learn from and hang out some of the finest minds in Open Source software. You Know You Want To!

Tuesday May 12, 2009

KCA2009 - registrations are NOW LIVE (also, agenda posted)

About 15 minutes ago I was ecstaticly pleased to see that the KCA2009 website had gone live with the confirmed agenda, speaker bios and most importantly, registrations.

Wednesday May 06, 2009

KCA2009 - we've finalised the accepted presentations

After a lot of back-n-forth in the committee, we've finally come up with a list of presentations that we have accepted for the 16 speaking slots at Kernel Conference Australia 2009. Hopefully all the presenters will get back to me asap so I can finalise the schedule and say who they are.

Saturday May 02, 2009

KCA2009 - Call for Papers is now CLOSED

I'm pleased to announce that the official Call for Papers for Kernel Conference Australia 2009 is now closed. Over. Finito!

We've had a great response to the CfP, and the review committee will be meeting in coming days to nut out just which presentations we'll take.

Personally, I'm very happy that given the event is new, and economies around the world are in turmoil, we've had so many presentation proposals (and of a very high quality) submitted.

The review committee will be working hard to ensure that we get the accepted papers finalised as soon as possible, but either way we've still got enough of the schedule mapped out already (keynote speakers and panels) that when registrations open on Monday you'll have a fair idea of how it'll all go.

If you are planning to register and attend, please join our Facebook Event.

Monday Apr 27, 2009

Less than ONE WEEK to go for KCA2009's Call for Papers

I woke up with a shock this morning, realising that there's less than ONE WEEK to go before the KCA2009 Call for Papers closes!


If you're going to submit a presentation proposal, please get it in as soon as possible - you don't want to miss out on being part of this conference.

Here are the links, just link case you'd misplaced them:

Official Conference Website

Call for Papers (all pretty)

Call for Papers (original, dry and technical)

The Facebook event

If you're a potential sponsor, please contact myself, Claire or Gabriele (details on the Official Conference Website).

I've also put together a flyer which you could print out and post around your workplace. Please do - the more the merrier!

Don't forget, registrations open in one week, on 4 May 2009. Save The Date

Friday Mar 27, 2009

KCA2009 - event website NOW ACTIVE

I'm really pleased to announce that the KCA2009 conference website is now active. Registrations open on 4 May 2009. The Call for Papers / (original CfP page) is still open and we're really keen to get your presentation suggestions. We're also very keen to get more sponsors on board - this isn't just a Sun event, it's an Open Source event - and Innovation Happens Everywhere(tm).

Monday Mar 09, 2009

KCA2009 - facebook event and IRC channel now active

Over the weekend I realised that I should organise both pre-Web1.0 and a Web2.0 thingies for KCA2009, so I've now registered the


channel on, and

Kernel Conference Australia 2009 event on

Please join!

Friday Mar 06, 2009

Kernel Conference Australia - Call for Papers issued

More movement at the station when it comes to KCA2009 - we've now got a Call For Papers which I've started emailing to various people and groups.

I'm still on the hunt for sponsorship to help things along, so if your org can help, please contact me directly.

One other thing - you might look at all the details for KCA and think "but I don't use OpenSolaris why should I bother?" - to which the response is that we're interested in Open Source, which is not limited to just one kernel or company. So please, don't think KCA2009 is not for you.

Friday Feb 27, 2009

Kernel Conference Australia - it's coming!

Over the years it's been a source of frustration for me that the conferences which I wanted to attend were either too expensive (time, travel, registration etc) or not covering topics I was interested in.

Late last year I realised I should stop grumbling about it and fix it myself.

So it is with great pleasure that I can announce that this July 15th to 17th in Brisbane, there will be an Open Source kernel-focused conference: Kernel Conference Australia. We don't have absolutely all of the organisational bits together yet, but here's what we do have:

World class speakers

Fantastic location

Our venue is the Queensland Brain Institute, within the University of Queensland.

Excellent climate

The University is situated in Brisbane, Queensland, Australia. Yes, it will be winter... but winter in Brisbane is a beautiful time to visit.

Call for Papers

The Call for Papers is not quite ready (still a few details to be ironed out), but if you'd like to be considered for a presentation spot you should be thinking about a topic in any of these areas:

  • Cross-architecture kernel development

  • Porting an OS to a new architecture

  • Filesystems

  • System performance visualisation (DTrace, SystemTap?)

  • Image visualisation (GPU kernels)

  • Fault Management

  • (globally) Distributed kernel development - how to make it work

  • Virtualisation

  • Clustering (HPC and High Availability)

  • Distributed systems

  • Kernel Testing - methodologies, interesting problems found

  • Traps and pitfalls found when porting drivers between OSes

  • Realtime performance and scheduling

  • Embedded OSes and drivers (including control systems)

  • Patents and Open Source

  • The state of OS kernel research / what's new / work in progress

Apart from the above, we expect that pretty much any kernel-focused topic for an Open Source licensed OS will be considered by the organising committee.

Target OSes

If you would like to help with sponsoring the conference, please let me know via email (jmcp at my employer . com).

Tuesday Feb 24, 2009

It's a small thing, but pretty

For a while now I've been wanting to switch from CDE to gdm as my login manager - part of the whole "getting ready for OpenSolaris" thing. When I LU'd to 106 I thought I'd give it a try, but svcadm disable cde ; svcadm enable gdm just wasn't giving me any gui lovin'.

Today in #opensolaris on freenode, alanc pointed me towards 6768573 GDM refuses to start if Xinerama is present but not active, which referred to a workaround in the nVidia doco:

Section "Device"
Option "NoTwinViewXineramaInfo" "True"

I added this to each Device section of my xorg.conf, logged out, ran

svcadm disable cde ; svcadm enable gdm

and lo + behold, gdm started up!


A quick change of login theme accomplished via running /usr/sbin/gdmsetup and I'm a heckuvalot happier. Of course, I can't get a command-line login screen option any more (gdm limitation), but I think I'll live :-)

Thursday Jan 01, 2009

The quietest NYE ever...

This bundle is the reason is why

Wednesday Dec 31, 2008

Guess what's on the front page?

The group that I'm part of has developed a really handy netbeans plugin module to help with writing device drivers for OpenSolaris. It's called Starfish and I was delighted to see that it's the current feature "article" at the top of

There are pointers to two flash demos (demo #1 and Driver Workshop) as well. I saw one of the demos at last year's Beijing Tech Day, and was really impressed not only with the module and its presentation, but also with just how many of the attendees at the session really got stuck in with using it there and then.

Well done to Ada and the team!

Tuesday Dec 30, 2008

Under the heading of "Why didn't I do this before?"

J and I have a Lexmark Optra S 1855 with mucho extra memory and a duplexer, got it from EBay in late 2003 when we needed to print out her OT honours thesis. It used to be the case that you could get a Solaris package of utilities and drivers for it from, but sadly the only ones available from these days are for their non-EOL'd printers.

I emailed Lexmark printer support asking whether I could get a copy of the old package, but the response was less than satisfactory:

Thank you for contacting Lexmark Email support. I apologize for the delay
in response.

In reply to your email, this printer Optra S 1855 is supported on Unix AIX
system. However,there are no specific x86 drivers available on Lexmark website.
The available drivers are print-drivers-aix5-sysv.pkg.Z and drivers-aix.pkg.Z.
You need to install these drivers. Alternatively, you can install a system
generic driver.

... after a request which specifically asked for the Solaris package.

So for the last few builds I've been printing to file as postscript, then dumping the .ps direct to port 9100 on the printer's IP.... slack, I know.

Since it's the slack time of the year I figured I'd have a go at seeing whether I could get the printer setup in my non-global zone using the lp family of commands. Not pretty, and in fact, rather painful to get anywhere. Following the instructions on gave me this:

-bash-3.2# lpadmin -p fishdbl -v /dev/null -m netstandard
-bash-3.2# lpadmin -p fishdbl -o protocol=tcp -v /dev/null
-bash-3.2# lpadmin -p fishdbl -T PS -v /dev/null
-bash-3.2# lpadmin -p fishdbl -I postscript -v /dev/null
-bash-3.2# lpadmin -p fishdbl -s prfish
-bash-3.2# lpstat -t
scheduler is running
no system default destination
-bash-3.2# lpadmin -d fishdbl
fishdbl: undefined printer

And yes, "prfish" is the name of the printer on the network.

Stopping and re-starting print/server didn't help either, so I went looking for help. The gnome-ish stuff for OpenSolaris all talked about autodiscovery, which is great except that it requires hal to work, and we don't find hal in a non-global zone. Then I found the Presto project, which lead to Printing community and that mentioned CUPS.

By now I was just totally over it all, and I wanted it to just work, dagnabbit! so I enabled CUPS in the non-global zone, configured my printer queues there, and then ran these commands in the global zone:

# lpset -a printer-uri-supported=lpd://prserv/printers/fishdbl fishdbl
# lpset -a printer-uri-supported=lpd://prserv/printers/fishsgl fishsgl

.... and it was all good!

Well, except for the annoying banner page, but the comments on Dan Anderson's post mentioned that you need to add -o job-sheets=none to the invocation for cupsd and that problem goes away. A quick edit of /var/svc/manifest/application/cups.xml (look for cups-lpd), then a re-import and restart of the cups services, and I'm now really happy.

Of course, I should hassle Norm for a slightly better solution than just hacking the manifest, but I'll do that later.

Thursday Nov 13, 2008

How to write a plugin for Pluggable fwflash(1m)

As I alluded to a few months back, I've been working on backporting Pluggable fwflash(1m) to Solaris 10. Part of the process for getting a backport integrated includes writing what we call a "TOI" - Transfer Of Information. Some people might use the term "technical deep dive".... but I'd look sideways at you if you said that's what I'd been delivering :-)

While I was in Beijing with the rest of our group last month, I delivered several TOIs, foremost of which was the one I'd developed on Pluggable fwflash(1m). The others were on Areca SAS/SATA RAID card driver, and on my stmsboot redesign.

I'm very pleased to be able to provide the Open version of the fwflash presentation via this blog.

I hope you'll find it useful - please let me know.

Thursday Oct 16, 2008

On stmsboot(1m)

When I started working with SAS (November 2006), our group's project (MPxIO support in mpt(7d)) was already off and running. The part that I was responsible for was booting - which meant fixing stmsboot(1m). Initially I was disappointed that I'd been given what I thought was such a small part of the problem to work on, but I quickly realised that there was a lot more to it than my first impression revealed.

Since we were under some pretty tight time pressure, I didn't really have time to do a redesign of stmsboot to make it more sustainable. The expected arrival of ZFS root meant that there was also some uncertainty about how that would tie in - nothing was nailed down, so I had to make some guesses and keep my eyes peeled for when ZFS root eventuated and then see what further changes needed to be made. We putback those changes into snv_63, and had a few followups in subsequent builds, and all seemed ok.

Then in February 2008 there was a thread on storage-discuss about how to obtain a particular device's lun number after running devfsadm -C (or boot -r, for that matter). I did a little digging and figured out that it would indeed be possible to provide that information - if you were willing to do a little digging and make use of a scsi_vhci ioctl() or two. Using hba-private data, unfortunately, so quite unsupportable. But it got me thinking, and I logged 6673281 stmsboot needs more clues as a placeholder.

Then a short while later I noticed that the -L or -lX options to stmsboot(1m) were now broken, as of snv_83 (nobody had worked on stmsboot(1m) since I made my changes in build 63). Since this is an essential part of the actual interface, I figured it was important enough to log (6673278 stmsboot -L/l is broken on snv_83 and later) but was unable to do much about it until I got Pluggable fwflash(1m) out of the way first. I was also annoyed to find that there were problems with updating /etc/vfstab, too (6691090 stmsboot -d failed to update /etc/vfstab with non-mpxio device paths... things were not looking good, and I was watching code rot for real. Staggering!

The kicker was (6707555 stmsboot is lost in a ZFS root world, and so I knew what I had to do - redesign and rewrite stmsboot from scratch.

The Redesign

I started with 4 guiding principles:

  • require only one reboot

  • listing of MPxIO-enabled devices should be \*fast\*

  • minimise filesystem-dependent lookups, and

  • use libdevinfo and devlinks as much as possible.

I then looked at the overall effects that we need to achieve with the stmsboot(1m) command:

  • enable MPxIO for all MPxIO-capable devices

  • enable MPxIO for specific MPxIO-capable drivers

  • enable MPxIO for specific MPxIO-capable HBA ports

  • disable MPxIO for all MPxIO-capable devices

  • disable MPxIO for specific MPxIO-capable drivers

  • disable MPxIO for specific MPxIO-capable HBA ports

  • update MPxIO settings for all MPxIO-capable drivers

  • update MPxIO settings for specific MPxIO-capable drivers

  • list the mapping between non-MPxIO and MPxIO-enabled devices

  • list device guids, if available

What does the old code do?

The code makes use of a shell script (/usr/bin/stmsboot), a private binary (/lib/mpxio/stmsboot_util) and an SMF service (/lib/svc/method/mpxio-upgrade) which runs on reboot.

The private binary does the heavy lifting, providing a way for the shell script and SMF service to determine what a device's new MPxIO or non-MPxIO mapping is. The old private binary also walked through the device link entries in /dev/rdsk when called with the -L or -l $controller options, printing any device mappings. Finally, the private binary handles the task of re-writing /etc/vfstab.

The shell script (stmsboot) is the user interface part of the facility. Its chief task is to do editing of the driver.conf(4) files for the supported drivers (fp(7d) and mpt(7d)), and to set the eeprom bootpath variable on the x86/x64 platform if disabling or updating MPxIO configurations. (Failing to do this would prevent an x86/x64 host from booting). The shell script also makes backup copies of modified files, and creates a file with instructions on how to recover a system which has failed to boot properly after running the stmsboot script.

The SMF service is armed by the stmsboot script, and runs on reboot. It mounts /usr and root as read-write, invokes the private /lib/mpxio/stmsboot_util binary to rewrite the /etc/vfstab, updates the dump configuration and any SVM metadevice (/dev/md) device mappings, and then (in the old form) reboots the system.

What has changed

The new design makes use of a private cache of device data (stored using an nvlist) gathered from libdevinfo(3LIB) functions, and obviates the requirement for a second reboot since the vfstab rewriting function is reliable - we use the kernel's concept of what devices it has attached so we're always consistent. In addition, the new design provides a significant improvement in execution time when listing device mappings: we don't need to trawl through device links on disk but instead use libdevinfo functions and our private cache to provide the required information.

The data that we store in the cache for each device attached to an MPxIO-capable controller is

  • its devid (eg, id1,sd@n5000cca00510a7cc/aS_________________________________________3QF0EAFP/a

  • its physical path (eg, /pci@0,0/pci10de,5c@9/pci108e,4131@1/sd@0,0)

  • its devlink path (eg, /dev/dsk/c2t0d0, which becomes c2t0d0)

  • its MPxIO-enabled devlink path (eg, /dev/rdsk/c3t500000E011637CF0d0,
    which becomes c3t500000E011637CF0d0)

  • whether MPxIO is enabled for the device in the running system (as a boolean_t B_TRUE or B_FALSE)

These are stored as nvlist properties:

#define NVL_DEVID "nvl-devid"
#define NVL_PATH "nvl-path"
#define NVL_PHYSPATH "nvl-physpath"
#define NVL_MPXPATH "nvl-mpxiopath"
#define NVL_MPXEN "nvl-mpxioenabled"

When we've found an MPxIO-capable device, we check whether it exists in our cached version, and if not, we create an nvlist containing the above properties and keyed off the device's devid. This nvlist is added to the global nvlist. In order to speed operations later, we also add some inverse mappings to the global nvlist:

devfspath -> devid
current devlink path -> devid
current MPxIO-enabled path -> devid
device physical path -> devid

This allows us to search for any of those paths and get the appropriate devid back, the nvlist of which we can then query for the desired properties.

When the mpxio-upgrade service is invoked, we need to determine the mapping for the root device in the currently running system and mount that device as read-write in order to continue with the boot process. We do this by reading the entry for root in /etc/vfstab and finding the physical path of that device in
the running system. We mount /devices/$physicalpath as read-write, then re-invoke stmsboot_util to find the devlink (/dev/dsk...) path for root, which we then remount. This two-remount option is required because the devlink facility is not available to us at this early stage of the boot process devfsadm is not running yet) - until we can determine what the root device is and mount it as read-write.

Once root and /usr have been remounted, we can then invoke stmsboot_util to re-write the vfstab. This is a fairly simple process of scanning through each line of the file and finding those which start with /dev/dsk, determining their mapping in the current system, and re-writing that line. As a safeguard, the new version of the vfstab is written to /etc/mpxio, and we let the mpxio-upgrade script take care of copying that file to /etc/vfstab. Once the vfstab has been updated, we run dumpadm, and if necessary, metadevadm. Finally, we re-generate the system's boot archive - which in fact is the longest single operation of all!

After this, we can disable the svc:/system/device/mpxio-upgrade:default service and exit.

When the mpxio-upgrade script exits, the svc:/system/filesystem/usr:default service takes over and the boot process completes normally - with the new device mappings already active and working. No second reboot required!

I'm not going to claim that the new form of stmsboot(1m) is a beautiful thing, but I do believe that the architecture and implementation that it has now are much more solid and should be easier to extend in the future if required.

Update (17 October 2008, 07:08 Australia/Brisbane): Jason's comment reminded me that I should have mentioned - I pushed these changes into build snv_99 and you can see them in the changelog.

See also links:

Solaris Express SAN Configuration and Multipathing Guide

Solaris ZFS Administration Guide

Linker and Libraries Guide


devfsadm(1m) stmsboot(1m) zfs(1m) zpool(1m)

libdevinfo(3LIB) libnvpair(3LIB) libdevid(3LIB)

fp(7d) mpt(7d) scsi_vhci(7d)



I work at Oracle in the Solaris group. The opinions expressed here are entirely my own, and neither Oracle nor any other party necessarily agrees with them.


« April 2014