Wednesday Mar 30, 2011

Solaris Online Forum

I'll be participating in an online forum on April 14th, where we'll be talking about Solaris 11. If you're interested in listening in, there's more info at the registration site.

Tuesday Oct 13, 2009

testing ON changes with OpenSolaris installers

Installers are funny things. They boot a reduced version of the OS in order to install it, usually in some bizarre context like netboot, or with a live CD. They have all sorts of restrictions -- not being able to write to certain parts of the filesystem, or needing to be bootstrapped from tftp or other ancient crufty protocols. Furthermore, the OS bootstraps itself with less information it has about the system's hardware and configuration than it will have in normal operation -- it needs to put together enough information on the fly and prepare the system for normal operation. But, fundamentally, the initial installer for most OSes boots essentially the same version of the OS it's about to install.

For a developer in any core part of the system that the installer uses, that means certain types of changes must be built and tested in install context. smf(5) is certainly one of those subsystems where a subtle change may have unintended consequences on the gymnastics the OS is doing in install context. Testing is key. But assembling an install image to test for Solaris 10 and earlier used to be a horribly complicated and arcane process. And most changes don't tend to require testing in install context, so people wouldn't do 'just in case' testing. They'd only test install if they really really needed to.

Recently, I needed to figure out how to do testing of nightly ON bits in install context. I started Saturday with my ON nightly repository in hand, memories of the Solaris 10 procedure fresh in my mind, and a sense of dread. But this isn't Solaris 10 anymore, and the Install team has done a great job of changing a painful procedure into something downright delightful. I decided I wanted to test a live CD, so I...

$ pfexec pkg install distribution-constructor
$ cp /usr/share/distro_const/slim_cd/slim_cd_x86.xml ~
$ pfexec zfs create rpool/dc

I modified ~/slim_cd_x86.xml to point at my ON repository first, and then find any packages it couldn't get there from a repository containing the previous build of OpenSolaris. I also didn't install the entire package, as I'm using ON development bits (don't do this on a supported system). And then ran

# /usr/bin/distro_const build ~lianep/slim_cd_x86.xml

About an hour later, I had a LiveCD image and a USB image, ready to test. So much for my sense of dread. One quick note: Distribution Constructor currently requires that you first image-update the system where you're running DC to the same install as you're attempting to build. The DC team knows it's an issue, and the workaround is quite straightforward as it's just "pkg image-update" to your repositories.

And that's pretty much it. The diffs for the DC manifest look like:

--- /usr/share/distro_const/slim_cd/slim_cd_x86.xml  Fri Oct  9 16:29:26 2009
+++ slim_cd_x86.xml  Sat Oct 10 16:53:39 2009
@@ -57,8 +57,8 @@
 		-->
 		<pkg_repo_default_authority>
 			<main
-				url="http://pkg.opensolaris.org/release"
-				authname="opensolaris.org"/>
+				url="http://ipkg.sfbay/on-nightly"
+				authname="on-nightly"/>
 			<!--
 			     If you want to use one or more  mirrors that are
 			     setup for the authority, specify the urls here.
@@ -75,15 +75,12 @@
 		     If you want to use one or more  mirrors that are
 		     setup for the authority, specify the urls here.
 		-->
-		<!--
-		     Uncomment before using.
 		<pkg_repo_addl_authority>
 			<main
-				url=""
-				authname=""/>
+				url="http://ipkg.sfbay/dev"
+				authname="opensolaris.org"/>
 			<mirror url="" />
 		</pkg_repo_addl_authority>
-		-->
[...]
                <packages>
-                       <pkg name="entire"/>

For cleanliness, I also updated the post_install_repo_default_authority and post_install_repo_addl_authority to point at the same URLs.

The install team does recommend running DC on a system that's already been updated to that package level. e.g. to build a 124 LiveCD, they suggest first image-updating the system to 124, then running distro_const(1M). That shouldn't always be necessary, but it's also easy to do given that DC requires a repository as input that you can just image-update from.

It's pretty nice that the install team has fundamentally converted a task that was challenging for ON developers before into something that's really quite easy. If you're interested in more, there's plenty, including documentation at the Caiman Project over on opensolaris.org.

Sunday Oct 11, 2009

building an ON IPS repository

I've been working with the gracious help of Mark on making the ON consolidation create an pkg(5) repository as part of the build process. If you build the ON consolidation from source ever, this is probably interesting to you.

Our changes are destined to be integrated into the main ON gate, which should happen in November 2009 sometime (though that's subject to change and doesn't constitute a promise). We've tried to make it easy for folks to build their own ON IPS repositories for testing in advance of integration of our changes. You can access the latest instructions for building your own ON repository in the README which lives in our development mercurial repository.

If you do want to try this out, I strongly recommend subscribing to on-ips-dev@opensolaris.org, as that's where we're answering questions, giving heads-ups about important changes, and having development conversations. We've got some sizable changes coming over the next few weeks, including a protocmp which works on IPS manifests.

I'm really enjoying using the same tools as we expect our customers to use. It's now pkg image-update to update my ON development bits rather than the development-only tool bfu. pkg image-update is at least as fast, if not faster than bfu, especially over a slower link. That's because only the bits which have actually changed between versions are downloaded and updated by pkg(5). Nice. And nice that our normal upgrade experience is now as blindingly fast as ON developers have come to expect.

Tuesday Jul 31, 2007

playa programming -- OpenSolaris at Burning Man

For my eighth year at Burning Man, the two-by-four of inspiration whacked me upside the head and stole away my summer free time. On evenings and weekends I've been working feverishly on an art installation called The Belligerent Blooms. With lots of help and indulgence from Jan, Bart, and encouragement from the rest of my campmates, the project is starting to take shape. There's been plenty of programming, hacking, drilling, sawing, soldering, and even a little bit of welding. The whole project is centered around an OpenSolaris-based system driving audio for this embedded application. There's no electricity out at Burning Man, and generators are noisy -- so with a bit of begging, we've managed to borrow 2 solar panels to charge the battery which runs the whole installation. For those playing along at home, that means we'll be running Sun on the Sun.

This post will (hopefully) be the first in a series about how the software and hardware of this art project was put together. The series will likely come in fits and starts as time before the event is precious -- we depart on August 26, so blogs may have to wait until we return. But, before I begin the series, there's some more begging to do...

The Belligerent Blooms need your audio contributions!

The Belligerent Blooms are a garden of cranky electro-mechanical flowers, accosting passers-by with their deeply rooted beliefs. We need your unique voices to contribute to the cacophony.

Consider what you'd say if you were a belligerent bloom -- share your experiences as part of nature, opinions about humanity in general or Burning Man participants in particular, or any other flowery invective you have to offer. Try to keep the comments short and pithy. Get a giggle, challenge world views, add your voice! Above all, be belligerent!

Contribute your flowery invective by sending mail to belligerent.blooms@gmail.com and including .wav (preferred, and easily generated by audiorecord(1)) or other audio files. Putting your quips in separate files will greatly please the gardeners. Contributions made by August 19, 2007 will be included.

If you, or someone you know attend Burning Man, the Belligerent Blooms make their home at the 'Blacklight Aquarium' theme camp. Habitat and 7:30; just look for the big spinning fish sign in the sky.

Technorati Tags: , , , and .

Thursday Jul 26, 2007

zone out and speed up your development cycles

The other day, as an aside in a conversation about how often developers use certain OS features, I was asked how often I use Solaris Zones. At least weekly, and daily if I'm lucky enough to be spending my time on code.

Surprised? Userland developers on Solaris shouldn't be.

I spend a great deal of time modifying libraries and daemons started really early in the Solaris boot process. While SMF tries to dump you at an sulogin prompt if you've introduced a bug, it's still a bit of a pain to recover from some nasty failure or hang you've coded into init(1M), svc.startd(1M), or libscf(3LIB). Zones make the deploy and reboot cycle go really really fast, and recovery from late-night programming errors is a breeze!

Here's what I do:

  • Create a whole root zone. Takes up more disk space, but allows me to replace any system binary:

            # zonecfg -z test
            test: No such zone configured
            Use 'create' to begin configuring a new zone.
            zonecfg:test> create -b
            zonecfg:test> set zonepath=/test-zone
            zonecfg:test> commit
            zonecfg:test> exit
            # zoneadm -z test install
            # zoneadm -z test boot
            # zlogin -C test
               ... answer sysid questions, and log in 
    
  • Make sure the bits I'm compiling are relatively close to the bits installed on my desktop. Live Upgrade and I are close friends so that I can keep my desktop as up-to-date as my workspaces.

  • Create a script to dump my modified bits into the zone root. Something like this fragment:

            gate="/net/coupe/builds/lianep/templates/usr/src"
            zone="/test-zone/root"
            if [ ! -d $zone ]; then
                echo "no zone root here: $zone"
                exit 1
            fi
            cp  ${gate}/lib/libscf/i386/libscf.so.1 ${zone}/lib
            cp  ${gate}/lib/libscf/amd64/libscf.so.1 ${zone}/lib/amd64
    
  • Compile, run script, test, debug, fix, repeat.

After my code is all basically working, then I move on to testing on the bare metal. But, the fast reboot times of zones and the easy ability to replace a broken library with a library broken in a new and different way is invaluable to making very rapid progress. I've rebooted my zone at least 15 times today. Compiling the library takes longer than the zone deployment and reboot! Every few months I mismatch libraries and commands from different workspaces and foul up my zone badly enough that it needs to be re-installed. But, zoneadm -z test uninstall; zoneadm -z test install provides a convenient excuse to go get coffee and then I'm back in business.

Technorati Tags: , , , and .

About

Liane Praza

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today