Sunday May 04, 2008

CommunityOne: here and there

Sunday I spent at Moscone, teaching laptops and projectors to get along. Saturday, Nathaniel, Benjamin, and I dropped in for lunch at the Developer Summit. I managed to talk with a few people before the boys found large whacking sticks, and then it seemed best to drive to Pescadero for some beach time.

I'll be busy for the morning of CommunityOne. For Rich's keynote, I'll be running some of the less violent demos. Almost immediately after that, lead modernizer David Comay and I will be going into more detail in our session

S297399 Getting Started with OpenSolaris™; New Features & Building OpenSolaris™ Packages; 11:00 a.m., Moscone South/Esplanade 300.
We've got some additional demonstrations, including a worked example of package publication using some pre-release tools, which could be exciting.

I'm hoping to have some time for questions during the session but, if not, I'll be circulating during the afternoon and happy to talk to people about 2008.05, image packaging, or whatever. And, of course, there will be time to talk at the party after the day's sessions. In any case, I should be easy to spot: I have a new tie.

I'm told you can still register on-site—it's not too late.

[ T: ]

2008.05: More ways to get it

As I did for the preview releases, I'll collect links to mirror sites here. These will also get links on the various download pages out there.

Bart and I just finished updating the package repository with the new packages we'd received the past few weeks, and that means 2008.05 is out the door. (Thanks to everybody who tried the release candidate, filed a bug, shuffled a package, wrote or proofread a document, or just spent energy anticipating the bits.) You can get the ISO image, suitable for burning to a 700 MiB CD or immediate use in virtual environment, directly from the following locations:

Reading the logs, and talking with some of our mirror sites, we know we all served out a lot of downloads for the previews; if you're interested in being a mirror, please let me know. (2008.05 remains completely redistributable.) We're using a bigger download complex this time, but every mirror helps.

If you already downloaded and installed Preview 2, it's more complete, easier, and probably faster to update directly using image packaging: see the update guidelines. These instructions involve a small script to safely update a clone of your installed system, and then switch to that on a system reboot. (If you were running Preview 1, you should update that to Preview 2, and then go on to 2008.05.)

Update 1: My thanks to Tobias Lundquist, who's once again mirrored via FTP and HTTP (Internet 2) in Sweden.

Update 2: My thanks to Luca, who's put up an HTTP mirror in Romania.

Update 3: Simon Valiquette has put up an HTTP mirror (Internet 2) at the Université de Sherbrooke in Québec, Canada. Merci, Simon.

Update 4: Bart Muijzer has arranged for mirrors by the Netherlands Unix Users Group (NLUUG) and Thanks, Bart.

These links are for the gzipped CD image, which contains the 12 "primary languages". It installs quite a bit faster, particularly on systems with slower CPUs. There is also an LZMA-compressed image, which has localization support for 42 languages, including those primary ones. It's available from,,,, and as a torrent. (Consult the language lists for specifics.)

[ T: ]

Thursday May 01, 2008

Live chat on #os200805

We're talking about OpenSolaris 2008.05 over on IRC, using the #os200805, from now until 2 p.m. Pacific Time. If you haven't got an IRC client, you can use freenode's Java applet, which worked great (until Danek kickbanned me). Enter a nickname, and then ask some questions.

Apparently the load from 60 000 users blew out the chat system, so we moved to IRC.

Thursday Apr 17, 2008

OpenSolaris: Bug dependencies and release management

Right now, if you're subscribed to any of the Installation and Packaging community group or project lists, you'll see a lot of commit notifications as the various teams attempt to fix various bugs noted since the second Developer Preview release. We've been using the trial Bugzilla instance—which is becoming the default defect tracker for it appears—and trying out various features.

For tracking release completeness, we're using "blocker bugs" or "tracker bugs", which are synthetic bug entries that we mark various important bugs ("stoppers") as blocking. That means that we end up creating a little dependency graph that shows what unfixed bugs are stopping us from reaching some initial set of release criteria. We have two tracker bugs

that we're monitoring to make sure we've got a handle on things.

Bugzilla has two nice summaries for showing this information, in addition to the default bug status page. I'll use 571 as the example tracker bug, since we've made 1190 block it—which leads to a more nteresting graph. The tree view is a useful and succinct representation, where indentation shows dependencies. The graph view is a bit unwieldy for this bug, but might be useful if the tree view became too long.

A useful technique if you're trying to bring a release together.

[ T: ]

Thursday Feb 21, 2008

pkg(5): Reverse proxying your depot with Apache HTTPD

As part of the changes to get Developer Preview 2 ready, we decided to rejigger the HTTP handling on so that we could have more options as more people attempt to use the early versions of image packaging. Previously, we ran pkg.depotd directly on port 80, in its read-only mode; now we use Apache HTTPD to listen on port 80, and use mod_proxy to proxy those incoming requests to a pkg.depotd instance listening on a separate port. With a couple of different approaches

Proxying and rewriting is one of those endlessly fun activities that somehow actually ends up being productive. Last time, proxying fun led Steve and I to fiddling around such that we ended up with proxy and rewrite patterns to enable the country portals for

If you want to share the top-level component of your URL space, you'll need to watch pkg(5) developments, as you have to map the list of operations one-by-one—and I know there are some new operations forthcoming. That would involve adding something like the following to a VirtualHost directive in your Apache configuration.

ProxyRequests On

Redirect /index.html

ProxyPass /abandon
ProxyPass /add
ProxyPass /catalog
ProxyPass /close
ProxyPass /feed
ProxyPass /file
ProxyPass /filelist
ProxyPass /manifest
ProxyPass /open
ProxyPass /search
ProxyPass /versions

ProxyPass /css
ProxyPass /logo
ProxyPass /icon

ProxyPass /status

Configuring your server in this fashion allows you to mix an image packaging server in with your other site content. You can easily deliver static content alongside your depot, for example.

If you don't mind pushing your package repository down one level in your URL space, then the above simplifies to

ProxyRequests On

ProxyPass /pkg/

(which should be a hint on how to create a repository farm under a single URL). To use the latter, you would use pkg(1)'s image-create subcommand

$ pkg image-create -F -a /path/to/image

to connect your image to your reverse-proxied packaging depot.

In the two examples above, you should of course replace machine names like and port numbers like 10000 with values appropriate to your own installation.

Happy proxied package serving!

Feel free to share your alternative configurations or approaches with other HTTP servers here, or on

[ T: ]

Tuesday Feb 12, 2008

Indiana: More ways to get to Preview 2

As I did for Preview 1, I'll collect links to mirror sites here, as well as on the project page.

So we just released the second Indiana preview ISO, which you can get from the following locations:

We had a lot of downloads for Preview 1; if you're interested in being a mirror, please let me know. (Indiana remains completely redistributable.) We're using a bigger download complex this time, but every mirror helps.

If you already downloaded and installed Preview 1, it's more complete, easier, and probably faster to update directly using image packaging: see the update guidelines on These instructions use ZFS and pkg(5) to safely update a clone of your installed system, and then switch to that on a system reboot.

Update 1: My thanks to Tobias Lundquist, who's once again mirrored via FTP (Internet 2) in Sweden.

Update 2: My thanks to trisk, who's once again put up an HTTP mirror (Internet 2) on the East Coast of the USA.

[ T: ]

Wednesday Jan 23, 2008

OpenSolaris: Finding work(ers)

A relatively new forum on is opensolaris-jobs, a community-moderated list where OpenSolaris-related positions can be posted and which should give access to the people most passionate, most expert, and most interested in making OpenSolaris work their job. In the past few weeks, there's been two postings you might find interesting:

Have a look, or—if you're hiring for OpenSolaris knowledge—get in touch with great candidates.

[ T: ]

Monday Nov 05, 2007

pkg(5): Fueling the next steps

Most Fridays, I spend the two hours before lunch assisting at our co-operative daycare. The hour and a half before are pretty good thinking hours, most of which lately have been spent on packaging. As the year passes, and we move into fall here in Northern California, the thinking's been best assisted by having a brief, and warming, snack.

Still life with packaging

The notes being written are about some of the points raised about the use of hashing, but then I buckled down and wrote an outline for our ARC inception materials, which will probably take more than one or two Friday sessions.

Location: Café Borrone, Menlo Park, California.

[ T: ]

Friday Nov 02, 2007

Indiana: More ways to get it

So there are a few ways to get a copy of the Indiana preview ISO:

Both our mirrors are in Northern California, so—since it's a completely redistributable set of software—if you're interested in being a mirror (even in California), please let me know. If you're a torrent fan, come join me and seed.

Update 1: My thanks to trisk, who's put up a fast mirror (Internet 2) on the East Coast of the USA.

Update 2: My thanks to Tobias Lundquist, who's put up an FTP mirror (Internet 2) in Sweden.

Icon Design by IconBuffet (from "Durango Office" and "Durango Research").

[ T: ]

Wednesday Oct 31, 2007

Indiana: VESA if you need it

Over the past week, as we kept reassembling the distro constructor, image packaging, and slim install, we tested installs on a bunch of laptops. This manual operation let me rehearse how to get things working if the Preview LiveCD doesn't have a graphics driver that will work. So: here's the VESA workaround.

  1. Boot text mode. One of the non-default lines in the GRUB menu should say "text"—pick that one.
  2. Login as jack. User is "jack", password is "jack".
  3. Generate a representative xorg.conf. Become root and have Xorg generate an initial configuration. The root password is "opensolaris". (Enjoy that `root`'s default shell is 'bash' for a bit.)
    $ su
    # /usr/X11/bin/Xorg -configure
    This procedure should create /jack/
  4. Change the driver. As root, you'll continue by editing that file in vi(1). Search for the Device section, and modify the Driver line to
        Driver  "vesa"
  5. Make it go. Stay root and hand-launch GNOME.
    # /usr/X11/bin/xinit /usr/bin/dbus-launch gnome-session -- \\
            /usr/X11/bin/Xorg -config /jack/ :0
  6. Launch the installer. Bring up a terminal, and invoke the installer, by typing install-lan &.

If your install fails, please file a bug report at If it succeeds, but graphics doesn't work, you can follow the same steps, but you'll have to edit the GRUB entry. (Roughly, type 'e', add "-s" to the end of the line with kernel/unix, and log in with the new root password you set during install. I haven't tested this portion—let me know if it fails.)

Last night, I had to do this for my trusty VAIO T370P, but tonight it's fine:


Please let us know how it goes.

[ T: ]

Saturday Oct 13, 2007

pkg(5): Talking in the redwoods, talking on the beach, ...

Danek and I are at the 2007 OpenSolaris Developer Summit at UC Santa Cruz, talking about and, I hope, later demonstrating the image packaging prototype so far. I gave a brief overview of the project's goals and status, and mentioning some of the unmentionables we've encountered—some particularly undisciplined configuration files, some apparently important but encumbered drivers, and so on.

View the slides

[ T: ]

Thursday Sep 27, 2007

pkg(5): project opens, development continues

I asked my teammates to take a brief break from prototyping, and we jumped from a collection of systems inside Sun to the clean project hosting at

As the migration registers, you should be able to access

If you're interested, please come check it out.

[ T: ]

Friday Sep 07, 2007

pkg(5): a no scripting zone

In my previous two posts, we examined two packaging system options—installer-specific knowledge and integrated build system—that I believe present costs that exceed their benefits. Here, we will again examine a design choice from a negative perspective: package-associated scripting.

System V packaging is rich with scripting hooks; scripts named checkinstall, preinstall, postinstall, preremove, request, and the class action scripts. Each of these scripts can do anything they like. Scripting, even in a relatively primitive shell, is an open-ended program—opaque to the invoking framework. It's difficult to catch an incorrect script prior to package publication time, which blocks our intent to prevent propagation of bad package versions. With a more limited set of actions—potentially with that limit enforced or marked—a class of incompletely known resource handling mechanisms can be kept off the most conservative systems.

One goal we have is to preserve or improve the hands-off behaviour associated with package operations. Legacy packaging allows hands-off by imposing a series of tasks on the deploying administrator. The pkgask(1M) tool can enable the deployer to develop a response to the request script; coming up with an appropriate admin(4) can restrict the framework's built-in interactive queries. (Interaction with signed packages also requires the deployer to modify their pkgadd invocation.) Removing the scripting degree of freedom means that obstacles to hands-off behaviour come solely from an interactive installer or from interactive services acting during system startup.

There's some amusingly egregious violations of the hands-off principle across the space of known packages. Less fun is that these set a bad example for later package developers.

A particularly error-prone aspect of the scripting interface in packaging comes from the variety of contexts the package developer must understand (and test within). It is legitimate to install packages on live systems, in alternate filesystem hierarchies of the same or different architecture, and in whole-root and filesystem inheriting zones; in fact, you have multiple choices about how your package should install in a zone.

We can expect the proliferation of virtualized systems, via the various mechanisms like LDOMs and xVM, to keep all of these contexts relevant as degrees of sharing make virtualization even more appealing. Making sure that the package system operates safely in these shared contexts is critical—another of our goals.

Returning to the zones case, the example pseudo-script in pkginfo(4)—a series of nested shell if ; then blocks to navigate some of these contexts—is helpful, but misleading. There is much more variable state a package developer needs to consider to reach correctness. In fact, if you aren't required to rediscover or reinvent a set of resource-handling cases for each components your package delivers, it becomes substantially simpler to make the package and return to improving the software it contains. Reducing the set of steps reduces developer burdens associated with packaging.

Two particular resources stand out: device drivers and smf(5) services. Although some limited amount of awareness—or at least easily duplicated code—makes these resources somewhat well-behaved during package operations, there are still problems that scripting presents: the addition of new contexts, the provision of multiple genealogies of copied code, and the failure to discover an associated best practice for any particular kind of resource.

There are other resources, of course; as a start, you could duplicate our survey of the ON postinstall and class action scripts.

I believe the key counterargument supporting scripting is that the set of configuration patterns on Unix-like systems is large, and that the easiest means of upgrading each of these potential patterns is to allow a complete programming environment to the package developer. Probably true, but if we look at service and application configuration with respect to when a correct configuration state is required, the update step appears to separate into three classes:

  1. Correct at system startup, no runtime context needed. These are the configuration settings that the various low-level boot components, the kernel, and the drivers need to bring the system to its running state. This class of configuration is generally limited to a specific set of resources, potentially established by a packaging system via corresponding resource-handling actions—or by an installer.

  2. Correct at system startup, requiring runtime context. These are settings where the manipulating agent might be influenced by policy or require some form of interprocess communication to effect configuration changes. smf(5) is an example of the latter, and handles its configuration evolution via the manifest-import service. Manipulation of the various local name service tables, like passwd or the RBAC configuration is another example, since data about potential principals must be correct for a group of affected services. Since such configuration can be required on the system as a result of package operations, these resources must also be handled via packaging, or require the use of an appropriate installer.

  3. Correct prior to service startup. Most service and application configuration falls into this class. It's not necessary, for instance, to bring a web server's configuration up to date if the service has no enabled instances. There seem to be a number of avenues for handling this kind of configuration: leaving it to the service or application, providing assistance via a configuration mechanism, or giving a hook where such updates can be made as needed. But the packaging system needn't provide this hook—there are a number of possible facilities, of varying suitability.

I should point out that David is making the smf(5) configuration update scenarios much more capable and precise with the Enhanced Profiles project. So, at least, a "configuration mechanism with assistance" is likely to be present soon.

Since the first and second classes and how their configuration manipulations vary in the various operating contexts are generally known, elimination of the third class makes precise, no-scripting packages a viable design choice.

That's a long series of arguments in favour of a scripting-free package system. It would be reasonable to ask: "can you actually do it?" So, as a check on our prototype, we used the branded zone capability to let us create a pkg(5)-based whole root zone. Here's a transcript

# zonecfg -z pkg_test
pkg_test: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:pkg_test> create -t SUNWipkg
zonecfg:pkg_test> set zonepath=/export/pkg_test
zonecfg:pkg_test> commit
zonecfg:pkg_test> \^D
# zoneadm -z pkg_test install
Preparing image
Retrieving catalog
Installing SUNWcs SUNWesu SUNWadmr SUNWts SUNWipkg
Setting up SMF profile links
Copying SMF seed repository
Done (115s)

There's dependency following, but no constraint handling; there's no filtering or snapshotting, but also none of the obvious performance optimizations has been implemented (for our 211MB resultant image). But the main point is: it works—installs, boots, upgrades, and still boots—with no scripting. Time for a project proposal.

[ T: ]

Friday Aug 24, 2007

pkg(5): Leaving the build system "out"

I've been busy the past weeks with school transitions and with getting the community defect tracking system requirements into a shape where we can start evaluating candidates.

Identifying the boundaries of a system during the design phase affects feasibility critically, perhaps more than any other choice. This choice reduces to "know what you're trying to create". As with my previous post, I'm going to describe something we're not doing—and explain why. I envision one more negative post, and then we'll get onto more positive expressions.

So: this packaging system does not contain a build or compilation component in its core architecture. There are pragmatic reasons for this choice, as well as technical ones.

One way to look at a packaging system is, similar to how we saw it as the connecting layer between the installer and lower-level OS services, as a way to collect and organize the set of components, binary and otherwise, into an always "bootable" flow of change. That is, various groups of people are, using their systems, emitting binaries, documents, images, and so on, that other groups of people combine into releases (or atoms of a release, like an updated package). The packaging system has to assist the latter group in that combination, by providing visibility into the completeness of the outputs of the former groups. It's less clear that the latter group needs to have the ability to construct the objects, as long as they can assess that the environment the objects need to execute can be realized.

Just as an aside, we're treating as critical design input exactly who touches and how they touch a software component as it proceeds through development, localization, release engineering, sustaining, and so forth. These touches might be reduced as the development becomes more open, but at present, they're a useful constraint on any tendency to make the system overly rigid (or overly monolithic).

The OpenSolaris consolidations—and related open source outputs like OpenJDK or OpenOffice, among others—don't presently share a common build system. In fact, one of the real impacts of Sun's progression to an open software development culture is that we're moving from being the originator or primary maintainer of 90% of the software we deliver, to a much more modest percentage—let's say 30 – 50%. Forcing a unified build system upon all of these disparate products is asserting the need for a long series of difficult conversations over many months.

Instead, let's defer those discussions, and see if we can get the same benefits while only managing the end outputs of each of the participating (or aggregated) publishers. (A byproduct is that injecting received binaries into the system is the common case, rather than a strange or special workaround.) As we noted in the earlier post, we have goals about safety, developer burden, and stopping "bad stuff" as early as possible.

Static analysis can get most of the dependency cases correct. Binaries, whether they are user applications, libraries, or kernel modules, contain a significant amount of dependency information. It turns out that many scripting languages can be roughly interpreted to determine their module requirements. Similarly, Java class files and JAR files—and even smf(5) manifests—contain a great deal of information that lets us determine the self-consistency of a system.

Of course, a program can evade these dependencies: beyond use of dlopen(3C), it can implement its own linking or overlay mechanism, or simply be implemented in a language unknown to the packaging system. The point is we can drive out most of the inconsistencies that a purely manual dependency statement allows. In fact, we can warn a developer about the incompleteness of their dependency statements, potentially correcting them: adjusting version requests, inserting omissions, even asking about possibly superfluous dependency correctness.

That said, it might be that an ideal build system is lurking out there to be layered atop this system; we'll leave room for expressions, like stating a build dependency on a certain tool, like lex(1) and yacc(1), say, so that a build system (or systems, if folks can't close those difficult conversations I mentioned) can benefit from the metadata discovered about each component. (If you're interested in constructing such a system, we thought a little about requirements for the SFW consolidation.)

[ T: ]

Monday Aug 06, 2007

pkg(5): No more installer magic

I thought I would continue probing some of the problems that present themselves to any packaging system that might follow the System V packaging. For the next few topics, I'll phrase the problem in terms of an outcome I believe we want to avoid. Here we discuss aspects of eliminating special metadata from installers.

If you're familiar with the packaging and installation components involved with Solaris Express, Developer Edition, or any of the OpenSolaris distributions that Sun produces, then it won't surprise you that there's a large amount of upgrade specific knowledge in the installation layer. For instance, to upgrade a package installed with Solaris 10 to its currently delivered version in a recent build, there's the set of files associated with the package—and then there's the package history, which collects additional information, like files no longer delivered, or the responsibility of another package. The presence of this file, and the absence of a true package upgrade operation in System V packaging, mean that any kind of upgrade requires some form of installer: the operating system installer for upgrading the entire installation; an application-specific installer for upgrading a group of related application packages.

There are a number of other metadata files trapped in the installation layer, but the most important are the metacluster files, which group the package clusters into large installation profiles, and the package clusters themselves, which group sets of packages along feature boundaries, approximately. It seems evident that these groupings are merely another kind of dependency, much like how a System V package can, naïvely state a dependency on any other System V package.

The System V dependencies also show that another critical piece of metadata—the versioning vector, or "arrow of time"—is encoded in the installer.

All of this information, if we are to allow packages to upgrade from one version to another in some linear fashion, needs to be pulled out of the installers and moved into some aspect of the packaging system. This change in responsibility means that the role of the installer becomes more precise: it must prepare a location for software installation, optionally lay down some initial, and possibly stale image, and collect any required configuration information. Subsequent upgrade operations are driven primarily by the packaging system, which can utilize the version history and dependencies in a manner at least equivalent to what the historical metadata allows.

The other reason that we wish to push that historical metadata into the packaging system is so that it becomes accessible to a new class of application: the distro construction toolkit, which needs the dependency and versioning information to simplify the construction of self-consistent installable images. That leads us to an architectural diagram like

pkg Layering

where we've suggested some internal structure to an updated packaging system. This separation shouldn't be that surprising: in the current system, lumake(1M) and most of LiveUpgrade as well as zoneadm(1M) for Zones are both performing image operations on either side of any packaging operations they might invoke. Designing a packaging system that makes the constructions of distributions and their installers substantially simpler will require such an API layer.

There are some open questions that come to mind:

  • How much variation is there in the initial preparation an installer offers for the image location? Is space management a policy owned exclusively by the installer, or should the image operations layer have the ability to influence that allocation?
  • Is there a marshallable image description, beyond the package dependencies? Is there a marshallable image format, or are we restricted to end-use formats, like ISO images or archived zones?
  • What OS services should we consider modifying, beyond taking advantage of ZFS's current capabilities?

If you're interested in these questions, or the potential architecture of a packaging system, we'll be discussing these topics further in the Install community group.

[ T: ]




« March 2015
External blogs