Thursday Nov 09, 2006

Tip: Mercurial atop ssh

On, we host Mercurial and Subversion via their ability to tunnel through an SSH connection. A slight gotcha with Mercurial is that it turns off compression when tunnelled through SSH, as described in hg(1):

$ man hg
    - Mercurial doesn't use its own compression via SSH; the right thing
      to do is to configure it in your ~/.ssh/ssh_config, e.g.:
        Host \*
          Compression off
        Host \*
          Compression on
      Alternatively specify "ssh -C" as your ssh command in your hgrc or
      with the --ssh command line option.

For pulls of a large repository, such as ON, compression changes an approximately 51 minute operation to a 17 minute operation. So you'll want something like

$ cat ~/.ssh/config
    Compression yes
in your SSH configuration.

[ T: ]

Tuesday Nov 07, 2006

OpenSolaris: Documentation tutorial buffet

Each of the community groups that make up OpenSolaris is experimenting with different styles of introductory materials (or procedures), so that potential contributors don't get blocked on the details and can focus on their real intention—contributions. The Documentation community group has been energetically pursuing a complete offering: they have a large document tree, they're using a "scratchpad" on the Genunix wiki, and they have at least one mailing list. They even have a Flash animated tutorial, illustrating how to use a DocBook XML toolchain. Slick.

[ T: ]

Monday Nov 06, 2006

OpenSolaris Day, Buenos Aires

I've been back from Buenos Aires for a week now, and just wanted to comment on my trip, and show the few pictures I managed to take. Like its sister events, Buenos Aires TechDays was a densely packed three day conference, with multiple tracks available on OpenSolaris, Java, NetBeans, SunStudio, Glassfish, and other technologies and platforms. I updated the "Building and Deploying OpenSolaris" talk, to cover the convenient Starter Kit, which has the tools and source (and OS images) and to mention the ONNV Mercurial repository on

The talk was simultaneously translated into Spanish, using some slick IR emitter arrays and receivers. Russ, I think, was much more effective, as he gave his presentations in Spanish.

The event was held at the Hilton Buenos Aires, which is located in the recently redeveloped Puerto Madero area. When Teresa and I walked around on the first day, we saw activity in the harbour, and paused to watch (along with a few hundred other pedestrians), as this tugboat


helped this coast guard ship


turn out of the relatively narrow channel.

Back at TechDays, we enjoyed the chance to have further technical discussions, with many resolutions


to study new problems, reengage with old ones, or just return to Argentina.

The Inquirer's Fernando Cassia has more on the NetBeans portion of the event.

In Argentina, but missed TechDays? There's an OpenSolaris user group—check it out.

[ T: ]

Tuesday Sep 19, 2006

OpenSolaris: Gary's in Brussels

Gary Pennington is attending EuroOSCON, being held this week in Brussels. He's reporting on the various sessions. Worth a read, particularly if you've not been to an open source conference in a while...

[ T: ]

Thursday Sep 07, 2006

OpenSolaris Day, Seattle

It's nice to be in Seattle. I spent a couple of weeks on the UW campus a dozen or so years ago; on customer visits, we're ususally in a suburb. Enjoying the chance to wander in a different downtown, I decided to walk from the hotel to the TechDays site, at Marion Oliver McCaw Hall, in Seattle Center.

This let me do a few errands—like eat a quick breakfast at Specialty Café and replace the broken SD card for my camera—and walk past a well known building

space needle

and under the disappointingly-testing-only-service monorail


Has no one learned from North Haverbrook?

We had an audience split roughly between developers and administrators at the first of a global series of OpenSolaris TechDays. After an introduction by Teresa, I gave Frank's new presentation, "Building and Deploying OpenSolaris", which is an overview of what's out there and how to build some of it. Post-presentation, we got some good questions: on switching easily between the various in-progress development project variants; on various forking scenarios; )and on arriving at a shared development methodology. I think the talk actually has some room for covering one or two more topics, but I've posted the presentation in the meanwhile.

Nicolas, Glenn, and Jim gave great technical talks on Virtualization, Security, and Performance for OpenSolaris, respectively. Slides for all of these talks should be at the TechDays developer site.

We finished with an open question-and-answer session.

On Thursday's TechDays session, there is a Solaris track which should provide folks with additional information about Solaris futures—derived, of course, from OpenSolaris—as well as development techniques available atop OpenSolaris systems. I'll be around until late afternoon, when I'll take #194 back to the airport.

Thanks all those who came, plus nods to Clayton for the conversations as well as a fine restaurant recommendation, the fellow at the airport who pointed out the fast bus to downtown, and to the folks who drove 350 miles round trip from Portland for the day.

[ T: ]

Wednesday Aug 09, 2006

A not-quite-isochronous upgrade

In the midst of many pieces of work, a new machine showed up on my doorstep. It's pretty sweet, and I wanted to get it running latest Solaris Nevada bits. So I repartitioned the disk to make room for an additional boot environment (BE), updated the Live Upgrade support, upgraded (to Build 44), and got back to work.

But this week I saw something pretty strange. I was working on making some software more portable, updating some Makefiles as well as changing some of the source. On a rebuild, my gcc compilation failed with

collect2: ld terminated with signal 9 [Killed] ld: fatal: version `SUNWprivate_4.2' not found (required by file /usr/ccs/bin/ld) ld: fatal: open failed: No such file or directory ld: fatal: relocation error: file /usr/ccs/bin/ld: symbol ld32_main: referenced symbol not found

After checking with a few neighbours about any recent linker fixes, and reviewing package installation times, we finally started looking inside the binaries. The version from the install server showed Build 44:

$ mcs -p reloc/usr/ccs/bin/ld

@(#)SunOS 5.11 snv_44 October 2007

But the version installed on the machine was from the future:

$ mcs -p /usr/ccs/bin/ld

@(#)SunOS 5.11 snv_45 October 2007

What were we running on this box?

$ cat /etc/motd
Sun Microsystems Inc.   SunOS 5.11      snv_44  October 2007

I suppose if this were an Encyclopedia Brown mystery, you would have to flip to the end of the book to discover the solution to the mystery. Unfortunately, I haven't given you quite enough information--you need to know how our install servers share new images.

As you might expect, there is a large set of install servers available across Sun. Some are run by IT operations, some are in development labs—but all pull their images across at different times. It turns out that the most recent build is linked to a directory called "latest", so that one can reinstall a system every two weeks and have it running the most recently assembled version of Solaris.

Generally, the action to get a copy of the newest image and update the links happens automatically, and outside of business hours. But sometimes in a development lab, someone wants an image early, perhaps to upgrade a collection of test machines for verification purposes. And, it turns out, luupgrade(1M) works just fine across such a rename, if it's not specifically accessing the filesystem during the image changeover. So my Live Upgrade installed a swathe of packages from 44 followed by a smattering from 45.

One more Live Upgrade (using the numbered path, rather than latest) to Build 45, and it's back to porting.

This problem isn't going to be seen very often—particularly if you don't have a multi-version install server setup like we do—but it seems appropriate to document the underlying issue: Live Upgrade doesn't detect an image change during the operation. I'm not sure it should, but it's pretty clear that the underlying package operations are incomplete with respect to versioned dependencies.

[ T: ]

Tuesday Jul 25, 2006

OpenSolaris: Noted on zfs-discuss

Since the break, I’ve been working with Gary, Steve, Derek and Alan—plus Dermot and Bonnie, neither of whom blog—on the source code repository hosting for, which I expect will come out of Beta pretty soon. The hosting model we worked out showed its versatility last week, when we added Mercurial support to the test site with only a few hours of work. (We had been doing the initial development using only Subversion repositories.) There are still a few bugs to fix and a few features to add, but adding this capability will take our open development effort a sizeable step forward.

In fact, I need to put the SCM work aside soon and join in the structuring of the final review of the OpenSolaris constitution. If you’re interested, see the Governance category at

But I’m still keeping up with the mail flows. One particularly interesting alias is zfs-discuss, which is a mix of the ZFS developers, early adopters, and potential deployers. There’s been discussions on reliability, Thumper, and a few issues people have had using zfs(1M) and zpool(1M). Occasionally, something more curious pops up, like Jim Connors’s posting, where he reveals

… I have a version of Solaris 10 U2 with ZFS functionality with a disk footprint of about 60MB. Creating a miniroot based upon this image, it can be compressed to under 30MB. Currently, I load this image onto a USB keyring and boot from the USB device running the Solaris miniroot out of RAM.


[ T: ]

Wednesday Jun 14, 2006

OpenSolaris: on one year

OpenSolaris 1 Year Anniversary

Anniversaries provide a convenient excuse to be contemplative—not that I would struggle to justify contemplation. So, I'll take a moment to look both back and forward. At one year, the OpenSolaris effort has exhibited steady growth across the entire period; Patrick's community metrics have shown that the interest in participating isn't waning. The community has passionate evangelists and communicators, and their efforts in blogs, at conferences, and in various forums are unflagging. Just as valuable have been the reasoned critiques of OpenSolaris's technical gaps (and other challenges)—although there has been some concern about the conflict between advocacy and criticism.

As long as we keep this conflict vigorous and civil, and avoid dogma, it will be constructive. I know that the technical contributors have examined and adjusted for every concrete criticism they've read; these realizations I expect will emerge as more projects move to developing on directly. On the other hand, hearing these critiques allows the evangelist to amend their presentation—to discover workarounds, to make counter-recommendations, and so forth.

I've written at length about the internal transformations underway as we move from an open-within-the-institution development model to one that is open to all. It's been just as gratifying to see the changes in the community-wide conversations, as the formerly-Solaris-only jargon evolves into a wider community idiom that we all share. The dropping of the most awkward or provincial terms is no cause for sorrow; the acquisition of new and useful expressions—like the Apache-style "+1" for "I agree"—has been a pleasant surprise. (Expressions that are unambiguous in email seem to be rare and therefore valuable.)

One of the things I'm looking forward to is unanticipated use. Jason Hoffman's entertaining presentation on their use of Solaris Express in production probed the boundaries of surprising use; I'll enjoy getting caught off guard by at least one project in the coming year. Or so I predict.

Don't get me wrong: obstacle-free anticipated use is good, too. But CDDL is so rational and relaxed compared to previous Solaris source arrangements that the bulk of previously impossible collaborations are now legit—so I want to be surprised by their output, not by their existence.

Whether or not my hoped-for dramatic novelty arrives, the coming year will bring change. Jim's off to a new stage on the other side of the Pacific; the community will hold elections for a new Governing Board, operating under a new and newly ratified Constitution; new requirements, be they technology-driven or technology-driving, will present new challenges for software systems. We might even get a new lead-in image on

There's no doubt it's taken much time and work to get here. There's also no doubt that it's been worth it. Enjoy the year past and the year to come—there's much to do.

[ T: ]

Thursday Jun 08, 2006

smf(5): Recent (and open) developments

Keeping up with the technical discussions on probably requires a full time team. Even with Glynn's weekly news and Eric's list summaries, I'm still not able to track it all—which shouldn't be a surprise, as I couldn't track everything when it was all closed (and somewhat smaller). In the various areas close to smf(5), there have been some developments worth mentioning.

David Bustos unveiled the Enhanced SMF Profiles project. There have been some clear shortcomings in our current use of configuration snapshots across upgrade operations, and enhanced profiles will clear these up very nicely. (But, like any well thought out feature, enhanced profiles will make a number of other configuration manipulation problems much more tractable.)

One such consumer will be the Network Auto-Magic project, which John Beck has been pursuing. The NWAM team recently released an initial prototype, which is letting them explore some of the issues—architectural, usability, generality—with making such a broad change.

I need to review the source to the iwi wireless driver to see whether it issues the appropriate DL_NOTE_LINK_UP and DL_NOTE_LINK_DOWN notifications to ip

Finally, we've long discussed extending the <template> section of the smf(5) service manifests with various forms of metadata that we believe might be useful to management software. Dave Powell wrote and released a draft discussion of template extensions under the auspices of the Visual Panels project. These properties will be useful to other smf(5) commands as well—svccfg(1M) being one of the most obvious target consumers.

Not all smf(5) work can be done virtually. Last month, Liane had a good trip to SANE 2006. She's posted the slides from her tutorial as well.

Probably need to get out there talking again soon myself.

[ T: ]

Tuesday Apr 25, 2006

Progress in open development

Just prior to this week's news, Glenn had an all-hands for the Operating Platforms Group. Since the all-hands is performed once for the Pacific through European business day and again later for Asia through Pacific working hours, I thought I would compose my presentation in full in the hopes of being consistent, and knowing that I could share it here as well. I fear the later presentation will become known as the "apple juice" talk, as my not-quite-two-years-old son Nathaniel came to the phone a few times to inform the audience of his preferred remedy to a sudden thirst. Your comments are welcomed—we'll be expanding and revising the talk and its material as the program and its tools mature. The talk benefitted from criticism and suggestions from Jim and Karyn.


Thank you for this opportunity to speak with you. Over the next thirty minutes, I thought that we could review how software development for the various consolidations will change into open development, on I will attempt to be clear about what you can do today and what's yet to come. Along the way, I'll also share what I believe are candid criticisms of what we have so far.

In the interests of lending some structure to this talk, we'll look at typical tasks an individual developer tends to do in the course of working on an operating system component. On a given day, a developer might

  • read and comment on a modified bug report,
  • update a child source tree,
  • change some code in their source tree,
  • build the binaries in their source tree,
  • solicit or provide feedback for a code review, or
  • make or evaluate a request to integrate reviewed code into a consolidation's gate,

as actions that involve small group of people. Our developer might also be taking actions that involve a larger number of people, up to the entire OpenSolaris community. For instance, they could be

  • asking or answering questions in a public forum,
  • proposing or commenting on a new interface or implementation choice, or
  • proposing or joining a new area of investigation.

I intend to touch briefly on each of these examples, which I hope will make it clear that we're committed to making it feasible—even easy—to perform the complete set of development tasks in an open environment.

Bug reporting

The first part of my morning is still spent sipping a hot cup of tea and reading new and update bug reports in the categories I'm interested in. For any of the reports I might wish to comment on, I can use the Java client to connect to the bug database and make appropriate updates, perhaps making connections to related bugs or adding the email of a developer I suspect might be concerned about this issue. Unfortunately, none of these actions is presently convenient for a developer without that access: bug submission and manual querying is available at, but ongoing reporting and updates are not.

There are two adjustments to make if you're using the bug database. First, use the Description field for the bulk of your updates, as all other fields are not published. This means the end of the "See Comments" bug idiom. And also, check Hook 6, which will contain the OpenSolaris ID of the submitter; if you make a significant status change, send the submitter an update.

We've given input into one effort to examine the bug database's role in open development, and we're funding work to improve the functionality of But if you have Bugster access and can help a contributor who can't make an update, please do so.

There are additional complications for the present bug database with respect to upstream and downstream relationships—with other communities whose software we rely upon or whose rely upon ours—but those aren't part of a typical developer's day.

One aspect of bug reporting we have noticed is that the bite sized bugs, with the keyword "oss-bite-size", provide a very effective starting point for developers new to OpenSolaris. I encourage you all to start marking introductory bugs with this keyword; because this keyword has been so effective, we're going to start tracking the marking and fixing rates as a metric for the program. For experienced developers, I appeal to you to respect some kind of rough law of "conservation of bite-sized bugs"; if you fix one, mark another one.

Source code management

Usually my tea is still warm when I'm deciding whether or not my current source tree is sufficiently up-to-date. Generally, the tea is tepid once my bringover is done.

If you're following the tools-discuss list on, you'll know that we recently selected a source code management tool called Mercurial to be the distributed source code management utility for opensolaris consolidations. We will be working with the Mercurial community to get a few features implemented, and I expect that most of you will be pleased with the performance of this very nice utility. I expect in most cases my tea will still be pleasantly warm.

There will be an interim period where we'll be publishing Mercurial bundles externally, but still using TeamWare inside Sun. This period will mostly be spent on tools updates, but in most cases we should see all developers on an equal footing, whether their primary consolidation uses Subversion or Mercurial.

Editing code

Once I've synced up, I'm in position to start making changes. I promise not to recommend a specific editor. Generally, expectations about code have not changed: the C style is expected and enforced, other conventions will be verified by your code reviewers.

Practices across internal and external contributors are common for this stage.

New files written by contributors to ON should contain the CDDL copyright block; other consolidations may have a different default license. An example is given for common types of files in

$SRC/prototypes/prototype.{Makefile, c, csh, h, java, ksh, man, s, sh}

in the ON consolidation. If your work involves integration into a Solaris release prior to 14 June, then the CDDL block should not be carried back into that source tree.

The "wx cddlchk" invocation, which is also part of "wx nits" and "wx pbchk", will check that the license block is correct and up-to-date.

Building a consolidation

Typically, building your consolidation or a portion thereof takes some time, usually based on your system's performance or how many people you share that system with. Once you've started your build, this is generally a good time to stretch or make a new cup of tea.

The various consolidations will tax build systems differently. With the recent integration of GCC shadow compilation, ON build system administrators need to be aware that there are now potentially two compiler invocations for each changed C file. On aggressively configured systems, this may result in paging.

For full builds, the experience of an internal and external developer should be equivalent.

A key difference at present is the availability of test suites. As you might suspect, there are many test suites that depend on the Sun corporate network environment; the result is that the suites are not presently available. Under the present sponsor process, a code submission is tested by the sponsor, or testing is arranged by the sponsor. There are a number of paths to balancing the test responsibility: the Test Community is exploring a hosted self-service test run, and would probably welcome discussions about what tests a developer should be able to run in their own environment to validate their changes.

Code review

With validated changes in hand, a developer is ready to start down the path towards integration. This commences with getting one or more developers to review your changes to the code. Of course, because you want your reviewers to find actual problems, you already made certain that your changes build cleanly and have no obvious style errors.

Finding a code reviewer often involves checking who's around, either physically or by various network protocols. The webrev tool allows you to construct a small web tree that contains your changes in a variety of formats so that a reviewer can work through your modifications and make appropriate suggestions. This makes it more convenient to have reviewers who aren't physically contiguous to you.

Since webrev generates a tree of HTML documents, all you need to publish your code review materials is a place to put them. This is easiest if you have an external site, and is an area where a developer primarily using the Sun corporate environment is at a disadvantage.

In the interim, Dan Price and Steve Lau have provided a "code review swamp" hosted at to offer a place to host your webrevs. The swamp is set up to handle publishing current format webrevs today; instructions are present on that site.

Now, I've also asked the Tools community to examine enhancing webrev to put the changes into a single document and use browser features to make the review experience equivalent. In this case, publication becomes substantially easier, and even blogs become a possible location for publishing code reviews.

Even with publication eased, finding reviewers can still end up relying on personal networks, which is difficult for a developer new to the community. I also believe that the request for a code review needs to be narrowcasted to those fellow developers who are interested in the area that is changing. So I believe we'll need to develop code review workflow support on the site. If you're interested in how this feature might work, we will be discussing it in the Tools community.

Code integration

With support for your changes from your code reviewers, and after building and testing your possibly modified code once more, you are ready to make your request to integrate—which has long been known by its acronym, RTI.

At present, the "request to integrate" is made via a tool only accessible on the Sun network. That, and the current source code management situation, led to the current sponsor process.

It's worth noting that each time I mention an aspect where Sun corporate network access is required to access a resource, the sponsor is performing the work on behalf of the contributor. That can add up to a substantial amount of work for some changes. Please be supportive and respectful of both the contributor and the sponsor; if you've the time to be a sponsor, or assist a sponsor-contributor interaction by participating in a review or running tests or shepherding a build, please consider helping out.

Once the various consolidations move to external writable consolidations, we'll see direct integration by all committers, but for new developers, the sponsor process will remain in place. Aspects will become simpler: the new source code tools make passing patches around substantially easier than before. As new capabilities are brought online on, like code review hosting and self-service testing, we'll see the sponsor-contributor process distilled down to its core: mentoring on technical participation in operating systems development.

Forum participation

Of course, technical guidance isn't only provided on a one-on-one basis, and not everyone is intending that their changes target mainline. In the various discussion forums, you'll see questions on how best to do something, or where it's presently done, or whether a particular choice has drawbacks.

You should engage in these conversations. Questions, and the context behind them, is valuable to all community members, whether they are planning to develop code or merely consider that feature in their deployment. A considered answer has value beyond the immediate conversation; all accurate technical content, be it a forum response, project documents, or blog entries, will increase the search engine cross section for As with any public forum, your participation should be confined to the technical aspects of the discussion; private information concerning your organization is not suitable as a topic for public discussion. That's not a new restriction—merely common sense.

I'll talk about forum creation in a moment. But first, I want to talk about a few special forums on

opensolaris-discuss is, in an extremely loose metaphor, the town square of The various procedures that the community's Governing Board has developed generally involve making an announcement or request for discussion on opensolaris-discuss. But, like any public place, there are conversations on pretty much any topic; it's a high traffic alias.

opensolaris-code has in recent days had its role clarified: it's a purely technical discussion alias on any aspect of the various components of OpenSolaris, architectural, implementation, or otherwise.

Architectural process

The third forum I want to mention is the ARC community and its associated mailing list, because it brings us back to our typical development activities.

Occasionally, in the course of fixing a bug or considering a problem, we realize we need to add a new interface to some OpenSolaris component. The process for doing this is to take the interface through architectural review. We've had a couple of contributors take their new interfaces through the fast track process, and reviewed some of the difficulties associated with each of those cases.

We also have found that the site isn't well structured for the frequent publication required to track the currently active ARC cases, or to publish key historical cases.

So, while we have an awkward way of making the process participatory, the correct course appears to be to take the fast track process for open consolidations and modify it such that it can be hosted on If you're interested in this topic, we'll be starting this discussion in the ARC community.


But sometimes, you realize that the changes you're envisioning are sufficiently large that they will require a group effort. This effort usually needs a forum of its own, plus a tree of documents, plus a source code repository. can handle this with its project hosting support, which will shortly offer the source code hosting feature. To get a project, you just propose the project on opensolaris-discuss. Over time, your project may garner interested individuals and communities, whose association will become a way for community members to identify projects they may find interesting. In the meanwhile, your project is well underway.

New projects and even existing projects for Nevada should use for hosting their project; please don't wait until you're ready to integrate. That's far too late. As you might suspect, we're developing a metric to monitor the progression towards external hosting.

The other kind of forum that we can host on is called a community, which has been confusing in a manifold sense. The term itself is overloaded: we can talk of the OpenSolaris membership as a whole as a community, or of a specific OpenSolaris community. And the role of a community in OpenSolaris governance is still being debated by the governance working group.

But I can tell you a few aspects of communities that might help you determine whether your initiative is a potential community. First, a community is not a program, strategy, or lifestyle. If you haven't heard these terms in reference to software development, you may wish to plug your ears for a little while. The programs, and so forth, are mechanisms for organizing projects with respect to the business priorities associated with the development of one distribution of OpenSolaris. Second, those distribution groupings are likely to have a finite period of applicability whereas a community, because it will be identifying its contributors to the larger OpenSolaris membership, is expected to be a long lived group.

Finally, a community is expected to appeal to a fairly wide subset of the membership; it attempts to gather combine a collection of interests into a coherent group. That's fuzzy and it's made a bit worse by the fact that there are some communities that were grandfathered from being proposed prior to the introduction of a community proposal process or prior to the introduction of project hosting. Reorganizing some of the communities will be an area of effort over the next few months, but the basic guideline is propose your effort as a project.


I hope this review of typical developer activities, how they are changing as we develop in an open community, and how certain aspects of these activities are simply hard to do if your workstation is not on Sun's network. None of these deficits is likely new to those Sun developers who work from a remote location or from a campus distant from California. And of course there are other aspects to collaboration that we need to explore, but that would be another talk entirely. It's my intent that we get to a situation where a developer, without reference to affiliation or location—but possibly dependent on their available bandwidth—can participate equivalently in opensolaris development. That will require ideas, feedback, code, and action from all of us.


Your questions, please.


A short set of slides were made to accompany the talk: PDF, ODP.

Creative Commons License
This talk and the slides are each licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License.

[ T: ]

Tuesday Apr 11, 2006

Program technical status, 11 April

[Repost from forum thread.]

Since my previous status summary, written in December, a number of technical aspects to the program have made progress. (It's becoming clear to me that I only write these when we actually deliver stuff.)

1. Website capabilities.

Basic community and project hosting seems to functioning reasonably well. There are 28 approved projects, some of which are still preparing their initial content, and 41 communities. It's probably worth noting that we can easily support multiple mailing lists for each project or community, if the project or community leadership requests additional lists.

There is still ongoing development to simultaneously relax and tighten our Mailman moderation settings, so that a registered email address can send to any list and so that an unregistered email address will always bounce. Mailing list moderators will appreciate the latter.

2. Source code management (SCM).

The software to add repository hosting to projects is getting close to completion. Internal testing with the Subversion-only option suggests that we should be ready to do a beta deployment in the next six weeks. Project leads interested in hosting their source using Subversion as part of this beta should contact me, directly or via website-discuss.

We're awaiting further comments on the tentative selection of Mercurial as the distributed SCM selection. Extending the hosting support to handle Mercurial repositories is straightforward, and I expect will be ready--with the then-current version of Mercurial--shortly after the deployment of the Subversion hosting beta.

The DSCM selection and related documents about hosting are available in the Tools community at

In terms of Mercurial use for the ON consolidation, I would like to discuss releasing Mercurial bundles, with per-integration granularity, as part of the weekly drops Steve Lau has been doing. The drop frequency will hold at weekly, until we are hosting read-only ON on On tools-discuss, we'll be talking about specific tools that need to be updated (or have the opportunity to be improved) because of the switch in the DSCM.

3. Continued source publication.

I believe that, since December, the Developer Tools, Documentation, Install, Network Storage, Solaris Freeware, and X Windows consolidations have all released portions or the entirety of their source trees. In many of these cases, an active community and/or project is available to discuss how these components can evolve.

The split ON tree has made deliveries easy, and Steve and others have continued to move code that we know to be unencumbered out of usr/closed and into usr/src (which is what you get from for ON source).

4. ON GCC readiness.

Keith Wesolowski implemented shadow compilation with GCC as a means to keep the ON consolidation warnings-clean with respect to both the Sun Studio and the GNU C compilers. The choice of shadow compilation means that a typical developer's test burden is still centered on the binaries produced with the primary compiler for the platform; at present, Sun Studio is the primary compiler for both SPARC and x86. Individuals and teams interested in a pure GCC build for those platforms or using GCC for ports to other platforms will still need to examine and test the binaries produced, but should not have to do "basic GCC cleanup".

The ON GCC status, and related links, are available in the Tools community at

5. Governance development.

On February 8th, the CAB approved the OpenSolaris Charter. Glenn Weinberg, Vice-President of the Operating Platforms Group signed it, on behalf of Sun, on February 10th. The Charter is linked to by the CAB community at

For those of you enthusiastic about historical documents, however trivial, Jim Grisanzio made a virtual exhibit of the charter Glenn signed.

(Whether we do a theatrical all-signatories-signing event is dependent on their schedules permitting them to physically congregate, I expect.)

The now-OGB (OpenSolaris Governing Board) is now working on the Constitution and/or By-Laws for the community. The cab-discuss alias is one forum for discussion; the developing document(s) are being composed in the wiki at, specifically:

Input into this drafting process is, I believe, always welcomed.

After we get through the various repository deployments, there are a number of software components that I would like to host, pretty much immediately: we need to improve many aspects of the site, including voting support for the OGB election, code review notifications, per-user repositories, and community, project, and user file management. Plus we could really use some new pictures for the front page. For all these developments, I expect website-discuss and tools-discuss to be the primary mailing lists. Please join in.

As always, please share your concerns; I am happy to receive them privately or on the list.

[ T: ]

Monday Feb 27, 2006

Gates, projects, and developers in ON

If you're following the tools-discuss alias on, then you're also aware that the ON consolidation uses a distributed source code management tool to track changes to the ON source (which includes the kernel as well as many device drivers, libraries, and commands). I thought I would review the current relationships between code repositories in typical development scenarios in ON, in part to practice using tool-neutral terminology in describing how code changes move from a developer into the main repository for the development release, but also to reveal some of the assumptions behind the distributed source code requirements and evaluation.

To stay neutral, the various invocations of a fictional 'scm' tool below are hypothetical; different SCMs might treat some of these command sequences as a more monolithic operation or might provide even more primitive operations.

For reference of the relationships we discuss below, here's a simple diagram, in which the arrows indicate the flow of code changes. The two shaded repositories are the first to receive changes from their respective children, which we'll call "gates". (The unshaded repositories receive their changes from a gate or via a child pulling from a gate.)

Typical repository relationships

Let's consider first an individual developer addressing one or more bugs in ON. He or she will need a copy of the source to work on, which we choose to store in /path/to/fix-0.

$ scm clone /path/to/onnv-clone /path/to/fix-0

For ON, this clone (or initial pull or initial bringover or initial checkout) operation involves the creation of a little over 34 000 files and nearly 7 000 directories.

Although we use file system paths in these examples, most of the candidates can perform their operations over the network, via `ssh`, `http`, or a native protocol.

The developer then makes his or her changes to the files requiring correction. During that period of time—editing, compiling, and testing, as well as devising a fix—the user may make intermediate commits of their current work and pull new changes (made by the other ON committers—let's say a thousand). This operation looks like

$ scm commit
$ scm pull
$ scm merge

with all operations going to onnv-clone.

Finally the developer is ready for final integration. Their code has been tested, code reviewed, and an integration request has been approved. Their final sequence is

$ scm reparent /path/to/onnv-gate
$ scm sanitize
$ scm commit
$ scm pull
$ scm merge

until they are completely up-to-date with respect to onnv-gate, at which point, they can publish their changes

$ scm push

The "sanitize" operation removes all of the intermediate commits made in the developer's repository—from merges made along the way—so that the main gate only sees the important portion of the history of their work: what specific defects, via bug IDs, are addressed by this changeset.

`sanitize` isn't typically implemented in source code management tools; it is a local addition that might be seen as controversial in some circles. ON isn't a pull- or patch-driven integration process, so if we didn't sanitize, we would have commit logs with the merge records resulting from 1 000 developers merging at different rates.

At this point, it's probably worth commenting on the existence of the onnv-clone repository, which is provided as a read-only child of the main development repository, onnv-gate. All of the operations in our scenario would have been sensible if we had used onnv-gate as our parent repository (and omitted our reparent operation). So why have a 'clone' at all? There are two reasons, one technical and one social.

The technical reason is to make sure that contention for the gate's repository locking is limited to developers integrating code. Integrating into ON carries with it a substantial burden of testing, watching for other integrations, rebuilding, and so on: getting blocked by an initial pull of the gate repository taking the various repository locks is a difficult scenario we want to avoid.

The social reason is that the clone represents a steadying rhythm for developers: unless you are preparing to integrate that day, you know that the pull/merge you do from the clone is, on some best effort basis, safe. Small build mistakes have been fixed and large ones undone prior to potentially impacting your development repositories.

So a clone is a useful technique when a DSCM's locking might be overwhelmed by the number of child repositories needing to be kept up-to-date.

Our second scenario, where a few developers are pursuing a project that involves a large batch of coordinated changes to ON, is an example where a clone repository isn't necessary, as the three developers are unlikely to contend on the repository lock. But the role of project-gate is similar: to provide a repository which is regularly built and whose binary products are regularly tested (and may even be shared with potential users or act as parents to other projects' gates), so that the developers can make confident forward progress.

One of the developers (or more) will, on some schedule, do pulls and merges from onnv-clone and integrate them into project-gate. This "resync" can be done from the project gate itself or (and more preferably, but not illustrated) from an additional child. In the latter case, this sequence looks like

$ scm parent
$ scm pull /path/to/onnv-clone
$ scm merge
$ # test build, corrections, with possible a commit/pull/merge
$ scm push

where we have pulled from onnv-clone to push to our project repository.

When the development team is ready to integrate their changes, their invocations from the project-gate repository are similar to their colleague's from fix-0 above:

$ scm reparent /path/to/onnv-gate
$ scm sanitize
$ scm commit
$ scm pull
$ scm merge
$ scm push

For a project, this action is usually followed by the consumption of one or more celebratory beverages, while the most nervous member watches for mail from one or more gatekeepers regarding build nits or more serious omissions. We'll have to work on collaboration tools so that this team experience can be shared across the network.

[ T: ]

Monday Dec 12, 2005

LISA05 Friday

After commiserating with my room's ceramic bunny—his feet are glued to the cabinet's top and so he has limited opportunities for conversation—I finished packing and headed out for the last few hours of LISA05.

No way out bunny

First up was the last papers session, with Liane presenting on smf(5), in addition to papers on a spreadsheet-based scripting environment and a whizzy DNS script for medium sized sites. There was setup before the session, with Luke Kanies outlining the basics of how the sessions run:

Liane pre-session

The talk was well delivered and Liane answered the audience questions adeptly so, after a group lunch, we headed off to watch Liane, Bryan, Dan, and Bill lead a "guru" session on Solaris 10.

Guru session

My flight time meant that I wasn't able to stay for the entire session, so I can't assess the total guru-osity displayed over the 90 minutes. But I certainly think that this year's trip to LISA was worthwhile.

[ T: ]

LISA05 Thursday: Reception(s)

Thursday evening was a series of sponsored events. The LISA reception was a combination of dinner, desserts, drinks, and a miniature casino floor:


A fascinating part of the reception was the balloon assemblies produced by the man pictured; here he's making a birthday cake hat:


People lucky enough to receive wings, a palm tree hat (complete with balloon monkey), and the like were good enough to wear them around the ballroom so that we might all see what objects can be constructed from inflated rubber.

After the reception, Sara hosted an OpenSolaris hospitality suite, which came furnished with a very generously pouring bartender. The crowd was pretty good:

As the crowd thinned out, I noticed that a complimentary laptop servicing was being handled by a group of engineers:

Ultimately, the laptop was restored and the evening ended happily.

[ T: ]

Sunday Dec 11, 2005

LISA05 Thursday: Cost models and wiretaps

I went to the midday configuration management theory session as well as to Matt Blaze's plenary. (I missed the invited talks as I lost track of time during a late lunch.)

The talk I enjoyed out of the theory session was by Alva Couch and colleagues from Tufts, who presented steps towards a cost model for system management. They worked through over a year's worth of RT logs from their site, working out the background effects of holidays and weekends, and then being able to model alternate staffing choices and their impact on response time. Alva also made some provcative statements about best practices; with respect to a time-sharing/interactive user dominated site, these statements likely hold. (Less applicable to server farms, though.) It was pleasant to see some math; I plan to work through this paper carefully to see what might be extracted.

Matt Blaze gave an energetic talk about his team's recent work on wiretap technology and how the various implementations can be evaded or confused. There were complimentary copies of the issue of IEEE Privacy and Security that contained the full article, which was a nice touch.

[ T: ]




« June 2016
External blogs