Sunday Jul 25, 2010

Revelations: 145

Phoronix published a sensationalist article last week claiming that my regular e-mail updates of our biweekly builds somehow signified some out of the ordinary newsworthy event, without bothering to do even the most basic of fact checking. While I pointed this out in their forums within hours of publication, I'm still seeing it cited by other web magazines that don't bother to fact check, as well as in various e-mails and blogs, so am publishing a more complete explanation here of why it really is a non-event.

The article claimed:

As the first email of its kind in months, Alan Coopersmith who is a known X.Org contributor and longtime Sun Microsystems employee now working for Oracle, has written a new email entitled "IPS distro-import changes needed for X packages for nv_145." Alan immediately began this public email by saying, "Just when you thought you'd never see another one of these biweekly mails..."

Sadly, all they needed to do to disprove the claim that it was the “first of its kind in months” was simply follow the links from the e-mail archive page they linked to, to see that I had sent a similar message two weeks earlier for the previous biweekly build nv_144. In fact, if they checked the archives for previous months, they would have found that, except for missing build 143 (a mistake on my part), I've sent these approximately every two weeks for every biweekly build for a very long time.

Perhaps I'd confused the article's author with the offhand comment he seems to have misinterpreted, but explaining that requires a bit of background explaining what these e-mails are and why I send them in the first place.

As many OpenSolaris users know, over the course of the last couple of years, we've been transitioning from the old SVR4 package system used in Solaris 2.0 through Solaris 10 to the new Image Packaging System (also sometimes known as “IPS” or “pkg(5)”) being developed by a team of Solaris engineers and community members. Initially, we maintained two parallel distros, Solaris Express: Community Edition (SXCE), which was built using the SVR4 packages and the old Solaris installer that worked with them, and OpenSolaris (originally codenamed “Project Indiana”), which was built using the IPS packages and the new Caiman install software designed to work with them. The teams providing the package contents continued to deliver SVR4 packages, and the OpenSolaris distro team converted those to IPS.

One of the goals of the OpenSolaris distro was to provide a set of ISO install images and package repository that was completely, freely redistributable, so that it could be easily mirrored, copied, and downloaded without having to deal with the various encumbrances required by some of the third-party licenses in the traditional Solaris and SXCE packages. Unfortunately, at that time, we had not yet finished separating the encumbered code from the open source code in our X packages so that they could be included, since when Sun made its proprietary fork of X11R6 in the early 90's, the engineers never figured we'd be open sourcing Solaris a decade later and need to easily separate out the encumbered bits they were merging into the main code base.

The initial Developer Preview releases of the OpenSolaris distro thus included a set of X packages that Moinak Ghosh built from the Fully Open X (FOX) project work we'd done to rebuild our source trees from the ground up using the open source code from the X11R7 modular releases in order to ensure everything was either open source or cleanly separated out. Over the first few months, we migrated from those to the packages our team delivered to the OS as we integrated the FOX project work into our main source tree. Because these changes weren't always obvious to the external observer, I started sending notes for each biweekly build to let the team maintaining the SVR4 to IPS conversion tables know which parts of our packages they could now include, as well as any other changes they needed to know about, such as version number changes. (Our SVR4 X packages all used the same version number, a holdover from the monolithic X source days, but we've migrated to using the upstream version numbers as much as possible in the new X IPS packages.) I did this not only to try to help the distro building team, but also to help myself keep on top of the changes they made in converting our packages, so that we could understand issues users hit and know how to help them, and to learn better what we'd need to do when it came time for us to start building the IPS packages ourselves.

Last year, Sun announced that the time had finally come to start converting the builds to generate IPS packages directly, taking the next step in transitioning off SVR4 and ending the production of the SXCE distro, since the SVR4 packages used to build it would no longer be made. The ON consolidation went first, converting the packages containing the kernel, drivers and core utilities in build snv_136. The Install consolidation went next, in build snv_143, and X followed in build snv_144.

So, when I wrote “Just when you thought you'd never see another one of these biweekly mails...”, I simply meant that after build 144, all the X packages are already delivered in IPS format, so there are no SVR4 to IPS conversion files to update for them any more, so I won't need to send those - except in cases like we had in 145, where I relocated one of the files that was listed as a dependency in one of the other packages still being converted by the distro builders, so they needed to update the dependency statement for it to list the new path to the file.

Despite what Phoronix seems to have assumed, I was not in any way referring to the limbo state the OpenSolaris distro is currently in (and unfortunately, as much as I'd like to explain that, I can't), nor stating anything about build 145 that is fundamentally different than the previous builds. It should come as no surprise to anyone that while build 134 was the last build to be publicly released, we have continued work on the Nevada builds after that - after all as we've said since 2005, Nevada is the code name for the development branch in which we're working towards the next full release of Solaris (i.e. not another Solaris 10 update release, but the one we may someday call Solaris 11) - while that's been released under various forms, as the OpenSolaris open source code, or via the binary releases of the original Solaris Express, Solaris Express: Community Edition, Solaris Express: Developer Edition, or the OpenSolaris 2008.05, 2008.11, and 2009.06 distros, we've always kept driving towards the same goal, with biweekly builds assembled to test the current progress.

My mail is hardly the only externally visible sign of this - you can see changelogs for the major consolidations (ON, SFW, X, and Desktop/JDS) for build 144 (the last fully finished build, as 145 is just finishing pre-integration testing now, with a delivery deadline of Monday for packages to be included, and the distro build process starting after that. Of course, the sources are also available, and there's plenty of activity on the various commit notification and discussion mailing lists showing that we're continuing to work away on Nevada.

So unfortunately, Phoronix succeeded in making a mountain out of a molehill, confusing their readers and fellow webzine authors, but likely meeting their goal of driving more traffic to their site to generate page views for their ad revenue as people passed the link around twitter, IRC & email or linked to it from their blogs & articles. As others have pointed out, checking the facts or contacting the developers to find out the story is less juicy than it seems doesn't play well with that business model (and that's not just true for Phoronix - look at any number of the columnists for other web-based "trade publications" that generate traffic via controversial posts, and the more outraged the community gets over them and angrily passes them around to denounce, the better their numbers are - you can just imagine how many of their articles are designed to bait Groklaw or Slashdot readers).


Engineer working on Oracle Solaris and with the X.Org open source community.


The views expressed on this blog are my own and do not necessarily reflect the views of Oracle, the X.Org Foundation, or anyone else.

See Also
Follow me on twitter


« July 2016