pkg(5): No more installer magic

I thought I would continue probing some of the problems that present themselves to any packaging system that might follow the System V packaging. For the next few topics, I'll phrase the problem in terms of an outcome I believe we want to avoid. Here we discuss aspects of eliminating special metadata from installers.

If you're familiar with the packaging and installation components involved with Solaris Express, Developer Edition, or any of the OpenSolaris distributions that Sun produces, then it won't surprise you that there's a large amount of upgrade specific knowledge in the installation layer. For instance, to upgrade a package installed with Solaris 10 to its currently delivered version in a recent build, there's the set of files associated with the package—and then there's the package history, which collects additional information, like files no longer delivered, or the responsibility of another package. The presence of this file, and the absence of a true package upgrade operation in System V packaging, mean that any kind of upgrade requires some form of installer: the operating system installer for upgrading the entire installation; an application-specific installer for upgrading a group of related application packages.

There are a number of other metadata files trapped in the installation layer, but the most important are the metacluster files, which group the package clusters into large installation profiles, and the package clusters themselves, which group sets of packages along feature boundaries, approximately. It seems evident that these groupings are merely another kind of dependency, much like how a System V package can, naïvely state a dependency on any other System V package.

The System V dependencies also show that another critical piece of metadata—the versioning vector, or "arrow of time"—is encoded in the installer.

All of this information, if we are to allow packages to upgrade from one version to another in some linear fashion, needs to be pulled out of the installers and moved into some aspect of the packaging system. This change in responsibility means that the role of the installer becomes more precise: it must prepare a location for software installation, optionally lay down some initial, and possibly stale image, and collect any required configuration information. Subsequent upgrade operations are driven primarily by the packaging system, which can utilize the version history and dependencies in a manner at least equivalent to what the historical metadata allows.

The other reason that we wish to push that historical metadata into the packaging system is so that it becomes accessible to a new class of application: the distro construction toolkit, which needs the dependency and versioning information to simplify the construction of self-consistent installable images. That leads us to an architectural diagram like

pkg Layering

where we've suggested some internal structure to an updated packaging system. This separation shouldn't be that surprising: in the current system, lumake(1M) and most of LiveUpgrade as well as zoneadm(1M) for Zones are both performing image operations on either side of any packaging operations they might invoke. Designing a packaging system that makes the constructions of distributions and their installers substantially simpler will require such an API layer.

There are some open questions that come to mind:

  • How much variation is there in the initial preparation an installer offers for the image location? Is space management a policy owned exclusively by the installer, or should the image operations layer have the ability to influence that allocation?
  • Is there a marshallable image description, beyond the package dependencies? Is there a marshallable image format, or are we restricted to end-use formats, like ISO images or archived zones?
  • What OS services should we consider modifying, beyond taking advantage of ZFS's current capabilities?

If you're interested in these questions, or the potential architecture of a packaging system, we'll be discussing these topics further in the Install community group.

[ T: ]


I understand the desire.. "need", even... for the ability for admins to do more organized, "cluster" aware installs. Believe me. I've been toying with the idea of having something like that for blastwave packages, for YEARS now.

The thing is, the only reasons we dont have that, are because I dont have the spare time to write the improved installer layer any more (too busy with the base day-to-day 'running' of blastwave packages), and because no-one else has had the commitment to work with me and write it either.

It's not because we "cant do it on top of SVR4 pkg format". We can.

There's nothing wrong with a layered approach to packaging and installation. In fact, there's something positive to be said about it. Seems like most times people try to stuff EVERYTHING into one big single architecture, it ends up causing more problems than having smaller, more modular components.
There was a category-based debian installer, that seemed to work quite well, and was on top of, not instead of, apt-get. (sorry dont remember name)

Heck, SUN ITSELF had one: swmtool. Wasnt pretty, but it was fairly functional. It had some bugs in it, but nothing that couldnt be fixed. I'm quite irritated that sun doesnt ship it any more.

So, once again, we're back to "why throw out SVR4 pkg format?"
Sure, you 'could' write a whole new back-end, to make it 'easier'. But for all us enterprise customers out here who actually value sun's stability of interfaces, the question is: do you \*HAVE\* to?
As far as I can see so far, the answer is no.

You say you "want to eliminate metadata from installers". you dont say why. If the answer is, "because you want to allow people to write "fancy" installers, but not have to dup efforts, all that means is that you need to provide an improved, standard, system-level tool for them to hook into.
That tool can be layered on top of pkgadd, rather than replacing it.

Posted by Philip Brown on August 07, 2007 at 03:04 AM PDT #

The reason for moving the metadata from the installer into the packaging is so that upgrades can be piecemeal instead of from one entire OS point release to another.

I don't think moving metadata from the installer to the packaging layer implies dumping SVR4 packaging. It does imply at least improving it.

I certainly do think that dropping SVR4 packaging altogether isn't worthwhile if we can improve it and reuse the majority of the SVR4 packaging we do today. But at least some very large packages (e.g., SUNWcs\*) will likely have to be split up into lots of very small ones, and that is \*may\* mean that the work to adapt to moving metadata from the installer will turn out to be comparable to the work in moving to a new packaging technology. Still, I think it's likely that we could simply keep and enhance SVR4 packaging at lower cost than moving wholesale to something else.

Posted by Nico on August 08, 2007 at 05:50 AM PDT #

I would like to understand more, why there cant be "piecemeal upgrades" already.

(and, what do you have in mind by the term "upgrade"? I can think of multiple potential contexts for it)

If you mean the full-on "OS upgrade" style upgrade
Seems like "all"(harhar) you would have to do, would be to do the exact same proceedure you do now, for quarterly updates... except have a smaller selection of target packages.
After all.. the current quarterly releases have "initial" vs "upgrade" options already.

Perhaps the sticking point there, is that perhaps the current "upgrade from cdrom" proceedure, tends to remove ALL packages, and that is undesirably slow if not needed. Is that correct?
[sidebar: would be nice to have a wiki page on this somewhere, rather than a blog, so that you could fill in the problems in detail, and update with more details as requested. Easier to read a clean summary, then re-read a full thread of discussion]

If this is what you are referring to... there is certainly the issue of "which packages are 'safe' to remove/update, and in what order?" However, there are multiple ways to approach this problem.
The first step in finding a "best" method, is, again, to actually write up a full description of the problem, with detailed issues.

I am fairly well versed in the issues myself, being the overseer of blastwave package upgrade policy, but I would be very interested in reading a detailed problem description from the perspective of sun packaging folks.

Posted by Philip Brown on August 09, 2007 at 03:37 AM PDT #

Piecemeal upgrade: upgrade OS components without having to upgrade all of it, just all the depedencies and affected dependents of the component being upgrade.

And yes, the pkgrm+pkgadd approach to pkg upgrade is rather inefficient.

Posted by Nico on August 09, 2007 at 04:50 AM PDT #

What is inefficient about "pkgrm+pkgadd approach"? The fact that it updates individual file listings? No way around that. Flat file updates? well, you tried to change that in sol10 express pre-FCS release, but reverted that. Why? (and why wouldnt those reasons also stick to any "total package management rewrite" approach?)

As for the other bit... yes, that can be a tricky problem, for "large" multi-pkg components. How about a wiki page documenting the issues that need to be solved for it?

Posted by Philip Brown on August 09, 2007 at 07:15 AM PDT #

So "pkgrm; pkgadd" takes package A through

A_v -> (nil), (nil) -> A_v'

while linear upgrade is equivalent to

A_v -> A_v'

There's a significant loss of state in the former path. (The intra-5.10 database introduction and removal was tied to performance criteria that were not met--it ended up being faster for queries, but slower for package add and remove operations.)

-- Stephen

Posted by Stephen on August 24, 2007 at 09:08 AM PDT #

Well, we've handled the "pkgrm; pkgadd" upgrade cycle for years now at through pkg-get. It takes a little more attention by the package maintainers, but it's doable in relatively clean fashion. 90% of packages dont need anything at all. 9% need 10 lines in postinstall/preremove. the other 1% are going to be ugly any way you look at em.

[and to answer some off-line email from Stephen]...
As far as drivers go... we only have one or two. they didnt require any customization, so no comment from me there. I'd be happy to discuss any specific examples if you like.

I will grant that smf services can be nastier... we've had some in-depth discussion, and there is an area of complexity around trying to judge intent on whether a sysadmin meant to temporarily disable, or permenantly disable, a service, prior to upgrade. (the main problem being trying to determine whether sysadmin wishes to auto-enable the service on pkgadd, or not).

but there are still workarounds.. and its not really so much a matter of "cant", but rather, choices on "which way do we think would be the most expected way to handle it".

Preserving "state" (of configuration) is still handled in the same way that we handle non-smf demon packages: by existance of a customized configuration file. (whatever config file the app normally uses)

hmm. That's one of the few areas of CSW packaging that isnt written up in our standards pages. To summarize(skipping some finer points i'd be happy to detail): If a user-customized prog.conf file exists, leave it be. It remains, even after pkgrm. Then if a pkgadd of a new version happens, it will automatically pick up the old conf file.
That is usually enough "preserved state" to make our users quite happy, and I dont recall any complaints about leaving a single .conf file laying around, even if the user's intent was to actually remove the package permenantly. Maybe one complaint in 5 years, across a couple-hundred-thousand users.

pkg-get does not implement a "purge" command like apt-get does, to "purge all related files".
It COULD (i'm imagining a possible implementation right now)... but I have not seen sufficient demand from our users to warrant spending time to implement it.

Posted by Philip Brown on August 27, 2007 at 10:31 AM PDT #

Hello Stephen,

I would also suggest that you cross-post in indiana-discuss in order to reach a greater audiance.

Posted by Francois Saint-Jacques on August 27, 2007 at 02:09 PM PDT #

State turns out to be more useful than just retaining package-specific configuration, but even there, it's very relevant for device driver handling. I suppose the relevance of driver delivery for the core OS is safe to assert.

A related problem are would be: do you have a package that requires the system, or a significant subset, be restarted? -- Stephen

Posted by Stephen on September 09, 2007 at 03:05 AM PDT #

kernel driver packaging handling, is a very very rare edge case. There are lots of potential ways to handle this, that don't justify rewriting package management from the ground up.

Sun already has some kind of handling for this, with patch/patchcluster installs.
There seems to be something where patchadd or at least install_cluster refuses to run, if there is a pending kernel "must reboot now" patch.

So, those rare packages that require such a thing, can have a smidge more preinstall/postinstall logic. They have to have major preinstall/postinstall handling ANYWAY, reguardless of which package management system you choose to use, so this isnt some major burden or obscenity in that area.

Got any further issues that involve more normal packages, rather than driver packages? ;-)
For your comments about requiring a "significant subset of the system to be restarted": "restarting", implies a service to be "restarted"... Ah... ISNT THAT WHAT SMF IS FOR?!?! ;-) Service management belongs at the "service management facility" level, not the package management level.

Posted by Philip Brown on September 10, 2007 at 05:44 AM PDT #

Actually, I was thinking of system libraries and other shared components, and not services, since we do handle that case well with smf(5)--as designed. If you believe that Sun and its customers are well-served with the patch tools and their attendant problems, then I guess our input data differs substantially. -- Stephen

Posted by Stephen on September 10, 2007 at 06:15 AM PDT #

I'm not saying that sun's patches are perfect. As a loong time enterprise customer, i'm well aware of the problems there :-) But, the patching issue, is a little more complex than straight-up package management.

Getting back to what you specifically said:
if you were not talking about restarting "services", and you're not talking about "reboot", then what kind of "restart" were you talking about?

Posted by Philip Brown on September 10, 2007 at 08:53 AM PDT #

I for one, vote keeping pkg SVR4 in O.S. but make enhancements to make up the differences with modern Package Manager that can do auto dependency installation, digitization of software build,package build and software distribution system in repeatable XML files.

Another comment is "why reinvent the wheel" to solve the shortcomings(above) of SVR4 ?, it is fixed already by TWW Inc. ;)


Posted by T.J. Yang on September 28, 2007 at 04:15 AM PDT #

Also, I have no grudge against SVR4 since I use TWW's Hyper package manager. but I am concern if packages format and SVR4 got drop in the future. said switch to debian(rpm,openpkg,ports ...), a total different PM system.

Please spare poor system administrator for this kind paradigm shift.

Can we start a wiki page about pkg SVR4 (on new O.S. wiki site)?
we can list out current issues and possible solutions.


Posted by T.J. Yang on September 28, 2007 at 04:44 AM PDT #

I'm also all for keeping SVR4 packaging around. There's bajillions of SVR4 packages already out there, and, for the most part they Just Work.

The biggest complaints I think I hear with packaging are not with the packaging system, but with how it is used.

For example, packages that are either too coarse (SUNWcsr, SUNWhea) or too fine (SUNWhmdu versus SUNWhmd).

SVR4 packaging has dependency tracking builtin. Just most folks don't bother with it.

So far I have yet to see a comprehensive package manager (RPM, DEB, NetBSD's ports tree, etc.) that solves all problems... they all have limitations, and some of them suffer really badly.

My gut reaction is that naive attempts to engineer away the problems that are created when we people screw up, are doomed to failure. I'd much prefer a KISS approach.

But like I said elsewhere, I happen to \*like\* SVR4 packaging.

Posted by Garrett D'Amore on October 04, 2007 at 02:02 PM PDT #

Post a Comment:
Comments are closed for this entry.



« July 2014
External blogs