verexec(1): A simple execute-most-recent-version utility

We've published three of the approximately-six-monthly distribution releases now, and we're starting to refine pkg(5)'s capabilities to handle automatically an ever larger set of configuration scenarios. (Read: "edge cases".) Bart implemented actuators in 2008.11, and that let us handle any resource that could be configured or reconfigured via an smf(5) service instance. In future releases, we'll build out the set of instances that handle standard configuration.

In this post, I'd like to talk about a tool—verexec(1)—to assist with executable version selection. One of the common questions that maintainers of the various language platform packages is "how can I make sure that /usr/bin/[language] points to the latest version of the package?". Attempts to satisfy this goal have led to some of the largest preremove and postinstall script pairs in historical Solaris.

Largest, and most difficult to get completely correct.

We can make this problem much simpler—and scripting free—if we accept the minimal performance penalty of a runtime-based decision.

Basic function

verexec(1) works in a similar manner to isaexec(1), in that dispatch is based on the target command being hardlinked to verexec(1), which then, based on the name of the target command and additional information, will in turn invoke the correct version of the target command.

Like most pkg(5)-driven modifications, we want to make the identification of the newest version based on the presence or absence of a filesystem object delivered into a predetermined location. For verexec(1), one delivers a symbolic link in /etc/verexec.d/[target], pointing to the appropriate executable. For example, for Perl 5, /usr/bin/perl is delivered as a hardlink to /usr/bin/verexec, in SUNWpl5. [TK new name?] The Perl 5.8.4 package would deliver the symbolic link, /etc/verexec.d/perl/5.8.4, pointing to /usr/perl5/5.8.4/bin/perl. (The version ordering matches that of pkg(5).)

When /usr/bin/perl is invoked via exec(2), verexec(1) would open—subject to various override mechanisms—/etc/verexec.d/perl and search through the links, looking for the maximal version. The link is then read, and that program is executed, with the arguments passed to the original invocation.

This approach handles directly the delivery of multiple versions, via separate packages. If we were delivering two versions, then removal or installation of either link would change the default version. For more than two versions, the default version only changes if the currently highest version is changed. If we deliver no versions, then verexec(1) displays an informative error message.

Refinements and commentary

Overrides. Since, at some sites, verexec(1) may be useful to provide access to a variety of local command versions, we provide a number of environment variable-based override mechanisms. First, we allow identification of an alternative symbolic link directory to /etc/verexec.d via the VEREXEC_ALTDIR variable. Second, we allow per-command interception via the VEREXEC_[command] pattern. Thus, in our Perl example, I could make /usr/bin/perl point to my local Perl installation via:

export VEREXEC_perl=/home/sch/bin/$(uname -p)/perl

instead of the system installation.

Native ISA selection. We could combine the function of isaexec(1) with verexec(1), by allowing links to have the pattern [version]-[bits], as returned by isainfo -n.

Search impact. Using a technique like this impacts the search functionality, in that a search request for /usr/bin/perl in our example, will actually return the package delivering the verexec(1) link, and not an actual Perl interpreter. We can address this somewhat by ensuring that (a) the actual Perl interpreter is delivered in a package with "perl" as a basename, and (b) the link-delivering package is named and documented as a supporting package.

Delivery, while delaying change in default. Since this mechanism is so much simpler than the scripting previously required, it means that we should reexamine the transition to a new version of a significant language platform. In the past, because correct packaging was a significant amount of the work involved in integrating a new version, we've generally introduced a new version and made that version the default in a single integration. Since we don't want to introduce unnecessary risk, we have generally lagged integrating the latest version of these interpreters. With verexec(1) and pkg(5) in place, we might pursue an alternative, of introducing the new version early, and then changing the default in a subsequent, small integration later in the release.

Performance. If we find that the overhead of the additional exec(2)—one for verexec(1) and one for the link target—is too high, there are a number of performance optimizations. One approach would be to use verexec(1) to send a message to a privileged daemon, which would loopback mount the appropriately versioned executable at the verexec(1) location. Any loopback mount-based technique makes a trade-off: more efficient execution for more complex handling of updates and deletions, which would require the daemon (or privileged command or service) to inspect the existing loopback mounts and evaluate them for their continuing correctness.

I've a refinement-lacking prototype of verexec(1) available for review; we're discussing it further at pkg-discuss@opensolaris.org.

[ T: ]

Comments:

Interesting idea. A couple of questions:

How do you determine the ordering of version numbers? Some packages use a combination of digits and numbers, so it isn't always straightforward, for example how do you handle something like '1.2.3beta1' and '1.2.3'?

How do you handle the case where od/even numbering is used to distinguish between release and development versions?

Is there a per-user override that doesn't depend on environment variables, for example can I have a private ~alanbur/.verexec directory?

Posted by Alan Burlison on August 21, 2009 at 08:41 PM PDT #

"verexec(1) works in a similar manner to isaexec(1), in that dispatch is based on the target command being hardlinked to verexec(1), which then, based on the name of the target command and additional information, will in turn invoke the correct version of the target command."

What if the target binary executable is on a different filesystem? Hard links cannot span filesystems.

This is basically the same problem which isaexec(3C) faces today.

For this reason alone, a third party software vendor would need to keep a private copy in /opt, which, ironically, would disable having the latest revision of verexec(1)...

Posted by UX-admin on August 22, 2009 at 02:14 PM PDT #

@Alan: We use the same ordering as pkg(5), so only "dot sequences" are legitimate. Since we would use verexec(1) for deliveries into a release, I don't think we would want to ship a development version with a verexec(1) link. That is, we can ship the development version, but wouldn't have the link in /etc/verexec.d present--it would never become the default. I wasn't planning to have $HOME/.verexec.d as a default option.

@UX-admin: Yes, that's a flaw with all these hard link-based schemes. Since isaexec and verexec are intended to be open source, third party vendors should feel free to take copies and ship them. Let me know if you have other suggestions, though.

Posted by Stephen Hahn on August 25, 2009 at 08:35 AM PDT #

Post a Comment:
Comments are closed for this entry.
About

sch

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
External blogs