Thursday May 27, 2010

Penultimate

Today's my last day but one in MPK17. I've spent the last week or so transferring ideas, status, and hopefully-not-too-incomplete sketches to my colleagues. It's been just over twelve years since I finished school and began working on problems in and around Solaris, with lots of excitement along the way. And now it's time to look for new opportunities, at least after a short break.

I've set my blog here to redirect to my personal blog after a short delay; most of my contact information should be on the Internet somewhere.

Thursday Sep 24, 2009

2010.next: a better netbook for meetings

I recently borrowed one of the 10" netbooks that came out this year, and tested Build 123 on it. Here's a shot of Toshiba's sleek black NB205 displaying a clean Device Driver Utility run.

nb205

The NB205 has a keyboard on par with the r500 I use regularly—large for a netbook—and has a superior trackpad. Suspend-resume is reliable; compiz runs well; the extra vertical inch means fewer application compromises over last year's netbook crop. (For instance, rather than rearranging panel to the sides, use of gnome-panel's auto-hide feature is enough to let applications run well when maximized.)

If I don't have to give it back, it'll be my new "netbook for meetings".

Netbook styling credit to Dave Powell.

[ T: ]

Friday Aug 21, 2009

verexec(1): A simple execute-most-recent-version utility

We've published three of the approximately-six-monthly distribution releases now, and we're starting to refine pkg(5)'s capabilities to handle automatically an ever larger set of configuration scenarios. (Read: "edge cases".) Bart implemented actuators in 2008.11, and that let us handle any resource that could be configured or reconfigured via an smf(5) service instance. In future releases, we'll build out the set of instances that handle standard configuration.

In this post, I'd like to talk about a tool—verexec(1)—to assist with executable version selection. One of the common questions that maintainers of the various language platform packages is "how can I make sure that /usr/bin/[language] points to the latest version of the package?". Attempts to satisfy this goal have led to some of the largest preremove and postinstall script pairs in historical Solaris.

Largest, and most difficult to get completely correct.

We can make this problem much simpler—and scripting free—if we accept the minimal performance penalty of a runtime-based decision.

Basic function

verexec(1) works in a similar manner to isaexec(1), in that dispatch is based on the target command being hardlinked to verexec(1), which then, based on the name of the target command and additional information, will in turn invoke the correct version of the target command.

Like most pkg(5)-driven modifications, we want to make the identification of the newest version based on the presence or absence of a filesystem object delivered into a predetermined location. For verexec(1), one delivers a symbolic link in /etc/verexec.d/[target], pointing to the appropriate executable. For example, for Perl 5, /usr/bin/perl is delivered as a hardlink to /usr/bin/verexec, in SUNWpl5. [TK new name?] The Perl 5.8.4 package would deliver the symbolic link, /etc/verexec.d/perl/5.8.4, pointing to /usr/perl5/5.8.4/bin/perl. (The version ordering matches that of pkg(5).)

When /usr/bin/perl is invoked via exec(2), verexec(1) would open—subject to various override mechanisms—/etc/verexec.d/perl and search through the links, looking for the maximal version. The link is then read, and that program is executed, with the arguments passed to the original invocation.

This approach handles directly the delivery of multiple versions, via separate packages. If we were delivering two versions, then removal or installation of either link would change the default version. For more than two versions, the default version only changes if the currently highest version is changed. If we deliver no versions, then verexec(1) displays an informative error message.

Refinements and commentary

Overrides. Since, at some sites, verexec(1) may be useful to provide access to a variety of local command versions, we provide a number of environment variable-based override mechanisms. First, we allow identification of an alternative symbolic link directory to /etc/verexec.d via the VEREXEC_ALTDIR variable. Second, we allow per-command interception via the VEREXEC_[command] pattern. Thus, in our Perl example, I could make /usr/bin/perl point to my local Perl installation via:

export VEREXEC_perl=/home/sch/bin/$(uname -p)/perl

instead of the system installation.

Native ISA selection. We could combine the function of isaexec(1) with verexec(1), by allowing links to have the pattern [version]-[bits], as returned by isainfo -n.

Search impact. Using a technique like this impacts the search functionality, in that a search request for /usr/bin/perl in our example, will actually return the package delivering the verexec(1) link, and not an actual Perl interpreter. We can address this somewhat by ensuring that (a) the actual Perl interpreter is delivered in a package with "perl" as a basename, and (b) the link-delivering package is named and documented as a supporting package.

Delivery, while delaying change in default. Since this mechanism is so much simpler than the scripting previously required, it means that we should reexamine the transition to a new version of a significant language platform. In the past, because correct packaging was a significant amount of the work involved in integrating a new version, we've generally introduced a new version and made that version the default in a single integration. Since we don't want to introduce unnecessary risk, we have generally lagged integrating the latest version of these interpreters. With verexec(1) and pkg(5) in place, we might pursue an alternative, of introducing the new version early, and then changing the default in a subsequent, small integration later in the release.

Performance. If we find that the overhead of the additional exec(2)—one for verexec(1) and one for the link target—is too high, there are a number of performance optimizations. One approach would be to use verexec(1) to send a message to a privileged daemon, which would loopback mount the appropriately versioned executable at the verexec(1) location. Any loopback mount-based technique makes a trade-off: more efficient execution for more complex handling of updates and deletions, which would require the daemon (or privileged command or service) to inspect the existing loopback mounts and evaluate them for their continuing correctness.

I've a refinement-lacking prototype of verexec(1) available for review; we're discussing it further at pkg-discuss@opensolaris.org.

[ T: ]

Tuesday Jul 07, 2009

MPK17: Outcome of a long discussion?

The coffee stations in MPK17 are popular choices for technical discussions. Some go longer than others, and occasionally result in byproducts. I wouldn't want to hazard a guess about how long a discussion resulted in this byproduct, which popped up sometime at the end of last week:

Many stacked disposable cups

I wonder if the original discussion ever resolved...

[ T: ]

Tuesday Apr 21, 2009

2009.06: A netbook for meetings

It's been a busy few months, but things are clearing up now.

The past few weeks, I've been using an Acer Aspire One for coverage when my trusty Toshiba r500 is recharging. It's my computer for meetings, I suppose. The Aspire One is a great little netbook, and 2009.06 should recognize every device on the system. Here's an action shot of this Atom-based system on my file cabinet:

Acer Aspire One action shot

The window shown in the display is the Device Detection Utility—the absence of red bars in that list indicates that every hardware component on the system is known to OpenSolaris. The main limitations of a standard single core, dual thread (but dual-core) Atom netbook like this are vertical screen real estate and memory usage. In the screen shot, I've moved the GNOME panels to the left and right sides of the display; I've also disabled unneeded services by using svcadm(1M).

If you've an Aspire One, or a similar netbook, you might want to give 2009.06 a spin. The usbcopy utility, from the SUNWdistro-const package, can be used to prepare a bootable USB stick to use for the installation.

[ T: ]

Tuesday Dec 23, 2008

2008.11: terminator is my favourite application

There's a lot of new applications in 2008.11—cheese is useful for testing your USB webcam and making self-portraits for avatars and profile pictures and I'm very intrigued by gobby—but I already have my favourite: terminator. Here's a scaled screenshot:

terminator thumbnail

I used to use ion as my window manager, so getting some tiling back is great. A side effect of those days is that I expect F2 to start a terminal, which is easy with a little gconf editing:

$ gconftool-2 -s /apps/metacity/global_keybindings/run_command_1 \\
    -t string 'F2'
$ gconftool-2 -s /apps/metacity/keybinding_commands/command_1 \\
    -t string '/usr/bin/terminator'

There's a little bug in handling the title bar I'd like to see fixed, but I can ignore that in return for greater terminal density.

Update: Col likes it, too.

[ T: ]

Wednesday Dec 10, 2008

2008.11: A roundup

With the launch over, I took a few minutes to read through today's blog entries, and pulled out a list of the either technical or personally significant entries. If you're looking for the highlights, Calum's overview seems to be a great place to start. He's got links to the best summaries and screencasts.

  • First, if you're running 2008.05 and considering upgrading—recommended strongly—please watch Markus's screencast. There are one or two tricky steps to pay attention to. (We got caught offguard by some misinteractions between GRUB, ZFS, and early startup; won't happen again.)
  • Erwann's screencast is a great introduction to Time Slider, and I'm pretty confident no turkeys were harmed during filming.
  • Peter has screen captures of top(1) on a couple of older laptops, including one with 447 MiB available to the OS. Dave and the Install team worked hard to keep 512 MiB systems booting on the release; here's evidence.
  • Doug's pkgfactory/roboporter discussion and Nico's explanation of the repositories show an extremely promising way to make sure there's a wide variety of software available for OpenSolaris—make the computers do the grunt work. Doug and Nico, and Jim Walker, have been pushing hard on improving porting efficiencies.
  • Knut explains using JavaDB as a managed service on 2008.11. Since Knut's post combines pkg(5) and smf(5) invocations, it easily wins a spot in any summary I might write. (And so it appears in this one.)
  • Jerry describes the foundational work for integrating zones, Snap Upgrade, and pkg(5). The changes to filesystem layout to support that integration place limitations on zones across the 2008.05 to 2008.11 transition, so be sure to review Jerry's post if you've been experimenting with ipkg zones on the release.
  • John reviews 2008.11 as a suitable paravirtualized guest on xVM, with some careful discussion of how to handle the graphical console.
  • Shawn's the sole member of the image packaging team to get a technical entry up. Shawn talks about the depot refacing work he's done and planning on doing for 2009.next. (Shawn modestly omits the fact that he also implemented "pkg history" and operational intent in the client.)
  • Finally, I'm giving Fintan the nod for best screenshot, even if I prefer to see this release on the metal.

There are a lot of entries under tagged with opensolaris; you'll find coverage of many other aspects of the release.

[ T: ]

2008.11: Live chat action

We're just wrapping up the live chat for the launch, with interested folks moving to freenode in #os200811. Here's a quick photo of Danek, Dave, and Bart responding intently to the questions and comments earlier this morning.

Launch chat action

As you can see, every question is taken seriously...

[ T: ]

Tuesday Dec 09, 2008

2008.11: Everyday projectors

I regularly use my for presentations and demonstrations, which in the past has involved some configuration gymnastics. If your laptop has NVIDIA graphics, the "NVIDIA X Server Settings" Preferences panel lets you configure for most projectors, but if you didn't have an NVIDIA graphics adaptor, configuration was arduous and fragile. With 2008.11, we get GNOME 2.24; the 2.24 Screen Resolution panel, reachable from the desktop background menu, makes configuration accessible to a wider range of adaptors, including the Intel graphics in my Toshiba r500.

So I made another little film:

Again, you can retrieve this video directly from channelsun.

[ T: ]

2008.11: More ways to get it, and more ways to share it

As I did for 2008.05, I'll collect links to mirror sites here. These will also get links on the various download pages out there.

Construction of 2008.11 was faster, more efficient, and generally more predictable than 2008.05. Image packaging, snap upgrade, and the distribution constructor—and the notorious distro importer—saw many fixes and features that resulted in the ability to rapidly turn out corrected images and packages. As problems were found during testing, we could spin up new test variants in hours or less. (Thus, the existance of RC1.5, which we were able to inject into the schedule because of this newfound speed. Dave's written more about the constructor.) You can get the ISO image, suitable for burning to a 700 MiB CD or immediate use in virtual environment, directly from the following locations:

As Glynn notes, we've set up anonymous rsync to make acquiring and mirroring the ISO images. In addition, I've posted draft instructions on how to be a content mirror for pkg.opensolaris.org/dev. We've tested the pkg(5) content mirroring, as well as the rsync service, but would be happy to get feedback on pkg-discuss-AT-opensolaris.org.

These links either take you to the site's page or directly to the osol-0811.iso CD image, which contains 11 "primary languages". It installs quite a bit faster, particularly on systems with slower CPUs. There is also an LZMA-compressed image, which has localization support for 42 languages, including those primary ones. It's available from dlc.sun.com, genunix.org, and as a torrent. (Consult the language lists for specifics.)

[ T: ]

Wednesday Nov 05, 2008

2008.next: demonstrating suspend-resume

Randy has been blogging on the suspend-resume feature and how it's been improving on a variety of x86-based systems. A few weeks ago, I was sitting on the USGS Menlo Park campus, and decided to record a demonstration of my own laptop being suspended and resumed multiple times:

(You can also access the video on channelsun.)

The demonstration is using Build 99, so there are a few differences and improvements in the release candidates soon to be published. With careful timing of your battery discharging and recharging, you can shoot for uptimes in days to weeks.

I'm also demonstrating my ignorance of all video-making rules, in that I used a Panasonic Lumix LX-2 in movie mode, outdoors, near traffic, and then edited the video using OpenSolaris and open source tools. But I'll note that, even in real life, you'll have to peer at your r600 closely to see the orange suspension indicator.

[ T: ]

Sunday Nov 02, 2008

2008.next: Recovering a dropped data pool

As you update your pkg(5)-based system to build 100a, you might see a message on console during startup. It will look something like

WARNING: pool 'zeta1' could not be loaded as it was last accessed by
another system (host: rosseau hostid: 0x581d45e). See:
http://www.sun.com/msg/ZFS-8000-EY

and in /var/adm/messages, you'll find it again:

Oct 27 14:55:21 rosseau zfs: [ID 427000 kern.warning] WARNING: pool
'zeta1' could not be loaded as it was last accessed by another system
(host: rosseau hostid: 0x581d45e). See:
http://www.sun.com/msg/ZFS-8000-EY

zeta1 is a data pool on my system—the boot pool, rpool, upgraded normally—and recent changes in the hostid implementation mean that we have to import (or reimport) any non-boot pools on the system. That is,

$ pfexec zpool import -f zeta1

and our zeta1-based filesystems are again online.

[ T: ]

Thursday Sep 18, 2008

2008.next: Easy install for docutils

It's been a busy summer, mostly trying to manage conflicting schedules at home while staying loyal to the six month release schedule for OpenSolaris at work. Schedules everywhere, it seems.

I've been probing deeper into the ecosystem around Python, and have been trying to determine whether restructured text might have some value in the workflow around OpenSolaris development: it can output to a variety of formats, including XML, LaTeX, and HTML, and doesn't have an onerous list of dependencies.

I've scars from building AsciiDoc 8.

docutils isn't available via image packaging yet, but it's easily retrievable using easy_install. That means the question is: how do I get easy_install? I'm running Build 98, so I'll restrict my search to packages on the development branch for that build:

$ pkg search -r easy_install | grep 0.98
basename  file  usr/bin/easy_install pkg:/SUNWpython-setuptools-AT-0.5.11-0-DOT-98

Okay, easy_install is part of setuptools—you probably knew that already. Let's install the appropriate package using pkg(1):

$ pfexec pkg install SUNWpython-setuptools
DOWNLOAD                                    PKGS       FILES     XFER (MB)
Completed                                    1/1       81/81     0.30/0.30 

PHASE                                        ACTIONS
Install Phase                                  93/93 
PHASE                                          ITEMS
Reading Existing Index                           9/9
Indexing Packages                                1/1
$ which easy_install
/usr/bin/easy_install

Having setuptools in place, we can now attempt installs of any Python module available in the Python Package Index (pypi), via the easy_install command. (easy_install --help will display a usage message for easy_install.) Let's install the docutils package:

$ pfexec easy_install docutils 
Searching for docutils
Reading http://pypi.python.org/simple/docutils/
Reading http://docutils.sourceforge.net/
Best match: docutils 0.5
Downloading http://prdownloads.sourceforge.net/docutils/docutils-0.5.tar.gz?download
Processing docutils-0.5.tar.gz
Running docutils-0.5/setup.py -q bdist_egg --dist-dir /tmp/easy_install-57yD32/docutils-0.5/egg-dist-tmp-Ip1xgp
"optparse" module already present; ignoring extras/optparse.py.
"textwrap" module already present; ignoring extras/textwrap.py.
zip_safe flag not set; analyzing archive contents...
docutils.parsers.rst.directives.misc: module references __file__
docutils.writers.html4css1.__init__: module references __file__
docutils.writers.pep_html.__init__: module references __file__
docutils.writers.s5_html.__init__: module references __file__
docutils.writers.newlatex2e.__init__: module references __file__
Adding docutils 0.5 to easy-install.pth file
Installing rst2xml.py script to /usr/bin
Installing rst2pseudoxml.py script to /usr/bin
Installing rst2html.py script to /usr/bin
Installing rst2s5.py script to /usr/bin
Installing rst2latex.py script to /usr/bin
Installing rstpep2html.py script to /usr/bin
Installing rst2newlatex.py script to /usr/bin

Installed /usr/lib/python2.4/site-packages/docutils-0.5-py2.4.egg
Processing dependencies for docutils
Finished processing dependencies for docutils
$ which rst2html.py
/usr/bin/rst2html.py

And now I can start experimenting with restructured text.

[ T: ]

Sunday May 04, 2008

CommunityOne: here and there

Sunday I spent at Moscone, teaching laptops and projectors to get along. Saturday, Nathaniel, Benjamin, and I dropped in for lunch at the Developer Summit. I managed to talk with a few people before the boys found large whacking sticks, and then it seemed best to drive to Pescadero for some beach time.

I'll be busy for the morning of CommunityOne. For Rich's keynote, I'll be running some of the less violent demos. Almost immediately after that, lead modernizer David Comay and I will be going into more detail in our session

S297399 Getting Started with OpenSolaris™; New Features & Building OpenSolaris™ Packages; 11:00 a.m., Moscone South/Esplanade 300.
We've got some additional demonstrations, including a worked example of package publication using some pre-release tools, which could be exciting.

I'm hoping to have some time for questions during the session but, if not, I'll be circulating during the afternoon and happy to talk to people about 2008.05, image packaging, or whatever. And, of course, there will be time to talk at the party after the day's sessions. In any case, I should be easy to spot: I have a new tie.

I'm told you can still register on-site—it's not too late.

[ T: ]

2008.05: More ways to get it

As I did for the preview releases, I'll collect links to mirror sites here. These will also get links on the various download pages out there.

Bart and I just finished updating the package repository with the new packages we'd received the past few weeks, and that means 2008.05 is out the door. (Thanks to everybody who tried the release candidate, filed a bug, shuffled a package, wrote or proofread a document, or just spent energy anticipating the bits.) You can get the ISO image, suitable for burning to a 700 MiB CD or immediate use in virtual environment, directly from the following locations:

Reading the logs, and talking with some of our mirror sites, we know we all served out a lot of downloads for the previews; if you're interested in being a mirror, please let me know. (2008.05 remains completely redistributable.) We're using a bigger download complex this time, but every mirror helps.

If you already downloaded and installed Preview 2, it's more complete, easier, and probably faster to update directly using image packaging: see the update guidelines. These instructions involve a small script to safely update a clone of your installed system, and then switch to that on a system reboot. (If you were running Preview 1, you should update that to Preview 2, and then go on to 2008.05.)

Update 1: My thanks to Tobias Lundquist, who's once again mirrored via FTP and HTTP (Internet 2) in Sweden.

Update 2: My thanks to Luca, who's put up an HTTP mirror in Romania.

Update 3: Simon Valiquette has put up an HTTP mirror (Internet 2) at the Université de Sherbrooke in Québec, Canada. Merci, Simon.

Update 4: Bart Muijzer has arranged for mirrors by the Netherlands Unix Users Group (NLUUG) and surfnet.nl. Thanks, Bart.

These links are for the gzipped CD image, which contains the 12 "primary languages". It installs quite a bit faster, particularly on systems with slower CPUs. There is also an LZMA-compressed image, which has localization support for 42 languages, including those primary ones. It's available from dlc.sun.com, genunix.org, ftp.df.lth.se, mirrors.xservers.ro, and as a torrent. (Consult the language lists for specifics.)

[ T: ]

About

sch

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
External blogs