Monday Mar 16, 2009

faster imports

Steve Peng has done something really cool. He's taken the slowest part of the first-boot experience and made it many \*times\* faster than it was. This happened quite a while ago in Nevada/OpenSolaris, so lots of folks have probably already noticed, but I wanted to take a few moments and call it out explicitly.

In onnv_84 (in February, 2008), Steve integrated the fix for

   6351623 Initial manifest-import is slow

During the first boot of Solaris, SMF doesn't know yet what manifests have been populated on the system, so it imports everything below /var/svc/manifest. On systems with slow disks, this can take a long while, but generally folks are used to (and frustrated about!) seeing the slow progress of:

   Loading smf(5) service descriptions: 36/189

During that time, SMF is carefully taking all the manifests from /var/svc/manifest and committing the data to persistent storage. Which takes a while because we weren't doing a bulk import, and instead doing every single property update as a separate transaction to sqlite, each of which needed to commit to disk, which can be very slow.

Steve changed things to instead do the import into tmpfs, where commits are fast because they go to memory rather than all the way to disk, and then switch the repository back to using persistent storage after the imports have completed.

This is a huge deal for performance of the manifest-import service. It significantly improves performance for the first boot after install, for the first boot of virtualized (zones, XVM, VirtualBox, etc.) deployments, and on system upgrades.

In a quick and dirty test on a bare-metal Solaris install, I found manifest-import was 3.15 times faster at importing the ~189 manifests currently included with SXDE. Steve's seen even better performance on many machines -- more like 6.6 times faster!

For those deploying diskless clients, the change in first-boot performance should be nothing less of remarkable.

Thursday Jul 26, 2007

zone out and speed up your development cycles

The other day, as an aside in a conversation about how often developers use certain OS features, I was asked how often I use Solaris Zones. At least weekly, and daily if I'm lucky enough to be spending my time on code.

Surprised? Userland developers on Solaris shouldn't be.

I spend a great deal of time modifying libraries and daemons started really early in the Solaris boot process. While SMF tries to dump you at an sulogin prompt if you've introduced a bug, it's still a bit of a pain to recover from some nasty failure or hang you've coded into init(1M), svc.startd(1M), or libscf(3LIB). Zones make the deploy and reboot cycle go really really fast, and recovery from late-night programming errors is a breeze!

Here's what I do:

  • Create a whole root zone. Takes up more disk space, but allows me to replace any system binary:

            # zonecfg -z test
            test: No such zone configured
            Use 'create' to begin configuring a new zone.
            zonecfg:test> create -b
            zonecfg:test> set zonepath=/test-zone
            zonecfg:test> commit
            zonecfg:test> exit
            # zoneadm -z test install
            # zoneadm -z test boot
            # zlogin -C test
               ... answer sysid questions, and log in 
    
  • Make sure the bits I'm compiling are relatively close to the bits installed on my desktop. Live Upgrade and I are close friends so that I can keep my desktop as up-to-date as my workspaces.

  • Create a script to dump my modified bits into the zone root. Something like this fragment:

            gate="/net/coupe/builds/lianep/templates/usr/src"
            zone="/test-zone/root"
            if [ ! -d $zone ]; then
                echo "no zone root here: $zone"
                exit 1
            fi
            cp  ${gate}/lib/libscf/i386/libscf.so.1 ${zone}/lib
            cp  ${gate}/lib/libscf/amd64/libscf.so.1 ${zone}/lib/amd64
    
  • Compile, run script, test, debug, fix, repeat.

After my code is all basically working, then I move on to testing on the bare metal. But, the fast reboot times of zones and the easy ability to replace a broken library with a library broken in a new and different way is invaluable to making very rapid progress. I've rebooted my zone at least 15 times today. Compiling the library takes longer than the zone deployment and reboot! Every few months I mismatch libraries and commands from different workspaces and foul up my zone badly enough that it needs to be re-installed. But, zoneadm -z test uninstall; zoneadm -z test install provides a convenient excuse to go get coffee and then I'm back in business.

Technorati Tags: , , , and .

About

Liane Praza

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today