Live Upgrade on a Laptop ?

It is fascinating that one of the most interesting features in Solaris isn't new - in fact it's been around for quite some time. I'm referring to Live Upgrade which allows the installation or upgrade of an alternate boot environment while you are doing productive work in another. And why would I want to do this, especially on a laptop ?

I can think of several reasons.

Live Upgrade can be completely driven from ISO disk images of installation media. This means that I don't have to burn DVDs or CDs, or even better - no more late night runs to CompUSA or Fry's before the big presentation in the morning. And I won't miss the sound of the DVD reader grinding itself to death every time I install Solaris.

It simplies running multiple versions of Solaris, significantly. One of the most frequent questions we encounter in Installfests is which OS to run ? Solaris 10, OpenSolaris Community Edition, Solaris Developer Express, the nightly OpenSolaris BFU. And the answer is all of them. OK, maybe not for everyone - but even if you are going to choose one OS, upgrading to the newest update or applying a set of patches safely, while keeping the older environment as a fallback is a good thing, especially if you are on the road.

And best of all, Live Upgrade is quite simple as long as you follow the rules - and fortunately there are only a few.

The plan

As with most things, the critical first step is the plan - and the earlier the better. For Live Upgrade this means the disk layout prior to installation. You will have to reserve some space for your alternate boot environment(s) when you initially install Solaris. While not strictly true, the mobile laptop user with a single disk drive will find the task of relocating data and repartitioning your disk after the fact rather complicated. In other words, you are better off planning in advance.

Here is an example of a multi-boot partition table that has been designed for Live Upgrade.

So how many boot environments do you really need ?

I use three - one for my customer facing Solaris 10 environment, one for my Open Solaris Community Edition daily driver, and one for whatever super-cool-gotta-have OpenSolaris project that is only available via BFU (today that is Xen, but it has been other things in the past and will likely be so in the future). And we all know that BFU means Better Forget Upgrades - but I have found that Live Upgrade is a simple way of preparing for a BFU. This third boot environment is also used as a backup during major Solaris updates (yeah, I would really like a 4th and 5th, but at some point you have to stop). Three is good.

How big do these need to be ?

That depends on how much stuff you carry around - specifically how much /opt will grow. There are some techniques you can use to reduce the storage requirements (sharing /opt directories across boot environments), but these tricks make other things more complicated (like zone creation). 6GB makes a good root file system, perhaps a bit larger if you start installing packages from Blastwave. Solaris Developer Express with the Studio development tools and Netbeans environment slightly larger - 8GB to 10GB.

Familiarization and Reading the Furnished Materials

The next step is to familiarize yourself with the process. There is a wealth of information to be found in the Solaris Documentation on the web, specifically Solaris Live Upgrade and Upgrade Planning in the Release and Installation Collection. Even more important is Infodoc 72099 which lists all of the patches required for Live Upgrade. An excellent community reference is Derek Crudgington's Live Upgrade for Idiots.

At first glance you may find the documentation a bit intimidating. Perhaps this is why more people aren't adopting Live Upgrade in their environment. Live Upgrade can do all sorts of spectacular things such as installing from a flasharchive, splitting and joining file systems, breaking Solaris Volume Manager mirrors - none of which apply to our current situation. In fact when you strip Live Upgrade down to the few parts that we will be using, it is amazingly simple. Here is an outline of the steps you will perform (after thoroughly reading the documentation - I have to say that).
  1. Check Infodoc 72099 to make sure all of the prerequisite patches are installed (and remember to do this each time as Live Upgrade is an evolving product).
  2. Create the new boot environment if it doesn't already exist. You can name your current boot environment the very first time you do this.
  3. Download the DVD ISO image for the new Solaris release or update
  4. Mount the ISO disk image
  5. Install the packages SUNWluu, SUNWlur (and if present, SUNWlucfg) from the installation media (in Solaris_nn/Product) in your current boot environment. This is a critical step - incorrect Live Upgrade software is the most common cause of failure.
  6. Run luupgrade -u to upgrade your new boot environment
  7. Check the upgrade logs in /var/sadm/system/logs in the new boot environment. The boot environment is mounted as /a during the upgrade, or you can use lumount after the upgrade is complete.
  8. Activate the new boot environment using luactivate
  9. init 0 (also critical as K0 RC scripts are installed to perform important steps like updating your bootloader configuration, syncing system files, and building a good boot archive.
It's really that simple. Let's see this all in action.

Preparing for Live Upgrade

I'm starting with a Solaris 10 6/06 system, freshly installed with my root file system (aka the primary boot environment) in c0d0s3. From my disk partition table you will see that I have two other boot environments available: c0d0s0 and c0d0s4. I plan on putting Solaris 10 11/06 in c0d0s0 and the latest OpenSolaris Community Edition (aka Nevada) in c0d0s4.

Since I've already read the documentation a few hundred times, I can proceed to patch analysis.

After comparing the patch list in Infodoc 72099 with what I have in Solaris 10 6/06 (showrev -p), I see that I am behind on four patches: 119255, 118855, 112653, and 121003. I fire up our new best friend updatemanager and see that I also need 121119 and 119255. These two will point updatemanager at the new repository.

So I apply 121110-08 and 119255-27, and restart updatemanager. This lets me find the rest of the patches that I will need.

I then apply 119255-32, 118855-19, 112653-04 and 121003-03. None of these required a reboot, so we can proceed immediately to the next step.

Creating the new boot environment

Since I will be upgrading from media and not a flash archive, I have to create a complete new boot environment, based on my current installation. This step will essentially clone the current system. As a side note, there are lots of fascinating uses for this that we will explore later.

Since this is the first time I am using lucreate, I also get to name my current boot environment. Future invocations of lucreate will be done without the -c argument.

# lucreate -c sol10u2 -n sol10u3 -m /:c0d0s0:ufs
This magic sequence told Live Upgrade to name the current boot environment sol10u2 and to create a new one called sol10u3 with the new UFS root file system on /dev/dsk/c0d0s0.

One more time, with output.
# lucreate -c sol10u2 -n sol10u3  -m /:/dev/dsk/c0d0s0:ufs
Discovering physical storage devices
Discovering logical storage devices
Cross referencing storage devices with boot environment configurations
Determining types of file systems supported
Validating file system requests
Preparing logical storage devices
Preparing physical storage devices
Configuring physical storage devices
Configuring logical storage devices
Analyzing system configuration.
Comparing source boot environment  file systems with the file
system(s) you specified for the new boot environment. Determining which
file systems should be in the new boot environment.
Updating boot environment description database on all BEs.
Searching /dev for possible boot environment filesystem devices

Updating system configuration files.
The device  is not a root device for any boot environment.
Creating configuration for boot environment .
Source boot environment is .
Creating boot environment .
Checking for GRUB menu on boot environment .
Creating file systems on boot environment .
Creating  file system for  on .
Mounting file systems for boot environment .
Calculating required sizes of file systems for boot environment .
Populating file systems on boot environment .
Checking selection integrity.
Integrity check OK.
Populating contents of mount point .
At this point all of the system files from the current root file system are being copied to the new boot environment. On this laptop and with a rather complete Solaris install, it takes about 45 minutes.

Upgrading Live Upgrade

If you have not already done so, download and unpack the DVD ISO image for Solaris 10 11/06 (or whatever media you wish to use).
# lofiadm -a /export/iso/sol10u3/solarisdvd.iso
# mount -o ro -F hsfs /dev/lofi/1 /mnt
# pkgadd -d /mnt/Solaris_10/Product SUNWluu SUNWlur        (Remember to install SUNWlucfg if present)
And answer Y when asked if you wish to overwrite existing files.

Perform a Live Upgrade

We will now run luupgrade(1m) to upgrade the new boot environment. Since Live Upgrade doesn't support non-global zones you should delete all installed zones in the target boot environment. This restriction is relaxed in the latest OpenSolaris snapshots. Since I'm starting with a clean Solaris 10 6/06, there are no non-global zones, so I can proceed.
# luupgrade -u -n sol10u3 -s /mnt
This magic sequence says to run upgrade from the media source /mnt (mounted in an earlier step) against the boot environment sol10u3. The process will go something like
  1. Extract the miniroot from the source media
  2. Mount the new boot environment as /a
  3. Remove all of the packages from the new boot environment
  4. Install new packages from the source media
  5. Install a new boot archive
  6. Unmount the new boot environment
On this laptop the process takes about 2 hours. During the upgrade you can watch the process by doing something like
# tail -f /a/var/sadm/system/logs/upgrade_log
When the upgrade is complete, check the logs for any failed package installations. Some are harmless, but all should at least be investigated. Also look at /var/sadm/system/data/upgrade_cleanup for a list of files that had conflicts, and what Live Upgrade did to resolve them. Most of these are harmless, but you should check to see if anything looks out of place.
# more `lumount sol10u3`/var/sadm/system/logs/upgrade_log
# luumount sol10u3
# more `lumount sol10u3`/var/sadm/system/data/upgrade_cleanup
# luumount sol10u3

Activating the new boot environment

The last step is to activate the new boot environment. This is all done by a K0 script, so after activation it is imporant that you use shutdown or init 0. Do not use reboot as this bypasses the normal shutdown process and the K0 scripts will not run. This is also an FAQ!
# luactivate sol10u3
# init 0
At this point some serious magic will be invoked. The K0 script will update your bootloader configuration (in this case /boot/grub/menu.lst on the boot partition), a new boot archive will be built (just in case), and commonly modified system files will be synced. This last step is driven by /etc/lu/synclist.

Extremely important GNOME warning: GNOME configurations are not generally compatible across releases. If you are switching between Solaris 10 and Nevada, OpenSolaris, or Solaris Developer Express you are advised to maintain separate home directories so that your GNOME configurations (.gnome, .gnome2, .gconf) don't collide. If you choose different Solaris users then you are done. If you are just changing your home directory in /etc/passwd then Live Upgrade will gladly send your old directory setting to your new boot environment - and that's not at all good. Either remove /etc/passwd from the file synclist or remember to log in via command line or failsafe session once you've activated and booted the new boot environment and manually change /etc/passwd.

And that's really about it. Recovery is rather simple. After activation you will have all of your boot environments available for selection. Skipping the activation step isn't recommended as it bypasses various sanity checks and the file sync process, but when recovering a broken boot environment I think I can deal with things like old passwords.

Practice makes perfect

One more time, this time putting a nevada build 57 in c0d0s4. The only change here is that nevada adds a new LU package: SUNWlucfg. The magic sequence looks something like
# lucreate -n nevada57 -m /:c0d0s4:ufs
# mount -o ro -F hsfs `lofiadm -a /export/iso/nevada57/solarisdvd.iso` /mnt
# pkgadd -d /mnt/Solaris_11/Product SUNWluu SUNWlur SUNWlucfg
# luupgrade -u -n nevada57 -s /mnt
# luactivate nevada57

Upgrading existing Boot Environments

Once you have used all of your boot evironments your next upgrade will require you to either delete one of them and start again or upgrade an existing boot environment. The differences are minor (ludelete and lucreate versus lurename and lumake), but there are a couple of things have to consider - if you don't then this will come back to haunt you wnen you least expect it.

Since there is no livedowngrade, you can only upgrade from an older version to a newer version. The implication is that you can't use Live Upgrade to install Solaris 10 from a Solaris 11 based system (Nevada, Solaris Express Community Edition, Solaris Express Developer Edition). That's not exactly correct, you can install a Solaris 10 from a flasharchive, but that's an installation, not an upgrade.

Now you know why I have three boot environments. I leave an old Solaris 10 around as a Live Upgrade source and then alternate the other two between whatever is the most interesting - today that is Solaris 10 11/06 as the legacy and Solaris 10 8/07 and Nevada in the ping pong set. Now that Solaris 10 8/07 has been released, it will become the legacy Solaris environment and I will go back to alternating between Nevada builds with and without xVM.

Another consideration, and common source of problems, is that Live Upgrade expects that the target enviroment match the source environment. As an example, lets assume that I am driving a Live Upgrade from Solaris 10 to one of my Nevada ping pong environments. It seems attractive to just upgrade one of the boot environments as it is - but it doesn't match the source. Some of the time, even most of the time, this will seem to work - but there will be times when drivers will not get replaced and you will have a hard to debug mess. The simple solution is to remember to lumake before you luupgrade.

One last thing - remember to check back to Infodoc 72099 as it does change occasionally - generally as a result of a Solaris release. But do check back to make sure your patch levels are correct.

One last example - my current boot environment is s10u3 and it is running Solaris 10 11/06. I have an nv69 and nv71 running Solaris Express Community Editions 69 and 71. I want to put Solaris 10 8/07 in the nv69 boot enviromnent and call it s10u4. After installing patches 119255-42, 121429-08, 126539-01, and 125419-01, the magic sequence goes something like

# lurename -e nv60 -n s10u4
# lumake -n s10u4
# mount -o ro -F hsfs `lofiadm -a /export/iso/s10u4/solarisdvd.iso` /mnt
# pkgadd -d /mnt/Solaris_10/Product SUNWluu SUNWlur SUNWlucfg
# luupgrade -u -n s10u4 -s /mnt
# luactivate s10u4
# init 0
And when a new Solaris Express Developer Edition or the next update of Solaris 10 is available, just repeat all of the steps above. I hope that this helps you understand an amazing Solaris capability and that you start using this on all of your systems, including your laptop.

Technocrati Tags:

Excellent write-up ! If someone needs handy slides as a reference for LU here they go:

Posted by Vladimir Kotal on March 05, 2007 at 07:05 PM CST #

Bob, thanks a lot for the info!

Unfortunately, the instructions does not seem to work with Nevada build 63...

Trying to upgrade from Solaris 5.10 Generic_125101-06 to Nevada build 63, I consulted infodoc 72099 and installed all the necessary patches. Then I did:

lucreate -n sol-nv-b63-x86 -m/:c0d0s7:ufs
pkgadd -d /mnt/Solaris_11/Product SUNWluu SUNWlur SUNWlucfg
luupgrade -u -n sol-nv-b63-x86 -s /mnt

which resulted in error message:

Copying failsafe kernel from media.
Uncompressing miniroot
Creating miniroot device
miniroot filesystem is <ufs>
Mounting miniroot at </mnt/Solaris_11/Tools/Boot>
Validating the contents of the media </mnt>.
The media is a standard Solaris media.
The media contains an operating system upgrade image.
The media contains <Solaris> version <11>.
Constructing upgrade profile to use.
Locating the operating system upgrade program.
Checking for existence of previously scheduled Live Upgrade requests.
Creating upgrade profile for BE <sol-nv-b63-x86>.
ERROR: You must install the Zones tool patches to use this set of Live Upgrade
packages to manipulate the boot environment at </>.
ERROR: Unable to create ICF file for boot environment <sol-nv-b63-x86>.
ERROR: Unable to determine root slice for BE <sol-nv-b63-x86>.

Posted by andrei on May 18, 2007 at 02:21 PM CDT #

This is because you have non-global zones either configured or installed. Detach your zones using zoneadm detach and then remove the configurations with zonecfg delete. Once your non-global zones are gone then you can use Live Upgrade. Once your luupgrade is complete your non-global zones can be reattached (zonecfg create -a and zoneadm attach). This step will no longer be required once all of the zone live upgrade pieces get delivered into Solaris 10.

Posted by guest on May 18, 2007 at 02:44 PM CDT #

I upgraded from Solaris 10 5/09 s10s_u7wos_08 SPARC
to Solaris 10 10/09 s10s_u8wos_08a SPARC

but when I booted the new BE, it gave me a message: loading 21 of 21 service descriptors - Why so few? Why only 21?

/usr which was a different FS was integrated into / and the swap slice remained the same.......

Posted by Dariusz Dolecki on October 19, 2010 at 07:42 AM CDT #

Post a Comment:
  • HTML Syntax: NOT allowed

Bob Netherton is a Principal Sales Consultant for the North American Commercial Hardware group, specializing in Solaris, Virtualization and Engineered Systems. Bob is also a contributing author of Solaris 10 Virtualization Essentials.

This blog will contain information about all three, but primarily focused on topics for Solaris system administrators.

Please follow me on Twitter Facebook or send me email


« April 2014