The Xen of DomU Installation

More virtualization from me, moving on from the emulated Linux environment provided by Solaris Containers for Linux Applications, to Xen!

Let's say you've got this brand spanking new OpenSolaris release that you've downloaded, configured and installed on your machine. Now what?

Well, the whole point of virtualization is to allow you to run a guest OS on your machine, just as if you had two (or more) machines, so let's walk through the installation of a guest domain, otherwise known as a DomU, on your machine running Xen, otherwise known as a Dom0.

To make things as easy as possible, I'll assume that your machine, like most today, does not have hardware support for running a guest OS (otherwise known as a HVM DomU.) Instead, let's install a paravirtualized guest. As I'm biased, let's install a second copy of OpenSolaris.

The provided documentation explaining how to do an install is pretty complete, so I'm not going to rehash the details here except to consolidate a few potential gotchas from the Release Notes and the configuration pages:

  1. If you install a DomU with the OpenSolaris installer's default settings, which were designed to allow installation on a system with a minimal configuration, you will not be able to upgrade the instance to a newer release of OpenSolaris later because of a lack of disk space.

    To avoid this situation, you should make sure to create a root ("/") partition of at least 1 GB in size, or 1.5 GB if you want to play it safe.

  2. Xen currently gets really cranky if it runs short on memory. Symptoms can range from simply refusing to boot a guest domain to madly trying to resize Dom0 to find memory to boot the DomU, with the result that your Dom0 may become catatonic for a long period of time, refusing to respond to inputs and even closing down network connections until it determines that it can't in fact shrink the Dom0's memory space to accommodate the DomU's memory requirements. Not good.

    Instead, it's best to avoid this situation by using a rule of thumb, namely that the combined memory usage of all DomUs you intend to run concurrently should never exceed approximately 55% of the physical RAM installed in your machine. Pretty straightforward stuff.

OK, now it's time to put that advice into action and install your first guest operating system instance.

There are two different interfaces you can use to create and run guest operating systems, xm, and virt-install. The documentation I've linked to above all references xm, so I'll stick with that method.

First, as in the documentation, I'll create a simple Python configuration file containing information regarding the DomU I want to install.

For my sample, install, I'm going to save the OpenSolaris ISO image I downloaded to /export/images/66-0624-nd.iso.

Once I've done that I will create a little infrastructure as I like to keep my disk images, Python config files for installation and Python config files for booting installed DomUs separate.

Feel free to come up with your own hierarchy; I like:

  • /export/images - ISO disk images
  • /export/config/install - Python config files for installation
  • /export/config/run Python config files for installed DomUs
  • /export/root - Image files used as file systems for DomUs
So I'll create the Python configuration file /export/config/install/, containing:
    name = "solaris-install"
    memory = "1024"
    disk = [ 'file:/export/images/66-0624-nd.iso,6:cdrom,r', 'file:/export/root/sola
    ris_domu.img,0,w' ]
    vif = [ '' ]
    on_shutdown = 'destroy'
    on_reboot = 'destroy'
    on_crash = 'destroy'
This file basically states that I want to boot the ISO file /export/images/66-0624-nd.iso as if it were a boot disc (in this case, a DVD), and that I want to use the disk image file /export/root/solaris_domu.img as the "hard disk" for my DomU. It also states that in case of a shutdown, reboot or crash, I just want to xm destroy the domain - no muss, no fuss.

Next, I'll create the file to be used as the DomU's "hard disk" using dd; I think 8 GB makes for a nice compact "drive" size for a DomU:

    dd if=/dev/zero of=/export/root/solaris_domu.img bs=1024k seek=8k count=1

Feel free to use whatever size you'd prefer; you can also use an entire dedicated disk partition or a zpool. The aforementioned documentation will tell you how.

Now it's time to create the new DomU using the OpenSolaris installer, via the xm create command:

    xm create -c /export/config/install/

When you run this command, you'll see a text-based OpenSolaris installer, the same as if you were installing your Dom0.

Simply follow the installer's prompts, the same as if you were installing a new machine, except you're really installing the new guest domain.

To make things easier, I suggest that you tell the installer to not eject CDs automatically when the installation is complete; that way you can walk away from your install and come back later and see that the install completed the way you want:

    - Eject a CD/DVD Automatically? ------------------------------------------------
      During the installation of Solaris software, you may be using one or more
      CDs/DVDs. With the exception of the currently booted CD/DVD, you can choose
      to have the system eject each CD/DVD automatically after it is installed or
      you can choose to manually eject each CD/DVD.
      Note: The currently booted CD/DVD must be manually ejected during system
                [ ] Automatically eject CD/DVD
                [X] Manually eject CD/DVD
         F2_Continue    F3_Go Back    F5_Exit

When the installer gets to the point where it detects disks, it should see your specified disk file/partition/zpool as a "hard drive" of the size you specified named c0d0:

    - Select Disks -----------------------------------------------------------------
      On this screen you must select the disks for installing Solaris software.
      Start by looking at the Suggested Minimum field; this value is the
      approximate space needed to install the software you've selected. Keep
      selecting disks until the Total Selected value exceeds the Suggested Minimum
      NOTE: \*\* denotes current boot disk
      Disk Device                                              Available Space
      [X]    c0d0                                              8173 MB  (F4 to edit)
                                         Total Selected:   8173 MB
                                      Suggested Minimum:   4727 MB
         F2_Continue    F3_Go Back    F4_Edit    F5_Exit    F6_Help
When you continue, tell Solaris to go ahead and automatically lay out your file systems; here's what I do:
    - Automatically Layout File Systems ----------------------------------- --------- On this screen you must select all the file systems you want auto-layout to create, or accept the default file systems shown. NOTE: For small disks, it may be necessary for auto-layout to break up some of the file systems you request into smaller file systems to fit the available disk space. So, after auto-layout completes, you may find file systems in the layout that you did not select from the list below. File Systems for Auto-layout ======================================== [X] / [X] /opt [X] /usr [X] /usr/openwin [X] /var [X] swap -------------------------------------------------------------------------------- F2_Continue F5_Cancel F6_Help
This will result in a layout that looks something like this:
    - File System and Disk Layout ----------------------------------------- --------- The summary below is your current file system and disk layout, based on the information you've supplied. NOTE: If you choose to customize, you should understand file systems, their intended purpose on the disk, and how changing them may affect the operation of the system. File sys/Mnt point Disk/Slice Size ======================================================================== / c0d0s0 792 MB /usr/openwin c0d0s1 815 MB overlap c0d0s2 8181 MB /var c0d0s3 509 MB swap c0d0s4 517 MB /opt c0d0s5 1066 MB /usr c0d0s6 4471 MB -------------------------------------------------------------------------------- F2_Continue F3_Go Back F4_Customize F5_Exit F6_Help
Now, because of what I mentioned a bit earlier, if you want to avoid problems upgrading your DomU in the future to a newer release of OpenSolaris, you should edit your partitions to make the / partition at least 1 GB in size.

I also like to take the opportunity to bump up the swap size, as I find the default to be a bit small.

Here's a configuration I've had good luck with:

    - Customize Disk: c0d0 ------------------------------------------------ --------- Boot Disk: c0d0 Entry: /opt Recommended: 628 MB Minimum: 534 MB ================================================================================ Slice Mount Point Size (MB) 0 / 1262 1 /usr/openwin 815 2 overlap 8181 3 /var 509 4 swap 776 5 /opt 705 6 /usr 4102 7 0 ================================================================================ Solaris Partition Size: 8181 MB OS Overhead: 8 MB Usable Capacity: 8173 MB Allocated: 8169 MB Rounding Error: 4 MB Free: 0 MB F2_OK F4_Options F5_Cancel F6_Help
When the installation completes, you will be asked to "eject" the disk and reboot the system. When that occurs, you will simply be returned to your shell prompt.

Now, you can run your installed DomU!

However, if you just use the configuration file mentioned above, you'll rerun the installer; instead, you'll need to create a new Python configuration file referencing your newly installed DomU.

Here's what I use; let's call the Python configuration file /export/config/run/

    name = "nevada"
    vcpus = 1
    memory = "1024"
    root = "/dev/dsk/c0d0s0"
    disk = ['file:/export/root/solaris_domu.img,0,w']
    vif = ['']
    on_shutdown = 'destroy'
    on_reboot = 'restart'
    on_crash = 'destroy'
Since I tend to use machines that have at least 2 GB of RAM installed and only run a single DomU, the "memory" line above denotes that I want to give my DomU 1 GB of RAM. This falls within the "55 %" guideline mentioned near the top of this post.

Now, the guest domain can be booted using the command:

    xm create -c /export/config/run/
It should boot just as if you were connected to the serial console of a second machine running OpenSolaris, except you don't need to fuss with BIOS or boot loaders; here's what mine looks like:
    # xm create -c /export/config/run/
    Using config file "/export/config/run/".
    Started domain nevada
    SunOS Release 5.11 Version xen-nv66-2007-06-24 64-bit
    Copyright 1983-2007 Sun Microsystems, Inc.  All rights reserved.
    Use is subject to license terms.
    ip: joining multicasts failed (4) on xnf0 - will use link layer broadcasts for m
    Hostname: chaser-guest
    NIS domain name is boulder.Central.Sun.COM
    /dev/rdsk/c0d0s6 is clean
    /dev/rdsk/c0d0s5 is clean
    chaser-guest console login:

I know this has been a lengthy post, but it's really much simpler than my verbosity would indicate.

Why not give it a try?


Can't this process be pretty quickly wrapped in a gui container via scripts like zfs currently is? That would be quite useful for folks that want to get started and certainly would raise the usage level.

Posted by ylon on August 25, 2007 at 11:44 AM MDT #

By the way, isn't this a big win for the Xen folks (I guess Citrix now...) in that you have dtrace in place and really can use this to squeeze performance out of various portions of Xen now based upon tracing? Just kinda curious.

Posted by ylon on August 25, 2007 at 11:46 AM MDT #

Post a Comment:
  • HTML Syntax: NOT allowed



« October 2016